I am new to KNIME and was wondering if there is a way to have a workflow to create a backfill dataset as such:
BigQuery Connection – > Run SQL Script (This has a DECLARE statement for date) – > Writes Results into a Table – > Changes the INTERVAL component of the date (For example from 1 week, to 2 weeks – > Re-run the SQL Script – > Append again – > Loop until a condition / date end.
I am basically trying to create a historical backfill over the span of 2 years, week by week. The result of the script is a weeks worth backfill, so I am trying to run this script 100 times, each time to change the script interval by 1 week.
The script works fine on BigQuery, I just don’t want to tediously run it 100x, and wanted a way to learn how to create historical backfills more easily for this, and for the future.
Thanks again for any help or tips.