I have a dataframe with around 2.000 rows.
I loop through these rows and perform a series of calculations for each of them, so it takes a long time to compute.
At some point I need to shut down Knime, and I would like to restart the workflow wherever I left it. In order to do this I though:
First step. (Executing the workflow the first time)
- Write a csv file where the rows entering the loop are appended (this is actually done through a flow variable).
Second step. (Relaunching the workflow after stoping it)
- Read the csv file and get the last row.
- Find this row in the dataframe that is entering the loop.
- Ask the loop to start at that given row.
@RoyBatty296 that sounds like a good idea. With CSV writer you have the option to append if the file already exists so you could collect the lines (RowIDs) that you already have processed.
I generally am able to find ideas, the problem is I don’t know how to implement them.
I was thiking of using an If switch depending on whether the csv reader is empty or not.
But do you have any suggestion on how to implement it?
@RoyBatty296 maybe you can take a look at this example:
The current RowID added to a ‘tracking’ CSV that you can use to reference the already processed lines at the re-start of your loop.
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.