Re-start a loop at a given row

KNIME Analytics Platform
1

I have a dataframe with around 2.000 rows.

I loop through these rows and perform a series of calculations for each of them, so it takes a long time to compute.

At some point I need to shut down Knime, and I would like to restart the workflow wherever I left it. In order to do this I though:

First step. (Executing the workflow the first time)

  • Write a csv file where the rows entering the loop are appended (this is actually done through a flow variable).

Second step. (Relaunching the workflow after stoping it)

  • Read the csv file and get the last row.
  • Find this row in the dataframe that is entering the loop.
  • Ask the loop to start at that given row.
1 Like
2

@RoyBatty296 that sounds like a good idea. With CSV writer you have the option to append if the file already exists so you could collect the lines (RowIDs) that you already have processed.

2 Likes
3

Thanks @mlauber71.
I generally am able to find ideas, the problem is I don’t know how to implement them.
I was thiking of using an If switch depending on whether the csv reader is empty or not.
But do you have any suggestion on how to implement it?
Thanks!

4

@RoyBatty296 maybe you can take a look at this example:

The current RowID added to a ‘tracking’ CSV that you can use to reference the already processed lines at the re-start of your loop.

image

2 Likes
5

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.