My Loop Runs too fast

KNIME Analytics Platform
#1

Hi 

I have a workflow that runs extremely fast and that should not be a problem under normal instances, but I am doing a simple loop that creates a different excel file after every iteration. My problem is that the excel writer node takes sometime to complete at the end of each loop, but the next iteration starts immediately. Sometimes the excel file is not fully produced giving me an error.  

Is there anyway of slowing the loop down so that the excel file has a chance to be created.  I could of course add some additional nodes to slow the loop down, but is there a more elegant way?

#2

Hello,

Maybe you should decrease the memory Knime can use, by modifying the knime.ini file.

I don't know if it is going to work.

#3

Hi Macca,

your Excel Writer must be inside the loop so it must be connected to the loop end.

You can do this by connecting the flowvariable port from the excel writer with the loop end node.

Cheers, Iris

#4

I tried this but I get ther error:

"WARN      XLS Writer   Errors loading flow variables into node : No output filename specified."

I am using a variable in the XlS writer to create dynamic path for each file.  Any other suggestions?

#5

Hm, that's strange, it should not have any influence of the incoming variables.

Can you post a screenshot of the loop?

#6


#7

Has merging the flow variables from the Row Filter and the TableRow To Variable nodes failed? 

You could try merging them with the merge variables node and then passing them into the XLS writer. 

 

 

Although it looks like this could warrant the use of a different setup. Are you using group by to get a set of unique values and then iteratively filtering based on this value? Have you tried using the group loop start node? This may allow you to not have to merge flow variables. 

Cheers

Sam

#8

I guess this warning (not error!) only occurs before the first iteration. The variable does not have a valuew until the first row of the input table is read. This is where the warning from the XLS writer comes from. As soon as the loop start is executed this warning should go away.

The same holds for the row filter.

#9

You can also try to replace the java snippet, the tablerow to variable and (maybe) the table creator with one java edit variable. Which directly creates the flow variable (which is the name of the xls file I guess)

#10

Thanks for all the advice, especially Iris

I replaced the "Table Creator", "Java Snippet" & "TableRow To Variable" node with a "Java Edit Variable".  This increases the speed to generate the file path thereby the file can be created before the start of the next loop iteration.

So thank you all again I hope to see you all at the KNIME UGM next week!