Table row variable looping - can this be applied recursively?

KNIME Analytics Platform
1

Hi all,

 

I'm wondering if this is possible (yet) in KNIME?

I need to loop recursively over a 1-column table; each iteration performing a string replacement and using the transformed output for the next replacement. This can be done easily with the recursive loop start/end nodes. The tricky part is that the lookup pattern needs to be handled dynamically: I have defined a list of lookup patterns that all need to be replaced by the same replacement pattern (missing value). That in itself I can handle easily with a table row to variable loop start (1 iteration for each lookup pattern, that is passed to the string replacer node as a flow variable), but this starter node is then not compatible with the recursive loop end. 

So, is there any way to either:

- add recursiveness to the table row to variable loop

- or add variable handling to the recursive loop?

My end-goal is an extension of following topic (https://tech.knime.org/forum/knime-general/find-and-replace). I too need to find and replace a pattern over an entire table, the difference being that I have multiple lookup patterns. One solution I found is to feed a dictionary to the cell replacer node (here I can use the table row to variable loop and don't need recursiveness because I haven't combined the columns), and do this for each column. But looping over a lot of columns seems a lot more computationally expensive then looping over a much smaller list of lookup patterns. My original table is 400K+ rows x 50+ cols...

PS: A recursive loop end node that is multi-compatible to any kind of loop start node would be sweet! (No idea if I'm dreaming of something that is programmatically impossible)

Any thoughts on this would be appreciated!

Regards,

Elissa

2

Hi Elissa,

when you go to this Video https://www.youtube.com/watch?v=0zhDVBd2ihk starting around 5:00 I talked about this.

The flow is on the example server under 011008_recursive_replacements.

Does this help?

Best, Iris

3

Hi Iris,

That is such an elegant solution, thank you for sharing!

The only sad thing is that after the transformation I still have to loop all columns a single time with a Java snippet to obtain missing values as replacement pattern. Replacing with "" in the recursive loop only yields empty cells upon splitting the transformed string back to the original columns (even though there's a setting to handle missing values, but neither of the settings yield missing values oddly enough), and they're hard to do anything with in KNIME. But empty string handling is an issue that has been discussed elaborately in other corners of this forum ;)

You have my gratitude!

Regards,

Elissa

 

EDIT: writing the intermediate results to disk and reading them back in with the File Reader node allows you to replace the dummy missing value replacement with real missing values in way that I believe to be quicker then column looping.