Recursive loop with rows updated in chunks

[Jaqen79]

Hello everyone,
I can’t find a solution to a tricky problem, that I guess needs to be addressed with a recursive loop combined with a chunk loop, but despite several attempts I’ve been unable to configure it. The solution must use just native KNIME nodes, no R, no Python.

Here’s the issue:
Imagine you have a matrix with K columns (where K can change) and N rows (where N can change). The values in the matrix are floats. For example, something like this (4 columns, 14 rows):

For each pair of columns, I have to apply a transformation that changes the initial values, updates the values in the starting matrix, and continues until a pre-defined number of iterations.
Using the matrix in the example, since I have 4 columns, there are 6 possible combinations (AB, AC, AD, BC, BD, CD) and the steps must be as follows:

Loop 1, six steps:

1. select columns A and B → values transformation → update the values of A and B
2. select columns A and C → values transformation → update the values of A and C
3. select columns A and D → values transformation → update the values of A and D
4. select columns B and C → values transformation → update the values of B and C
5. select columns B and D → values transformation → update the values of B and D
6. select columns C and D → values transformation → update the values of C and D

Loop 2: I repeat the steps of loop 1 with the modified values.

Loop 3 - Loop n: same steps repeated multiple times

Any idea?
Thanks

Daniele

[HansS]

Hi @Jaqen79 , Daniele

This was not easy. At least for me. But I think I got it working for you. I started with the recursive loop, but it turned out that that was not the way to go. So the wf writes and read a KNIME table every loop.
The number of columns and the column names does not matter, it is all made dynamic by using flow-variables.
Take a look and see if it helps you out. not_so_recursive.knwf (116.8 KB)

gr. Hans

[Jaqen79]

That’s brilliant! Thank you, I insisted that the recursive loop was necessary, but the solution lay elsewhere… I have tested your solution and it works fine.

Do you think it would be possible to change your workflow to avoid writing/reading the external table? I want to embed everything in a component and so it’s better not to write any table.
Thanks again,
Daniele

[HansS]

Hi @Jaqen79 / Daniele

Glad I could help you with a solution. Because I couldn’t put the TableRowToVariable Loop inside the Recursive Loop, I ended up with this solution of writing to and reading from a file. And that is indeed not an ideal situation.

What you can do is to use a relative path in the TableWriter and TableReader nodes. That would mean that the workfow as a Component could run on any machine.

gr. Hans