Parallel Chunk: Add option for more table ports and sort variating column order

Hi,

using the Parallel Chunk Nodes it can be tricky to ensure table specs are identical. I once found an approach but frankly speaking, that adds quite some maintenance overhead i.e. when the table specs change on purpose.

https://forum.knime.com/t/parallel-chunk-end-w-changing-table-specs/14452/8

Therefore, I’d like to suggest to either allow changing table specifications, a more blunt approach, or, similar to the pretty much all loop end nodes allow to add more ports.

image

PS: I later on found that chunks can give back the same table but with different column sort order. Complementary to the Concatenate node I would propose to auto-sort/align columns if their super class matches instead of allowing variating column specs.

Best
Mike

Hi @mwiegand ,
thank you for the suggestion for the voting!
Best,
Daria

1 Like

I ran into this as well. Lucky for me, all my nodes I was running in parallel were streamable, but that’s not the case all the time.

I would say that all loop ends should have the option to allow for changing table specs. :slight_smile:

Adding one more thing to this request as I encountered that issue:

(Collection of: String) and does not comply with its supposed superclass Set (Collection of: String)

It’s cause is that the column order gets scrambled during execution. Though, the Parallel Chunk Nodes do a concatenate of the tables. However, compared to the regular concatenate nodes, the Parallel Chunk End does not take care of the column order.

Hence, my suggestion to auto-adjust the column sort order as long as their corresponding super class does match.

That would cater for the scenario of variating table specs and ease the feature implementation substantially.

Best
Mike