Strange `Ungroup` node behaviour with mix of missing and empty collections

I have just encountered some very strange behaviour with the Ungroup node when there are multiple columns being ungrouped and some contain missing and / or empty collections.

Consider:
image

(Which contains all combinations of ?, [] and a single-membered list [Foo] (for simplicity))

Ungrouping on only Collection 1 works as expected - i.e. if neither Skip missing or Skip empty collections options is checked, then all input rows are represented in the output table, rows with ID starting 0_ are all lost if we select Skip missing, rows with ID starting 1_ are lost if we select Skip Empty collections, and only row with ID = 2_2 makes it through if we select both options.

Howeverā€¦
If we ungroup on both columns, then we should still expect all input rows to be represented in the output table with neither ā€˜Skipā€¦ā€™ option selectedā€¦, but:
image

Weā€™ve lost ID = 0_1, which had on each of ? and []ā€¦ Not nice, and not what I was expecting!

If we select either Skip Missing or Skip Empty Collections options, then we lose the expected ID rows (0_0 or 1_1 respectively - both these have only the skipped type, ? or [] in both columns) AND we also lose the ā€œrogueā€ ID = 0_1 row in both cases, which has one of each filtered value - not really expected, because I would expect it to be retained because it is not excluded rather than the other way round, butā€¦ open to interpretationā€¦ maybeā€¦!

Now the final case - select both Skip Missing and Skip Empty Collections, and now we get really unexpected againā€¦
image

Now the output contains only the rows with neither value present in either column - i.e. we have lost data from those rows with (? or []) and [Foo] -thatā€™s really not expected to my mind?!

Example workflow which may explain better at

Thanks!

Steve

Moved this to Feedback and Ideas section :-). Thanks for what looks very comprehensive documentation which Iā€™m sure the KNIME team will appreciated!

1 Like