Weird column type behavior

Feedback & Ideas
1

Dear Knimers,

I guess you guys have grown tired of my attempts at comedy, so allow me to switch to magic tricks…

Here we have a table with two columns of type Double:
image

I can transpose it, which gives me a table with 2 rows and 4 Double columns:
image

Let’s remove the second row with a Row Filter. This yields a table with 1 row and 4 Double columns:
image

Transposing that one gives me a table with 4 rows and a single Double column:
image

Now transposing that one AGAIN (drumroll)…
image

…gives me a table with 4 different types: MissingCell, Double, Boolean, Integer!!

How is this possible? The original table was constructed using Concatenate on tables with identical column names but different column types. Apparently, even though the columns are converted to type Double in the combined table, the cells still somehow remember their original types. This is totally invisible to the user, though.

Best
Aswin
KNIME_project22.knwf (18.2 KB)

6 Likes
2

By the way, applying a “Table Row to Variable” node on the third table yields 4 Double flow variables…
image

…but using using a “Create Collection Column” on the the same table somehow induces a type change:
image

Applying a “Table Row to Variable” node on the final table fails completely:
image

1 Like
3

Starting with the final table of my original post, I can unpivot, which gives me a column of Doubles…
image

…which I can then pivot again, using the “First” aggregation method, which gives me 4 Double columns:
image

However, when I use the “Concatenate” aggregation method, I get this:
image

1 Like
4

Great find @Aswin !!

1 Like
5

So far, I have found the following ways in which these “hidden types” can be created:

  • Concatenation of 2 tables with identical column names but different column types
  • Transposing a table with different column types
  • Unpivoting a table with different column types
  • Creating a collection column from columns with different types and subsequently ungrouping this collection (this works with both the Column Aggregator and the Create Collection Column nodes)

It looks as if every operation that somehow moves values from multiple columns with different types to a single column results in these “hidden types”.

How to make the hidden types appear?

  • Transposing a column with hidden types (only when there are no other columns that “overrule” the hidden types)
  • Grouping values from a single column into a list and subsequently apply Split Collection Column (with “Determine most specific type” checked) on this list
  • Pivoting (for some but not all aggregation methods)

Operations that interestingly do NOT make these hidden types appear:

  • Chunk Loop Start - RowID - Loop End (Column Append)
  • Column to Grid

However, even in those cases the hidden types are still there; they are just remain hidden.

Best
Aswin

2 Likes