Bug? report: Transpose "forgets" column type for missings

Dear Knimers,

Maybe this is a bug, maybe there is a reason for this behavior that I simply do not understand.

Consider the following table:
image

All the values in the single column of this table are obviously of type String.

However, when I apply the Transpose node on this table…
image

…Knime seems to be unable to figure out the type of column “Row2”, even though it can only be a String. This can be a problem, for example when applying the Missing Value node on a table like this because it is not possible to set a Fixed Value on a column of unknown type.

Best,
Aswin

Edit: obviously it is possible to create a column with a missing value of type String directly with a Table Creator node:
image

Hi @Aswin,

While transposing, Having no data type in a row from the initial table results in “MissingCell” column type and having multiple data types results in “Non-Native”.

multi0

multi1

multi2

The Missing Value node knows these column types:
missing

As you said, one cannot use a fixed value for these columns using the Missing Value node. Instead, as all the values in a “MissingCell” column are missing, you can use e.g. the String Manipulation node to assign a fixed value to the column.

I think the actual problem is with the “Non-Native” column type where neither the Missing Value nor the Rule Engine node (if “Replace Column” option is selected) can be used to replace missing values with some fixed value.

One can use the “Append Column” option in the Rule Engine node to handle missing cells of “Non-Native” type or the Column Expressions node to replace the column.

:blush:

4 Likes

Dear Armin,

In your transposed table ALL columns are now of type “?”. As you say, this is logical, because your original table had both a String and an Int column. After transpose, an Int and a String end up in the same column, but one column cannot simultaneously contain Strings and Ints, hence type “?”. It is interesting that there are different types of “?”, but usually they should all be avoided, I think.

However, the Transpose node is almost exclusively applied to tables that only have columns of one single type. Tables with all columns of type String, or all columns of type Int, etc.

My problem arises when a row of such a one-type table only contains missing values. After transpose, the row of missing values becomes a column with missing values. Knime could assign the same type to this column as to all the other columns, but for some reason it does not do this. Instead, it unnecessarily assigns the type “?”. I think this is a bug, or at least undesirable behavior.

Best,
Aswin

I have to confess that handling missing values in multiple “MissingCell” or “Non-Native” columns all at once is a problem. But I still agree with having “MissingCell” column type. It seems rational to me. But I think there should be more support for these types in other nodes.

:blush:

Oh, I have absolutely no problem with the type MissingCell existing

My problem is with the MissingCell type unnecessarily appearing after transposing a normal table of a single regular type.

Edit: I should emphasize that it is perfectly possible to have a column of type String/Int/whatever containing only missing values. The Transpose node, however, refuses to produce such a column.

1 Like

Hi there @Aswin,

Only offering a workaround: Add additional column of type your new column should be. After transposing filter out last row. Idea from other user here.

Br,
Ivan

4 Likes

Another workaround: treat the transpose as a special case of pivoting:

image
Start:
image

Result:
image

All columns in the result are of type “String”, as desired. This also works when you start with multiple columns.

3 Likes

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.