Type aware replacing

I’m working with a dataset that has missing values and therefore use the Missing Value node to to replace missing values with reserved ones depending on the column type. I do this to be able to merge on missing values.

Now i want to undo this replacement before writing to file, but i struggle to find a way to do this depending on the column type.
Using a Multi Column String Manipulation node works well if i convert each cell to string, but i lose type information and ids are converted to decimal representation in a string type column.

How can i achieve type aware manipulation similar to the missing value node?

Hi @Ellison, I’m not sure I fully understand what you mean by “undo this replacement”. Are you trying to put missing values back, or find a different way to remove them?

As you have mentioned, the missing Value node can perform type-aware fixed replacement for Double, Integer, and String which should retain type, so is it the reversal of this that you are trying to do?

(i.e. turn specified fixed values back into missing values ? )

Could you upload a small sample dataset and describe/demonstrate what you are trying to do with it.

If you have a table of String fields, and you are trying to cast them back to their original data types, I do have a component that might be able to help you:

Feed your “Strings” table to the upper port, and the original table to the lower port, and see what happens :slight_smile:

Exactly, i want to reverse it back to missing values. I’ll have a look at your node, i think it should do exactly what i need.

Hi @Ellison, depending on the specifics of your use case, if the data that came from the original table with missing values is otherwise unchanged (e.g. you have simply added new columns through a join process or other manipulation), you could make use of the Table Updater node (KNIME 5.x only), to put the original values back.

It would look like this:

Original data:

After Missing Values node and other processing (addition of new column):

After Table Updater (original table values are returned):

The caveat here is that the RowID in both the original table and the new table have to be the same, since Table Updater keys by RowID.

It’s only taken me 6 months to find an actual use case for the Table Updater node! :wink:

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.