Remove empty columns

KNIME Analytics Platform
1

Hi,

I'm sure there must be a simple solution to this, but I can't think of it.

I have a file with multiple columns (>250), a lot of these columns have no data in them for any of the rows.  Is there a workflow/node that will examine each column, inspect it for any data in any of the rows and discard/filter the column if all the rows are empty to leave only columns with data left.

Thanks,
Angus

2

Why not use the groupby node. In the node, don't group on anything, but in the options tab, aggregate on all columns using Sum. This now condenses everything to one row, showing if any data is present.

then use the transpose node, and use the row filter to only leave those rows with data present. So choose to exclude those with missing data. 

then transpose back.

Finally you now have the column names which contain data, use this with a reference column filter node on the original table.

hope that helps,

simon.

3

Thanks Simon, that worked.

Angus

4

The XLS reader has the option Skip empty columns.  Maybe that could help.

Stephen

1 Like
5

There is also a Low Variance Filter node which can be used for this purpose as well.

 

6

Thanks Aaron, such an underused node on my part. I forget about this one.

simon.