deleting rows with missing values spread over multiple columns

[mgamer]

Hi all,
I have the following problem. A table with a lot of column has missing values in various rows, I now have to delete all rows containing a missing value in a column. My first attempt was to loop over the column an apply a row-filter (see workflow-screenshot). This works well for each iteration but then joining the columns together we have all the missing values back alive. Is there any way to get things done here?

[bruno29a]

Hi @mgamer , how many columns do you need to evaluate? And do they all have to be missing for the same row for the row to be filtered out?

For example, let’s say you have 3 columns to evaluate, Columns A, B and C, and you have the following data:

RowID | ColA | ColB | ColC | ColD | ColE
1     | ?    | ?    | ?    | ?    | ? 
2     | ?    | ?    | ?    | ?    | 123
3     | ?    | ?    | ?    | 123  | ?
4     | ?    | ?    | ?    | 123  | 123
5     | 123  | ?    | ?    | ?    | ?
6     | ?    | 123  | ?    | ?    | ?
7     | ?    | ?    | 123  | ?    | ?
8     | 123  | 123  | ?    | ?    | ?
9     | 123  | ?    | 123  | ?    | ?
10    | ?    | 123  | 123  | ?    | ?
11    | 123  | 123  | 123  | ?    | ?

Which of the RowIDs should be filtered out?

[HansS]

Hi @mgamer

Did you check the Column Aggregator node with the Missing Value Count option?

gr. Hans

[mgamer]

It is about 20 columns and all the rows, that contain a missing value should be filtered out.

[mgamer]

Maybe a good idea, I’ll give that a try!

[mgamer]

tried it - worked… very cool idea, thank you very much!

[ipazin]

Hello @mgamer,

and what about Remove Row option from Missing Value node?

Br,
Ivan