Bug on missing value filter on last release?


I’m using missing value filter on a “set” typed column, and it seems it is removing columns even if I have data inside them. It happens since the last upgrade, is it a known bug ? (sorry new to the forum, there might be a place for that)


Cannot reproduce the problem on columns of type “Set (Collection of: Number (double))”. Does the “Value Counter” node report missing values?

Working with a set of strings, and it is not always happening after some tests
Value counter gives 699 missing values et 35 set values

The missing value column filter removes columns that have more than a specific percentage of missing values. The default value is 90% or 95% I think, and with 35 sets and 699 missings you are above the threshold. Maybe it should be set to 100% for your purposes?

(Edited for clarity)


Thank you @Aswin for the explanations!

Yes you need to set it to 100% if you want to unsure only columns with missing values only will be filtered.

Cheers, Iris

Oww my bad, I’ve never actually opened the configuration window for this node and was assuming it only removed columns where all values were mising! Learned something today, thanks!

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.