Show duplicates within a column exclude blanks

[mlauber71]

@Darius you could first mark the three columns in question if they have a MISSING value with the Rule Engine. Then exclude these rows that have all columns missing. You could then replace the missings with another value with the missing values node.

Then you might concatenate the three columns into one (new) column. So if one row is unique in all three columns the new column would be identical. And then deal with the remaining duplicates.

School of duplicates - and how to deal with them

Dealing with duplicates is a constant theme with data scientist. And a lot of things can go wrong. The easiest ways to deal with them is SQL’s GROUP BY or DISTINCT. Just get rid of them and be done. But as this examples might demonstrate this might not always be the best option. Even if your data provider swears your combined IDs are unique especially in Big Data scenarios there might still be lurking some muddy duplicates and you should still be able to deal with them. And you should be able t…

You might want to make sure you have a unique RowID or something in oder to be able to bring back your results to the original data (and deal with the remaining columns you want to keep).