Issue filtering column names

[Lauri_Tuumi]

Hey! I’m trying to use a CSV input table to filter columns from my input and I’m having issues with the column filtering.

The workflow:

So we’re taking data in at ‘Input Statistics’, renaming some columns. Then we want to filter our columns down to the list of columns from ‘Data Model Template’.

I cannot seem to be able to get the filtering to work

Here’s a screenshot of the ‘Variables connection’ of ‘Table Row to Variable’

And here’s a screenshot of the ‘Filter Excess Columns’ Flow Variables dialog:

And finally a screenshot of the output from ‘Filter Excess Columns’:

Columns like Fuel Surcharge, Mobility Package Surcharge, Marpol Surcharge and Maut are not a part of the Data Model Template and thus should be filtered out.

I tried to mimic the workflow from Column filter flow variable – KNIME Community Hub

But the filtering just does not seem to work. What could be the issue?

[takbb]

Hi @Lauri_Tuumi , can you show what you have as the main config dialog for the Column Filter, and which version of KNIME are you using.

Basically if you have the include names being set as a flow variable, you want the main config dialog to look like this, with no columns in the “included” list, and “enforce inclusion” set. So even columns that you will ultimately want included (contained in the flow variable) should be put in the “excludes” list on this dialog.

The equivalent in the newer Column Filter in Modern UI is this:

After that, as long as your flow variable has the correct column names, it should work.

(welcome to the KNIME forum!)

[Lauri_Tuumi]

Thanks for the prompt reply! This solved my issue Definitely not a clear interaction between the dialog and the flow variable for a new user such as me.

[takbb]

You’re welcome @Lauri_Tuumi . You’re right that this particular one is not obvious and I had the same problem when first using that node in this way.

But the issue arises because we assume (understandably) that having told KNIME to include a specific set of columns, it means it will exclude everything else. But it doesn’t quite work like that.

Effectively having told knime to include all the columns in the flow variable, it then looks to see what you had configured for all the other columns that aren’t in the flow variable and if the manual configuration says to include them, you get them as well…

So yes it’s easy to get caught with this one but hopefully it’s now a little clearer, and I’m glad you got it working. Thanks for marking the solution.