I have been successfully using a flow variable to select the included column from the ‘right’ table in a Joiner node.
I have updated KNIME to version 5.5.1. The Joiner node has been updated and when I click the flow variables button for the Bottom input (‘right’ table) section I get a message saying ‘The following set flow variables are deprecated: “model.rightColumnSelectionConfig.included_names”…’
It’s pretty annoying for an update to change nodes in a way that breaks workflows, but what’s even more annoying is that I don’t seem to be able to fix it!
I’ve created a flow variable containing the name of the column I want to select and applied it to rightColumnSelectionConfigV2.manualFilter.manuallySelected but the named column is not being joined. I’ve tried adding a flow variable containing the string ‘MANUAL’ and applied it to rightColumnSelectionConfigV2.mode but this hasn’t helped.
Can anyone tell me what I’m doing wrong? Or is there a bug in the new node?
t certainly looks like the underlying configuration for the Joiner node has changed in the 5.5 release, although I have not been able to find any reference to the change in the 5.5 changelog.
I just created a demo workflow in 4.7.8 which I then imported into 5.4 and 5.5.
In 4.7, the flow variable config for two included column selections looked like this:
In 5.4, with Modern UI config, I can only view a small section of the config at one time, so the Bottom Input column selection config appeared as this:
In 5.5, you are right that the config has changed. As far as I can see, the node still works using the “deprecated” config that has been imported, but if you need to change the config of the flow variables, you will first see this:
**To make flow variable config changes, it is necessary to click “unset” then reconfigure the flow variables and then configure using the new settings **
e.g.
I don’t think that this is a bug as such, but I agree that this isn’t ideal especially if there are a number of different config items being set by flow variable, and you have to find the new names for each of the items! Again not ideal.
It looks like the change to the flow variables config has likely come about with the inclusion of “Type” selection as a method of choosing output columns in the 5.5 release.
Thanks, that’s a useful summary of the changes. The problem I’m having is that I add a flow variable (containing the name of a column in the right table) to the Overwrite manualFilter.manuallySelected box, but when I execute the node the column is not added to the output table. Does it work for you?
What is it you are seeing in the output? (e.g. all columns or no columns?)
Just a thought but before setting the flow variables, did you make sure that all columns + any unknown columns" was set to be excluded? (i.e. just on the specific tables where you are going to specify flow variables):
Final thought… you did have the selection set to “Manual” didn’t you (not Wildcard, Regex or Type) and the case of the column name is exactly right?
That could accidentally affect the result.
Thanks. It appears that the ‘Any unknown columns’ setting is crucial. If this is on the ‘Includes’ list then the flow variable override doesn’t work.
The behaviour of flow variable settings for this part of the node configuration has changed. It used to be possible to put columns into the ‘Includes’ list in the configuration window and use a flow variable to add further columns. Now the flow variable completely overrides the list, and only the columns in the flow variable get included. Which is a pain, as now I either need to find a way to add other column names to the flow variable, or do the join twice, once with the columns I always require, and once with those set by a flow variable. Not sure that any of this is an improvement!
Hi @simonrayner, I hadn’t noticed a change in behaviour from a cursory check, but I’ve not tested exhaustively.
I’ll see if I can confirm this behaviour based on what you’ve described but to be sure would it be possible for you to upload a small demo workflow that demonstrates the problem you’re seeing that can be tested in a previous version of KNIME and compared with 5.5.
Incidentally what was the previous version you were on?
I just tried building a demo in 5.4.4 and there the ‘All unknown columns’ setting doesn’t affect it!
I don’t have an older version of KNIME than this, but in the version where model.rightColumnSelectionConfig.included_names had not been deprecated the flow variable added columns to the include list rather than overwriting the list.
Thanks for the info @simonrayner . I haven’t yet tested this in 5.5, but I concur with your view that in earlier versions the supplied flow variable appended to the included column list rather than replacing it.
