At the moment column filter handling is quite restrictive in which you can
only specify an inclusion list or exclusion list. As new columns arrive in
workflows which always happens in life sciences industries as new assays (and
therefore new data) arrives its how this new data is handled in workflows
with ideally minimal user intervention.
Can additional column filter nodes be provided where they are filtered by a
string match (like the row filter node abilities), and also column filter by
column type (I.e. string, double, sdf, smiles etc).
Whilst I’m on the subject of workflow automation, please can the column
resorter node be modified so it can handle new columns without causing an
error, I.e. just appending any new columns to the end of the table.
The column splitter node also seems to result in new columns to disappear
unless node is reconfigured. Ideally node needs an option to specify whether
new columns are parsed to top port or bottom port, I.e. based on the
inclusion/exclusion list logic from column filter node
These are common and simple problems which come up time and time again and it
would be great if more options are available to handle new columns introduced
into a workflow after it was generated.
For me this handling of new column data is one area knime needs to improve
on going forward, it’s a common frustration I hear from users who build
complex workflows and one new column introduction causes untold errors which
novice users have no idea how to fix.
Simon.
I agree. We have this on our list for the next release and will subsequently adapt most of the nodes using the new Column Filter panel. This makes all those nodes more flexible in the sense, when using them in an environment with changing columns.
That’s great news. One thing I would say is most people have no idea what the inclusion / exclusion logic means in the column filter node, and yet its really powerful. If you are incorporating this in to other nodes (which is brilliant news) I would also try and better explain its usefulness in the node description. even some of our knime developers didn’t realise what it did!
Thanks
Simon.