I use a wrapped node, which protects the scope of any flow variables within it, and use quickform nodes to provide configuration options. These allows for parameters and selections of input table values.
I have found that with this any corruption of flow variable from different parts of the workflows can be prevented. I also use call local workflow node to create factories so I can have a generic workflow that passes control to local workflows to deal with anything specific. This keeps the scope of the flow variables precise to where it is needed.
It is not strictly necessary to delete them. But it would be nice if that were possible.
I am working on a fairly large workflow and there are many variables that are only specific to certain parts of the workflow but they still show up in the setup for nodes that are further downstream.
Certainly, wrapped nodes/metanodes can help, but they don't always provide what I want. Sometimes I want to export a specific variable from a metanode while discarding all other variables. I don't think this currently possible.
For me it is mostly an issue of having a clean workspace while still being able to express myself with any variables that I would need to describe what I want to influence. In most programming languages you also have local and global variables that you can use to give names to intermediate or permanent values. And the local variables are always gone after a function returns.
Hi, it could be necessary in some situations. For example, if I use the extract context properties node, it creates incompatibility with the Python nodes that throw the error "Requested string value from cell with type: STRING".
So if I use the David Ko trick wrapping the extract context properties and keep only the variable I need from context and rename it the error disapear.
I try having a more canonical view on that:
A variable is defined by a name, a type and a visibility. “name” is the unique identifier within the “visibility”. “type” defines the range of valid values and allowed operations. “visibility” tells something about who is allowed to read/modify a variable (realm).
A variable from a realm may under no circumstances be accessible from another realm (security) - else wise the value is no more trustworthy and/or a security leak may arise.
From that point of view, a kind of “garbage collection” or “limited visibility” makes sense!
@aborthwick: A common approach to circumvent these situations is by using realm-bound prefixes (e.g., variables used during data load are prefixed “dl” while variable used within processing are prefixed “pc”).
I would use wrapped metanodes for encapsulation flow variables.
But, there is one usecase, which prevents using wrapped metanodes:
I have hundreds of interactions with databases and I can’t using database nodes within wrapped metanodes, if I want to start my workflow as a scheduled workflow on knime server.
The reason is, that I can not get the workflow-credentials into the wrapped metanode! The only way to do this, is to use quickforms for setting username/password. But this won’t work with scheduling…
So I have to deal with all flow variables along the complete workflow…
Another thing is: I created some generic Nodes for our organisation which contains database-stuff. But with wrapped Metanodes we cannot use these (see above). And so, all colleges get flow-variables they won’t have…
The inability to filter flow variables really causes hassles in creating large workflows that have duplicate process components or node collections. It builds up a massive confusing backlog of flow variable clutter, it causes duplication of flow variables when reading and writing variables to memory, and in general it is just bad practice to leave a confusing mess of “single process use” flow variables in the larger workflow pool at risk of being used in error on a later date… I don’t get why anyone wouldn’t want the ability to filter variables? Is this still not possible?
From my side and regarding the issue with ‘overwrite Path Variable’ , as mentioned in the topic:
“All these workarounds could be avoided with the implementation of an ‘Overwrite Variable’ function within the ‘String to Path (Variable)’ node.”
An alternative option could be to release a node update with an ‘Overwrite Variable’ option, as it is included in ‘Java Edit Variable (simple)’ node, see picture . It would avoid some workarounds.