Difference in flow variable handling between Metanodes and Components

I realised today that there is a counter-intuitive difference between the way that Metanodes and Components handle multiple flow variable input ports.

It’s probably a scenario that isn’t that common, but if you ever need this functionality in a Component, you will find you hit a wall as I did.

I would therefore request a modification to the Component Input node, so that we can decide whether the flow variable flows should be Merged or kept separate within the Component.

This example will hopefully better describe and demonstrate the problem.

Here I have three demonstration flows. It is a contrived minimal example to demonstrate the issue.

The first is taking two variable flows.

Both flows contain a string variable called “message”. In the upper branch the message is “Hello”. In the lower branch the message is “World”. It performs a String Manipulation on each variable to make them upper case, then turns them into a table row and finally concatenates the rows to form.

For good measure, I also added a unique variable to each of the flow branches “top flow” and “bottom flow”

The concatenation generates the following output table:

Having built this “complex” :wink: workflow, I decide to turn it into a metanode to tidy things up. The second example shows the metanode version, and again it generates the required output as above.

Internally the metanode is just like the original flow:
image

Now, being the type of person who thinks that this may have application elsewhere, I decide to turn the metanode into a component.

In the component’s Component Input node, I instruct it to allow in all variables, and internally, the component looks very similar to the metanode.

So similar, that you would probably expect them to behave the same way.

However there is a behavioural difference. The resultant output from the component is still 2 one-row data tables, but when concatenated it forms:

This comes about because the Component Input node currently acts like a “Merge Variables” node, and it is not possible for this component to ever see the message “WORLD” internally that came on the lower flow.
image
We can see that variables came in from both ports, but where the same variable name appears on both, the upper value takes priority. The variables “seen” on both input flow ports of the Component Input node are the same, no matter what the actual input flow variables were.

What I would like to see is a change to the Component Input node, where a configuration option allows us to choose whether it should Merge Multiple Input Variable Flows (i.e. existing/backward compatible behaviour) or Separate Variable Input Flows (i.e. the more intuitive and logical behaviour), compatible with the way that Metanodes work.

A possible alternative solution would be to allow variables from each input flow to be prefixed with a different value for each input,

e.g.
in1:message
in2:message

but personally I’d rather see them stay separate, thereby being an exact behaviour match for the metanode it is supposed to be replacing.

To be honest, I’m not sure what would be a better solution to this problem.

Components are meant to be encapsulated and independent. In theory, they should not accept any external flow variables by default unless explicitly specified. If we follow this theory, the current situation should not change.

I agree that that by default we have no external variables entering the component, and I agree that components are self contained. But components can be configured to accept “outside” flow variables on multiple input ports, and where we do elect to let them in, I feel that it should be for the internals of the component to determine what to do with them.

My specific use case here was that I wanted to make a component that acted like a Merge Variables, but would allow the user to choose which flow variable stream had priority (top or bottom), so as to enable optional overriding (through configuration).

image
Unfortunately it is impossible with the component to differentiate variables on the upper port from those on the lower port because they get automatically merged.

My component, which I would have a use for elsewhere, had to be rewritten to work with variables pulled in on data ports instead, where it then chooses how to merge them and then outputs them back as variables again.

image

It is “always” possible to generalise a metanode into a component with the same ports, thus allowing it to be configured and made shareable.

“Always”… except in this case :wink: , which I do realise is not a common usage…

2 Likes

@takbb Good point! I totally agree in this case!

1 Like