Apologies if this has been raised before but I couldn't see it, or if I've missed a solution.
I'm experimenting with Simple Streaming & Wrapped Metanodes in 3.1.2 as the amount of data we're processing has recently increased substantially and have come across an issue/bug.
When the Concatenate (Optional-in) node is in a wrapped metanode configured for streaming it returns an empty table, but with no errors or warnings.
I've attached an example of this which shows a simple process where i) two tables are concatenated, ii) the same process is collapsed into a simple metanode, iii) the metanode is wrapped iv) the wrapped metanode configured for streaming in Job Management.
If I've missed something, please let me know, if this is a genuine issue/bug could someone let me know if it's known and if there's a timeline to resolve it?
Streaming looks to me to be a really big step forward, really happy it was implemented.
the problem here is that with streaming you cannot see inbetween results. So what you are typically used take a look at the data tables after each node is not available in streaming. The reason is that with streaming the node never sees the whole data table at once, it just gets small chunks and afterwards can forget them again. So inside a streamed wrapped node there is no data available. You can see the data only outside of the node.
You therefore need to connect the "last" port to the inport of the Wrapped Node output. In case there are none (like in your example) you can add outports via rightclick -> Wrapped Node -> Reconfigure and add e.g. a Data Port in the first tab of the reconfigure dialog.
Thanks for your explanation, I've added the connection in the example workflow and can see the output dataset at metanode level, I'll re-review the original flows also to see what had happened when I collapsed the existing flow into a metanode as it should have worked.
This also explains why the Value Counter nodes we had in the process to create operational meta-data also aren't working. I'll add multiple data ports and see if I can get that to work.
Is there any detailed documentation on Streaming & Wrapping? From my perspective (as an ex-SAS user now fully converted to knime & r) the potential here is huge as the elapsed time of processes is substantially reduced and I'd really like to get my head round how to optimise our large data analysis flows using both wrapping and streaming.