Stream vs Parallel


This is a companion discussion topic for the original entry at https://kni.me/w/i_OxTnxBC6_ALVU7

After playing around with this for not a lot of time it does seem as if the streamed execution is much faster than the Parallel Chunk Loop. I assume that simple Math Formulas are optimal for Parallelism, which allows excellent results with the streamed execution. I am also aware that the Parallel Chunk Loop allows to parallize in different ways, but I’m starting to get the feeling that the way each parallelization option works is not really competing with each other, meaning that in most scenarios it’s obvious which option should be used.

I’m trying to put that into a simplistic view with few words: The streamed execution seems to be useful primarily for large dataset where you need to do lots of operations, whereas the Parallel Chunk Loop allows to parallize a whole workflow, so functions on a different level than what streaming is intended for, which can be useful for example in scenarios where the bottleneck is not the amount of interface connections you can have at any given moment, but rather the latency within each individual connection.

Maybe I’m thinking wrong about this, I’m sort of trying to open a discussion on the topic, so I can understand it better myself.

1 Like