Keep in all in memory option + Java Code Snippets

Dear All,

My question might be slightly off topic, but I would very much appreciate any insights you can give me.

Keep in all in memory option + Java Code Snippets, how does this work?

When I write my own Java code, every time I execute a new process, I have to reload my data into memory. So each time when I make some small modification to my code that generates the statistics I need, I need to relaunch my process, reload all data into memory, and do all the computation to generate the statistics. I never have the option to keep things alive in memory between runs.

But in KNIME I can do exactly this.
I can load a data set, and use the Snippets operator to compute the statistics I need. Instead of having to reload all the data, I only have the re-execute the snippets operator.

So how does this work? Does KNIME use shared memory? Or socket communication between processes?

Best regards,

Wessel

I believe the data in a node is serailized, when you run your next node if the data is still in memory no deserialisation is needed. However, if you data has been serailsized the next node will deserialize the data before running your code.

I think the keep in memory option is only for the running of that node. The memory will be released once the node has completed. 

Sam