Workflow frozen during saving

Hello guys,

I have two strange problems.

Problem number 1:

When I finish my workflow (correctly, all outputs are written) I would like to save everything and that’s all.

Unfortunately saving activity takes so much time and I have never seen success. I tried to keep it once more than one hour but nothing happened. Still the same windows.

image

Can someone tell me why? Is it normal behavior? i do not think so…

Problem number 2:

On top of it if my workflow is done and no activities are provided, but KNIME continually writing to disk until the disk is completely full. I do not know what is wrong.

Do you have any idea?

Just theoretically is there any limit how many Nodes could be in one workflow?

Thank you many times.

Hi @sm0lda,

Could both your problems be related because

During execution of a workflow each node tries to keep results in memory, or when this is too much, writes the results to disk. During execution this is done to temp.
When you save a workflow all these results are stored to disk as well (all in zipped format, maybe this zipping requires additional diskspace as well?), this time on the same disk as where you specified your workspace.
So if you work e.g. under Windows and have your workspace in C:… too, the results are stored once in temp and once in the workspace. This might become too much.

Possible workarounds I found when running into similar issues are:

  • Try to define the workspace on another disk then where the temp is located. You might get a worse performance because of this when this disk is on a networkserver. It’s one or the other evil…
  • Execute the workflow partially, save workflow, restart Knime, run rest. This does not require everything to be double stored on disk. This is cumbersome though.
  • Split your workflow in several workflows and use stored data to “communicate” between the workflows. This is a bit like the previous option.
  • If possible try to combine several nodes into one (like e.g. a Java Snippet). This reduces the amount of results to be stored.
  • If you retrieve data from some database and apply actions on this data by several nodes, which could be done by the database too, let the database do it (e.g. run a query and let Knime sort the data; you can sort in the query too). Additional advantage: works often faster too.

I don’t know if there is a limit on the number of nodes, but the larger a workflow becomes, the less overview you have on what is going on in it (especially if something goes wrong).

2 Likes

Hello @JanDuo

thank you for your answer. You are right with particular savings etc. it looks like I have not enough space due to lack of SSD size.

I have to figure out somehow. Sometimes even you have working workflow the background kick you down because of lack of performance etc.

Once again thank you

Jiri

When an ssd is close to full, it’s performance can get really, really really bad. So always a good bet to leave a couple GB free.

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.