Could not save workflow: null

Hi together,

i have a problem with one of my workflows. It is a workflow that correlates customer data within a loop.
The loop executes fine as expected. The problem occurres when i try to save the workflow in an executed state.
The error message is “Could not save workflow: null”. When i reset the workflow i can save it. As soon as i execute the loop i can’t. I attached the KNIME Log in hope you can help me.

BTW: I had a previous problem with KNIME Desktop freezing under certain conditions. I applied the Hotfix mentioned in this post:

It might be relevant, because the error message in the Log file prints an SWT exception again.

Hi,
Once before I had a issue somehow similar to yours.
The problem occurred when I deleted some temp files while the workflow was being saved.
Here I can suggest you to:
1- Reset the whole workflow, then save the workflow and exit KNIME.
2- start KNIME and execute the workflow (wait until the execution process for all the selected nodes is completed) and then try to save the workflow again.
If this didn’t work go to plan B and after completing the first step in the previous solution, Open the workflow (don’t execute) and save it as a new workflow using a new name and then do the second step for this new workflow.

I hope this solve your issue. That’s all I can suggest based on my own experience.

Best,
Armin

Hi @armingrudd, thanks for your help. I followed your instructions. But unfortunatelly i still get the same error.

If it’s OK, you may share the workflow here. Then maybe I can save it in executed state and resend it to you.
That’s the best I can do to help you.
Just remember to attach the data file or at least execute the nodes which read data before saving.

Hey @armingrudd, thanks for your offer that is really kind of you. Unfortunatelly i can not share the data with anyone, because it’s customer data. But i can export the processed data into an external files and continue working with that, after i executed the workflow.
I think the problem is generally undesirable and should be fixed in case it is an error of the new KNIME Version 3.7.

1 Like

The SWT exception is the save progress trying to update the display of a save that failed - which ain’t great, but isn’t the root of this problem.

I noticed that the file path of some of your node directories are already at 255 characters (for example “C:\Users\Michael Höhn\knime-workspace\ZF\Data Analysis - comp results\Correlate (#103)\Linear Correlation (#104)” ) so maybe there is some filesystem related problems being run into here (indeed - the root of the error in the stack trace is the a file stream failing to be created as part of the ZIP file creation.)

My text editor was telling me something about character position that i mistook for length of selected text.

On a lark, could you try making a new workspace in some place with a short name like “C:\foo” and then, importing the reset-and-saved version of this workflow, executing it, and saving it in your new workspace?

The exception is still something file system based, though. Something silly like disk space availability?

You have one warning about low memory earlier in the Log but that maybe is not connected

2019-01-07 09:01:55,650 : DEBUG : Service Thread : MemoryAlertSystem : : : Memory usage below threshold (88%) after GC run, currently 2% (0,13GB/6,51GB)

Then maybe you could tell us something about your operating system and hard drive. Is this a virtual machine (they tend to have problems). It might be that your temp folder is on another virtual drive than your knime-workspace. In which case knime would have to move the temp files to the workspace and this might lead to problems. Also you might want to check if you have any quotas and what ist the space left on the drive.

Another candidate could be a very aggressive virus scanner although I doubt that in this case.

It seems the error occurred while replacing some folders in the process of storing the files, and could it be that this is a metanode with a lot of subnodes?

I experienced problems with a One Note folder once not being able to handle some so special characters Knime is so fond of.

And knime does not like it sometimes if you try to save the workflow while it is still running, even if that is permitted in general.

And of course it is possible that there is a problem with knime.

——
And another note: there is some information in this log about your name, company, methods and variables used; although I am aware of the Barbara Streisand effect you might want to check if you are 100% comfortable with sharing them.

1 Like

Hi together,

thank you for your help.
@quaeler: I have a couple of GBs left on my hard drive and i saved other bigger workflows after the one in question. So i think this is not a problem.

@mlauber71: thanks for beeing so thorough. I’m working on a Laptop with 64 bit- Windows 10 Pro with 16 GB RAM, no virtual environments involved. My workspace is on my local hard drive (475 GB space).
This is the first time i ever encountered this problem and i am working with the Laptop and KNIME for months now without any major configuration changes, that is why i doubt it’s a problem in the software setting like virus scanners etc.
There are some metanodes but i think at most 2 encapsulated levels.
The workflow was fully executed when i tried to save it. No parts of it running and it happens every time i attempt to save it, not just once.

And another note: there is some information in this log about your name, company, methods and variables used; although I am aware of the Barbara Streisand effect you might want to check if you are 100% comfortable with sharing them.

Thanks for pointing that out. I was not thinking about that, good to know.

I realized one more thing. It seems like whenever i close and reopen the workflow the Linear Correlation Node in one of my metanodes of the workflow is missing (even if i save it in unexecuted state). But i can not re-produce the error in other workflows.

This suggestion sounds like a pain in the ass, but what if you make a new workflow and then copy and paste the old workflow in chunks to the new workflow? If you do it in order from start nodes to, finally end nodes, and execute + save after each paste, at what point does it stop saving correctly?

I already copy pasted the nodes into another workflow with no effect. Problem is, that most of the workflows nodes are contained within a Loop. If i copy the without the loop i can execute and save them. If i copy them with the Loop and execute it, i can’t save the workflow anymore.

Hi @MH,

Some ideas that could help narrow down the problem:

  1. As suggested by @quaeler - have you tried individually executing the nodes in the loop one after another, saving the workflow after the execution of each node? This way, we might be able to determine at which point the workflow becomes unsaveable.
  2. Can you give us information on (a) what kind of loop you’re using and (b) what types of data are processed within the loop (Strings, numbers, Documents, images, chemical compounds, …)?
  3. You could try changing the data storage format in KNIME to Parquet. To do so, you’d have to install the extension “KNIME Column Storage (based on Apache Parquet)”, then restart KNIME and activate the format via “File -> Preferences -> KNIME -> Data Storage”. Maybe this helps or at least leads to a more helpful error message than “null”…

Best,

Marc

2 Likes

Hey @marc-bux,

thank you for your suggestions.
1.) I tried. All works (saves) fine, until i start to execute the loop (by excuting the loop end node). As soon as the entire loop finishes i am not able to save the workflow anymore.
2.) I’m using a Table Row To Variable Loop Start Node (cause i need to read individual excel files) together with a Loop End (2 ports) node. The data within the loop can be either a Local Date Time, String or numeric data type. But the Loop End Node collects numerical data and one string column.
3.) Changing the column storage to Parquet has no effect at all. I’m still getting the same error message.
image

Ok - so it seems like we are reasonably concluding that it is something about the nodes involved in the loop that are going off the rails. Can we restrict the problem further by:

  • Making a workflow that contains as little more than the loop as possible?
  • Then restricting the input to the loop (i.e try having the loop read one of the excel files - execute, save; then two of the excel files - execute, save; …)

Does this provide any more information as to when the problem is happening?

Hey @MH,

I’ve built a minimal workflow that processes data of the types you mentioned in between a Table Row To Variable Loop Start node and a Loop End (2 ports) node, but I wasn’t able to reproduce the error.

The error seems to occur as KNIME attempts to copy a temporarily buffered table to a target file in the workflow destination folder and finds the target file not properly initialized. This behavior is certainly odd, but should be fairly easy to fix once we’re able to replicate it locally.

In addition to the (very promising) suggestions by @quaeler, other things you could do / questions you could answer to help us isolate the source of the problem are as follows.

  • Can you try executing the nodes in the loop individually (without executing the loop end node) and saving the workflow after having executed each individual node? Alternatively (if this doesn’t help), could you reduce the amount of nodes in the loop step by step and see at what point it lets you save the workflow?
  • Just about how large are the tables handled within the workflow (rows x columns)? If they are not too large, can you try and set the memory policy of all nodes within the loop to “Keep all in memory.” and try to save the workflow then?
  • Does the issue also occur in an earlier KNIME version (e.g., KNIME 3.6.3)?

Best,

Marc

Hi @quaeler and @marc-bux,

thank you for your suggestions. I narrowed it down to the nodes that seem to be responsible for the problem.
image
Weird as it is. As soon as i experiment with this node combination.
1.) Copy paste it outside of the Loop and execute it
2.) Copy paste it outside of the metanode
3.) Execute it with different sets of data
It seems like sometimes the error occurres and sometimes it does not. But i can never find a combination of nodes/configurations that always fails (apart from the original workflow i constructed).
One example, when i copy the original combination of the IF and correlation node, execute and save it, that fails. When i plug the correlation node behind the lower port of the IF node it does not fail and i can save. Reattaching it to the upper port it does not fail either (wtf).
Also when i copy the part that originally fails into another workflow with similar data it does not fail either. It is so strange.
And i think it already cost too much of our time.
But answering at least part of your questions:
1.) All nodes execute fine and can be saved. The problem is executing the Loop entirely. I realized that the last run of the Loop creates an empty table before the correlation step. Thats why the upper branch of the IF node is deactivated in the last run of the Loop. And i’m sure this is at least part of what is causing the problem.
2.) I can execute and save the Loop when i limit the data. I only need to exclude the last row and it works. If i execute the workflow with just the last row, i can not save the workflow. That’s why i think the problem is not necessarily the loop node.
3.) The issue also occurres in previous KNIME Versions (at least with 3.6.2).
I doubt we will find the root of the problem. It seems to be a complex combination of a distinct set of steps that is causing the issue.

Thanks for your help. Very much appreciated.

3 Likes

Thanks for your great investigation. This is definitely a perplexing situation :- /

I’ll make a JIRA ticket for it that captures everything you’ve discovered so at least we have a place to reference should this be encountered by another user.

Great idea, thank you!