Manually parsing a node's data.zip (undesired loop context)

KNIME Analytics Platform
1

Dear KNIMErs,

I've inadvertently opened a loop which I failed to close, which was not a problem until I re-opened the workflow on my new laptop after migration. In perfect compliance with Murphy's Law this branch is one of the longest-running I've ever built for non/semi-experimental purposes, performing a brute-force parameter search which took my previous laptop about 28 hours to complete.

Is there a way to avoid resetting the (undesired), not fully executed loop, and just use it as the workflow branch it's meant to be? If the answer is no, is there at least a way to recover my time-stamped benchmark table from the respective node's data.zip? Several attempts of migrating the data.zip to something like an internal KNIME table have failed so far. The best I achieved was to have the table reader read an entirely empty table with an otherwise correct spec – guess the data.bin needs patching, too. Should I get a hex editor? :)

Thanks a million,
E

P.S.: Of course I could just take screenshots of the node table within KNIME and OCR the result - but that'd be quite crude, wouldn't it? :-D

2

Hi Mark, 

That is quite unfortunate.  It is very difficult to parse this data.zip file since it contains references to other files as well, but there are a couple of other approaches that you can use to get back at the data. 

First, can copy the data out of the table?  If you are just using basic types, you can paste the results into a table creator or text file.  You will lose column names, but maybe this is acceptable (and maybe not). 

The other option is to do a bit of hacking on the settings.xml file to make it forget about it's current loop context.  To do this, look for at most 2 elements in the settings.xml file for your node of interest that look like:

<config key="Loop_Execute_3">

<entry key="type" type="xstring" value="loopcontext_execute"/>

</config>

and/or

<config key="Loop_0">

<entry key="type" type="xstring" value="loopcontext"/>

</config>

Remove these and re-open your workflow.  You should then be able to attach a table writer to the workflow and save the results of your current iteration. 

As a general useage tip (if not for you than for others reading), when I am executing workflows where execution time is the limiting factor in my analysis, I will often use a table writer in my loop body to cache intermediate results so that if I need to close KNIME while in the middle of a long running loop for some reason, I can later resume execution from where I left off without too much lost time. 

I hope this helps (and good luck!)

Aaron

 

 

 

 

3

\o/

Aaron,

Many thanks, I'll try this immediately once I'm back home (kids, do try this at home). Forcing workflow elements to "forget" their loop context is something desirable at times, no matter how hacky the implementation.

Cheers
E

4

Aaron,

Thanks again, it worked fine. For the sake of clarification for everone else, my findings are documented below:

  • Copy & paste with standard OS shortcut works (includes classic Ctrl + Ins and current Ctrl + C in Windows), pasting into spreadsheets such as Excel works as well. Be careful though, it copies over in string format only, which makes all double colums lose their decimal separator. Pasting back into KNIME and manually setting the columns back to double will restore them, but that obviously won't work in Excel and others.... too bad, actually! Just a short step away from a fully funcional cross-application copy & paste (with table view menu entries for user awareness, etc.) Easy to add that feature? :-)
  • If you want to fiddle with the seetings.xml, you'll only find the loop context stuff in the loop nodes as such - I initially misread Aaron's "node of interest" statement as the one containing the view / transformation of interest. But obviously these inherit the loop setting only (good DRY practice), so you need to eliminate the undesired loop creator node's loop context only. Sounds awkward, is awkwward, but works just fine for every other node downstream, as the "zombie loop start" no longer concerns them from functioning normally. You'll still want to start over with a cleaned-up workflow for subsequent executions, though.

Cheers
E

5

Thanks for the followup and clarification. That's quite weird about excell though, the decimal is gone?  Is this also true if you just paste into a plain old text editor (I don't have Excell!).

Thanks again, 

Aaron