Wrong data connections? Memory mixups?

Hi, I'm trying to read 6 csv files and concatenate them. (KNIME 2.8) Should be simple, and always used to be, up to 2.5.4 at least. Somehow all the csv readers read the csvs using the settings as set in the last CSV file I have reset. But not consistently so. Something similar happens with concatenation. So I end up resetting nodes one by one hoping the right CSV files end up in the right place. A lot of work for a simple job.

I still have the overview of what's happening to my data at this point, but at some point, I'll have to start trusting KNIME to just read the data as I configure it, and move the data along the right connections.

Is there a setting that might help? Is there a bug report on this already? Does anyone experience something similar? Should I just switch back versions?

 

Cheers,

Maarten

ps as I don't think the actual files matter, (and are private data) I won't post them here, and I won't make dummy files either, as it won't really matter, I suppose.

Hi Maarten, 

 

I tried to reproduce this by making several CSV files, then using a list files node to read these files inside a tablerow to variable loop and it seems to work fine.  

 

Attached is the example workflow that I used.  It generates and reads the data without input once you specify a writable directory in the Create Directory node.

 

Would it be possible for you to give this a run and let me know if it works?  You need the Create Directory node which is part of the File Handling feature in KNIME Labs as well as the KNIME testing Application if you want to have the tables compared automatically (otherwise you may delete the last node).

 

Regards,

 

Aaron Hart

KNIME.com

 

Interesting, I was noticing some weird behavior like this yesterday in 2.8. I have multiple parallel streams that are all doing the same thing except on different subsets of the data. At some point I'm renaming columns and I'd change the name in one of the streams and the same new column name would show up in a different stream (even when it had already been executed I believe). Sometimes other nodes, like the math node, would also just change a column name on its own. I was able to notice it because at the end I'm using the concatenate node and I was getting duplicate row ID's ("_dup") and there shouldn't have been.

 

Hi Natasja, Hi Maarten,

Thanks for the problem description. I sat down with Aaron and we can reproduce the problem now. We believe it only occurs when you duplicate a node multiple times using copy&paste. If you copy a node and paste it twice, both new copies of the node use the same configuration (the one, which was changed last). If you open the configuration dialog, you still see individual/different node configuration but whenever you OK the dialog the new config is applied to both nodes.

Can you confirm this is what you did (or believe you did)? Or is there anything else that we are not aware of? You should not see this behavior on old workflows and it should also go away when you save the workflow and reopen it.

We'll fix this problem for 2.8.1. The workaround is: don't paste a node twice but take a new copy (ctrl-c).

Thanks,
  Bernd

Hi Bernd

 

I colleague has noticed similar issues. We had a loop iterating over some columns, he then copy and pasted (multiple times) to connect to a different SDF reader. 

 

The pasted loops appeared to be updating the earlier (disconnected) loop. 

 

The empty table switch node also doesn't behave correctly. We had a table switch which passed through a column filter, then a concatenate (to replace a table when an empty row is found). The red cross made it to the column filter but not any further causing the rest of the loop to show the red light. 

 

We can try and recreate the problem if needs be?

 

Cheers

 

Sam

Hi,

I'm glad it's reproducible, and yes, I was copy/pasting a few times I think. 

Thanks,

Natasja

 

Hi Sam,

Yes, please send a workflow so that we can figure out what's wrong.

Thanks,
  Bernd

Hi there,

Sorry I didn't fill in sooner with feedback, I indeed used multiple copy/pastes of the same node. Loops can circumvent the problem, but in my case I have to select different excel sheets for each execution of the node, which I can't do in a loop. Copy-paste would be the best way to set the correct settings each time, (and should just be possible) but the handling is not correct. I'm glad its reproducible, I hope that helps catching the bugs. Found anything yet? ;) I'd like to know when an update will fix this... If you want me to try or explain things, please let me know, I'll check more often for replies this week.

Cheers,

Maarten

Hello Maarten,

Have you tried 2.8.1 yet? It seems the 4404 bug description fits your problem and was fixed.

Cheers, gabor

OK, did that, and had no luck trying to reproduce the error, so great! Thanks!

Cheers,

Maarten

Files now works as expected. I appreciate your  good work and creativity. Real Datpiff Services