CSV reader does not work in batch mode

KNIME Analytics Platform
1

Hi all,

I want to run a very simple workflow in batch mode:

There is one CSV reader, one table creator and one database reader (connected to a MySQL connector). All of the nodes are excecuted (i.e. green) when I start the process.

The code of my batch-file is the following:

"C:\Program Files\KNIME_2.10.3\knime.exe" -reset -consoleLog -noexit -nosplash -application org.knime.product.KNIME_BATCH_APPLICATION -workflowDir="..."

The table creator and the database reader are reset and re-executed just as I want it to be - I know that because the SQL code in the database reader is simply "select now()". The CSV reader however is reset but not re-executed.

Where is my mistake? It would be great if you could help me here :)

Thanks a lot in advance!

Best

Claudia

 

2

How are the nodes connected? 

 

3

Hi David,

only the database reader is connected to the MySQL connector. The table creator and the csv reader are not connected to any other node.

Best

Claudia

4

Hi Claudia,

Not sure how flows with two starting points trigger in batch mode. You could try to make a variable port connection between the MySQL connector and the CSV reader to force-linke the latter's execution to the former. This should work, I'd think.

Cheers
E

5

Hi E,

thanks for your help! Unfortunately your solution did not work for my workflow.

I don't think that having more than one starting point is a problem in general, since I've just tried having two MySQL connectors (each connected to a database reader, but not connected to each other) and both were reset and re-executed using the batch file from above.

Best

Claudia


 

6

Mh, bugger. :) Some question occur to me:

  • Did the CSV format change?
  • How you do track the start of he flow? By having the Platform open while executing the batch file? If so, have you tried closing it all down and determining success through other means (e.g. new "success CSV" creation)?
  • Are the database nodes set to "skip configure"? If so, what if you change that? CSV readers cannot skip this step.

I'm suspecting something along the lines of open workflow locks, as you may see.

Best,
E

7

Hi E,

  • I'm not quite sure what you mean, unfortunately - not being a native speaker doesn't help ;). I did not alter anything regarding the CSV file.
  • Knime is closed while the batch file is executing. I'm not tracking the start at all, but since there is a database reader in the workflow that requests the exact time (now()), I always know whether the process was executed at all. I integrated a CSV writer node and that works perfectly fine...
  • Yes, the database readers were set to "Run SQL query only during execute, skips configure". When I remove the tick mark here, however, the database reader is still reset and re-executed, as it is supposed to be...

If you have any other idea why every node is working except for the CSV reader, please let me know! I'm getting kind of desperate ;)

Best

Claudia

8

I'm actually getting closer to the problem:

If I use another csv file as input, suddenly everything works. I couldn't figure out why though; both have commas as column delimiter and the settings of the two csv readers are identical.

Any ideas? :)

9

Thanks Claudia,

And sorry for my lack of follow-up! Been too busy these days to return to the forums...

-E

10

I solved the problem - in case this is helpful for anyone:

The problem was that the one csv file was on the same server as Knime and the other one wasn't.

However, it's not in general a problem if Knime and the document that you are referring to in a CSV reader (or writer) are on different servers; you just have to add the server to your file path like this:

\\servername\filepath.csv