External tool Node output synchronisation issue

Hi, 

I am using KNIMe version 2.11.3.

I am trying to get the external tool nodes output from its output port. My external exe is generating an output file(output.txt) and this file is configured as external nodes outfut file. The problem is output file content is not appearing in the outpurt port of the external node.

When i run the workflow again, output file populates with new content, this time the outpurt port shows the output.txt file's previous content.

 

It appears like external tool node reads the content of the output file and send to output port before the file writing is completed by the exe

Could you please help me on this synchronisation issue?

 

Thanks

Nipin

 

Have anybody faced this issue? Or any problem with my node settings?

Why 'External node' not waiting for output file to populate before reading it?

 

Thanks 

Nipin

Hi Nipin,

the external tool node waits until the external process is terminated, then reads in the output file. Unless you run your executable on a remote machine, it is unlikely that this leads to synchronisation issues. 

Please keep in mind that this node has a chunking capability and might - depending on its parameters - start multiple instances of your executable if the input table is large enough. This could explain the described behaviour, in particular in combination with a hard-coded output file name.

Nils

Hi weskamp,

 

Thanks for the answer.

I am using the 'External tool' under Miscellaneous category. The synchronisation issue is there.

I think you are mentioning about 'External Tool' under KNIME (labs) category. I prefer the first one for my requirements. As i am not able to get over the issue, I am trying for developing a custom node for running executables.

 

Thanks,

Nipin

Both nodes definitely wait until the external process has finished. If the file contains only partial data then the process terminates too early.

Hi Thor,

Let me describe my workflow:

I am using 'chunked loop' node for reading a file and supplying its each row as chunk to 'external tool' node .

The external tool is running an exe, for example to find sum. i.e if my input row is 10 20, them sum.exe will find 

the sum (10+20 =30) and write to an output.txt file(which i configured as output file of external tool).But when i run the workflow,output port is showing empty table,though the output.txt is populated with the result.

 

So i suspect external tool is reading the out file before it is populated.

 

Thanks

Nipin