We have a remote linux server which is doing some geoprocessing task, I would like to integrate that with KNIME workflow. Following are the steps I want to do :
1) Ideally I want to load data from KNIME into one of the folders on Linux system
2) Initiate the shell script that does geo-processing
3) Check if the process is finished
4) Get the results back from the output folder on Linux into KNIME
Not sure how to achieve this with KNIME nodes. Any help would be appreciated.
I found a partial solution to my problem using "External SSH Tool"..couple of issues: (Attached is the screenshot for reference)
1) Passing a single input file and single output file is too restrictive. I don't really care about input file (my shell scripts handles the part of checking if any input files present), but it seems to be a requirement for this node.
2) Reading multiple output files. The script produces multiple output files so I cannot read multiple files outputs. give me erro saying "Execute failed: Copying multiple files, but destination is missing or a file."
3) This node deletes both input and output file, how can i prevent from deletion ?
4) Not stable. Sometimes it give errors saying "Didn't get any value for column(s) with index #0. Please verify column type(s)" . even though the I ran the same file multiple times.
5) I don't have a way to check if process is finished, thankfully the node gives intermediate message saying "waiting for the node to finish" which is fine but I don't think it keeps track of spawned sub processes.
I don't know the program you're using, but you should be able to tell it where to store the output-files and where to get the input data from. Then you can simply write your data to a certain location, pass that information via the call to the external software, and when that's finished read in the output files from the specified location (you could for example use a temp folder, there are KNIME nodes to handle these).
By this, you can use as many input files as you want (or none, just pass a dummy or so), as many output files as you want, and both are not deleted if you don't want them to.
However, this is a very generic answer because I know neither the software you are using nor what exactly you want to do (what your are passing back and forth for example)