Problem to launch a workflow from a Python script

Hi guys,

I try to launch a Knime workflow from this Python script:

import argparse
import subprocess as sp

def runKnimeWorkflow(workflowPath, workflowFile):
    """
    Run selected workflow in KNIME (batch mode)
    input:
    workflowPath -> Entire workflow path + name to run """
    print("Invoke KNIME to run selected workflow: ", workflowPath.split("/")[-1])
    sp.run(["/home/giuseppe/Downloads/knime_4.7.0/knime", "-nosplash", "-consoleLog", "-nosave", "-reset", "-noexit",
            "-application org.knime.product.KNIME_BATCH_APPLICATION"
            "-workflowDir=\""+workflowPath+"\" -workflowFile=\"" + workflowFile+"\""])
                    
    return

def main(config):
    """
    Invokes a KNIME workflow in batch mode
    input:
    config -> contains a list of command line arguments with respective values
    """
    runKnimeWorkflow(config.workflowPath, config.workflowFile)
    return

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument("--workflowPath", type=str, required=True)
    parser.add_argument("--workflowFile", type=str, required=True)
    config = parser.parse_args()

    main(config)

This script take as input the path of workflowPath and the workflowFile.
But it opens the Knime window and it doens’t execute the workflow…Why?
As you can see I use the option -nosplash but it seems ignore it.
There is a bug on Knime Linux?
Can someone help me?
Thank you.

@giuseppeR

In your first function definition you have triple quotes to open a docstring, but no triple close to close that docstring. Therefore, virtually the entire script is being read as a docstring.

DiaAzul
LinkedIn | Medium | GitHub

Hi @DiaAzul,

this is an error to paste and copy code…now it is correct with the command to launch Knime workflow.
Is it know some bug on Linux?
Thanks.

@giuseppeR

My apologies, I am not an expert on running subprocesses (only reading code and commenting on it :rofl:)

In your sp.run command you pass a list of arguments. Some of these arguments you pass as individual elements in the list (first line) and others you concatenate into a single string (third line) the second line is missing a comma, so the strings will concatenate (I think). Have you tried to make these all consistent. Finally, your arg list may require preceding by an asterisk to indicate it is a list of *args.

So I would expect to see something like (I’ve used f-strings instead of concatenation to make it more readable):

 sp.run([
    "/home/giuseppe/Downloads/knime_4.7.0/knime", 
    "-nosplash", 
    "-consoleLog", 
    "-nosave", 
    "-reset", 
    "-noexit",
    "-application org.knime.product.KNIME_BATCH_APPLICATION", # Add comma
    f"-workflowDir=\"{workflowPath}\"", # Use f-string and split with comma
    f"-workflowFile={workflowFile}", # use f-string (remove slashes?)
])

I’m just punting as I don’t have a system in front of me to test this on.

DiaAzul
LinkedIn | Medium | GitHub

@giuseppeR there already is a Python package to run KNIME workflows called “knimepy”. That should do what you want. Maybe you can explore. Also the option to execute workflows on a KNIME server from within Python.

Also KNIME and Jupyter notebooks can work together both ways. KNIME calling Jupyter and the other way round.

3 Likes

@mlauber71 thank you for your suggestion but I need to launch a python script without launch Knime interface.
@DiaAzul I try a simple python script in this manner:

import subprocess as sp

print("Invoke KNIME to run selected workflow: ")

sp.run([
        "/home/giuseppe/Downloads/knime_4.7.0/knime",
        "-nosplash",
        "-consoleLog",
        "-nosave",
        "-reset",
        "-noexit",
        "-application org.knime.product.KNIME_BATCH_APPLICATION",  # Add comma
        f"-workflowDir=\"/home/giuseppe/knime-workspace/DTLinux\"",  # Use f-string and split with comma
        f"-workflowFile=/home/giuseppe/knime-workspace/DTLinux/DTLinux.knwf",  # use f-string (remove slashes?)
    ])

where I define the 2 variables workflowDir e workflowFile to understand if the problem is the variable passed as argument of the python file but it is not the problem because it launch Knime interface.
Any other suggestion?
Thanks for your precious help.

Well this is what this package does. For me it seems you are trying to recreate the functions, which of course you can try to do.

Thanks @mlauber71 , great post!
I need to try this. Does the example (2) work the same way with the update knime scripting lib or is it still the knime package if used in python only?
br

@Daniel_Weikert I might have to update the example to the latest syntax conventions

@Daniel_Weikert I have updated the workflow - though the part with the transfer of data between KNIME and Jupyter notebooks keeps crashing under Apple M1 … will have to investigate or try on Windows machine.

2 Likes

@giuseppeR just to wrap this up. You could run a KNIME workflow from Python. The software would have to be installed on the machine (obviously) and it would run in the background

# https://pypi.org/project/knime/
# pip install knime

import knime
import pandas as pd

# Change the executable_path to point at a particular KNIME install.
# May alternatively be set via OS Environment Variable, 'KNIME_EXEC'.
knime.executable_path = r"/Applications/KNIME 4.6.0.app/Contents/MacOS/knime"

# Use a with-statement to set the inputs, execute, and get the results.
# https://hub.knime.com/-/spaces/-/latest/~bWh_i0_N_OyhEJf9/
# you might have to adapt the workflow path
with knime.Workflow(r"00_Random_Forest_knimepy") as wf:
    wf.execute()
    output_table = wf.data_table_outputs[0]  # output_table will be a pd.DataFrame

output_table.head()

I am not sure about the knio packages in later KNIME versions I can not find these functions. But this would work. You might have to figure out how to address more than one table output. There are also examples how to use input files (GitHub - knime/knimepy).

4 Likes

@mlauber71 thank you for your replies!
I’m sorry for my late reply…After you first reply I haven’t see the knime.py github repository…it suggested the solution:

knime.executable_path = r"/home/giuseppe/Downloads/knime_4.7.0/knime"

print("Invoke KNIME to run selected workflow: ")

with knime.Workflow("/home/giuseppe/knime-workspace/DTLinux") as wf:
    wf.execute()

and according your Jupyter it work’s fine!
Thank you and I’m happy to ear about this python library for Knime.

1 Like

Hi @mlauber71
Where exaclty is the Workflow “Random Forest” located regarding the executable path? Is it the name of a folder?
I assume you could also “inject” variables? Those would probably be input_table 1 and 2 aka 2 dataframes from jupyter?
br

it is just the (absolute) path of the workflow on the drive. It has nothing to do with the location of the executable. It is a folder technically but would appear as a workflow within KNIME.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.