Replacing legacy Input nodes on Business Hub workflows

[frenemie]

Hello all, we are doing something similar to @ooobii’s awesome git hub actions market place item:

Call KNIME Service Deployments from GitHub Actions!

Hi Everyone! I thought I’d share a creation that could prove useful to others. My employer recently developed a need to trigger KNIME Workflows as a step in GitHub actions. To fulfill this need, I’ve created a GitHub action that can call service-deployed workflows! GitHub: ooobii/execute-knime-hub-workflow-service Full details on how to utilize this GitHub action are available in the README. Please feel free to open issues for bug fixes or pull requests for improvements! …

This works via sending parameters into the business hub job API and has been working great.

i.e. the javascript snippet for executing the job with parameters from input nodes

var executeJobUrl = new URL( `${baseUrl}/jobs/${jobId}`);
    executeJobUrl.searchParams.append('reset', resetWorkflow ? "true" : "false");
    executeJobUrl.searchParams.append('async', "true");
    executeJobUrl.searchParams.append('timeout', '-1');
    var executeJobResponse = await fetch(
      executeJobUrl, 
      {
        method: 'POST',
        headers: getHeaders(),
        body: JSON.stringify(JSON.parse(core.getInput('parameters') ?? "{}"))
      }
    );

And everything works great. However, it looks like the parameters are tied to the legacy Input nodes which look to be deprecated soon.

How would you do this with the newer configuration and/or widget nodes? I’ve been running around in circles on this one for a bit. I’ve only found references to the configuration nodes and not widget nodes which seem to be the right replacement for the older quick forms. If there is any documentation on this from the business hub side, that would also be fantastic.

[frenemie]

Well, apparently you can use configuration parameters with an undocumented api ‘wizardExecution=true’ which is what the KNIME Business Hub uses.

Ex

curl 'http://api.hub.fake.com/jobs?itemVersion=most-recent&workflowUri=*XXXXXXXXXX&wizardExecution=true' \
  -H 'Accept: application/json, text/plain, */*' \
  -H 'Content-Type: application/json' \
  --data-raw '{"actions":[],"executionContext":"xxxxxx","configuration":{"double-input-198":0.4,"string-input-201":"foobar","boolean-input-199":false,"integer-input-200":100,"integer-slider-209":50,"list-box-input-210":"foo\nbar\nbaz\nboo\nfu"}}'

Without wizardExecution=true the job stays in IDLE state.

So now I have two questions:

1. Is there another way to switch the job from IDLE to EXECUTING
2. Should I really be using this undocumented API?

Plus the how do I do the same thing with the Widget inputs.

Thanks in advance.

[kpowney]

@frenemie Thank you for your questions. I think you may be misunderstanding how to use the API to create and execute jobs. The POST /jobs endpoint is intended to create a job in the environment and should be followed up by a second API call to POST /jobs/{jobId} in order to trigger the execution of the job that was created. This second API call should move the job from an idle/loaded state to an executing state.

In addition to these two API calls you can also make calls to retrieve the status of the job or its results (GET /jobs/{jobId}), and to remove the job once it is no longer needed (DELETE /jobs/{jobIs}). Ideally your app or script that is interacting with the API would make use of all 4 of these endpoints in order to create a job, check it’s status, retrieve output, and remove the job when finished.

[frenemie]

Okay, after a brief reprieve, what I was missing is that you need to append the configuration node id to the variable name. I.e. if your configuration node is string-input, and it is in node 5, the configuration name is “string-input-5”. The only way to know this is to inspect the workflow. So for the following contrived example:

The String Configuration is node 5 and has a parameter/variable name “string-input” so the configuration name is “string-input-5”

For posterity, here is how to use curl and jq to run a job and get the json output. (Note if the job can fail and this script will keep polling.)

username="XXX"
password="XXXXX"
workflow="XXXXX"

# create the job
curl -s -u $username:$password -X 'POST' \
  "http://api.hub.glysade-demo.com/jobs?workflowUri=$workflow&timeout=60000&asyncLoading=true&resetBeforeExecution=false" \
  -H 'accept: application/vnd.mason+json' \
  -H 'Content-Type: application/json' \
  -d '{"configuration": {"string-input-5": "this is the current output yeah yeah yeah"}}'> execute.json

jobid=`cat execute.json | jq -r '.id'`
joburl=`cat execute.json  | jq -r '.["@controls"].self.href'`

# execute the job
executeUrl="$joburl/?resetWorkflow=true&async=true&timeout=-1"

curl -s -u $username:$password -X 'POST' \
  $executeUrl \
  -H 'accept: application/vnd.mason+json' \
  -H 'Content-Type: application/json' > job.json

# poll the job
status=""
while [ "$status" != "EXECUTION_FINISHED" ]
do
 curl -s -u $username:$password -X 'GET' \
      $joburl \
        -H 'accept: application/vnd.mason+json' \
	> status.json
 status=`cat status.json | jq -r .state`
 sleep 1
done

cat status.json | jq -r .outputValues
									     
# delete the job

[frenemie]

As an aside, it would be wonderful if the error from job creation would indicate what variables were configurable. The legacy inputs nicely does this:

Could not set input on job 'XXX: Parameter name "string-input-asdfsdf" doesn't match any node in the workflow, valid parameter names are: ["string-input"]

But the configuration method doesn’t

Configuration 'string-input' is unknown.

Thanks for all your help.