Polling a REST API

I need to poll a REST API using GET until a status of “completed” is reached. How can I accomplish that?

Just to give a bit more detail of the process:

  1. I request a file from a REST API, eg. GET http://serviceurl/generatefile?par1=10&par2=20
  2. A JobID is returned in JSON: {JobID:“abcd1234”}
  3. I now have to poll the service for that JobID, eg. GET http://serviceurl/status?JobID=abcd1234
  4. The status is returned in JSON: {status:“in progress”}
  5. I now need to poll this until the status changes: {status:“completed”}

I’m okay with steps one to four but I would appreciate help with step 5.

Thanks in advance
Jannetta

Hi @jannetta,
You can use the Generic Loop Start and the Variable Condition Loop End nodes for this, they allow you to let the node run until a condition is met, best combine them with a Wait node so that you do not send out to many unnecessary requests. You can find an example workflow for it here: https://nodepit.com/workflow/public-server.knime.com%3A80%2F06_Control_Structures%2F04_Loops%2F13_Usage_of_Generic_Loop_Start

best,
Gabriel

1 Like

Hi Gabriel

Thank you very much. That will work. But can you perhaps explain how the Constant Value Column works. I don’t quite understand how to use it. I have tried a few things but they didn’t work.

Currently my workflow looks like this:
Table Creator -> String Manipulation -> GET Request -> Column Filter -> JSON to Table -> String Manipulation -> Generic Loop Start -> GET Request -> JSON to Table -> Constant Value Column -> Variable Condition Loop End

It does the following:
Set variables -> Create URL -> Call URL -> Get Column containing JSON -> Extract JobID from JSON -> Create get-status URL -> Start loop -> Call get-status URL -> Extract status from JSON (it can be “in progress”, “error” or “completed” -> if status=in progress then loop else stop

I hope my notation makes sense. My problem, off course, is how to use the Constant Value Column to extract the status value and then make the Variable Condition Loop End to respond appropriately. My thinking was to set a boolean variable “end” to false and then change it to true if the status changes to “error” or “completed” but I couldn’t get it to work.

Regards
Jannetta

You can do this in many ways, probably the easiest is to just use a Table Row to Variable node to get a flow variable with the status, then connect that flow variable with the Variable Condition Loop End node and configure that node to stop once the value is “completed.” To handle the error case you might need to manipulate the variable further, one way would be with the Rule Engine Variable node, where you can turn both “error” and “completed” into a stop-loop value.

best,
Gabriel

Hi Gabriel

I’m now really close but … the Variable Condition Loop End doesn’t work. Everything seems to run fine up to that point. I’m uploading a zip of my workflow directory. You won’t be able to run it because the REST service I use is on my localhost but if you wouldn’t mind looking at the workflow, you could perhaps tell me what I’m doing wrong.

.RecompDiff.zip (38.8 KB)

Thank you very much for your help so far. I really appreciate it.

Regards
Jannetta

Hi @jannetta,
The problem was that the missing input connection for the Variable Condition Loop End node:

That node needs an input table to run, that was all :slight_smile:
best,
Gabriel

1 Like

That worked :D. Thank you so much for your help. I have now added some “wait” nodes too and all works beautifully.

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.