Pagination on Get Request node

Hello everyone,

I’m new to KNIME and need to use JIRA REST API to analyze some data. I’m using the Get request node to extract the data from Jira and works well.

The problem is that I need do pagination on JSON output.

https://xxx.atlassian.net/rest/api/2/search?jql=sprint%20in%20futureSprints()&startAt=3&maxResults=100

If I open the REST API URL in the browser, I’m able to navigate in the JSON pagination with the parameter startAt=0startAt=3 and so on.

The REST API header give me this:

{"expand":"schema,names","startAt":3,"maxResults":100,"total":1660,"issues":[{"expand":"operations,versionedRepresentations,editmeta,changelog,renderedFields","id":"171417","self":"https://xxx.atlassian.net/rest/api/2/issue/171417","key":"xxx","fields":…

I think the Chunk Loop Start is one option to do that pagination, but I don’t know how to pass the parameter startAt=0

Anyone can give-me one example to do that?

Thanks in advance

Maybe you can provide us with a workflow that has a working extraction and/or a working URL that you know contains information you want. The URLs in the text seem not to be working and their special structure might be compromised by the limitations of this forum. So maybe attach them as a file or even better a KNIME workflow.

If it is just the “startAt=0” part you might create a loop that replaces the 0 with 1,2,3,4. If you get the maximum number of pages somewhere you might even set up a dynamic workflow determining the necessary number of loops.

1 Like

KNIME_project6.knwf (10.0 KB)
Hi mlauber71, many thanks for your help.

Unfortunately, I can’t give publicly the endpoints and credentials of our enterprise Jira Account.

However, I created a dummy account in Github and a Knime Workflow that do a very similar approach.

In this case I need to change the webservice pagination changing the flag page=1…2…(n).

https://api.github.com/user/repos?page=1&per_page=100

Not really sure about the results, but the structure of a loop could look something like this. You might have to adapt it to your needs.

kn_example_http_get_pagination.knwf (50.6 KB)

Hi Both,

another idea would be to use the Recursive Loops for this. It is very helpful if you get the next page in the JSON as a result. Because than you can continue the loop until there is not more next page available.

Cheers, Iris

1 Like

Many thanks mlauber71 and Iris, I already resolved the problem with something very similar.

Have a happy new year!

1 Like

Hi Peter29, I have the same issue with the same application you are connecting. Could you share the solution with us? Txs!

Hi Peter,

I am facing the same issue too. Could you please share the solution.
Thank you!