Peter29
December 28, 2018, 10:59am
1
Hello everyone,
I’m new to KNIME and need to use JIRA REST API to analyze some data. I’m using the Get request node to extract the data from Jira and works well.
The problem is that I need do pagination on JSON output.
https://xxx.atlassian.net/rest/api/2/search?jql=sprint%20in%20futureSprints()& startAt=3 &maxResults=100
If I open the REST API URL in the browser, I’m able to navigate in the JSON pagination with the parameter startAt=0 … startAt=3 and so on.
The REST API header give me this:
{"expand":"schema,names","startAt":3 ,"maxResults":100,"total":1660 ,"issues":[{"expand":"operations,versionedRepresentations,editmeta,changelog,renderedFields","id":"171417","self":"https://xxx.atlassian.net/rest/api/2/issue/171417","key":"xxx","fields ":…
I think the Chunk Loop Start is one option to do that pagination, but I don’t know how to pass the parameter startAt=0
Anyone can give-me one example to do that?
Thanks in advance
Maybe you can provide us with a workflow that has a working extraction and/or a working URL that you know contains information you want. The URLs in the text seem not to be working and their special structure might be compromised by the limitations of this forum. So maybe attach them as a file or even better a KNIME workflow.
If it is just the “startAt=0” part you might create a loop that replaces the 0 with 1,2,3,4. If you get the maximum number of pages somewhere you might even set up a dynamic workflow determining the necessary number of loops.
1 Like
Peter29
December 30, 2018, 2:02am
3
KNIME_project6.knwf (10.0 KB)
Hi mlauber71, many thanks for your help.
Unfortunately, I can’t give publicly the endpoints and credentials of our enterprise Jira Account.
However, I created a dummy account in Github and a Knime Workflow that do a very similar approach.
In this case I need to change the webservice pagination changing the flag page=1 …2…(n).
https://api.github.com/user/repos ?page=1 &per_page=100
Not really sure about the results, but the structure of a loop could look something like this. You might have to adapt it to your needs.
kn_example_http_get_pagination.knwf (50.6 KB)
Iris
December 30, 2018, 11:43am
5
Hi Both,
another idea would be to use the Recursive Loops for this. It is very helpful if you get the next page in the JSON as a result. Because than you can continue the loop until there is not more next page available.
Cheers, Iris
1 Like
Peter29
December 31, 2018, 5:33pm
6
Many thanks mlauber71 and Iris, I already resolved the problem with something very similar.
Have a happy new year!
1 Like
Hi Peter29, I have the same issue with the same application you are connecting. Could you share the solution with us? Txs!
Hi Peter,
I am facing the same issue too. Could you please share the solution.
Thank you!
Hello @Peter29 ,
I am facing a similar problem. Can you provide your solution?
Thank you in advance
Mareike
ipazin
October 6, 2022, 8:16am
10
Hello @marryah ,
if haven’t seen there are couple of topics dealing with this. For example check this one as it has references to others as well:
Hello Knime Community,
I receive Data via a Get Request. Due to restrictions not all Data can be retrieved, so beside Data the Request also delievers an URL to the next set of Data. I like to implement something like:
Send Request with initial URL
Extract URL if available (save initial url and extrated url)
Send Request with extrated URL, back to 2
No URL - variable loop over the collected URLs and retrieve the Data
I think the Variable Condition Loop Node would be the best fit?
Or maybe it…
Br,
Ivan
1 Like