issue with getting binary data on 2nd row from the table

I am having an issue with the REST GET node for Microsoft Graph API. For the particular GET API, the REST GET API is supposed to return a binary object in the body and the content type is jpg. It worked fine with the first row/record but on subsequent row it is missing value in the body, despite having a status of 200.

Hovering over the ? in the output of the 2nd row, Knime returns the following error message -> missing value (Duplicate key detected: “BinaryObject-0”). It seems like the REST node is trying to write the body content to BinaryObject-0 again? Wouldn’t this variable increment for the 2nd row?

Any help would be appreciated and I am pretty new to Knime so please excuse me if I misuse the Knime terms.

1 Like

I ended up using the HTTP retriever and HTTP result data extractor. It is now working as expected.

There seems to be a bug with the REST GET node to get past the first table row of retrieving binary body.

3 Likes

Hi @kkam -

Welcome to the forum, and thanks for posting your workaround. We’ll investigate further about potential issues with the GET Request node.

Hey,
@ScottF
I have the same issue here. I am trying to access PubChem via the substance ID in a call ike e.g. https://pubchem.ncbi.nlm.nih.gov/rest/pug/substance/sid/11468023/SDF to get the structures of chemical compounds. In ~300 out of 3000 cases (the first time at row numer 30, I use a chuck loop witz size 100 ), I get the Duplicate key detected: “BinaryObject-0” instead of the expected sdf (text based format file).
I am using the workaround right now as well (Thanks! @kkam!) , but would appreciate to get infos on whats going on as we work quite frequently with APIs.

Thanks! Jennifer

Hi @kkam and @jenniferh -

Do either of you have a workflow handy that reliably reproduces the problem? I’d like to send something on to our developers, if possible. Thanks!

Hi,

@ScottF took me quite a while to get something reproducible and sharable but I have attached a workflow which shows the problem. It contains both solutions, the lower is the problem for the REST node and the upper is the HTTP retriever solution. What I found interesting is that with the rest I either get response 200 or empty, but in the HTTP retriever node I quite often experience a 503 timeout error and I am not sure how this comes about. It would be great to know if its the server or if there is something else I need to consider for the HTTP retriever as so far, unfortunately both options are not really working for me right now.
In my original workflow I have about 3000 URLs, and I use a chunk of 100 for one request. The API specifies that no more than 5 requests per second should be made.

Thanks for your help!
Jennifer

REST_problem.knwf (48.3 KB)

1 Like

Thanks - I’ve created a ticket for this in our system. I’ll let you know when there are any updates.

2 Likes

Bumping this thread to note that this issue should now be fixed in KNIME AP 4.2. If you still run into problems here - please let us know!

2 Likes