Hello Sir @armingrudd,
I have been using your GET REQUEST PLUS reliably for many months. Since several days ago, the workflow has had some difficulties completing and I found these errors. I am feeding in several thousands URLs to make the REST API calls and I am multi-threading with Parallel Chunk and your Get Request Plus. Am I multi-threading too much? I searched up “Failed to asynchronously serialize object data.” but couldn’t find much.
I’m sorry to be contacting you directly for something I’m doing. I’m at a loss with this error. You have helped me in the past and any guidance will be appreciated greatly, Sir. I’m willing to rework this workflow myself if something is not ideal. Thank you again in advance for your time and guidance.
ERROR Parallel Chunk End 0:3746 Execute failed: Failed to asynchronously serialize object data.
ERROR ResourceLeakDetector Error when attempting to close open closeable.
ERROR GET Request 0:3524:0:29 Execute failed: Failed to asynchronously serialize object data.
ERROR GET Request 0:3923:3553:0:29 Execute failed: Failed to asynchronously serialize object data.
ERROR GET Request 0:3527:0:29 Execute failed: Failed to asynchronously serialize object data.
I have been running more tests. Got new error message.
ERROR GET Request 0:3934:3527:0:29 Execute failed: Length must not be larger than capacity.
ERROR GET Request 0:3564:0:29 Execute failed: Length must not be larger than capacity.
I ran more tests and got columnar storage error as well. The disk space in C is running out although I have separate working directory. Clearing some space and rerunning it again. I’ll move the storage to cloud. Will report back later. Thank you!!!
I looked inside the components but could not find which 2 nodes are unconnected. And I don’t get that error message with the other 3 Get Request Plus component nodes and I haven’t had that error in many months of using this. @armingrudd Sir or anyone else, any idea? Thank you so much!!!
Odd, but somehow one of the component inside the component node Get Request Plus had one missing node. I did not touch it at all when it started erroring out so no idea.
I am seeing a new error now.
ERROR Get Request Plus 0:3527 Caught “SWTException”: Invalid thread access
Fixed the missing node in the component and am running another test again.
To make sure that you are using the most recent and complete component, update them. Never disconnect and modify the component unless you need to add a new functionality and don’t care about future updates of the component.
If you are getting any error in the component or if you are getting unexpected results, please provide me with some sample inputs so I can reproduce the error.
Regarding your workflow, my first question is why are you using Get Request Plus inside the loop? Why don’t you send request first and then pass the results to the loop?
Or maybe it’s better to ask why are you using loops in the first place?
Hi @armingrudd , thank you Sir for your response. I’ll update the node copies from your Knime hub page.
I was originally using regular Get Request nodes and I wanted to use Parallel Chunk Start to “multi-thread” Get Request nodes. Right now I have 4 Parallel Chunk with custom chunk count of 5 in all of them. In Get Request Plus nodes, I have # of retries = 2, Delay = random (I am randomizing between 53-379 ms with variable), concurrency = 1 and timeout = 120.
I kept the Parallel chunk loop when I replaced regular Get Request nodes with your Get Request Plus. Should I remove the Parallel chunk look and just have one path with one Get Request Plus node and increase the concurrency (assume it’s the number of parallel instance you can run to multithread?)? This API takes about 40-60 seconds to return the json response and allows 1 request per second so I wanted to ask faster/more while waiting for responses.
In terms of component, I was running the workflow using Windows Task Scheduler daily so didn’t even open the Knime or workflow (much less the component) around that time. And only 1 out of 4 Get Request Plus nodes got corrupt. I don’t know how it got corrupt but I’ll pay more attention moving forward.
Thank you again so much for your time and support Mr. @armingrudd . Truly appreciate the help.
