knime server problems after multiple get/post requests

Hello everyone,

I was having quite a few problems with the knime server recently.
Either “RMI Executor could not be started” or other problems that prevented scheduled workflows from starting. Restarting the server (service on Linux headless server) always “fixed” the problem, but that does not help when most nightly jobs failed. There was also no useful information in the logs.

Since I stopped running a pretty ugly workflow in which I have to do thousands of get/post requests to external APIs (unfortunately that cannot be done differently) the problems are all gone. The timing also fit with the problems.
The get/post requests were not even running in parallel, but either linux or knime itself apparently cannot handle it.
Knime server version is 4.12 with Executor 4.3, Java 8.

Running the “get/post workflow” locally on windows machine is not a problem.
It would make no sense to provide the workflow since the called APIs and other data sources are within our internal network.

Does anyone have a solution or similiar experience?

Will knime server version 4.14.1 (once released) with Java11 fix it? Or should I just do it within python nodes?

another thing: Running multiple rows (let’s say 100+) through get/post requests nodes makes them often unresponsive; they cannot be cancelled while running (looping over each row instead works)

Thanks in advance!

Hello @tgoedeke,

We are not aware of any Linux specific issues regarding the execution of many REST calls. But the logs may contain a specific error or warning that reveals the root cause.

Could you please provide KNIME Server logs of at least one day you experienced this behaiour? Please send them to support@knime.com with a reference to this forum post.

I will check the other mentioned multiple row processing problem/experience with a developer. It might be related to the different handover of the data. Will get back to you as soon as I have more details.

Thanks,
Michael

1 Like