I’m having fun exploring the opportunities afforded by your great Azure Open AI integration, but have hit a snag.
I was looping through table rows to get responses, but it was a bit slow because I want it to produce output from eight queries on one page for a user to review. I created a component and made seven copies to parallelize the operation, but some of my components return this error:
Contains one node with execution failure (Chat Model Prompter #1825)
Chat Model Prompter #1825: Failed to initialize Python gateway.
Should I take this as as KNIME’s way of telling me to be patient and go back to using a loop, or is there a way around the issue?
(I know I could orchestrate using flow variable connectors, but that would be the same as a loop, just more ugly!)
I found a thread talikg about the same issue here. Could it be possible you are facing the issue due to less memory allocation? Can you try to run the workflow on system with more memory or try increasing the memory allocation to Knime by editing KNIME.ini file
The line for memory allocation starts with “-Xmx” and “-Xms”. (The flag Xmx specifies the maximum memory allocation pool for a Java virtual machine (JVM), while Xms specifies the initial memory allocation pool) -Xmx16g :- memory allocation to 16GB
If this doesnot solve the issue, can you share a sample workflow for us to try and recreate the issue?
I tried changing Xmx, from 16g to 20g to 26g…at which point it crashed Windows! It ran at 26g after a reboot, but it didn’t solve the problem. I’ve dropped it back to 16g
I don’t have an Xms flag in my knime.ini file, so I didn’t do anything there.
I’ve attached a sample workflow as requested. There is more data in the actual workflow, but it’s in the megabyte range, if that. I’ve rerun the sample and it does recreate the error when making the requests in parallel.
I hope you had a good break, I’m off on leave shortly too.
I’ve attached the latest log entries and it looks like it’s memory related from a glance. Looking at the task manager, KNIME does launch a lot of python instances when it gets to those nodes.
When I tried to run it the first time, KNIME crashed again and didn’t restart until I ended the orphan Python processes. It’s run a couple of times since, but with the Python gateway errors.
It appears that the error is due to the paging file (virtual memory) on your system being too small to support the operation. Based on a quick search, adjusting the size of the paging file might resolve the issue. You could try increasing the virtual memory and see if that resolves the problem as mentioned here.