Error when creating output from Python script node

KNIME Analytics Platform
1

Hi, I’ve been grappling with an issue for hours and I’d really appreciate some help on it:
In a Python Script node I run the Python code as seen in workflow I uploaded.

Error message:

When I run the code by clicking on “Execute script”, I get the following error:

C:\Users.…\miniconda3\envs\py3_knime\lib\site-packages\pyarrow\pandas_compat.py:358: FutureWarning: is_sparse is deprecated and will be removed in a future version. Check isinstance(dtype, pd.SparseDtype) instead.
if _pandas_api.is_sparse(col):
Caught exception: java.lang.reflect.UndeclaredThrowableException
jdk.proxy18/jdk.proxy18.$Proxy60.executeOnMainThread(Unknown Source)
org.knime.python3.scripting.Python3KernelBackend.lambda$13(Python3KernelBackend.java:646)
org.knime.python3.scripting.Python3KernelBackend.beautifyPythonTraceback(Python3KernelBackend.java:719)
org.knime.python3.scripting.Python3KernelBackend.executeAndCheckOutputs(Python3KernelBackend.java:646)
org.knime.python2.kernel.PythonKernel.executeAndCheckOutputs(PythonKernel.java:486)
org.knime.python2.kernel.PythonKernelManager.lambda$1(PythonKernelManager.java:208)
org.knime.python2.kernel.PythonKernelManager$3.run(PythonKernelManager.java:225)
org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:367)
org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:221)
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
java.base/java.util.concurrent.FutureTask.run(Unknown Source)
org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)

Additional information:

Weirdly enough I don’t get any error messages, when executing either the last line or everything above it separately.
However, as I said, I can’t execute the whole script and can’t run it either by clicking on the “Execute” button of the node.
To give some more context, I’m using the Conda Python environment configuration and as far as I know should have installed all the relevant packages.
The input of the Python script node is a table that comes from a JSON Reader node. This input is then converted from JSON format to a regular dataframe in the Python script. The only goal of the Python script is to convert the data from JSON into a dataframe format.

Thanks so much in advance for your help! I’ve also attached the workflow to this post in case that helps.

Soccer Analytics Student Challenge.knwf (75.9 KB)

2

Dear @GschichtIsAJene,

nice that you work with the Python Script node!
Could you send the workflow in an executed state? I cannot reproduce it as I do not have the JSON file available.

You pointed out that the issue seems to be the last line, i.e. knio.Table.from_pandas(final_df) and in the script you have the thought that the knio.Table.from_pandas function cannot handle some of your data types. I’m curious now which data types you use in the final_df :slight_smile:

Best regards
Steffen

1 Like
3

Hi Steffen,

Thank you so much for your reply!
In the end a colleague of mine made it work by splitting the output into two tables. So maybe the issue in some way was that the output table was too big (the initial file is huge). Either way we have solved the issue now.
Hope you have a great day and thanks again!

1 Like