Problems with Python Script (DB) install requirements

#1

I’m trying to use the Python Script (DB) node on a Win7(64bit) computer. I have the Python extensions installed in Knime and configured to Anaconda python 3.5. From the node description I know i need JayDeBeApi and JPype. I have JayDeBeApi installed but I’m having a hard time finding win64bit version of JPype. Can anyone point me to a 64bit version (or an alternative to JaydeBeApi for Python Script (DB). thanks

0 Likes

#2

Hi @rscottrowland2,

I followed the link in the node description of the Python Script (DB) node (https://www.lfd.uci.edu/~gohlke/pythonlibs/#jpype). This shows a number of different versions of JPype. This one seems to be for Windows 64bit: JPype1‑0.6.3‑cp37‑cp37m‑win_amd64.whl

is this what you were looking for?

Cheers,
Roland

0 Likes

#3

I saw that file. I’m assuming the amd64 refers to AMD 64bit chips and not Intel chips which is what I have. (I actually tried it anyway with pip and pip complains about not supported which is consistent with my belief its for AMD chips).

0 Likes

#4

Hi @rscottrowland2,

amd64 is commonly used in file names to refer to the 64-bit version of the x86 instruction set, which covers both Intel and AMD CPUs. There are some slight differences between AMD and Intel, but when it comes to installers those shouldn’t matter (else I would expect them to provide a version for both).

What is the error you are seeing when you tried to install?

Cheers,
Roland

0 Likes

#5

Hi,

i want to use a Python Script (DB) node.
I have installed Python 3 without Ananconda.
I also have installed jpype and JayDeBeApi.
When i try to run this workflow

Example workflow

i get this error:

ERROR PythonKernelManager Failed to clean up Python. See log for details.

ERROR Python Script (DB) 2:13 Execute failed: An exception occured while running the Python kernel. See log for details.

Can anyone help me, please?

BR,
ptrwlr

0 Likes

#6

Hi ptrwlr,

Could you please upload your knime.log file? Thanks!

Marcel

0 Likes

#7

Hi MarcelW,

when i open the Python Script (DB) Node the following is written to the knime.log file:

2019-05-17 08:54:26,906 : DEBUG : main : NodeContainerEditPart : : : Python Script (DB) 2:86 (IDLE)
2019-05-17 08:54:27,613 : DEBUG : Thread-91 : PythonKernel : Python Script (DB) : 2:86 : Using serialization library: Flatbuffers Column Serialization.
2019-05-17 08:54:29,269 : DEBUG : Thread-91 : PythonKernel : Python Script (DB) : 2:86 : Python PID: 11604

0 Likes

#8

Hi ptrwlr,

That doesn’t include the information I’m looking for, unfortunately. The root of the error is possibly buried in the knime.log file quite deeply, so it would be easiest for you to upload the entire file here. Please see here (third and fourth paragraph) to find out how to locate the knime.log file.
Also, please let me know if you have concerns uploading the entire file (for privacy/business reasons), then I’ll gladly point you to the right spots in the file but this might get more complicated.

Marcel

0 Likes

#9

Hi MarcelW,

does this include the information you are looking for?

2019-05-21 07:52:41,789 : DEBUG : Thread-15 : PythonKernel : Python Script (DB) : 2:129 : Using serialization library: Flatbuffers Column Serialization.
2019-05-21 07:52:43,577 : DEBUG : Thread-15 : PythonKernel : Python Script (DB) : 2:129 : Python PID: 1096
2019-05-21 07:52:57,876 : DEBUG : main : NodeContainer : Python Script (DB) : 2:129 : Setting dirty flag on Python Script (DB) 2:129
2019-05-21 07:52:57,882 : WARN : main : Node : Python Script (DB) : 2:129 : Database does not support insert or update
2019-05-21 07:52:58,391 : DEBUG : Thread-18 : ThreadUtils : : : Unnecessary call to runnableWithContext, because no context is available
org.knime.core.util.ThreadUtils$UnnecessaryCallException
at org.knime.core.util.ThreadUtils.runnableWithContext(ThreadUtils.java:381)
at org.knime.core.util.ThreadUtils.runnableWithContext(ThreadUtils.java:357)
at org.knime.core.util.ThreadUtils$ExecutorServiceWithContext.submit(ThreadUtils.java:94)
at org.knime.python2.kernel.messaging.DefaultTaskFactory$DefaultTask.run(DefaultTaskFactory.java:184)
at org.knime.python2.kernel.messaging.DefaultTaskFactory$DefaultTask.get(DefaultTaskFactory.java:202)
at org.knime.python2.kernel.PythonKernel.close(PythonKernel.java:1443)
at org.knime.python2.kernel.PythonKernelManager.close(PythonKernelManager.java:375)
at org.knime.python2.config.PythonSourceCodePanel$6.run(PythonSourceCodePanel.java:334)
at java.lang.Thread.run(Thread.java:748)
2019-05-21 07:52:58,392 : ERROR : Thread-18 : PythonKernelManager : : : Failed to clean up Python. See log for details.
2019-05-21 07:52:58,392 : DEBUG : Thread-18 : PythonKernelManager : : : Failed to clean up Python. See log for details.
org.knime.python2.kernel.PythonKernelCleanupException: Failed to clean up Python. See log for details.
at org.knime.python2.kernel.PythonKernel.close(PythonKernel.java:1452)
at org.knime.python2.kernel.PythonKernelManager.close(PythonKernelManager.java:375)
at org.knime.python2.config.PythonSourceCodePanel$6.run(PythonSourceCodePanel.java:334)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.Exception: Failed to receive message from Python or forward received message. Cause: Connection reset
at org.knime.python2.kernel.messaging.AbstractMessageLoop.throwExceptionInLoop(AbstractMessageLoop.java:77)
at org.knime.python2.kernel.messaging.DefaultMessageReceiverLoop.loop(DefaultMessageReceiverLoop.java:98)
at org.knime.python2.kernel.messaging.AbstractMessageLoop.doLoop(AbstractMessageLoop.java:176)
at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:328)
at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:204)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
… 1 more
Suppressed: java.lang.Exception: Failed to distribute message from Python. Cause: Message receiver loop terminated.
at org.knime.python2.kernel.messaging.AbstractMessageLoop.throwExceptionInLoop(AbstractMessageLoop.java:77)
at org.knime.python2.kernel.messaging.MessageDistributorLoop.loop(MessageDistributorLoop.java:93)
… 6 more
Caused by: java.io.IOException: Message receiver loop terminated.
at org.knime.python2.kernel.messaging.DefaultMessageReceiverLoop.receive(DefaultMessageReceiverLoop.java:82)
at org.knime.python2.kernel.messaging.MessageDistributorLoop.loop(MessageDistributorLoop.java:82)
… 6 more
[CIRCULAR REFERENCE:java.lang.Exception: Failed to distribute message from Python. Cause: Message receiver loop terminated.]
[CIRCULAR REFERENCE:java.lang.Exception: Failed to distribute message from Python. Cause: Message receiver loop terminated.]
Suppressed: java.lang.Exception: Failed to send message to Python. Cause: Connection reset by peer: socket write error
at org.knime.python2.kernel.messaging.AbstractMessageLoop.throwExceptionInLoop(AbstractMessageLoop.java:77)
at org.knime.python2.kernel.messaging.DefaultMessageSenderLoop.loop(DefaultMessageSenderLoop.java:88)
… 6 more
Caused by: java.net.SocketException: Connection reset by peer: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
at java.net.SocketOutputStream.write(SocketOutputStream.java:134)
at java.io.DataOutputStream.writeInt(DataOutputStream.java:197)
at org.knime.python2.kernel.messaging.DefaultMessageSender.send(DefaultMessageSender.java:80)
at org.knime.python2.kernel.messaging.DefaultMessageSenderLoop.loop(DefaultMessageSenderLoop.java:86)
… 6 more
[CIRCULAR REFERENCE:java.lang.Exception: Failed to distribute message from Python. Cause: Message receiver loop terminated.]
[CIRCULAR REFERENCE:java.lang.Exception: Failed to send message to Python. Cause: Connection reset by peer: socket write error]
[CIRCULAR REFERENCE:java.lang.Exception: Failed to distribute message from Python. Cause: Message receiver loop terminated.]
[CIRCULAR REFERENCE:java.lang.Exception: Failed to send message to Python. Cause: Connection reset by peer: socket write error]
[CIRCULAR REFERENCE:java.lang.Exception: Failed to distribute message from Python. Cause: Message receiver loop terminated.]
[CIRCULAR REFERENCE:java.lang.Exception: Failed to send message to Python. Cause: Connection reset by peer: socket write error]
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:210)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.DataInputStream.readFully(DataInputStream.java:195)
at java.io.DataInputStream.readFully(DataInputStream.java:169)
at org.knime.python2.kernel.messaging.PythonMessagingUtils.readInt(PythonMessagingUtils.java:91)
at org.knime.python2.kernel.messaging.DefaultMessageReceiver.receive(DefaultMessageReceiver.java:80)
at org.knime.python2.kernel.messaging.DefaultMessageReceiverLoop.loop(DefaultMessageReceiverLoop.java:92)
… 6 more

0 Likes

#10

Hi ptrwlr,

The warning

2019-05-21 07:52:57,882 : WARN : main : Node : Python Script (DB) : 2:129 : Database does not support insert or update

could be interesting in general. But given that you only want to run the (unmodified) example workflow, this shouldn’t be a problem. Other than that, I don’t see anything helpful, unfortunately. I hoped to find some Python error tracebacks that might help to lead us in the right direction. Could you please try and search the log for the term Traceback and post all of the matches here?

Also: Did you try to run any other Python example workflow (maybe one where no databases are involved)? This would help to narrow down the problem to either databases or Python in general.

Thanks for your assistance and sorry for the inconvenience!

Marcel

0 Likes

#11

Hi MarcelW,

a Python script without DB connection works fine. I talked to our technical support for the Hadoop Cluster. We do not quite understand how the Python code (DB) is executed via Hive when we use the node. I have a hive connector then used a Python Script (DB) node. Could you give me some information? Possibly the problem is not at the node itself.

Thank you very much.

BR,
ptrwlr

0 Likes