Is it possible to use pipenv to install python packages like Keras for use with KNIME?
I rather like the idea of pipenv and wonder if there are any pointers how to use it along with KNIME?
Thanks
Hi knick,
Yes, this should be possible (in fact, I just tried it myself and it worked ).
Here’s what I did (Linux):
pipenv --three # or --two
pipenv install keras tensorflow pandas
pipenv shell
pipenv shell
will output the command to activate its virtual environment. The command should look like this:
Launching subshell in virtual environment…
. /home/<user>/.local/share/virtualenvs/<project-name>-E4Euwy4K/bin/activate
You can now take the example script from our Python setup guide for conda and replace the line source activate py35_knime
(Linux/Mac) or the respective part of the the line @CALL activate py35_knime || ECHO Activating py35_knime failed
(Windows) by the command that was output by pipenv shell
. You can also remove the line where Anaconda is added to the PATH variable. Then save the script. If you’re on Linux/Mac, you’ll also have to make it executable.
In my case, the script for use with pipenv looks like this:
#! /bin/bash
. /home/marcel/.local/share/virtualenvs/marcel-E4Euwy4K/bin/activate
python "$@" 1>&1 2>&2
All that’s left to do now, is to point to that file via File > Preferences > KNIME > Python > Path to Python (2|3) executable. The preference page should give you immediate feedback on whether the environment was detected.
Please let me know if you experience any problems .
Marcel
Hi Marcel, thanks for the quick answer! Unfortunately I get stuck at Locking [packages] dependencies…
Might have something to do with my connection, on the road. Will try it again when I get home
Ok, installation finished and I did $ pipenv shell
and put the resulting path in a .sh file which I made executable and entered in knime. But then Knime shows an error: python cant open file
, and then the path to my knime plugins directory and the file PythonKernelTester.py
and [Errno 2] No such file or directory
Any ideas?
Hi knick,
Can you double-check whether the file PythonKernelTester.py
at the given path actually exists? Maybe something went wrong during the installation of the Python extension.
Can you also take a look at your knime.log
file and see if there is any additional information? Thanks!
Additionally, you could try to run the command you got from pipenv shell
outside of KNIME, e.g., by simply pasting it to a command line. Maybe there’s something wrong.
Marcel
The file exists and the command from pipenv shell
seems to work. Looking at the last lines in the logfile, it may contain related stuff. But It is 8mb!
Thanks for double-checking. Can you maybe upload these last lines of the file? Or delete the log file and reproduce the python cant open file
error again. Then only the relevant lines should be in the (now much smaller) log file.
This should be the relavant part
2018-10-27 13:25:55,201 : DEBUG : KNIME-ConfigurationArea-Checker : ConfigurationAreaChecker : : : Configuration area check completed in 0.0s
2018-10-27 13:27:24,467 : DEBUG : Thread-13 : PythonKernelTester : : : Error occurred during testing Python2 installation
2018-10-27 13:27:24,468 : DEBUG : Thread-13 : PythonKernelTester : : : Executed command: /home/user/st/ml/knime/keras-test/activate-env.sh /home/user/st/ml/knime/knime_3.6.1.linux.gtk.x86_64/knime_3.6.1/plugins/org.knime.python2_3.6.1.v201808311614/py/PythonKernelTester.py 2.7.0
PYTHONPATH=:
PATH=/home/user/.local/bin:/home/user/.nvm/versions/node/v10.1.0/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/home/user/built/bin
Error during execution: python: can't open file '“/home/user/st/ml/knime/knime_3.6.1.linux.gtk.x86_64/knime_3.6.1/plugins/org.knime.python2_3.6.1.v201808311614/py/PythonKernelTester.py': [Errno 2] No such file or directory
Raw test output:
2018-10-27 13:27:24,526 : DEBUG : Thread-14 : PythonKernelTester : : : Error occurred during testing Python3 installation
2018-10-27 13:27:24,526 : DEBUG : Thread-14 : PythonKernelTester : : : Executed command: /home/user/st/ml/knime/keras-test/activate-env.sh /home/user/st/ml/knime/knime_3.6.1.linux.gtk.x86_64/knime_3.6.1/plugins/org.knime.python2_3.6.1.v201808311614/py/PythonKernelTester.py 3.0.0
PYTHONPATH=:
PATH=/home/user/.local/bin:/home/user/.nvm/versions/node/v10.1.0/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/home/user/built/bin
Error during execution: python: can't open file '“/home/user/st/ml/knime/knime_3.6.1.linux.gtk.x86_64/knime_3.6.1/plugins/org.knime.python2_3.6.1.v201808311614/py/PythonKernelTester.py': [Errno 2] No such file or directory
Raw test output:
Does the log say anything interesting?
Sorry for the delay.
Not really. There’s a double quotation mark at the beginning of the path that Python complains about ('“/home/user/st/ml...'
). Maybe that’s just an error in the print output but it isn’t displayed when I try to execute some nonesense like this: python lala.py
(output: python: can't open file 'lala': [Errno 2] No such file or directory
). Can you try this yourself and see if there’s a double quotation mark?
Could you also share your activation script (activate-env.sh)? I just want to make sure everything’s alright there.
Also, did you already try to execute the entire command /home/user/st/ml/knime/keras-test/activate-env.sh /home/user/st/ml/knime/knime_3.6.1.linux.gtk.x86_64/knime_3.6.1/plugins/org.knime.python2_3.6.1.v201808311614/py/PythonKernelTester.py 3.0.0
from a shell?
Solved! The problem was I copied the below. For some reason " had turned into “.
Oh, I didn’t expect that. I fixed my post. Thanks for working it out!
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.