I have files placed on knime server. using Path to string node with “create KNIME URL file system” checked, to access the file as knime server location, to ultimately pass it as flow variable in python script to read the files.
I am stuck at this point on how to get the file path from the knime server URL.
As you can see, the workflow works fine with listing files and decompressing files whiten workflow area, but when executing python node, it shows the following error:
No such file or directory.
Any suggestions on how to get the right path syntax of file on KNIME server would be helpful.
Thanks!
Welcome to the KNIME community.
As the Python Script node is not able to process knime URLs you have to use the Path to String node with the “create KNIME URL file system” checked first and then the URL to File Path node to create an absolute File Path in String format.
Thanks Michael for your reply,
I have added the “URL to file path” node after “Path to String” node even tho i get the string format without it. I got the following error from “URL to file path” node:
As I mentioned, files are on KNIME server not local. I need to know what is the correct URL path format for a file on server looks like.
Anyone have a clue on how to pass a path of a file on KNIME server, and python script node will be able to access it?
note: both workflow and file under the same folder.
please see the attached workflow on how to access files that are present in KNIME as the Path type from python. The trick is to convert the knime urls (knime://...) to absolute paths with the combination of a Path to String and URL to File Path node
I’m happy to help, but I’d need more information to do so
From your error message I can conclude that you are trying to access a file via knime://knime.workflow/../../data/file.dat in a KNIME node. The folder structure should then look something like this:
Is that correct and intended? Or is there a ../ too many?
Does the workflow I sent you work (i.e. gets executed without errors) on your Server?
Does the error show up in your or in my workflow?
In which node do you get the error?
What are the settings of the node in which the error shows up?
Can you share (parts of) your workflow?
The workflow I sent should be a minimal example and the idea would be to get that one working first. Once it works, you can adapt your workflow accordingly.
ah, thank you, this doesn’t look so bad two things you can check:
the output of the URL to File Path node: is there a string available that looks like an absolute path? Something like C:/folder/... or /home/alamrf/... ? If so, remember the column name, it will be the one we try to access in the python script. Most likely, this column is names “File Path” (see screenshot below, green annotations)
In the python script, are you using the column that contains the absolute path? Your can check this by including a print(file_path) statement and hit “Execute Script” or “Execute selected lines” in the configuration dialog (red annotations).
If it still fails, can you send the relevant part of the python code? A screenshot similar to mine should do, so that we can see the code Input variables and the relevant part of the code.
Reading files from Current Workflow Data Area seems to work fine when passed to python script, thanks to you! However, I mainly need to read from Current Workflow repository, which the path of it looks sth like this: “knime://knime.workflow/…/files/data/Hyper/abc.hyper”
including the two dots and forward slash in the middle.
What is the workaround in order for this path to be reachable by python script?
That is good to hear, we are on the right track! Do you get some sort of error message when trying to convert such a string to an absolute path or where does it fail?
It might be an issue with the file system access - by default, workflows are not allowed to write somewhere on the server machine, due to security issues. Generally, we advice to not set that setting, but use e.g. a shared drive whenever possible. Still, you (respectively the KNIME Server Admin) add the line /instance/org.knime.filehandling.core/allow_local_fs_access_on_server=true to the executor settings (see here: Access to the local file system is not Allowed - #2 by bruno29a).
When trying to generate a path of workflow repository file, results in the path format in my last reply. basically the error from python script node is FileNotFoundError: No such file or directory.
Target files resides on the server, not local, so I am not trying to reach local FS from server, I am only not able to get the right path format of these files exist on the server.
I just double checked, indeed the URL to File Path node will fail when trying to convert the knime:// URL to an absolute path. This is a security mechanism, so that you don’t (accidentally? mischievously?) write/edit files on the server file system. However, if you want to write files to the server file system via any of the scripting nodes, you need the absolute path, and for that you need the URL to File Path node. Hence, if you trust the other users on the server not to be malicious, you (i.e. the KNIME Server admin) can set said allow_local_fs_access_on_server-setting to true.
i see here in your path “C:\Users\Lukas…”, you are only able to access the “temp space” which the python node originally operates in.
Are you able to access files that are completely resides on the server with python node? not in the WF Data Area.
If I understand you correctly, yes as long as you have the allow_local_fs_access_on_server=true, and the user has the proper permissions on the file share/server path to write or access the location, then they should be able to access any file path on the server.
Hi Zack,
thanks for your reply.
Please keep in mind that I need to access files that reside completely on KNIME server, Not local. as in this screenshot:
let’s say I want to reach the “binary_file_0.dat” file by a python node in a workflow located somewhere on the SAME server as the files, How to get the correct path that can be passed to python node?
sorry for the late response - it’s indeed a little bit tricky to get the absolute paths in python if the file is not within the workflow. The reason is this: The workflow repository on the KNIME Server and the KNIME Executor might not be on the same machine, thus my suggested approach fails. This is handled fine by the KNIME Nodes when they see the knime:// protocol, as they open streams to handle that. This is not done so in python, hence this cannot always be converted to absolute paths and hence it fails. When executing a workflow as a job on the executor, however, the whole workflow (including its data area) is copied over from the repository - and the trick works.
What can we do about it then? Two things come to my mind:
write the files to a shared directory (SMB, Google Drive, AWS, …) which is also accessible via python code
transfer the file from the workflow repository to the data area (and back) before and after the python script node, e.g. with the Transfer File Node
Do you think you can work with one of the options?
two things you can check:
