Bash Node: Command not found

PS: I suspect the path variables being at “fault” due to

Which I once encountered in

But also others

Cheers
Mike

Update: It is indeed as I thought that the PATH environment variable does not match.

Knime: PATH=/usr/bin:/bin:/usr/sbin:/sbin
OS: PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/Apple/usr/bin:/Applications/Wireshark.app/Contents/MacOS

Anyone knows how to adjust this as I could not find anything in the Knime preferences?

Many thanks in advance
Mike

Hi @baj,

I possibly posted this in the wrong forum section as I thought this is more a general question but it seems I was wrong. In search for a solution to instruct Knime respectively your bash Node to use the actual PATH pariable, I wonder if you could shed some light onto this?

Googling around to adjust the Eclipse Path variable I could not come to a solution (please see my prev. comment). Do you happen to have an idea?

Many thanks in advance
Mike

Hi @mwiegand,

with respect to the path issue, can you print the environment using “set” or “env”?

One quick way to solve it, is to use the full path…

Other things to consider are where you set your path in the login script or .profile… I am not sure which one JAVA is using to start the command.

It has been a few moons since I programmed this and haven’t had time to return to KNIME… What a shame!!!

Glad to see that people are still using the nodes.

Hope this helps.

Best,

Bernd

1 Like

Did you figure out the awk issue?

to me it sounds like you need either other quotes, or need to escape them with with 1-3 ""…

B

Hi Baj,

I haven’t continue working on the awk issue but with the recently gained knowledge I might do that. Though, that project I worked on back then was closed (I switched employers). For the sake of interest – and to satisfy my German urge to do things right :slight_smile: – I might come back to this in the future.

About the environment I printed the variables in a previous comment. When chekcing the env vars in terminal they differ from those available in Knime. Therefore, i suspect that Knime is using another configuration file.

However, I could not find anything. Even by declaring the PATH variable in the knime.ini it did not changed anything. The HOME and SHEEL Variables are identical, though. Therefore, I thought it’s using the same bash which I confirmed via

which bash true false

Setting the absolute directory for the command not found by first identifying the executable “/usr/local/bin” via (in terminal):

which nmap true false

Did not resolve the command not found issue.

Creating a .bash_profile and declaring the PATH variable, applying via source .bash_profile, didn’t resolve it either. I assume Eclipse / Knime is spun up in a separate user / config or so not utilizing the system nor my actual user configuration (to the full extend).

The answer from Stackoverflow, declaring a key in the Info.plist file of Knime did not work too.

I also declared the PATH variable in /etc/profile to make it accessible for all users to no avail. Knime was restarted each single time.

Best
Mike

PS: I believe it is less related to your bash node but more to Knime in general. using a Java Snippet to retrieve the PATh variable I too get the same result.

out_path = System.getenv("PATH");

Bildschirmfoto 2022-08-25 um 12.19.34

@ipazin you are always a great and knowledgeable source. Do you maybe have an idea?

Cheers
Mike

Where do you set your PATH variable in the shell?
(.bashrc, .profile)?
Bernd

Hi Bernd,

I tried both as per this explanation but neither did work: About bash_profile and bashrc on macOS – Scripting OS X

Is this actually reproducible on other OS or unique to OSX? Shall I start a VM for further testing?

Cheers
Mike

did you check that you are actually using bash? the newer OS is recommending something else.

just saw it… you are using zsh… I don’t use that one, so not entirely sure how to things there…

B

I suppose zsh is used per the SHELL variable shown in Knime

Even the TMPDIR variable matches which I’d suspect getting randomly regenerated with each session. There is a higher level of alignment but for some reason if ignores or overwrites the PATH variable.

:exploding_head:

Zsh Profile File (.profile, .zprofile, .zshrc) – LinuxTect.

I changed from zsh to bash but Knime did not pick that up. The shell profiles, whichever I configured, are not picked up either.

Despite being odd it further indicates that Knime is “minding it’s own business” not fully caring about OS preferences. WDYT?

did you log-out after changing and potentially reboot…?

To confirm mwiegand’s assessment of the problem:

  • it appears that the $PATH declaration made by a user in either .bash_profile or .zshrc (the profile for zsh) has no effect on the PATH established by the Bash node.
  • for this reason, any executable residing outside of that path variable definition is not found.
  • for some reason, variables (which use the “$” character) are not recognized, so one is prevented from executing either a temporary addition to the path or a ‘permanent’ change to the profile

At this point, I’ve all but given up. My last effort will be to determine whether the conda environment that prevails when I try to use the Bash node might have some means of affecting the ability to temporarily (for the duration of the Bash node session) modify the $PATH variable.

This same issue appears in my attempts with Cmdwinput node and External Tool node. All of them put you in $PATH jail.
= RM

1 Like

Just for completeness, I rebooted but it did not have any effect. I tried setting the PATH variable in Knime but it failed with Execute failed: Cannot execute command as well.

I think the only two options left at present are to:

  1. Create a shell script, which could get done via Knime, and make it executable OR
  2. Using Java to make bash calls like in Shell Command in knime

While attempting the two options I encountered an issue with approach #1. I can’t make the shell script executable. Setting POSIX permission fails.

Bildschirmfoto 2022-10-18 um 15.32.16

With approach #2 I noticed a possible cause of the command not found. Using the Bash Node to get all env variables it gave back a PATH variable. However, using the Java Snipped the PATH variable was not found. Could it be that the PATH variable is not properly setup / defined and Knime respectively Eclipse falls back to a default?

I put all bits and pieces in this workflow together:

NOTE: There is a mistake with the Java Snippet which I noticed only after upload. Will update the workflow once resolved.

There are a few additional interesting findings as well. Using the whereis YOUR-CMD command to locate the executable the commands which do NOT work still could be found. The where YOUR-CMD (without “is”) command also works in the terminal but not in Knime.

where and whereis CMD in Terminal
Bildschirmfoto 2022-10-18 um 17.02.20

where and whereis CMD in Knime

Since not everyone uses OS X, even though it’s linux, in case anyone wants to debug something, we could arrange a remote debugging session.

Best
Mike

Hello @mwiegand,

missed this one somehow. Unfortunately I can’t help here…

Br,
Ivan

Thanks anyways @ipazin!

The other thread showcasing an approach to execute shell commands via a Java Snippet Node is unfortunately dead I noticed.

Does maybe anyone here notice the flaw in the code I attempted to refurbish but which does not give any feedback but just an empty response?

The example is also in the workflow I shared in my previous comment.

Best
Mike

Just for the record. Thanks to @carstenhaubold there is a Python based approach of executing bash commands.

Unfortunately, though, it has the same issue that some commands are not found. That pretty much makes it abundantly clear that the PATH variable difference between Knime / Eclipse and vs. user under which it runs is at fault.

Bug Report

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.