I am guessing that you are using Windows Operating System with a Chinese locale. From your error, it appears that when you are printing your results Python is trying to convert the UTF-8 decoded string to a string with the same encoding as your operating system locale using the
gbk codec which converts to a Chinese encoding which doesn’t cover all of the Unicode characters. Therefore, when you get to a character that cannot be converted Python complains.
I am guessing, that it will work in VS Code if the terminal window supports UTF-8 characters and Python then has an encoding target that supports all of the characters in your source material. As to why this is so I cannot answer; unless I have more information on your machine and how the KNIME Python node generates its terminal window.
Language locales is a messy topic and creates all sorts of problems that Unicode is intended to resolve. However, because there are many legacy documents (files) which use non-Unicode characters and legacy software that uses them (written in Python) the path to enforcing UTF-8 as a standard for reading and writing files is long and painful. Note: This is for user files, the standard for code files is and has been for a long time UTF-8.
One suggestion I can make is to force Python to ignore the locale setting and use UTF-8 encoding for strings. This cannot be done within KNIME and needs to be set before the Python interpreter has started.
You can do that two ways:
A/ Open PowerShell and enter:
& 'C:\Program Files\KNIME\knime.exe'
This will set Python to use UTF-8 encoding for the session and then start the KNIME software. Note, you may need to adjust your path to the KNIME application. This may be a good way to test whether it works for you. If
$env:PYTHONUTF8=1 doesn’t work then try
B/ Set the environment variable
If the above works then you can set the PYTHONUTF8=1 environment variable permanently by opening system properties and adding it under Environment Variables…as a user variable.
Be aware that setting this environment variable to apply to every instance of Python that you will run may cause applications to break. Whilst there is a plan to make this a default setting in Python 3.15 there are still applications being used that may break. So, use with caution.
I’ve had to make a lot of guesses and assumptions, so there is no guarantee that it will work or is the root cause of your problem. But it may give you something to try and clues as to where to look for the answer.
LinkedIn | Medium | GitHub