UnicodeEncodeError: 'charmap' codec can't encode characters

[JayR]

Greetings,

I’m using Python Script node to run my Jupyter Notebook and then use one of its dataframes as data table in my workflow, but I’m getting the following error:
UnicodeEncodeError: ‘charmap’ codec can’t encode characters

I’ve checked the posts been posted before with the same error, but the suggested solutions didn’t work in my case.

I would truly appreciate your assistance!
Thanks

[mlauber71]

@JayR without having checked this out I would see these ways forward

• make sure the file in the jupyter notebook really is a pandas dataframe without any funny names or data types (first)
• experiment with the formats for communicating between python and knime (In the preferences)
• store the data from the jupyter notebook as csv or parquet file and load it back (you will have to deal with all the path things). Not very elegant but it might just work

I was planning on expanding this example to show data communication between knime and jupyter but have not done so yet.

Use functions and Python code from an external .PY file or a Juypter notebook...

[mlauber71]

@mlauber71

I was planning on expanding this example to show data communication between knime and jupyter but have not done so yet.

OK I have added a part to the workflow where KNIME, Python and Jupyter would communicate with each other. ‘Variables’ can be exchanged via a file and also data can be transferred via Parquet files. It would be great though to be able to transfer ‘real’ variables between KNIME and Jupyter (like in this idea).

Unfortunately on my Mac the export thru the new Python integration produces a strange error about index when trying to access the results. Some nodes (like Group by) although would work. I have tried several things with columnar backend, Flatbuffer communication and also re-setting the index in the pandas data frames. To no avail.

ERROR Cache 0:262 Execute failed: index: -1, length: 0 (expected: range(0, 364))

@MarcelW maybe you could investigate. My fear is that something with the arrow integration still is not right - admittedly I tried to challenge the parquet format somewhat with different data types (had to cut back on that already). But what is strange that the Parquet reader can handle that and also ‘within’ the Python node everything seems fine. What fails is the write_table:

knio.output_tables[1] = knio.write_table(df1)

But the ‘communication’ between KNIME, Python node in KNIME and Jupyter would just work fine.

The interesting feature being to tag only the cells in the notebook “test_data_all_types.ipynb” in the subfolder “/script/” that you would want be executed in KNIME. So you could combine some exploration and execution.

Use functions and Python code from an external .PY file or a Juypter notebook...

[MarcelW]

Hi @JayR,

Does your notebook contain any code cells with non-latin characters?
The following line in the error message you posted:

print("Failing notebook code:\n + code)

suggests that KNIME failed to load your notebook because one of the notebook’s cells failed to run. KNIME then tried to report the failing cell by printing out its content. This, in turn, failed, most likely because the cell contained text that could not be encoded when printing it out.

That is, the “UnicodeEncodeError” you see is probably not the root cause of your problem. I would suggest to open your notebook in Jupyter, run all cells (using a fresh kernel), and check if they all run through successfully. KNIME can only load your notebook if this is the case. In case you want KNIME to only run/load certain cells, you can tag them and instruct KNIME to only load those tagged cells, as @mlauber71 suggested.

Marcel