# Python Knio write error

**URL:** <https://forum.knime.com/t/python-knio-write-error/45333>\
**Category:** KNIME Analytics Platform\
**Created:** [July 21, 2022, 5:08pm UTC](https://forum.knime.com/t/python-knio-write-error/45333 "2022-07-21T17:08:45Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Daniel\_Weikert](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/d/e47c2d/32.png) [@Daniel\_Weikert](https://forum.knime.com/u/Daniel_Weikert)\
**Post date:** [July 21, 2022, 5:08pm UTC](https://forum.knime.com/t/python-knio-write-error/45333/1 "2022-07-21T17:08:45Z")

</div>

Hi I tried the new python labs node and have an issue when writing back

```auto
import knime_io as knio
import pyarrow as pa
# This example script simply outputs the node's input table.
print(type(knio.input_tables[0]))
df = knio.input_tables[0].to_pandas()
print(df.head())
df = df[['Cat','CustomerID']]
ar = pa.Table.from_pandas(df)
#does not work
knio.output_tables[0] = knio.write_table(ar)

#does work
#knio.output_tables[0] = knio.write_table(knio.input_tables[0])

```

In the Node both work but when I close the node and execute the workflow the upper throws me an error  
Any ideas? It just tells me exception occurred in the python kernel. Thanks

---

<div class="post-metadata">

**Author:** ![mlauber71](https://forum-cdn.knime.com/user_avatar/forum.knime.com/mlauber71/32/73590_2.png) [@mlauber71](https://forum.knime.com/u/mlauber71)\
**Post date:** [July 21, 2022, 9:02pm UTC](https://forum.knime.com/t/python-knio-write-error/45333/2 "2022-07-21T21:02:31Z")

</div>

@Daniel_Weikert could you give us more details for example what the error message says? Better even could you provide a workflow that would demonstrate the error?

Two possibilities would be date and time variables where there seem to be problems.

> [@Error with pandas in the Labs Python Integration](https://forum.knime.com/t/error-with-pandas-in-the-labs-python-integration/44473/9):
>
> @toscanomatheus this might be an interesting case. I will have to have a closer look at your specific file. I startet to look into the question pof the Python Labs nodes and Date and Time variables (again) and indeed there seems to be a problem with the handling of types. In a previous discussion I toyed around with several formats and then exported the data thru Parquet files and brought the data back to KNIME which worked (mostly): This time I tried two things with string formats exported …

Or might there be an issue with RowID and index in pandas?

> [@Error in Python script node](https://forum.knime.com/t/error-in-python-script-node/41439/5):
>
> Hi @mohini1329, as the others have already suggested, you should make sure that each row in the Pandas DataFrame has a unique index. Dropping/resetting the index is not the best option though, because KNIME uses the RowID (which is equal to the DataFrame index) to identify rows when you select / highlight individual cells in your data. If you reset the index, then this row identification will no longer work. The best solution would be to let the index of all values in your input table as they…

---

<div class="post-metadata">

**Author:** ![Daniel\_Weikert](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/d/e47c2d/32.png) [@Daniel\_Weikert](https://forum.knime.com/u/Daniel_Weikert)\
**Post date:** [July 22, 2022, 4:13pm UTC](https://forum.knime.com/t/python-knio-write-error/45333/3 "2022-07-22T16:13:48Z")

</div>

Thanks a lot for your comments @mlauber71 .  
I made sure to remove any datetime columns and resetted the index in the pandas df.  
It works smoothly as long as I have the node open. Only when running the workflow it fails.  
Error is  
Execute failed: An exception occured while running the Python kernel. See log for details.  
but the KNIME log does not show anything? What kind of log is this message pointing to?

Are the new nodes a huge performance boost? Just wondering because the old node works fine with this “testing 2 line script”  
br

---

<div class="post-metadata">

**Author:** ![mlauber71](https://forum-cdn.knime.com/user_avatar/forum.knime.com/mlauber71/32/73590_2.png) [@mlauber71](https://forum.knime.com/u/mlauber71)\
**Post date:** [July 23, 2022, 8:31am UTC](https://forum.knime.com/t/python-knio-write-error/45333/4 "2022-07-23T08:31:31Z")

</div>

@Daniel_Weikert oils you [check your python setup](https://kni.me/w/88FrmrDO5ZwlqByH) and maybe try with a fresh one following the [KNIME Python Guide](https://docs.knime.com/latest/python_installation_guide/index.html#_introduction) or the [provided yaml files](https://forum.knime.com/t/python-environment-creation-problem/41577/6).

Logs in debug modus:

> [@Files Corrupted During File Transfer](https://forum.knime.com/t/files-corrupted-during-file-transfer/43738/8):
>
> @djarrett I could not initially find any problems. You could get rid of the vernalis error messages by [installing RDKit](https://forum.knime.com/t/error-plugin-com-vernalis-knime-chem-mmp/30068/9). Could you create a LOG in DEBUG mode? Open the existing log and delete the content: [image] or delete the file from the workspace directory {workspace\_directory}/.metadata/knime/knime.log activate the DEBUG level. Make sure that it would not contain any sensitive informations and remember to set it back to WARN after you are finished: [image] You might want to include f…

Also you might create a minimal workflow that could be checked.

---

<div class="post-metadata">

**Author:** ![carstenhaubold](https://forum-cdn.knime.com/user_avatar/forum.knime.com/carstenhaubold/32/33947_2.png) [@carstenhaubold](https://forum.knime.com/u/carstenhaubold)\
**Post date:** [July 25, 2022, 6:44am UTC](https://forum.knime.com/t/python-knio-write-error/45333/5 "2022-07-25T06:44:35Z")

</div>

Hi @Daniel_Weikert,

could you try to run

```python
knio.output_tables[0] = knio.write_table(df)

```

instead of

```python
ar = pa.Table.from_pandas(df)
knio.output_tables[0] = knio.write_table(ar)

```

and see if that works? KNIME performs some additional type handling when converting between Pandas and PyArrow internally, but when you do it manually KNIME cannot do that for you. So that could be the culprit.

A minimal workflow using the same data types (but maybe randomized data) as your real workflow would help us to reproduce and understand the problem better.

Best,  
Carsten

---

<div class="post-metadata">

**Author:** ![Daniel\_Weikert](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/d/e47c2d/32.png) [@Daniel\_Weikert](https://forum.knime.com/u/Daniel_Weikert)\
**Post date:** [July 25, 2022, 5:11pm UTC](https://forum.knime.com/t/python-knio-write-error/45333/6 "2022-07-25T17:11:05Z")

</div>

> [@carstenhaubold](#):
>
> `knio.output_tables[0] = knio.write_table(df)`

Thanks a lot @carstenhaubold  
That worked. So it is never required to convert the dataframe back even it is converted to pandas after reading it? The output writer will always accept a pandas dataframe?  
br

---

<div class="post-metadata">

**Author:** ![carstenhaubold](https://forum-cdn.knime.com/user_avatar/forum.knime.com/carstenhaubold/32/33947_2.png) [@carstenhaubold](https://forum.knime.com/u/carstenhaubold)\
**Post date:** [July 26, 2022, 11:00am UTC](https://forum.knime.com/t/python-knio-write-error/45333/7 "2022-07-26T11:00:03Z")

</div>

Yes, as we allow to read input tables using `pyarrow` or `pandas` we also allow to return both of those types.

May I ask which data types are inside your data frame? Could you show us the output of `print(df.dtypes)`?

---

<div class="post-metadata">

**Author:** ![Daniel\_Weikert](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/d/e47c2d/32.png) [@Daniel\_Weikert](https://forum.knime.com/u/Daniel_Weikert)\
**Post date:** [July 26, 2022, 5:59pm UTC](https://forum.knime.com/t/python-knio-write-error/45333/8 "2022-07-26T17:59:19Z")

</div>

The dataframe hat several types including datime, float, category.  
But the output I converted back to pyarrow where only to categorical columns and with those only it already failed  
br

---

<div class="post-metadata">

**Author:** ![system](https://forum-cdn.knime.com/uploads/default/original/1X/ab3ccf34482a0329361734a18199390177204f15.png) [@system](https://forum.knime.com/u/system)\
**Post date:** [August 2, 2022, 5:59pm UTC](https://forum.knime.com/t/python-knio-write-error/45333/9 "2022-08-02T17:59:43Z")

</div>

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

---

<div class="post-metadata">

**Author:** ![carstenhaubold](https://forum-cdn.knime.com/user_avatar/forum.knime.com/carstenhaubold/32/33947_2.png) [@carstenhaubold](https://forum.knime.com/u/carstenhaubold)\
**Post date:** [August 3, 2022, 9:41am UTC](https://forum.knime.com/t/python-knio-write-error/45333/10 "2022-08-03T09:41:48Z")

</div>

Thanks for the feedback, we’ll have a look at categorical columns!
