# Export data from KNIME for processing in Python (externaly)

**URL:** <https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495>\
**Category:** KNIME Analytics Platform\
**Tags:** python\
**Created:** [September 23, 2022, 7:17am UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495 "2022-09-23T07:17:26Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![TheLeo](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/t/85f322/32.png) [@TheLeo](https://forum.knime.com/u/TheLeo)\
**Post date:** [September 23, 2022, 7:17am UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/1 "2022-09-23T07:17:26Z")

</div>

Hello all,

in our environment we often have the use case, that incoming data is prepossessed in Knime and then later analyzed using python.  
Now I know that you can run python within knime but that is not exactly what we need.  
In our case the data needs to be exported from knime and then analyzed on a different machine with python.  
The question now is how to do that efficiently? Saving the data as CSV is not optimal since it doesn’t store the column type.  
I know the best way would probably be a dedicated database but if that’s not an option what would you recommend to use?

Thanks a lot!

---

<div class="post-metadata">

**Author:** ![mlauber71](https://forum-cdn.knime.com/user_avatar/forum.knime.com/mlauber71/32/73590_2.png) [@mlauber71](https://forum.knime.com/u/mlauber71)\
**Post date:** [September 23, 2022, 7:44am UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/2 "2022-09-23T07:44:10Z")

</div>

@TheLeo you could use [Parquet](https://kni.me/n/jita8ax2pW9UyCI8) to keep the column types. Or SQLite database:

> **[use the new (KNIME 4.6+) Python Script node and bundled Python version to...](https://hub.knime.com/mlauber71/spaces/Public/kn_example_python_read_parquet_file_2021~G1kX4pbOlCeq56cH/current-state)**
>
> use the new (KNIME 4.6+) Python Script node and bundled Python version to read Parquet file into KNIME, export it again, put it into SQLite database and read i…

ORC could also be an option but it might be more complicated. Both ORC and Parquet would allow to read and write data in chunks.

> **[Read ORC file into KNIME's Python node – mlauber71](https://hub.knime.com/mlauber71/spaces/Public/kn_example_python_read_orc_file~uW1d55taxEotKFHO/current-state)**
>
> Read ORC file into KNIME's Python node If you want to transfer several files from outside sources into the Python environment and not loose column types ORC is…

If you absolutely must have a text only file. [ARFF](https://datahub.io/blog/attribute-relation-file-format-arff) is another option:

> **[use the new (KNIME 4.6) integrated Python extension to read and write ARFF...](https://hub.knime.com/mlauber71/spaces/Public/kn_example_python_read_arff_file~mp7Pv607rD0biVI5/current-state)**
>
> use the new (KNIME 4.6) integrated Python extension to read and write ARFF file into KNIME, export it again as Parquet, put it into SQLite database and read it…

If you want to explore more about exchange between KNIME and Python you might have a look here:

> **[Use functions and Python code from an external .PY file or a Juypter notebook...](https://hub.knime.com/mlauber71/spaces/Public/kn_example_python_import_module~BQEN92dR6W0cUM8R/current-state)**
>
> Use functions and Python code from an external .PY file or a Juypter notebook in KNIME's Python Source node This workflow demonstrates how modules in (external…

> [@UnicodeEncodeError: 'charmap' codec can't encode characters](https://forum.knime.com/t/unicodeencodeerror-charmap-codec-cant-encode-characters/39199/2):
>
> @JayR without having checked this out I would see these ways forward make sure the file in the jupyter notebook really is a pandas dataframe without any funny names or data types (first) experiment with the formats for communicating between python and knime (In the preferences) store the data from the jupyter notebook as csv or parquet file and load it back (you will have to deal with all the path things). Not very elegant but it might just work I was planning on expanding this example to sho…

---

<div class="post-metadata">

**Author:** ![TheLeo](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/t/85f322/32.png) [@TheLeo](https://forum.knime.com/u/TheLeo)\
**Post date:** [September 23, 2022, 8:38am UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/3 "2022-09-23T08:38:15Z")

</div>

@mlauber71  
Thanks!  
Text Only file is not necessary.  
In a best case scenario the file format would be:

- Easy/fast to export from knime
- Easy to get into a pandas dataframe in python without setting all datatypes manualy
- some sort of compression since csv files get big really fast just by having the same text in each row for example.

I’ll look at your proposals!

---

<div class="post-metadata">

**Author:** ![mlauber71](https://forum-cdn.knime.com/user_avatar/forum.knime.com/mlauber71/32/73590_2.png) [@mlauber71](https://forum.knime.com/u/mlauber71)\
**Post date:** [September 23, 2022, 8:59am UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/4 "2022-09-23T08:59:17Z")

</div>

@TheLeo then maybe Parquet is your best option. I works like this:

```auto
import pandas as pd 
import pyarrow.parquet as pq 

# in Jupyter / Python
df = pq.read_table("your_local_file.parquet").to_pandas() 

# in KNIME node (or again Jupyter / Python)
df.to_parquet("your_file_from_knime.parquet", compression='gzip')

```

---

<div class="post-metadata">

**Author:** ![TheLeo](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/t/85f322/32.png) [@TheLeo](https://forum.knime.com/u/TheLeo)\
**Post date:** [September 23, 2022, 9:07am UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/5 "2022-09-23T09:07:14Z")

</div>

@mlauber71  
Thanks for your input, I already tested it with “pd.read\_parquet()” which works but when I use your way I get an error: “UnicodeDecodeError: ‘utf-8’ codec can’t decode byte 0xb3 in position 11: invalid start byte”  
Is there a reason you didn’t recommend read\_parquet?  
Now when I look at the imported data (using read\_parquet) I discovered unfortunately that the column types in fact are not correctly transferred: An exported INT becomes after the import a float. Am I missing something?

thanks again for your help!

---

<div class="post-metadata">

**Author:** ![Daniel\_Weikert](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/d/e47c2d/32.png) [@Daniel\_Weikert](https://forum.knime.com/u/Daniel_Weikert)\
**Post date:** [September 23, 2022, 5:01pm UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/6 "2022-09-23T17:01:57Z")

</div>

Do you have a sample to share?  
br

---

<div class="post-metadata">

**Author:** ![DiaAzul](https://forum-cdn.knime.com/user_avatar/forum.knime.com/diaazul/32/48297_2.png) [@DiaAzul](https://forum.knime.com/u/DiaAzul)\
**Post date:** [September 24, 2022, 12:25pm UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/7 "2022-09-24T12:25:23Z")

</div>

@TheLeo

Your suggestion to use a database is the most appropriate solution and will save a lot of grief in the mid to long term. It doesn’t have to be difficult and I frequently deploy temporary containerised databases to test client workflows. Once you have a database you can work to optimise its structure and indexing so that downstream processing becomes more efficient.

As a half-way solution you might want to consider storing your data to an SQLite database. In KNIME you would extend your workflow to add an SQLite connector which defines the file into which the data is written; and a DB Writer that outputs the data to the defined table name. You could include more than one table in the SQLite database if that is required.

![Screenshot 2022-09-24 131528](https://forum-cdn.knime.com/uploads/default/original/3X/9/d/9d13fc9cd7b188c98d3d87b49f0f6bb73f3fceb1.png)

In Python you would use SQL Alchemy to import the data from the SQLite database. The following example loads the data into a Pandas DataFrame.

```python
import sqlalchemy as db
import pandas as pd

engine = db.create_engine("sqlite:///D:\\Data\\tempDatabase.sqlite")
with engine.connect() as db_connection:
    table_name = 'TestTable'
    df = pd.read_sql(f'SELECT * FROM {table_name}', con=db_connection)

```

Using this approach you do not need a network based database; however, if you do install one at a later date then both KNIME and Python code will need minimal changes.

DiaAzul

---

<div class="post-metadata">

**Author:** ![Geo](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/g/f4b2a3/32.png) [@Geo](https://forum.knime.com/u/Geo)\
**Post date:** [September 25, 2022, 12:43pm UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/8 "2022-09-25T12:43:52Z")

</div>

Depending on the data types you use in KNIME, another solution is to save the data using an R node. To retain the most common data types, you could resort to the package haven (R) and write e.g. a Stata file (dta). The Stata format has a decent set of data types (byte, int, long, float, double, str, Lstr, etc.).

This solution is a one liner in the relevant R node (easy) and should be pretty fast. Not sure whether the files will be small though. The SPSS format offers compression but its set of data types is also less refined than Stata’s.

---

<div class="post-metadata">

**Author:** ![TheLeo](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/t/85f322/32.png) [@TheLeo](https://forum.knime.com/u/TheLeo)\
**Post date:** [September 26, 2022, 7:06am UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/9 "2022-09-26T07:06:49Z")

</div>

@DiaAzul  
We’ve tested SQLite but unfortunately it will not recognize dates & date/times correctly (and has no compression at all).  
@Daniel_Weikert  
I’ll try to create example file to show the issues we’ve

---

<div class="post-metadata">

**Author:** ![TheLeo](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/t/85f322/32.png) [@TheLeo](https://forum.knime.com/u/TheLeo)\
**Post date:** [September 26, 2022, 7:43am UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/10 "2022-09-26T07:43:01Z")

</div>

So, now I’ve the examples:  
Workflow for the test data: [Data Export Example – KNIME Hub](https://hub.knime.com/theleo/spaces/Public/latest/Data%20Export%20Example~ovSsl-AvJFwZYtmy)

Import in python:

 ![image](https://forum-cdn.knime.com/uploads/default/original/3X/2/1/2166d603784a31ae47a06a58773cd7b1188faaa5.png)

If it would work, the columns should show up as Date, String/Object, Int, String/Object  
But in both cases, the date is not recognized correctly.

---

<div class="post-metadata">

**Author:** ![DiaAzul](https://forum-cdn.knime.com/user_avatar/forum.knime.com/diaazul/32/48297_2.png) [@DiaAzul](https://forum.knime.com/u/DiaAzul)\
**Post date:** [September 26, 2022, 11:26am UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/11 "2022-09-26T11:26:28Z")

</div>

@TheLeo

Thanks for the data, that helps a bit. Three thoughts/ideas for you to consider (this will be a long post as I will include examples).

I’ve uploaded the workflow that contains all the examples to KNIME hub [here](https://kni.me/w/vu9vs1pPEb8FC_98).

**1/ Convert data/time to ISO8601 string.**

When writing dates and times to file formats/ databases that don’t support a native date/time type, then ISO8601 format strings should be used. This is an internationally recognised standard and widely supported across software platforms/packages. When reading an ISO8601 string Pandas can be configured to parse the string into a Pandas datetime dtype.

In KNIME you would convert the dateTime column to a string.  
 ![Screenshot 2022-09-26 113711](https://forum-cdn.knime.com/uploads/default/original/3X/3/4/34810ac25708f78fd34c721f73209bfa899bf5f6.png)

And in Python you would configure Pandas to parse the string. The additional parameter compared with my prior post is `parse_dates` which is passed a list of columns containing ISO8601 format strings.

```python
import sqlalchemy as db
import pandas as pd

engine = db.create_engine("sqlite:///D:\\data\\sqlite.sqlite")
with engine.connect() as db_connection:
    table_name = 'Export'
    df = pd.read_sql(f'SELECT * FROM {table_name}',
                     con=db_connection,
                     parse_dates=["Date"])

```

**2/ Implement a data schema**

Your country column is a categorical column and has a lot of redundant duplicate strings. This is going to increase your output file size considerably. You might want to consider creating dimension tables holding which are referenced by the original (fact) table. In KNIME this is relatively easy to do.

In the following workflow, the a dimension table is created by grouping the `Country` column in `Create Testdata`. A counter generator is used to generate a set of sequential keys (integers). The table is then manipulated to rename the counter generated int column to `CountryKey`. This table is then output from the metanode as the dimension table (bottom leg of the workflow).

The dimension table is then joined to the original table matching on the country column. The original Country column is dropped and the CountryKey is added. This replaces the duplicate Country strings with an integer which references the actual country name in the dimension table. This will reduce the size of your output file significantly.

The benefit of this approach is (a) it is good practice if you move to an online database; (b) it will help pickup errors in categorical fields if there are slightly misspelled items, missing items and other corruption. The downside is that the end user will need to use dimension tables to add the string representation to the fact/data file (though with Pandas this should not be too difficult).

 ![Screenshot 2022-09-26 114450](https://forum-cdn.knime.com/uploads/default/original/3X/5/b/5b8c6f8ef1d22a3a7c2056e58937aaedb89f3548.png)

**3/ Consider HDF5 file format**

HDF5 file format provides a greater degree of flexibility compared with many other file formats. The downside is that KNIME does not have a native node supporting it (I’ve never understood why not). It is possible to use a Python node to export data to an HDF5 file though it will require a bit of work due to the evolving nature of KNIME’s Python implementation.

_Pre-requisites:_  
You will need a conda environment with the KNIME base packages, into which you will also need to add `pytables`. Note: There is another Python package `h5py` which provides more control, but for the purposes of keeping it simple I will stick with `pytables`.

_Why Python in KNIME is a pain point_  
Pandas is a wrapper around underlying data types. This gives it great flexibility in abstracting functionality from implementation. When an operation is defined using pandas then its implementation is passed to the underlying data type to implement. When a third party package consumes a Pandas dataframe (such as Pytables) it assumes data is represented using convention data types such as `int` for integers and `object` for strings and Python `datetime`. However, KNIME does not conform to the conventions and so it stores strings as `string` instead of `object` and `datatime` as a java inspired something. It’s a pain!

The reason for the above rantorial is to explain why the following is more complicated than it needs to be.

Before adding the Python node, the date column needs to be converted to an ISO8601 string.

![Screenshot 2022-09-26 121652](https://forum-cdn.knime.com/uploads/default/original/3X/2/e/2e2c360a53ed73800cdfd7ad639a95d3bb8182c4.png)

The Python script is as follows:

- Set the output file path and name of the table in the hdf5 file (hdf5 files can contain multiple tables in a folder like structure).
- Convert the ISO8601 date to Pandas datetime format, and convert strings to objects. Copy remaining columns to output file.
- Write output file setting compression mode and compression level.

```python
import knime_io as knio
import pandas as pd
from os import path

# Configuration
output_path = path.abspath("D:\\Data\\dataExample.hdf")
output_table_name = "Export"

input_table = knio.input_tables[0].to_pandas()
output_table = pd.DataFrame()

# Convert ISO8601 dates to DateTime format.
dates_to_convert = ["Date"]
for column in dates_to_convert:
    output_table[column] = pd.to_datetime(input_table[column], 
                            format="%Y-%m-%d")

# Convert strings to objects
strings_to_convert = ["ID", "Country"]
for column in strings_to_convert:
    output_table[column] = input_table[column].astype(object)

# Copy remaining columns to output table.
all_other_columns = ["Quantity"]
for column in all_other_columns:
    output_table[column] = input_table[column]

# Output to hdf file
# mode = write a new file (w)
# complib = compression library
# complevel = compression level (1=faster, 9=more compression).
output_table.to_hdf(output_path,
                    output_table_name,
                    mode="w",
                    complib="zlib",
                    complevel=5)

# Pass though input.
knio.output_tables[0] = knio.write_table(input_table)

```

In Python to read the file you would use the following:

```python
import pandas as pd
from os import path

input_path = path.abspath("D:\\Data\\dataExample.hdf")
input_table_name = "Export"

df = pd.read_hdf(input_path, key=input_table_name )

```

I hope this helps.

---

<div class="post-metadata">

**Author:** ![mlauber71](https://forum-cdn.knime.com/user_avatar/forum.knime.com/mlauber71/32/73590_2.png) [@mlauber71](https://forum.knime.com/u/mlauber71)\
**Post date:** [September 26, 2022, 1:34pm UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/12 "2022-09-26T13:34:34Z")

</div>

> [@TheLeo](#):
>
> We’ve tested SQLite but unfortunately it will not recognize dates & date/times correctly

You could try and implement something with date and time and SQLite but it may not be the best approach

> **[Play around with SQLite and Date and Time functions – mlauber71](https://hub.knime.com/mlauber71/spaces/Public/kn_example_db_sqlite_timestamp~lr2kArL8kWy1rdfw/current-state)**
>
> Play around with SQLite and Date and Time functions

---

<div class="post-metadata">

**Author:** ![mlauber71](https://forum-cdn.knime.com/user_avatar/forum.knime.com/mlauber71/32/73590_2.png) [@mlauber71](https://forum.knime.com/u/mlauber71)\
**Post date:** [September 26, 2022, 1:39pm UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/13 "2022-09-26T13:39:02Z")

</div>

@TheLeo there are some things to check when converting data with date and time variables between KNIME and Python (and Parquet). It should be possible to come up with a setting that should work for all your data variants. I might take a look at your example.

> [@Error with pandas in the Labs Python Integration](https://forum.knime.com/t/error-with-pandas-in-the-labs-python-integration/44473/9):
>
> @toscanomatheus this might be an interesting case. I will have to have a closer look at your specific file. I startet to look into the question pof the Python Labs nodes and Date and Time variables (again) and indeed there seems to be a problem with the handling of types. In a previous discussion I toyed around with several formats and then exported the data thru Parquet files and brought the data back to KNIME which worked (mostly): This time I tried two things with string formats exported …

> [@Error upon executing Python script (Rows can only be sliced by slice objects but got \<class 'type'\>)](https://forum.knime.com/t/error-upon-executing-python-script-rows-can-only-be-sliced-by-slice-objects-but-got-class-type/46899/3):
>
> @pt2501 I see several points. One is that there seems to be a problem with the date and time conversion. The Python Labs node is struggling with that somewhat ([Error with pandas in the Labs Python Integration - #9 by mlauber71](https://forum.knime.com/t/error-with-pandas-in-the-labs-python-integration/44473/9)). I imported the table first as PyArrow file and then converted specifically the Date columns to a pandas date time variable. It seems to work table = knio.input\_tables[0].to\_pyarrow() df = table.to\_pandas(date\_as\_object=False) df['Creation date'] = pd.to\_datetime(df['…

> **[Check out Date and Datetime variable types between KNIME and Python – mlauber71](https://hub.knime.com/mlauber71/spaces/Public/kn_example_python_date_time~j65pBtLZ09q6ugO0/current-state)**
>
> Check out Date and Datetime variable types between KNIME and Python A Jupyter notebook to toy around with data variable is in the foler /data/kn\_example\_python…

---

<div class="post-metadata">

**Author:** ![kienerj](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/k/ce7236/32.png) [@kienerj](https://forum.knime.com/u/kienerj)\
**Post date:** [September 26, 2022, 1:56pm UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/14 "2022-09-26T13:56:31Z")

</div>

Depending on the interactivity and “repeatability” (how often does the same exact thing happen?) of the analysis in python, instead of some manual import/export scheme I would consider one of the following:

1. Make your analysis a web service you can call from knime

This will still require some type of export/import but it can be fully automated. How exactly depends on the amount of data, eg. can it be sent in a http request or does there need to be some intermittent file stored and just a link sent.  
Upside: runs on the server the web service is installed  
downside: more involved that manual but if done daily, will likley become worth it quickly  
downside: depends on 2.

1. Make your analysis a proper python package

eg with build process, versioning and the likes. Then you can import it into a python script in knime and the only code you need is input, to call your package, output.  
Upside: you should do this anyway, even if used from Notebooks  
Downside: runs locally (which might or might not be an issue)

---

<div class="post-metadata">

**Author:** ![TheLeo](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/t/85f322/32.png) [@TheLeo](https://forum.knime.com/u/TheLeo)\
**Post date:** [September 26, 2022, 2:26pm UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/15 "2022-09-26T14:26:41Z")

</div>

@DiaAzul  
Thanks for your long reply!  
We’ve been working with a similar way than your nr.1 but I just wanted to make sure that I’m not missing some obvious better way. Since the datasets are different each time it somewhat time consuming to specify each date/datetime column manualy when importing. It’s not the end of the world I just thought there might be a format that stores all the information and can be imported in python easily.  
Regarding nr.2 I think if the file format like parquet supports compression that shouldn’t be a problem.

@mlauber71  
thanks for the example workflows/links. I’ll look into it!  
Generally speaking I think parquet is my favorite so far. Besides it converting some of the integers to float (and date/time issue) it has no downsides to a CSV but at the same time being a lot smaller due to compression.

@kienerj  
I’ll look into way nr.2 since the process is different almost every time.

Thanks all for your feedback so far!

---

<div class="post-metadata">

**Author:** ![TheLeo](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/t/85f322/32.png) [@TheLeo](https://forum.knime.com/u/TheLeo)\
**Post date:** [September 27, 2022, 9:14am UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/16 "2022-09-27T09:14:21Z")

</div>

So, quick update: I got Parquet working!  
Now to make it work you need to change the Type Mapping:

 ![image](https://forum-cdn.knime.com/uploads/default/original/3X/1/8/186c5020c53794607ddf9d0e2856f31e08920b56.png)

in Python during import you can use this as an example:

```auto

# Please check datatypes after import
### Import python libraries
import pandas as pd
import matplotlib.pyplot as plt
import pyarrow.parquet as pq
import pyarrow as pa
### Import Parquet file
#### Function that works as a mapper for int
def lookup(t):
  if pa.types.is_integer(t):
    return pd.Int64Dtype()
#### Imports using all available threads
pqimport = pq.read_table('C:/test.parquet', use_threads=True)
df = pqimport.to_pandas(types_mapper=lookup)
### Show first rows
df.head()

```

Using this it works at least for me to read in all the ints as int and dates as date without manually specifying columns.  
Thanks all for you amazing support!

---

<div class="post-metadata">

**Author:** ![DiaAzul](https://forum-cdn.knime.com/user_avatar/forum.knime.com/diaazul/32/48297_2.png) [@DiaAzul](https://forum.knime.com/u/DiaAzul)\
**Post date:** [September 27, 2022, 12:21pm UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/17 "2022-09-27T12:21:25Z")

</div>

@TheLeo

Is there a reason that you had to change the output mapping for the Local Date format? I ask because INT96 for timestamps was deprecated some time ago and replaced with the mappings which are default within the dialogue. Secondly, you are exporting as int96 and importing as int64 - which doesn’t match.

I ran you solution leaving the default mappings in the KNIME node and it appears to work as you intend. Therefore, is it necessary to use an INT64 mapping?

DiaAzul

---

<div class="post-metadata">

**Author:** ![TheLeo](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/t/85f322/32.png) [@TheLeo](https://forum.knime.com/u/TheLeo)\
**Post date:** [September 27, 2022, 12:35pm UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/18 "2022-09-27T12:35:12Z")

</div>

@DiaAzul  
I used int96 becasue it’s the only format that’s working when importing (all other formats are imported as String).  
And I don’t think it’s converted since it shows up after import as date[ns] and only the “real” ints are affected as far as I can see.

---

<div class="post-metadata">

**Author:** ![DiaAzul](https://forum-cdn.knime.com/user_avatar/forum.knime.com/diaazul/32/48297_2.png) [@DiaAzul](https://forum.knime.com/u/DiaAzul)\
**Post date:** [September 27, 2022, 1:31pm UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/19 "2022-09-27T13:31:41Z")

</div>

@TheLeo

The date is stored in parquet as a `date32[day]` object. When this is imported into Pandas it is converted to a `datetime.date` object (note not all object shown by Pandas `dtypes` are strings). Pandas shows the date correctly because it has a handler to convert the internal `date` object to the string shown.

If you want Pandas `datetime64[ns]` columns you can use the following option when converting from Parquet to Pandas dataframe:

```python
 df = pqimport.to_pandas(date_as_object=False)

```

This keeps the original mappings in KNIME and gives you the object type in Pandas you are looking for.

DiaAzul

---

<div class="post-metadata">

**Author:** ![PhilTroy](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/p/f4b2a3/32.png) [@PhilTroy](https://forum.knime.com/u/PhilTroy)\
**Post date:** [September 30, 2022, 12:12am UTC](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495/20 "2022-09-30T00:12:41Z")

</div>

Hi

It would be really cool (I guess I am dating myself) to build two new nodes, the first to export data from knime on one machine to the second which would import it on the other machine. The actual format used could be almost any, or there could be a parameter that specifies it.

Phil Troy

[Next page](https://forum.knime.com/t/export-data-from-knime-for-processing-in-python-externaly/47495.md?page=2)
