Parquet reader does not support all (common) datatypes by default

KNIME Analytics Platform
1

Some common datatype (of real world backend systems) are not supported for type mapping, neither automatic, nor manual. So parquet reader is useless for such purposes, needs to be rebuild python code :frowning:

e.g.: TIMESTAMP_MICROS (INT64) or DECIMAL (INT64).

Would be nice to have, as parquet is becoming more and more a file format of choice.

Knime output:
WARN Parquet Reader 0:3
org.knime.bigdata.fileformats.utility.BigDataFileFormatException: Input type mappings are missing for columns:
NET_VALUE (DECIMAL (INT64)),
CREATED_ON_SOLDTO (TIMESTAMP_MICROS (INT64)),
CREATED_ON_SHIPTO (TIMESTAMP_MICROS (INT64)),
CREATED_ON (TIMESTAMP_MICROS (INT64))

1 Like
2

Hello Thomas,
I have created a ticket to support these types with the Parquet Reader. Thanks for reporting this.
Bye
Tobias

2 Likes
3

Hi Thomas,
we have just released KNIME 4.0.1 which also contains support for more Parquet types (BD-955). For further details see the changelog. To update simply open KNIME and go to File->Update KNIME…
Bye
Tobias

4

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.