Parquet format reader/writer separate from Spark

Is there a node which supports the writing or reading of Parquet files without connecting to Spark? I have created a basic stand-alone Parquet Reader and Parquet Writer node, but they only handle basic Knime DataCell types (numeric and string) and can run out of memory when working with large Parquet files. Please let me know if there are other stand-alone options I can use to read and write Parquet files. I am similarly interested in other big data file format reader/writers, like Avro. Thanks!

Hi,

There are no nodes for this in the current release, but there is a set in development right now. This will include reader/writer nodes for Parquet, ORC, and Avro.

Cheers,
Roland

1 Like

Hello @RolandBurger,
Any details about the availability of these nodes?
Regards,
Sébastien

Hi there!

I’m no Roland but maybe this link will help:
https://www.knime.com/whats-new-in-knime-36#parquet-orc

Br,
Ivan

2 Likes

Indeed 3.6 has the nodes I was seeking!

1 Like