Is there a node which supports the writing or reading of Parquet files without connecting to Spark? I have created a basic stand-alone Parquet Reader and Parquet Writer node, but they only handle basic Knime DataCell types (numeric and string) and can run out of memory when working with large Parquet files. Please let me know if there are other stand-alone options I can use to read and write Parquet files. I am similarly interested in other big data file format reader/writers, like Avro. Thanks!
Hi,
There are no nodes for this in the current release, but there is a set in development right now. This will include reader/writer nodes for Parquet, ORC, and Avro.
Cheers,
Roland
1 Like
Hi there!
I’m no Roland but maybe this link will help:
https://www.knime.com/whats-new-in-knime-36#parquet-orc
Br,
Ivan
2 Likes
Indeed 3.6 has the nodes I was seeking!
1 Like
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.