Is there a node which supports the writing or reading of Parquet files without connecting to Spark? I have created a basic stand-alone Parquet Reader and Parquet Writer node, but they only handle basic Knime DataCell types (numeric and string) and can run out of memory when working with large Parquet files. Please let me know if there are other stand-alone options I can use to read and write Parquet files. I am similarly interested in other big data file format reader/writers, like Avro. Thanks!
There are no nodes for this in the current release, but there is a set in development right now. This will include reader/writer nodes for Parquet, ORC, and Avro.
Any details about the availability of these nodes?
I’m no Roland but maybe this link will help:
Indeed 3.6 has the nodes I was seeking!