AVRO file reader?


Is there are an AVRO file reader node, or plans to add one?
I see the big data extension includes Parquet and ORC reader/writers, but not AVRO.



Edit: AVRO is even mentioned in the “Blend data from any source” section of the Analytics Platform page. Am I missing something?

1 Like

I suspect that links means the " Avro to Spark" Node which might not really be what you are looking for.

Yeah, I’m after a local file reader/writer node. As I said, there are Parquet and ORC ones.

Hi Richard,

I made you an example how you can read and write AVRO files with KNIME directly to your local disk.


Cheers, Iris


Hi Iris,

Thanks for the suggestion. Unfortunately the Avro to Spark node is throwing:

at org.knime.bigdata.spark2_4.api.TypeConverters.getConverter(TypeConverters.java:121)
at org.knime.bigdata.spark2_4.api.TypeConverters.convertSpec(TypeConverters.java:162)
at org.knime.bigdata.spark2_4.jobs.genericdatasource.GenericDataSource2SparkJob.runJob(GenericDataSource2SparkJob.java:82)
at org.knime.bigdata.spark2_4.jobs.genericdatasource.GenericDataSource2SparkJob.runJob(GenericDataSource2SparkJob.java:1)
at org.knime.bigdata.spark.local.wrapper.LocalSparkWrapperImpl.runJob(LocalSparkWrapperImpl.java:121)
at org.knime.bigdata.spark.local.context.LocalSparkJobController.lambda$1(LocalSparkJobController.java:92)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

The AVRO contains quite complex objects, and the schema does permit null values in attributes.
I don’t think I can post the AVRO here, since it contains data that we’ve licensed. I can send it directly somewhere though.



This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.