Write in HDFS

Hi. Is there a node that i can use to write a local data into HDFS?

Hi,

you can use one of the the HDFS Connector nodes together with the Upload node to upload local files into HDFS. Have a look at the HDFS and File Handling Example.

Everytime we send a file to hdfs, we have to download it? Can't we send directly this file to spark?

Hi,

of course you can directly read files from HDFS into Spark without downloading them into KNIME first. We have reader nodes that allow you to read ORC, Parquet, JSON, text and CSV files (already residing in HDFS) into a Spark RDD. For ORC files, just use the "ORC to Spark" node, for Parquet files the "Parquet to Spark" node, and so on.

Best,

Björn

Ok! Thanks a lot Bjorn and Tobias!