How to read a File placed on an HDFS directory using KNIME

Is there any Node available in KNIME Platform, which reads a file placed on HDFS directory.Actually while working on one of the use case, I have across a scenario where I need read a file placed on an HDFS directory to get the value stored in it, and use it for my further processing.

See example here

2 Likes

Thanks for your reply on this.Actually I looked at the workflow which you have shared, but the example shows that it downloads the file from HDFS and then perform operations on it.But the file which i will be reading from HDFS will be having some credential details , which needs to be read at run time to make the connection with one of the server…So I am looking for some example which reads the file from HDFS directly, and fetch the value rather than downloading the file from HDFS and doing read operation.

Here some more examples


https://www.knime.com/knime-big-data-connectors

Hello ankur,
you can use the Parquet/ORC Reader/Writer nodes to do this.
Bye
Tobias

2 Likes

One option you could explore would be to use external tables if you coudl/want to do your manipulations within a Big Data system - and maybe download the results later. One benefit could be that you are already logged in with your credentials via the:

1 Like