How to read a File placed on an HDFS directory using KNIME

KNIME Extensions
,
1

Is there any Node available in KNIME Platform, which reads a file placed on HDFS directory.Actually while working on one of the use case, I have across a scenario where I need read a file placed on an HDFS directory to get the value stored in it, and use it for my further processing.

2

See example here

2 Likes
3

Thanks for your reply on this.Actually I looked at the workflow which you have shared, but the example shows that it downloads the file from HDFS and then perform operations on it.But the file which i will be reading from HDFS will be having some credential details , which needs to be read at run time to make the connection with one of the server…So I am looking for some example which reads the file from HDFS directly, and fetch the value rather than downloading the file from HDFS and doing read operation.

4

Here some more examples


https://www.knime.com/knime-big-data-connectors

5

Hello ankur,
you can use the Parquet/ORC Reader/Writer nodes to do this.
Bye
Tobias

2 Likes
6

One option you could explore would be to use external tables if you coudl/want to do your manipulations within a Big Data system - and maybe download the results later. One benefit could be that you are already logged in with your credentials via the:

1 Like
7

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.