How to read data from Azure datalake in knime?

KNIME Analytics Platform
1

I could see Azure Blob Store connection & the file picker. But I want to load data from Azure data lake. Is it possible?

2

Hi @Nithya -

Sorry for the delayed reply. If I’m understanding you correctly, you want to read data from an Azure data lake and save that data locally? If so, you can take a look at this workflow as an example. As you mention, the two Azure nodes are doing the heavy lifting here in the second branch.

We also have this blog post that might be of interest to you.

3

Hi @ScottF,

Thanks for your reply. The workflow and blog post tells only about the Azure datablob. But no where it is mentioned on how to take data from Azure datalake. This is my question.

Also there is no node designed in KNIME for fetching data from Azure datalake (But for data blob it exists).

My question is there any node exists for fetching data from datalake? If not how can we design it?

4

Have you tried using the WebHDFS Connection node in conjuntion with KNIME’s file handling nodes? See here: https://www.knime.com/nodeguide/big-data/big-data-connectors/hdfs-and-file-handling-example

5

Hi @ScottF

Thanks! Actually HDFS is no more related to reading files from Azure Datalake. Actually we have ‘Azure Blob Store Connection’ & ‘Azure Blob Store File Picker’ nodes in KNIME.
But we dont find any node for ‘Azure Datalake Store Connection’ and ‘Azure Datalake File Picker’. Also we are not able to connect to it via any other means. So I would like to
confirm on this before writing a custom node for the datalake connection.

1 Like
6

Hi @Nithya, any update on this? Did you find / write a connector for ADLS?

7

Please see my comment here.

1 Like