Create folder in azure datalake using azure blob storage connector

KNIME Extensions
,
1

I created workflow as below to transfer knime tables from local to Datalake.

image
image632×658 25.3 KB

Getting below error while trying to create a folder in transer files:

image
image1066×586 35.7 KB

I tried creating a folder using create folder node. Still getting same error:

Execute failed: The requested URI does not represent any resource on the server.

How do i create a folder in azure datalake from knime?

2

Hey @rajvenkatesh_k,

Is the “Create missing folders” option enabled in the Transfer Files node?

image

If that setting is enabled and you are still running into issues, I would recommend triple-checking the URI to ensure that it’s correct and verifying that the authenticated user has permissions to create directories in that storage container.

For the purpose of identifying the root cause of the problem, you may want to test if you are able to create files at all in that storage container before attempting to create a directory.

Cheers,

@sjporter

1 Like
3

Hi @sjporter ,

I enabled the create missing folders option.
I could transfer the files using the same URI if the folder is created manually in Azure datalake.
But when i create the folder from knime, i get the below error.

Execute failed: The requested URI does not represent any resource on the server.

4

Hi @rajvenkatesh_k,

are you using Azure Blob Storage or Azure Data Lake Storage Gen2? There is a new Azure Data Lake Storage Gen2 Connector that you have to use in the later case.

Cheers
Sascha

1 Like
5

Hi @rajvenkatesh_k,

another thing you can try is another folder name (without : in its name). Not sure if this makes problems.

Cheers
Sascha

6

Hi @sascha.wolke

I tried creating a simple folder with name as test. It didnt work.

Thanks,
Raj

7

Hi @rajvenkatesh_k,

did you try the Azure Data Lake Storage Gen2 Connector?

Cheers
Sascha

8

Hi @sascha.wolke,

we are using knime analytics platform version 4.3.1 and Azure Data Lake Storage Gen2 Connector is not available in this version.

Thanks,
Raj

9

Hi @rajvenkatesh_k,

if you like to write to a Azure Data Lake Storage Gen2 than you have to use the Azure Data Lake Storage Gen2 connector (added in 4.3.3), the Azure Blob Storage connector works only with Azure Blob Storage.

Cheers
Sascha

10

Hi @sascha.wolke ,

currently I can use only 4.3.1 knime analytics platform. using Azure blob storage connector, i connected to adls2 and transfered few files from local system to datalake. Only thing that is not working for me is folder creation.

Could you please check how i can create a folder in datalake using azure blob storage connector?

Thanks,
Raj

11

Hi @rajvenkatesh_k,

One of the benefits of ADLS Gen2 over Blob Storage is the hierarchical namespace that works like a file system with a directory/file structure. This means directories work different in Azure Blob Storage and Azure Data Lake Storage Gen2. Thats why you can upload files with the Blob Storage connector to a ADLS Gen2 bucket, but can’t create/read the directory/file system structure and why we created the new ADLS Gen2 connector. There is no workaround on this. You have to use a Blob Storage bucket with the Blob Storage connector or upgrade to 4.3.3 and use the ADSL Gen2 connector with a ADSL Gen2 bucket if you like to use directories.

Cheers
Sascha

1 Like
12

Thank you @sascha.wolke. Now i understand the problem. I will request my server team to upgrade to 4.3.3.

13

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.