Azure and local Spark

Hi @madape,
the Local Big Data Environment is not set up to connect to Azure Blob Storage, so I am not sure you can get it working, but let us try.
Adding the jars, would also be my first suggestion to you.
You would also have to add some spark configs:

" spark.hadoop.fs.azure", "org.apache.hadoop.fs.azure.NativeAzureFileSystem"
" spark.hadoop.fs.AbstractFileSystem.wasb.impl", "org.apache.hadoop.fs.azure.Wasb"
and
" spark.hadoop.fs.azure.account.key.YOUR_ACCOUNT.blob.core.windows.net", "YOUR_KEY"

I hope that will do the trick.
best Mareike

1 Like