Is it possible to use the KNIME Azure Blob Connector to connect anonymously to an Azure Blob store?
I’ve come across the same requirement twice in the last 24 hours on different projects. Both projects have switched to using Azure Blob stores to publish large quantities of regularly changing data. For example, the UK Meteorological office is publishing data to support Covid-19 research as described here:
Details fo the account and container are:
storage account: metdatasa
I can work around it in the short term by manually download, but automating the process is so much more elegant.
Hi @DiaAzul -
If there’s a way to do this, it’s not obvious to me. I’ve asked the Dev team for more info… I’ll let you know what I find out.
No that’s not possible at the moment using the Azure Blob Store Connection node. But for your case you could parse the html page and download the files using the HTTPS Connection node.
I created a workflow that does just that, all that needs to be changes is the target folder in the Download node.
@oole, many thanks for your post, and the exampe workflow. I have uploaded my workflow to KNIME hub as it is slightly different, and others may find it usefl. PHE provide an XML file which can be parsed (rather than the html list). I use this directory listing to find the most recent data file, download it, parse it and then pass it on as a KNIME data table.
Well, that is a very elegant solution. Thanks for sharing your solution!
I might be ugly, but it doesn’t mean I can’t write pretty code.