I had connected to Databricks using DBFS node and got the list of files available under dbfs location.
I need to read the csv files from dbfs to knime. When i tried to use CSV Reader node I am getting below error.
Any help will be really helpful for reading the csv file from DBFS location into knime
currently you can not read a CSV file directly from DBFS. You first need to download it locally and then read the downloaded file into KNIME. Please have a look at this workflow for a related example.
We are currently implementing a complete new file handling framework for KNIME which will allow you to read/write directly from/to various file systems including DBFS. However we do not have the corresponding Databricks connector as of today but it is planned for the winter release. So please stay tuned.
Thanks for the response @tobias.koetter.
But i need to make the file downloadable to the user using FileDownloadWidget.
In that case is there any way to make the file downloadable to user from DBFS location?
And while trying to pass the path of the downloaded csv file, I am getting error. Is there any efficient way to make the file downloadable to the user from knime workspace location.
Reading remote file handling feature is available in new version of knime?
yes the new file handling framework is available in KNIME Analytics Platform 4.3. However if you want to provide the CSV file to the user via the File Download Widget you still need to download it first using the Transfer Files node and the Databricks File System Connector.
I will create a feature request to support direct loading of a file from a remote file system as well.