# DATABRICKS(dbfs) file download/ read into knime

**URL:** <https://forum.knime.com/t/databricks-dbfs-file-download-read-into-knime/26347>\
**Category:** KNIME Analytics Platform\
**Tags:** database\
**Created:** [August 26, 2020, 12:06pm UTC](https://forum.knime.com/t/databricks-dbfs-file-download-read-into-knime/26347 "2020-08-26T12:06:11Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![mathi](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/m/3bc359/32.png) [@mathi](https://forum.knime.com/u/mathi)\
**Post date:** [August 26, 2020, 12:06pm UTC](https://forum.knime.com/t/databricks-dbfs-file-download-read-into-knime/26347/1 "2020-08-26T12:06:11Z")

</div>

Hi Team,

I had connected to Databricks using DBFS node and got the list of files available under dbfs location.

I need to read the csv files from dbfs to knime. When i tried to use CSV Reader node I am getting below error.

 ![image](https://forum-cdn.knime.com/uploads/default/original/3X/2/f/2fd12fb422b67be567c32be7334e89ce45dfcd3e.png)

WF

 ![image](https://forum-cdn.knime.com/uploads/default/original/3X/e/9/e9df001318409c611086ad09ea2baa9d66579f95.png)

Any help will be really helpful for reading the csv file from DBFS location into knime

---

<div class="post-metadata">

**Author:** ![tobias.koetter](https://forum-cdn.knime.com/user_avatar/forum.knime.com/tobias.koetter/32/56844_2.png) [@tobias.koetter](https://forum.knime.com/u/tobias.koetter)\
**Post date:** [August 26, 2020, 12:48pm UTC](https://forum.knime.com/t/databricks-dbfs-file-download-read-into-knime/26347/2 "2020-08-26T12:48:09Z")

</div>

Hello mathi,

currently you can not read a CSV file directly from DBFS. You first need to download it locally and then read the downloaded file into KNIME. Please have a look at this [workflow](https://kni.me/w/Y-EtOLZjJLSQFm0u) for a related example.

We are currently implementing a complete new [file handling framework](https://www.knime.com/whats-new-in-knime-42) for KNIME which will allow you to read/write directly from/to various file systems including DBFS. However we do not have the corresponding Databricks connector as of today but it is planned for the winter release. So please stay tuned.

Bye  
Tobias

---

<div class="post-metadata">

**Author:** ![mathi](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/m/3bc359/32.png) [@mathi](https://forum.knime.com/u/mathi)\
**Post date:** [August 27, 2020, 11:58am UTC](https://forum.knime.com/t/databricks-dbfs-file-download-read-into-knime/26347/3 "2020-08-27T11:58:25Z")

</div>

Thanks for the response @tobias.koetter.

1. But i need to make the file downloadable to the user using FileDownloadWidget.  
In that case is there any way to make the file downloadable to user from DBFS location?

2. And while trying to pass the path of the downloaded csv file, I am getting [error](https://forum.knime.com/t/file-download-widget-unable-to-download-map-a-csv-in-wf-location/26369). Is there any efficient way to make the file downloadable to the user from knime workspace location.

---

<div class="post-metadata">

**Author:** ![mathi](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/m/3bc359/32.png) [@mathi](https://forum.knime.com/u/mathi)\
**Post date:** [January 3, 2021, 9:31pm UTC](https://forum.knime.com/t/databricks-dbfs-file-download-read-into-knime/26347/4 "2021-01-03T21:31:43Z")

</div>

Hi @tobias.koetter,

Reading remote file handling feature is available in new version of knime?

---

<div class="post-metadata">

**Author:** ![tobias.koetter](https://forum-cdn.knime.com/user_avatar/forum.knime.com/tobias.koetter/32/56844_2.png) [@tobias.koetter](https://forum.knime.com/u/tobias.koetter)\
**Post date:** [January 4, 2021, 4:56pm UTC](https://forum.knime.com/t/databricks-dbfs-file-download-read-into-knime/26347/5 "2021-01-04T16:56:16Z")

</div>

Hi mathi,  
yes the new file handling framework is available in KNIME Analytics Platform 4.3. However if you want to provide the CSV file to the user via the File Download Widget you still need to download it first using the [Transfer Files node](https://kni.me/n/mzh7RbOHud8N4yI5) and the [Databricks File System Connector](https://kni.me/n/fQW2Nf8Px3KCP3_j).  
I will create a feature request to support direct loading of a file from a remote file system as well.  
Bye  
Tobias

---

<div class="post-metadata">

**Author:** ![rogerius1st](https://forum-cdn.knime.com/user_avatar/forum.knime.com/rogerius1st/32/38975_2.png) [@rogerius1st](https://forum.knime.com/u/rogerius1st)\
**Post date:** [July 12, 2023, 2:19pm UTC](https://forum.knime.com/t/databricks-dbfs-file-download-read-into-knime/26347/6 "2023-07-12T14:19:10Z")

</div>

Dear @tobias.koetter ,  
My situation is a little different from the one by @mathi : I have already downloaded all the desired DBF files to a specific folder in my local system, and I wish to read them as a loop.  
I tried the following sequence of Knime nodes:  
[Possible\_Loop\_for\_reading\_DBF\_Files.knwf](https://forum.knime.com/uploads/short-url/7s9QMapVJNQ4gYVoBhxZWmEruny.knwf) (19.3 KB)  
 ![image](https://forum-cdn.knime.com/uploads/default/original/3X/5/1/511620efb2f6db3bff8a429868adf3ca81fe1206.png)  
It all went very well on the two first nodes, but I could go no further (most likely due to a mistake while configuring the “File Reader” node). I tried a bunch of individual settings on it, including by checking the option “Support short data rows”, but it didn’t read the files. BTW, I set the “Path” from the output of the previous node (the “Table Row to Variable Loop Start” node) as a flow variable.  
Would you mind helping me with this reading of several (80) DBF files, using Knime nodes, instead of coding in a programming language, such as R or Python, for I’m not an IT professional or student (i.e., programmer)?  
Thanks in advance.  
B.R.,  
Rogério.

---

<div class="post-metadata">

**Author:** ![Daniel\_Weikert](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/d/e47c2d/32.png) [@Daniel\_Weikert](https://forum.knime.com/u/Daniel_Weikert)\
**Post date:** [July 12, 2023, 4:42pm UTC](https://forum.knime.com/t/databricks-dbfs-file-download-read-into-knime/26347/7 "2023-07-12T16:42:38Z")

</div>

If you hover over the error node you should get and explanation what the issue is  
br

---

<div class="post-metadata">

**Author:** ![rogerius1st](https://forum-cdn.knime.com/user_avatar/forum.knime.com/rogerius1st/32/38975_2.png) [@rogerius1st](https://forum.knime.com/u/rogerius1st)\
**Post date:** [July 12, 2023, 11:33pm UTC](https://forum.knime.com/t/databricks-dbfs-file-download-read-into-knime/26347/8 "2023-07-12T23:33:25Z")

</div>

Dear @Daniel_Weikert,  
I’ve already passed the mouse over the node with the error message (the “File Reader” node), and got this:  
 ![image](https://forum-cdn.knime.com/uploads/default/original/3X/1/4/14cc235aa8c6c4cb0dd296f95eb8dad5eb778662.png)  
When I tried several different settings (on the encodings, on the different options to be checked/unchecked, etc.), no change brought me the expected reading of the files (one by iteration along the loop).  
Besides, I tried another workflow (suggested on the Knime.Hub), from:

> **[Reading multiple files using loops – knime](https://hub.knime.com/knime/spaces/Examples/latest/06_Control_Structures/04_Loops/02_Example_for_Reading_a_List_of_Files~UPm1nNWmew1uu2L0)**
>
> This workflow demonstrates how to read multiple files using a loop.

and inserted the “Java Edit Variable” Knime node between the nodes “Table Row to Variable Loop Start” and “File Reader”. But I got no enhancements on the performance of this node for reading the files.  
Could you help me with this configuration? What should be the code for each setting on this latter node?  
Otherwise, do you know another node that could read several (DBF) files in a loop like the ones above? Especially one node with no coding (or “low coding”)?  
Thank you for any help.  
ATB,  
Rogério.  
P.S.: The DBF files I downloaded are public, but are large enough to be converted into CSV format. Would it be useful if I posted here the URL for accessing the original files?

---

<div class="post-metadata">

**Author:** ![mlauber71](https://forum-cdn.knime.com/user_avatar/forum.knime.com/mlauber71/32/73590_2.png) [@mlauber71](https://forum.knime.com/u/mlauber71)\
**Post date:** [July 13, 2023, 4:46am UTC](https://forum.knime.com/t/databricks-dbfs-file-download-read-into-knime/26347/9 "2023-07-13T04:46:17Z")

</div>

@rogerius1st I fear that reviving old threads will not give you a different answer (see below). If we are still talking about the same files (DBC/DBF) KNIME simply does not have a node to just import them.

If we are talking about [DBF](https://en.m.wikipedia.org/wiki/.dbf) files [LibreOffice](https://en.m.wikipedia.org/wiki/LibreOffice) still does support the format. That would be importing individual files.

> [@Widget to upload multiple files](https://forum.knime.com/t/widget-to-upload-multiple-files/69854/6):
>
> @rogerius1st I built an example how to import such files - DBC (compressed DBF) from DATASUS (Department of Informatics of Brazilian Health System): Due to the size of the data in the sample on the Hub there are only 3 result files with the first 1.000 Rows. But it did work for your larger DBC files you provided: Please also consult the [external resources](https://hub.knime.com/mlauber71/spaces/Forum/latest/2023/kn_forum_69854_dbc_dbf_import_datasus_br~VkImhhmw7SwUKo5b#externalresources). [kn\_forum\_69854\_dbc\_dbf\_import\_datasus\_br.knwf](https://forum.knime.com/uploads/short-url/jnNWit5NUkWqco5glHFSGL9lMYQ.knwf) (179.4 KB)

Please note. There seem to be several different formats with similar extensions but they are not the same. DBFS in this case is Databricks File System.

---

<div class="post-metadata">

**Author:** ![Daniel\_Weikert](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/d/e47c2d/32.png) [@Daniel\_Weikert](https://forum.knime.com/u/Daniel_Weikert)\
**Post date:** [July 13, 2023, 3:56pm UTC](https://forum.knime.com/t/databricks-dbfs-file-download-read-into-knime/26347/10 "2023-07-13T15:56:29Z")

</div>

doesn’t the file reader support the action “support short data lines” or sth. equivalent?
