CSV Reader Issue: "not a regular file"

I recently upgraded to KNIME 4.3.0 and I have begun to have issues with the CSV Reader node. All of my .csv files are failing to execute with the error ... is not a regular file. Please specify a file .

These files are perfectly legit and execute just fine with the File Reader.

I have noticed that it might be an issue with the Local File System and OneDrive : when I run a given csv with path as C:\Users\name\ I get no issues. But if I try to open the same file from my OneDrive path C:\Users\name\OneDrive - Company\, I get the error message above.

Like I said, everything is fine using the File Reader so that is the work around. But this CSV Reader issue is a bit of a PITA.

Thanks!

1 Like

Hi @scapuzzi

sorry that the CSV Reader is giving you this problem.

On the web we could find some cases, where files from OneDrive are represented as special files (“reparse points”) in Java. Such “special files” are currently ignored by the “Files in Folder” option of the CSV Reader node (and other reader nodes), because in most cases it is not possible to read them.

Internally we tried to reproduce your issue with OneDrive, but did not succeed. Could you be more specific about:

  • Your Windows version
  • Your One Drive settings

Thank you,
Björn

2 Likes

Hi Björn:

Thanks for getting back to me.

My Windows specs are
Edition: Windows 10 Enterprise
Version: 1709
OS Build: 16299.2166

My OneDrive specs are
Version: OneDrive 2020
OS Build: 20.114.0607.0002

1 Like

I had the same problem, I not to be able to open a Excel in a Sharepoint folder with the new node.
I solved this with the older Excel Reader node deprecated.

1 Like

I’m afraid I don’t have a solution, but just to say I have similar problems with the new CSV Reader node - many files that open with the old node, no longer open with this one.

Could you please share more details? Where are those files located? What is the error you’re seeing?

Best
Mark

On closer inspection, the CSV Reader node would error if it came across a value that was a different data type to that expected. There is one specific identifier I use whereby 99%+ of values are numeric, but some have characters in too, and the CSV node would fail on these.

By removing the ‘Limit data rows scanned’ flag on the ‘Advanced Settings’ tab of the node these problems were resolved.

Thanks, Dom

3 Likes

limiting the rows did not solve the issue for me. I am still seeing “not a regular file” for my files that are on One-Drive. The old csv reader works (see screenshot). I have tried several different combinations of settings in the new node reader without success.

image

@dpowyslybbe problem is totally unrelated to yours.

The problem is of technical nature as pointed out by @bjoern.lohrmann . We’re trying to fix this issue. Please excuse the inconveniences

Best
Mark

For the record, the same issue occurs with the Excel Reader node.

Thank for looking into this

Have there been any updates on this? Various readers do not work with files on One-Drive. For instance, the Model Reader, Parquet Reader, Excel Reader, CSV reader, all report ... is not a regular file. Please specify a file. The files will read initially but after a period of time, they get flagged with this error. When trying to reproduce, I recommend waiting a period of time and then seeing if the file can be read via these nodes from One Drive. This feels like a major bug.

Thx for reporting we are currently working on this issue and are trying to ship the fix with 4.3.3

Mark

3 Likes

@scapuzzi ,
we have fixed the problem when reading “special” files in the nightly build and will release it with the 4.3.3 bug fix release. We have also added a new option to the filter options when working with files in a folder to also include special files:
image

Could you do us a favor and download the nightly build from here and see if the problem is fixed for you.

Thanks
Tobias

1 Like

Awesome! Could you tell me under which extension I can find the fix? I don’t want to have install the whole nightly build for fear of corrupting my environment.

Hi Scapuzzi,
The changes are part of the basic KNIME extension which would update the major part of your installation which wouldn’t recommend.
I would suggest to either use the self extracting archive or the zip archive but not the installer. You can then extract the nightly build into a separate folder. In addition you need to use a new workspace. Simply create a folder within the nightly build folder and select it during startup of the nightly build. This way you will not change anything on your working environment. All changes will be within the nightly build folder including the newly created workspace.
Thanks for your help
Tobias

3 Likes