Controling File Reader via FlowVariables

We want to read in CSVs from within a component.
We have a particular - basically one of the two standard variants of CSV - where the CSV values are stored as follows:

  • commas separate values
  • double quotes encompass strings
  • dots are used as decimal separator
  • strings can span multiple lines

Now, I can happily use the CSV-reader in a component by clicking options and controlling the file name via a flow variable.

Except the CSV Reader does not allow for multi-line strings.

The File Reader can be used instead and manages to read the files.

But!

  • File Reader does throw away its option values when switching the file name - so there is no clicking the right options and only using flow variables to switch out file names.
  • On the other hand controlling all File Reader options via flow variables so far has led no-where - to me its not clear how this should work.

Now, is there a solution other than using other tools like R or Python (neither do I want to add dependencies nor do I want to wait for Knime <-> R/Python to exchange data)?
Reading CSVs and doing some automation on top really should not be that hard!

Example of what we want to do:
minimal_csv_example.knwf (55.1 KB)

@petermeissner you could try and use the new CSV reader with Autodetect. That seems to be able to handle these files.

3 Likes

Hi petermeissner,
as mlauber71 already mentioned above you can use the new CSV Reader that was introduced with KNIME 4.3 and set all the settings you mentioned via flow variables.
Bye
Tobias

1 Like

Thanks, @mlauber71 @tobias.koetter

switching versions is not an easy thing to do because I am working in an enterprise setting with centrally packaged software for client computers.

Nonetheless, glad to hear this is not an issue from 4.3 onwards and a good argument to upgrade from 4.0 to 4.3. :+1:

1 Like

You said you did not want to use R. But also the R package readr would give you good results:

Regarding an upgrade. You might want to wait until 4.3.1 since 4.3 at the moment has some problems especially with the Path variables

1 Like

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.