Loading multiple files

Hello, dear KNIME users.
Please, can you help me with loading multiple files
I have looked at different discussion. But seems like those nodes are updated and tried them, but for me they did not work
My workflow does not see the variable, that I m passing
Can you give me some hints, pls

KNIME_project.knwf (14.1 KB)
I have uploaded workflow

I suggest that you read the File Handling Guide.

4 Likes

Hello @Karlygash,

have you managed to load your files into KNIME? I suggest to use CSV Reader (more suitable for loops and based on new File Handling framework) in a loop or even better without loop if your files have same structure and you are concatenating them :wink:

Br,
Ivan

2 Likes


I ve done in this way)
Ok, I ll try to do it without loop

Thanks

3 Likes

Hello @Karlygash,

glad you made it. (Excel Reader can read multiple files without loop as well :wink: )

Br,
Ivan

4 Likes

hello, @ipazin
Do u mean by concatenating to read each file separately and then union them?

Hi!

No, the Excel Reader node has an option where, if the files have the same structure, you can read every file in a folder.

Quite handy!

2 Likes

ok, thanks! What if does not have the same structure?

Hello @Karlygash,

under Advanced Settings tab there is option Fail if specs differ which is by default enabled and won’t allow you to configure Excel Reader to read multiple files with different structure. If you uncheck it you will be able to read multiple files with different structure where same columns will be concatenated and where columns are missing missing values will be inserted. Also there is new option to Append path column which might be helpful in recognizing files which have different structure.

Br,
Ivan

1 Like

And if with all that it does not work, you might need to check the “Support changing file schemas” under the same tab:

Although I am not quite sure if this option is as reliable as with it unchecked…

1 Like

Hello @jorgemartcaam,

Support changing file schemas is for file(s) change between multiple executions. For example in loop processing to be able to read files with different structure within different iterations.

Br,
Ivan

1 Like

Ah!
I ended up using it sometimes when I was not able to read every file at once, because of small differences between files.
I guess now I know the reason why it was not that reliable. I was using it wrong :sweat_smile:
Thx for the info!

2 Likes

hi @ipazin @jorgemartcaam @elsamuel
Is it possible to get filenames using excel reader node?
I am reading from 10 files
I found component extract filename, but it is giving just null values

Thanks in advance!

Hi!

If you only need to get the filenames, here you have a way to do it:

Be sure to include all the flow variables in the “Variable to Table Row” node and to select the “Location” within the “URL to File Path” node. I also added the column filter to select the filenames.
Otherwise, this is good to go, you almost do not need any configuration within the nodes.

Best regards!

Hello @Karlygash,

with KNIME version 4.4.0 you can use Append path column option to have file identifier when reading one or more files into KNIME. See here for more:

Br,
Ivan

3 Likes

this option is available only for csv reader?

As @ipazin linked,

Best regards!

2 Likes

@ipazin , @jorgemartcaam :smiling_face_with_three_hearts:
thanks, I updated and used Column Expression as u advised

1 Like

Good Day, @ipazin , @jorgemartcaam

How we can when u read data from several files.
I mean i have some column extra and now I having an error, which says column number and name should be same in order to have concatenation.

How I can unselect extra column?