Reading Multiple Files with varying Columns and Rows

Hi,

I am working on a work flow to read multiple files with varying row and columns. I have built the work flow as below but when it runs it just reads the initial file selected in the reader node over and over for as many files as listed in the list file node.

The file has multiple sheets but to configure the reader node I selected the largest file and selected the one sheet I want to read from from the files. Because of the varying rows and columns I selected the columns to read (E to BF) and the rows (4 to 5000).

Any assistance would be greatly appreciated.

Hi @KevinD1,
Can you give a screenshot of your Excel Reader Node. Configuration?
It sounds like you did not configure the file path though the flow variable :thinking:

You told the Excel Reader node to read 1 specific range, in 1 specific worksheet, in 1 specific file, and that’s what it did.

1 Like

Maybe you could take a look at this collection of examples how to import files from Excel in various scenarios.

Numbers 050 and 150 might help you

2 Likes

Thank you for the responses. I am now just able to work on this project again and wanted to send the follow up to see if you could assist.

Below is the image of the configuration window on the excel reader. I did not select anything on the Flow Variables tab. I also allowed variable column types and changing table specifications in the loop end.

Why are you not using the flow variables you created in the previous node?

1 Like

Hello @KevinD1,

have you manage to figure this one out?

Br,
Ivan

Yes, thank you Ivan.

On another note maybe you can tell what is causing the below warning now that I am ready to write to a CSV file?

WARN CSV Writer 0:141 Input table must only contain String, Int, or Doubles

1 Like

Ivan,

Thank you for your help today. I figured out my last issue about the CSV Warning.

Greatly appreciate your help.

I would assume it means that you have data formats other than -well- string, integer or doubles. Maybe you could check that out.

If you absolutely must save to a CSV you could try and convert the data to string.

Another option could be to use R and see if this can handle your formats. But you might have to transfer the data to R first thru a R node (which uses wither data.frame or data.table).

1 Like

Thank you for the response. I was able to figure this one out. The qty in one column was showing as non-native to Knime. Simple solution to use String Manipulation node and convert column toInt(). CSV or Excel Writer recognized the column once executed.

Appreciate the suggestion.

2 Likes