I have a weekly CSV filing reading and concatenation project I’d like to automate.
On Monday I access the prior seven days of sales data on AWS S3 using the appropriate KNIME nodes to download the folders/data to a target local location.
Say a CSV file type is called ‘Orders’. Each day has a separate file in a folder labeled with that day’s date (yyyymmdd = folder name).
What is the basic structure using loops and/or file type list reading to have KNIME CSV import each ‘Order’ CSV from each of the seven folders for concatenation?
You could combine two lists. One with the (last three) directories and another with the relevant Excel files and then loop over them. You could change the conditions to your needs.
The files and folders would have to follow the same pattern every time.
So I have created a week definition data. Each data is assigned a week number. It is created in external File and read into KNIME.
Based on your input week number Row Filter node only provides 7 rows i.e. 7 folder names for that week. With String Manipulation node you create folder path for each folder and then you loop over it and read it.
Data is attached with workflow and all paths are relative so you should be able to run it with no modifications. Hopefully 2019_01_18_Automating_File_Reading.knwf (79.2 KB)
@ipazin This didn’t quite work for the problem at hand. I like your approach of creating a week number table to use, but @mlauber71 provided a solution that can be automated a bit more.