Playing with DCM data: (https://developers.google.com/doubleclick-advertisers/dtv2/reference/file-format)
This means that I have one CSV file per date and hour and this also means that if I desire to run some analysis of a display campaign in a specific month for instance, I should first make an UNION of those several files.
In my test, I'm using just files of the 25/01/2017 per hour and this means that I have 24 files and A LOT OF ROWS.
What is the easy way to make a UNION of all these files?
I found CONCATENATE node but it works just for 2 files per time and I would have to CONCATENATE 24 files just for one day and would be hard to do the same for several months...
you need to loop over the files and compine them using a loop end.
Here is a nice example which demonstrates how this can be done. https://www.knime.org/nodeguide/control-structures/loops/example-for-reading-a-list-of-files
Wow! I was impressed! I didn't use the METANODE with the CREATE TEMP DIR and the UNZIP FILES. Why should I create a Temp file?
In my case I'm just using LIST FILES and the LOOPS
I created a workflow without using a flow variable or determining the row prefix as in your example and worked.
I justed used this sequence of nodes LIST FILES>LOOP START>FILE READER>LOOP END, but what I noticed is that using your workflow I have more rows than using my workflow and I still dont know why.
