Just getting started with Knime (mainly use Tableau Prep but I need something with more functionality). I’ve downloaded several sample workflows that address this issue, but I’m unable to replicate it.
My use case is I have an input file that has X number of rows. For every 999 rows in the input file I need to create a separate output file with one synthetic row that summarizes the 999 other rows.
I was able to do most of this using brute force but now I’m trying the looping process. I’m able to get the loops to work but I can’t seem to get a dynamic file name to work.
I’m use the create file name node, and then storing the current iteration in the file name flow variable but when I connect that to CSV writer I do not see the output flow variable name as an option.
This seems to work for all of the sample flows I see so I must be doing something incorrectlyJournal Creation 2.knwf (12.7 KB)
I have checked your workflows and thing missing is connection between Chunk Loop Start and CSV Writer node. Without data connection (which “transfers” data from one node to another) this won’t work
@ipazin Thank you I appreciate it. I’ve now learned the difference between data nodes and variable nodes. You definitely fixed my first issue, but now I’m having difficulties meeting all of my requirements.
Requirements: Split a file into many separate files, each having 999 rows of the original file.
Read in a file of variable length in 999 row increments (File Reader + Chunk Loop)
Build a summary row and add it to the 999 making a 1000 row file below are columns and the values i want
a. Line description - Should be a variable (similar to the one i have for the file name)
b. 5 columns with hardcoded values
c. Each of the 999 rows have an amount field, i want this to be the sum of the 999 amounts x -1
d. More hardcoded values
Name the 1000 row file and write it somewhere.
The part that I’m struggling with is how to do I create a synthetic row, I was able to do it with table creator but that has no inputs so I can’t connect it to my loop. I’m able to sum the amounts of the 999 records using group by but how do I place that in a column of my synthetic row?
Attached is my workflow, I was trying to add the rows with add empty rows and then fill them out but I can’t figure it out.
Any help is appreciated, liking Knime thus far just seems a bit overwhelming at first.Journal Creation 3.knwf (17.9 KB)
Not sure I understood every requirement but in general you want to add summary row. My approach would probably be to use Table Creator node to input hard-coded values (in case they really never change; in case they change maybe think about Configuration nodes). Then using Constant Value Column (if value in table) or Variable to Table Column (if value in flow variable) node add calculated values. Don’t see need for Add Empty Rows node.
Try it out and un-check reset option while exporting in order to check workflow
Thank you for your help thus far. Yes I was using the Table Creator to create the row. However, the issue I’m having is the Table Creator has no input step, so I cannot figure out how to put it inside the chunk loop.
IE if I have an input file with 9990 rows, I want to chunk it with 999 rows per chunk (10 files) and for each of those 10 files input a new created row with the Table creator.
I’ve tried putting the table creator outside the loop and joining it back with concatenate but I get ‘Input table must contain only String, Int, or Double’. I think this is because it’s not in the loop. I’ve googled that error and that seems to be the issue.
So how would I put the table creator within the loop? I do not see examples of that anywhere. Any help is appreciated.
