4 files, 12k records, divided by 500 rows each files = 24 files

Hi there

as i mentioned in the title, i tried to split 4 files with 12k rows in total (file 1= 3000 rows; file 2= 4000 rows; file 3 = 2000 rows; file 4= 3000 rows) in multiple files with only 500 record each.

I merged all in with a csv reader and then i split with row filter limiting by 500 rows, then i put 24 excel reader. But i would ask u if there s a faster way, maybe with a loop but i can t find it.

Setting 24 row filters it s a bit long, anyone know a simplier and faster way?

THANKS SO MUCH!

Hi @Pippobaudo89 , this sounds like a job for a chunk loop, set to process 500 rows at a time and write them using Excel Writer, with the path defined by a flow variable, and based on the current iteration number.

Hope this example helps
Split file across multiple outputs.knwf (15.7 KB)

[Note: Depending on your version of KNIME, you may not have the Variable Expressions node. If you don’t, it will need to be replaced with something like a String Manipulation (variable) to create a string with the required file name, followed by a String to Path (variable) node to convert it to a path ]

3 Likes

ok thanks

i have two problems:

i need to change the directory where to save my files (but if it s a difficult step it s ok, i cut and copy from temp to my folder on desktop). The second passage it s that i need a csv writer, i tried to copy your setting from xls writer but it doesn t work…

i substitute your string in variable expressions with this

createLocalPath(“C:\Users\luca\Desktop\CARICAMENTI CONDOMINI\caricamento” + variable(“currentIteration”)+“.csv”)

but seems there are some problems…

You can add the filepath as another variable and determine where each file should be stored.
Normally the error message should give you some guidance on what the root of the issue is.
@takbb s solution works with csv files as well
br

1 Like

Hi @Pippobaudo89 , I purposely used forward slashes in the Column Expressions as it is easier :wink:

createLocalPath("c:/temp/myoutputfile_" + variable("currentIteration")+".xlsx")

You can use either, but if you use backslashes you probably need to double them up because a backslash is interpreted as an “escape” character.

so make the expression either this:

createLocalPath("C:\\Users\\luca\\Desktop\\CARICAMENTI CONDOMINI\\caricamento" + variable("currentIteration")+".csv")

or this:

createLocalPath("C:/Users/luca/Desktop/CARICAMENTI CONDOMINI/caricamento" + variable("currentIteration")+".csv")

and hopefully it should then work.

And yes as @Daniel_Weikert rightly says, you can split out and hold the destination folder separately from the rest of the file name. This was just for demonstration of the concept and I’d recommend holding folder locations separately to make it easier to configure later, once you have the concept working.

To use a CSV Writer in place, attach it into the workflow as follows:

The make sure you have execute the Variable Expressions node so that the outputfile path variable has been created. Then configure the CSV Writer node as follows:

3 Likes

MANY MANY THANKS…

i have to study a lot the variables, because are the key to solve a lot of problems

Thank u so much

2 Likes

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.