go through chunks of 500 and export to txt file

[kowisoft]

Hi, dear KNIMErs

I have a small problem which I though I could solve easily but somehow feel like I don’t see the obvious solution.

I have a file with 15,000+ line items. I want KNIME to go throught that file and export 500 line items at a time into a text file (txt), numbering it accordingly (like 00500.txt, 01000.txt, 01500.txt etc.)

I tried a combination of chunk loop start and excel / csv writer but have faced two problems.

1. How to extract the current chunk for file naming purposes
2. all the writer nodes have only an input but no output so how could they be looped?

Thank you in advance for your kind help.

[jenniferh]

Hi,

You can use the workflow variables to get the iteration. With these you can also use a variable loop end, hence you do not need a physical endpoint.
I have attached a workflow with a mockup dataset.

Feel free to ask if you need help. The variable edit node can be also replace by other edit nodes in case you do not want python or are familiar with other nodes already.

Best,
Jennifer

chunk.knwf (21.4 KB)

Edit: Johannes was a bit faster but I guess both options are valid

[Iris]

And now we need to decide who is the solver, I am happy I am not the one who decides:D

[EvanB]

@johannes.schweig

Johannes

^, as this solution does not require configuring Python.

[kowisoft]

strange… Johannes’ response seems to have disappeared…???

[kowisoft]

Thank you for proposing your solution @jenniferh

I exchanged the Python part for a little filename iteration “sub workflow”.

Took me a few minutes but now it runs perfectly fine.

[jenniferh]

Haha…totally agree next time ill try to stick to “basic nodes” but as I said Java variable edit or other “variable editors” are fine too.

@kowisoft great it works for you now!

[johannes.schweig]

Don’t know what happened. But you figured it out, great!