parallel loop ?

KNIME Analytics Platform
#1

Friendly greetings !

I'm a new user (since yesterday) and wrote this loop that read ~50Millions lines in 30 CSV files, do some simple processing and insert the result in a mysql database. (for easier processing later, and for learning purpose).

796×146

 

 

 

 

 

This loop could be parallelized without problem, but it isn't. (the bottleneck is the CVS Reader, CPU-Bound). Is there a way to explicitely parallelize this loop ?

 

Thank you very much, have fun :)

#2

The image is gone :(

here is the url : http://fractals.s3.amazonaws.com/vrac/csvloop.jpg

#3

ho nooooooo. i have only the last table in the database angry

#4

Did you check the "Append table" option in the Database Writer?

#5

i'm not sure what happened. but now it works.

(i rewrote the workflow many times)