Data load time -- .csv

KNIME Analytics Platform
1

I've got a powerful machine (24 core, 164 G RAM) with heap set to 8192 MB. Trying to read in a 3.8 G .csv. It's taking all morning... 

Is this normal? What should I expect from Knime when working with larger data sets like this?

 

Thanks!

2

Hi!

That's not normal. I just read in a 5.6 GByte CSV file. It took me 30min. (2 cores, 512MB heap).

How many rows/columns does your file have? Which version of KNIME do you use, which OS?

Could your network be a bottleneck? (You can also let the filereader read from a zip archive.)

 - Peter.

3

Hmmmm..... Well, I'll keep playing with the settings then...