I’m new here and since a few weeks I work with KNIME. Now I’m at the point that my computer does not have enough capacity for dealing with huge data sets over 2 million rows. Is there any recommendation how much RAM and disc capacity is necessary to work fluently with big data sets? I’m loocking for a computer with 32GB RAM and 1TB SSD disc - could that be enough computer power?
It is difficult to tell if the power would be enough to work with your data since this would depend on how large your data is and waht you want to do with it. 32 GB and 1 TB SSD although sounds solid at a first glance. Remember to give KNIME its fair share of the RAM. You would have to configure that in the knime.ini. For starters I would recommend 2/3 of the RAM if you plan on really large jobs. Depending on what other jobs run on the system you could increase the RAM for KNIME.
Also you might want to check out this collection about KNIME and performance, maybe starting with the official blog.
Thanks for your input. I try now to work with a computer wich has 32GB RAM capacity and I’ll see if I can work fluently whe I’m using files with many records. I keep you updated.