I’m curious as to what is the most important PC spec to speed up a KNIME workflow. The workflow I’m working with is rather large (3 queries of 20 million rows each, many joins, and group by nodes) and my laptop at work processes it very slowly (2 core CPU, 32 GB of RAM). I thought my home desktop might process the workflow significantly faster (8 core CPU, 16 GB of RAM), yet I was shocked when my home PC, despite having significant more CPU throughput, was slower than my work laptop.
Will having additional RAM always be more important factor for workflow processing time compared to a faster CPU? Is this true in all instances, or just for certain nodes?
Disk speed can also be quite important with KNIME as data is frequently buffered to disk. Are both machines using high performance SSDs? That said, with the recent performance upgrades in KNIME 4, it may be the case that ram is more of a bottleneck now.
Thanks Aaron, I’ll look through that article. Yes, my personal desktop machine has an M2 SSD, and my work laptop also has an SSD, though I don’t know its interface.
You need to take in account KNIME configuration on both PCs. Also, you need to provide very specific information on processor model and memory type to compare. Antivirus software may significantly slow down PC performance. So, it is not just CPU/Memory question.
For the nodes you mentioned, multiple cores are useless as Joins or group by are serial by nature and to gain anything from more cores. Certain nodes like IO and pre-processing (row filtering etc) can benefit from streamed execution (again joiner or group by can’t and if you think about it it makes a lot of sense that RAM helps a lot here).