From Words to Wisdom material - chapter 4 01 Frequency workflow

KNIME Extensions
1

Running KNIME 5.1.2 on Windows Pro 64 bit
Intel(R) Core™ i7-3770 CPU @ 3.40GHz 3.40 GHz 32 GB RAM

The Frequency workflow’s Term co-occurrence node is consuming all CPU cores at 100%. Time to process the Node was roughly 20 minutes.

Any thoughts as to why this is happening, is it hardware?

Thank you.

2

Hi @Onjai and welcome to the forum. Two things I would try here:

  1. increase the amount of memory available to KNIME in your knime.ini file
  2. change the Maximum number of parallel processes setting in the node configuration dialog

Can you give those a shot and report back?

1 Like
3

Hi @ScottF.
Thank you for the suggestions.

Increased memory to 24576m & max. # of parallel to 4 = 17minutes to process the node.
Increased memory to 30720m & max. # of parallel to 2 = 11.5minutes to process the node.
The node processed quickly to 90% in 4 minutes, then required 7 minute to complete.

Running an Intel i7-3770 CPU at 3.4GHz & 32GB RAM, not sure if a faster/newer processor will speed this up?
Onjai

4

Faster processor couldn’t hurt. Term co-occurrence is an expensive calculation, so it’s not surprising that it takes a few minutes, especially if you are dealing with larger documents.

5

Ran the co-occurrence node on an AMD Ryzen™ Threadripper™ PRO 3955WX Processor, 64GB RAM and node completed operations in 2 minutes 40 seconds. A vast improvement compared to 17 minutes. So yes, the faster processor worked. Thank you for all of your support.
Onjai

1 Like
6

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.