Training BERT on CPU

[ehsanT]

Hi everyone,

I am trying to fine-tune BERT in knime for emotion detection. I created a workflow which is working well on a laptop with a GPU.

However, when I am trying to repeat the experience on a laptop with no GPU, it is consuming the CPU fully (100% usage in windows 10) with a high memory performance (more than 80%)

Also I cannot see any training report in the console.

I tuned knim.ini based on the tips in Memory/Heap space issues in version 4+, but it was not helpful.

The training time has been increased significantly rather than when I was using the GPU.

I will appreciate any ideas to tackle this problem.
Thank you!!

[kienerj]

@ehsanT

I will appreciate any ideas to tackle this problem.

Use a GPU. even a laptop class GPU will be 10-100x times faster than a CPU. Forget the idea to train something like BERT on a laptop CPU. What you are seeing is the reason why people use GPUs for deeplearning. Simply too slow on CPU.