Knime is slowing down

KNIME Analytics Platform
2

That is certainly odd. To the best of my knowledge, there haven’t been any recent major changes to KNIME that are known to have a degrading effect on performance. From my experience, the most dominating performance factor in KNIME is the materialization of intermediate data tables on disk.

Have you noticed the performance of your hard disk degrading independently of KNIME?

While we have some performance improvements for KNIME in the pipeline, here are some small adjustments you can try out:

  • You can disable GZIP-compression of KNIME data tables by putting -Dknime.compress.io=false in your knime.ini. Or, better yet, you can install the KNIME Column Storage (based on Apache Parquet) data store and activate it in File -> Preferences -> KNIME -> Data Storage.
  • You can increase the amount of table cells KNIME keeps in memory before writing to tables to disk by putting -Dorg.knime.container.cellsinmemory=10000000 in your knime.ini (the default value is 100000).
  • You can configure the Windows Defender to ignore the KNIME workspace and/or KNIME temp directory. Obviously, do this only (a) on your own risk, (b) if you’ve set up a dedicated KNIME temp folder, and (c) you trust all data you’re working with in KNIME.
  • Maybe try a different workspace, ideally on another hard disk.

It might also be worth having a look at a blog post from 2017 that discusses optimizing KNIME workflows for performance: https://www.knime.com/blog/optimizing-knime-workflows-for-performance

Best,

Marc

6 Likes