Data passing to Python node still slow

KNIME Extensions
3

@Experimenter two things about that. With the version 4.5 KNIME launched a new Python file handling support that should greatly increase the speed.

You could try a sample here:

You will have to active the Apache/Parquet based columnar storage to take full advantage:

https://docs.knime.com/latest/analytics_platform_workbench_guide/index.html#table-backend

Since the new integration is yet in Labs status you could also try and use Parquet to bring data in and out of KNIME and Python. Maybe not as fast but might be worth a shot:

Which still is a possibility with the new integration also. As well as using SQLite to have a local database ready that both Python and KNIME can use:

3 Likes