Knime compressed database like Alteryx .yxdb

[mlauber71]

@ebarr the KNIME tables themselves are compressed. Then you have formats like Parquet or ORC that would also have compression both also support the addition and removal of parts.

And you have local SQL databases which you also can compact to a certain degree:

KNIME, Databases and SQL

H2 would also support larger datasets

DB Writer to Access DB Hangs at 99%

@qdmt one idea could be to test H2/SQL database: http://h2database.com/html/advanced.html#limits_limitations You could create a new database connection with a current H2 driver (from Maven) and add the “split” parameter to the connection string: jdbc:h2:split:[location=file?<file>][location=in-memory?mem:<database>] [image] You will have to select this new “h2_split” database driver in the H2 Connector instead of the built-in one: [image] The data would then be stored in a local H2/S…