Create Local Big Data Environment as actual database

@nxfxcom technically you could use the local big data environment and store data as CSV, ORC or parquet files underneath so you could also access them individually even if the environment is no longer there. You can get an impression here:

Blut I would advise against it since big data technology would make the most sense when used over several nodes and I think the KNIME implementation of Hive might be more for educational purposes than a fully maintained productive environment. For storing large amounts of data in a local environment systems like MySQL (or Postgres or MariaDB) might be better suited.

If you are looking for a single-file solution you might take a look at H2 (how to get the latest H2 driver) or SQLite but they might have some limitations when it comes to very large data sets.

3 Likes