I have various data imports I’m bringing in and not using a database, but writing deduped files as outputs to create a “master” table each day (import ‘master’ - join to last several days in the event data points changed, recent data overwriting the past, then pushing to the master table to bring in the next time). I used Alteryx for a number of years, and they had a “.yxdb” output that was essentially a compressed data structure. I would use that .yxdb as the master, and I could save it locally to my machine, or a server, etc. Does Knime have that same type of compressed file?
@ebarr the KNIME tables themselves are compressed. Then you have formats like Parquet or ORC that would also have compression both also support the addition and removal of parts.
And you have local SQL databases which you also can compact to a certain degree:
H2 would also support larger datasets
2 Likes
Hello @ebarr,
KNIME equivalent to Alteryx “.yxdb” file is .table file for which you need to use Table Writer and Table Reader node.
Additionally take a look at this topic:
Br,
Ivan
2 Likes
Thank you, that was easy enough and effective! I appreciate it.
3 Likes
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.