KNIME Output Data / Output-Data Type

[Christopher_Lozano]

Hi there,

i have a question due to the general outputs that are possible in KNIME.

In our compnay we have a workflow that runs daily and generates a forecast for several years including every month of it. Having round about 400k customers it geenrates a table of 16 millions.
Now we want to have a look for one year and analyse the data. With a Rule-based row filter so far so good but running an output in form of a csv-writer it generates a 700MB Table.

Is there any possbility to generate an RDS-Output Data or does even KNIME have an own data table format for an output which generates a very smaller table size. By Transforming it with R and generate an RDS Data the table “only” will have 59MB.

Greetings
Christopher

[elsamuel]

Here’s a workflow demonstrating RDS output by @mlauber71:

various methods to export data and graphics from KNIME with R – mlauber71

[mlauber71]

@Christopher_Lozano

Transforming it with R and generate an RDS Data the table “only” will have 59MB.

When transferring data between KNIME, R and maybe other systems like big data or python you also might want to consider Parquet format. It also can be split into several sub-files that can be handled together or individually.

use R library(arrow) to read parquet file into KNIME – mlauber71

The example also shows SQLite which also offers great flexibility and ARFF if you absolutely must have a Text only file but want to preserve column types.