knime server Jobs are holding too much size
every day I run almost 70 jobs and its occupies 50GB+ size of the disk
to free the disk I have to delete the jobs every day
Why jobs are holding too much data size?
How to reduce it?
The size of a job his highly depending on the amount of data it processes. If possible, try to keep volume low (filter and transform in DB or Big Data systems). Other than that streaming can not only increase execution performance, but also keeps size of the job low as data will not be stored persistently. And you could also discard the the job right after execution if that helps to keep usage of system’s volume low.
2 Likes
i know . but I cant discard jobs same day , i need that job atleast for 4 days.
i need another way.
Is it due to logs size ??