Hello,
is there any safe/good method to clean up old workflow instances or "left overs" as space is running out from time to time on our knime server (linux).
The workflow folder looks as follows:
12G jobs
8.4G runtime
4.0G temp
143M trash
Is it e.g. safe to delete the contents from time to time? Are there any scripts available or is there a best practice?
There shouldn't be any need to clean these directories manually. However, Server 3.8 had a problem that sometimes prevented deletion of jobs (if the server was restarted before they were auto-discarded) and uploaded workflows were not fully removed from the temp directory. This is may explain most files in "jobs" and "temp" and is fixed in 3.9. I'm wondering what the 8.4GB in runtime are, though. Can you look up the largest files in there?
If the server is shut down and no jobs were running you can deleted all files in "jobs" and "temp".
Hello thor,
thanks for your feedback. After the disk was full this morning, I deleted some stuff already - so now my runtime isnt that big anymore. Regarding the jobs... in KNIME 2.9 i could either choose to discard or to keep but nothing in between (eg keep for 7 days for debug reasons). Manually deleting via the interface is not really practicable for alot of scheduled Jobs.
I tried something like this for now, to delete everything older than 7 days:
find . -mindepth 1 -maxdepth 1 -type d -mtime +7 -exec rm -r "{}" \;
Regarding the runtime folder, it seems there are thousand of chart engine logifles.. her the first entries:
[knime@knimeserver workflows]$ du -a runtime|sort -nr|head -n 50 3022704 runtime 3018896 runtime/runtime_knime-rmi-1099 2674512 runtime/runtime_knime-rmi-1099/workspace 2674508 runtime/runtime_knime-rmi-1099/workspace/.metadata 2585632 runtime/runtime_knime-rmi-1099/workspace/.metadata/.plugins 2585592 runtime/runtime_knime-rmi-1099/workspace/.metadata/.plugins/org.eclipse.birt.chart.engine 344340 runtime/runtime_knime-rmi-1099/.metadata 338308 runtime/runtime_knime-rmi-1099/.metadata/.plugins 338204 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine 80056 runtime/runtime_knime-rmi-1099/workspace/.metadata/.log 47700 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2013_10_30_16_12_20_369.log 24816 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2013_08_19_17_43_11_140.log 11076 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_07_04_11_51_29_325.log 7764 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_05_30_14_51_58_173.log 7316 runtime/runtime_knime-rmi-1099/workspace/.metadata/knime 6740 runtime/runtime_knime-rmi-1099/workspace/.metadata/knime/knime.log 6740 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_03_25_18_05_24_800.log 5928 runtime/runtime_knime-rmi-1099/.metadata/knime 5408 runtime/runtime_knime-rmi-1099/.metadata/knime/knime.log 4592 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_04_01_13_05_38_427.log 4340 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_04_03_15_05_26_506.log 4268 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_04_11_11_05_33_402.log 4264 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_04_09_11_05_29_984.log 4252 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_03_28_11_05_21_750.log 4064 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_04_07_11_05_27_928.log 3988 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_01_06_09_41_15_369.log 3980 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2013_08_30_12_47_11_336.log 3912 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_04_05_15_05_34_818.log 3804 runtime/runtime_knime-rmi-1101 3516 runtime/runtime_knime-rmi-1101/workspace 3512 runtime/runtime_knime-rmi-1101/workspace/.metadata 3504 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_03_30_12_05_23_432.log 3404 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2013_12_04_17_29_18_517.log 3364 runtime/runtime_knime-rmi-1101/workspace/.metadata/knime 3268 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2013_12_13_16_33_09_365.log 3224 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2013_09_03_15_44_37_527.log 3216 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2013_10_14_17_43_16_007.log 3124 runtime/runtime_knime-rmi-1101/workspace/.metadata/knime/knime.log 3120 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_05_24_13_51_21_367.log 3104 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_05_28_14_51_23_992.log 3104 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_05_22_13_51_28_059.log 3064 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_05_20_13_51_25_965.log 3064 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_05_16_13_51_32_045.log 3056 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_05_18_13_51_21_758.log 3052 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_06_14_13_51_48_624.log 3052 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_06_12_13_51_31_385.log 3044 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_06_16_13_51_33_145.log 3044 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_05_14_14_00_16_895.log 3036 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_05_12_14_00_36_236.log 3036 runtime/runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2013_10_16_17_43_12_018.log
Hello,
I could swear I posted something already and it was in moderation first and then dissapeared. In the meantime i updated the server to 3.9 and deleted jobs as adviced.
Probably something like this would work too, to clean up from time to time:
find . -mindepth 1 -maxdepth 1 -type d -mtime +7 -exec rm -r "{}" \;
My runtime looks as follows:
[knime@svx runtime]$ du -sh *
4.6G runtime_knime-rmi-1099
1.6G runtime_knime-rmi-1100
3.8M runtime_knime-rmi-1101
[knime@svx runtime]$ find . -type f -exec du -a {} + | sort -n -r | head -n 20
866472 ./runtime_knime-rmi-1099/workspace/.metadata/.log
752092 ./runtime_knime-rmi-1100/workspace/.metadata/.log
47700 ./runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2013_10_30_16_12_20_369.log
24816 ./runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2013_08_19_17_43_11_140.log
11076 ./runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_07_04_11_51_29_325.log
9592 ./runtime_knime-rmi-1099/workspace/.metadata/knime/knime.log
7764 ./runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_05_30_14_51_58_173.log
7492 ./runtime_knime-rmi-1100/.metadata/knime/knime.log
6740 ./runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_03_25_18_05_24_800.log
5448 ./runtime_knime-rmi-1099/.chemaxon/lib/tesseract-linux-x64-3.01/tesseract.linux-x64
4592 ./runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_04_01_13_05_38_427.log
4340 ./runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_04_03_15_05_26_506.log
4268 ./runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_04_11_11_05_33_402.log
4264 ./runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_04_09_11_05_29_984.log
4252 ./runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_03_28_11_05_21_750.log
4064 ./runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_04_07_11_05_27_928.log
3988 ./runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_01_06_09_41_15_369.log
3980 ./runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2013_08_30_12_47_11_336.log
3912 ./runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_04_05_15_05_34_818.log
3504 ./runtime_knime-rmi-1099/.metadata/.plugins/org.eclipse.birt.chart.engine/org.eclipse.birt.chart.engine_2014_03_30_12_05_23_432.log