KNIME Batch running via the Windows Task Scheduler

KNIME Analytics Platform
1

I've a workflow that is running every night via the Windows Task Scheduler.

Even though I've ticked the box "Stop the task if it runs longer than 2 hours", the workflow no longer runs, but there is always a remanining knime process that never stops and that uses memory.  As the program is running every night, I got many non-ending KNIME processes that are taking a lot of memory if I don't kill them manually.

Can someone help to kill the knime process automatically after the program has run?

Thanks,

Amélie

1 Like
2

The Batch Executor process always terminates when the workflow execution has finished. If the process is still running then the workflow hasn't finished yet. Why the task scheduler doesn't kill the process correctly I don't know.

3

I have the same issue. Could it be anything to do with Windows 10?

4

I am facing the same issue - any resolutions found?

5

I’m getting the same kind of problem in a production machine running win server 2016.

Currently we have +20 jobs calling different Knime workflows at different hours of the day. I developed a script to kill the Knime processes at 23 PM every day to free up the windows memory. But in some months, we will probably have +50 jobs running at different hours of the day and this will be very hard to manage.

I don’t understand why so much knime processes stay running in task manager.

6

Is the task scheduler running a .bat that then launches KNIME? If so, some googling about task scheduler killing child processes does turn up some similar complaints - one SO self-solution was moving from batch to powershell, supposedly.

7

Hi quaeler,

In fact, we are using Rundeck to manage all these jobs. But every rundeck job call a .bat file that then launches KNIME workflows.

8

Unfortunately, there is a still open feature request from 4+ years ago that Rundeck better handle killing children processes: https://github.com/rundeck/rundeck/issues/1038

9

Thanks, quaeler.

I’m starting to think in buy the KNIME Server, as this also have a scheduling jobs.

But I need to be really sure that I will not have this kind of problems and be able to run for example 5 workflows at same time in parallel.

because I also have this kind of problems at the moment, I Have a workflow job running every 15 minutes and sometimes this provoke some other failed workflows/jobs.

My problem now, is I’m not relay sure if the main issues is with rundeck or KNIME free desktop application.

At the moment we have around +20 jobs/workflows but maybe in 6 months will have +100 jobs.

Any recommendation to setup a robust environment for this?

We are using AWS and Windows server 2016 in EC2 and Rundeck to manage Jobs scheduling.

Thinking to Buy the KNIME Server if this is really the best option but don’t want to go for Linux.

And what about running Knime as Cluster? Seems like this only work in Linux.