Feature request: knime server, discard workflow after execution

KNIME Server
1

Hello,

An idea for great feature for knime server, allowing workflow to be discarded after execution, unless there has been any a) warning, b)failure

I have some workflows that run very often, but if I check the discard workflow after execution checkbox, that won't catch failed executions right? 

Kind regards,

Joseph

2

Hi Joseph,

Thanks for contacting us with the feature request.

I've created a feature request in our ticketing system, I can't yet say when, or if that feature is planned to be available.

Best,

Jon

3

Thank you. Is it possible to track the progress on that feature ticket or is it an internal system? 

1 Like
4

I just wanted to put a big +1/upvote/shout out for this idea.

We too want to be able to see failed workflows – but discard successful ones. Similarly, it would really help for KNIME Server to be able to discard really old workflows (in some cases, really old means older than a few days, in other cases, that’s a month). We recently had a problem where KNIME Server just stopped working because the disk was full because we have no middle ground, and we found it was these old workflows that filled up our disk.

As is, because KNIME Server cannot discard appropriate workflows, KNIME Server is not truly self managing because we have to go in and check it regularly to make sure that it hasn’t used up our disk (which is large, but running several workflows many times can fill it up over time).

5

Hi there,

I just added your name to the list for the feature. With KNIME Server there is a feature to discard jobs that are older than a certain time. The default is 7 days.

With the KNIME Server 4.8.0 release yesterday we added a new feature that allows to discard jobs that execute longer than a certain time.

Hope that helps. and post back here if you have more questions.

Best,

Jon

3 Likes
6

We would need that feature too for our KNIME Jobs. We have some that are getting executed every minute and they load the server full with data.

7

Hi @stofffetzen

I will add a +1 to the ticket for you.

One possibility is to use a scheduled REST workflow to clean the server of those jobs.
With our REST API you can get a list of all jobs, filter the successfully executed ones and discard those.

Best, Iris

2 Likes
8

Hi @Iris, thank you for adding the +1 on the ticket.
Besides that I wanted to mention that we are already using the REST feature to discard Jobs with a DELETE Request. We have a job running every hour to discard all executed jobs without errors.
We do have problems with using this method because we have around 300 jobs executing per hour on our server and the Rest Service is very slow (1 Job to discard takes around 4 seconds).
Can’t the KNIME Server handle multiple (parallel) requests with the Concurrency feature of the REST nodes? It feels like the concurrency setting doesnt matter when using it with the KNIME Server.
Greetings
Chris

1 Like
9

We would need that feature too for our KNIME Jobs.
We have jobs that are getting executed every minute and they load the server full with data also Knime Analytics platform becomes un-responsive when you try to open the workflow history.

This is urgent for us to use keep using Knime server! Are there any update on this?

1 Like
10

@menayah You have two possibilities how to to this already.

  1. When scheduling a job, use the option Discard Workflow Job after Execution. This will directly discard them after they are executed.
  2. If you want to keep them for a while, you can discard them with a scheduled job. You can find the workflow for this here:
    https://hub.knime.com/knime/spaces/Examples/latest/09_Enterprise/01_Server/03_Using_KNIMEServer_REST_API/02_Discard_Failed_Jobs_on_KNIME_Server

I will also give the Feature Request a +1 from you.