# Model Monitoring workflow node

**URL:** <https://forum.knime.com/t/model-monitoring-workflow-node/27402>\
**Category:** KNIME Analytics Platform\
**Created:** [September 15, 2020, 7:51pm UTC](https://forum.knime.com/t/model-monitoring-workflow-node/27402 "2020-09-15T19:51:18Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Wizard\_dk](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/w/e68b1a/32.png) [@Wizard\_dk](https://forum.knime.com/u/Wizard_dk)\
**Post date:** [September 15, 2020, 7:51pm UTC](https://forum.knime.com/t/model-monitoring-workflow-node/27402/1 "2020-09-15T19:51:18Z")

</div>

Hi KNIME community,

I was going through a beautiful webinar on “Integrated Deployment :- How to move data science into production” on KNIME TV available at youtube([https://www.youtube.com/watch?v=MJq154i4O-0](https://www.youtube.com/watch?v=MJq154i4O-0)).  
In this Michael Berthold(presenter) use a workflow for demonstration which he mention that it is available on KNIME HUB ([https://youtu.be/MJq154i4O-0?t=723](https://youtu.be/MJq154i4O-0?t=723)) but somehow i am not able to find the same.  
I am currently working on similar kind of problem (specially the monitoring and re-train of the model part and deploying the chosen model).The above mention workflow can be very useful to me.

can someone share the workflow? it will be very helpful.

P.S :- Sorry, i am not sure this forum is suitable for this kind of question, please do let me know . i will refrain myself for posting such question in future,if required.

Thanks,  
Wizard Dk

---

<div class="post-metadata">

**Author:** ![ScottF](https://forum-cdn.knime.com/user_avatar/forum.knime.com/scottf/32/61316_2.png) [@ScottF](https://forum.knime.com/u/ScottF)\
**Post date:** [September 15, 2020, 8:20pm UTC](https://forum.knime.com/t/model-monitoring-workflow-node/27402/2 "2020-09-15T20:20:42Z")

</div>

Hi @Wizard_dk -

You may want to check the ongoing Integrated Deployment blog series by our data science team:

[https://www.knime.com/integrated-deployment-knime-blog-series](https://www.knime.com/integrated-deployment-knime-blog-series)

Right now there are three different posts that go into detail of how everything works, with links to several example workflows.

With respect to the particular component highlighted in Michael’s video - the one about monitoring and re-training - I believe that may not be published yet. Let me double check internally and see what I can find out.

---

<div class="post-metadata">

**Author:** ![Wizard\_dk](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/w/e68b1a/32.png) [@Wizard\_dk](https://forum.knime.com/u/Wizard_dk)\
**Post date:** [September 15, 2020, 8:52pm UTC](https://forum.knime.com/t/model-monitoring-workflow-node/27402/3 "2020-09-15T20:52:50Z")

</div>

Thanks @ScottF for prompt reply.  
I am going through given series of blog 🙂  
Yes, please check if the monitoring node and other nodes are published.

Thanks,  
Wizard Dk

---

<div class="post-metadata">

**Author:** ![paolotamag](https://forum-cdn.knime.com/user_avatar/forum.knime.com/paolotamag/32/1061_2.png) [@paolotamag](https://forum.knime.com/u/paolotamag)\
**Post date:** [September 16, 2020, 8:39am UTC](https://forum.knime.com/t/model-monitoring-workflow-node/27402/4 "2020-09-16T08:39:32Z")

</div>

Hi @Wizard_dk,  
at the moment we do not have an available Component for the monitoring part yet.  
We are working on it but I cannot guarantee any deadline.  
Sorry about that. If you use Integrated Deployment you should be able to re-execute pieces of workflows on demand. Something like:

1. Train your model on train set (generic Learner node)

2. Score model on test set (generic Predictor node)

3. Capture Scoring (generic Predictor node) with Integrated Deployment (node [1](https://kni.me/n/63TEO1y-39Dh5uQa) and [2](https://kni.me/n/fnDFDSaCZVy-J3LA))

4. Deploy Scoring as REST API on KNIME Server via Integrated Deployment ([Deploy node](https://kni.me/n/h0uaCFJJGvJD5SUL))

5. Capture from point 1 to point 4 (Learner node + Captured Predictor Node + [Deploy node](https://kni.me/n/h0uaCFJJGvJD5SUL)) with Integrated Deployment (node [1](https://kni.me/n/63TEO1y-39Dh5uQa) and [2](https://kni.me/n/fnDFDSaCZVy-J3LA))

6. On a separate workflow query for new data for which you have ground truth (maybe from a frequently updated database with some timestamp column)

7. Call Deployed scoring model (point 4) via [Call Workflow node](https://kni.me/n/M66aoaj-gqFbGR5j)

8. [Measure performance](https://kni.me/n/5tsiYT6vDKM7YChN) (and optionally plot it in a line plot)

9. Check if performance is below a threshold you decide

10. Execute previously captured workflow (point 5) to retrain workflow and also redeploying it

The cool part is that with point 10 by calling this main workflow you are executing a chain reaction to repeat from 1 to 10 and the whole process restarts.

It might be a bit tricky to wrap your head around this but it should do the job.  
I gave for granted you have access to a KNIME Server. If you don’t it should still work locally.  
Just replace the [Deploy node](https://kni.me/n/h0uaCFJJGvJD5SUL) with a [Workflow Writer node](https://kni.me/n/ouYgT_6spFNuvnv_).  
Let me know if you have any questions.

Cheers  
Paolo

---

<div class="post-metadata">

**Author:** ![Wizard\_dk](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/w/e68b1a/32.png) [@Wizard\_dk](https://forum.knime.com/u/Wizard_dk)\
**Post date:** [September 16, 2020, 9:06am UTC](https://forum.knime.com/t/model-monitoring-workflow-node/27402/5 "2020-09-16T09:06:37Z")

</div>

Thanks Paolo  
I am following your Integrated Deployment Blog Series  
I am doing the similar thing as suggested here 🙂 like comparing model performance metric with defined threshold and plotting metric on line plot.  
Things are smooth for now, will let you know if some help required.

Cheers !!  
Wizard Dk

---

<div class="post-metadata">

**Author:** ![paolotamag](https://forum-cdn.knime.com/user_avatar/forum.knime.com/paolotamag/32/1061_2.png) [@paolotamag](https://forum.knime.com/u/paolotamag)\
**Post date:** [October 2, 2020, 7:36am UTC](https://forum.knime.com/t/model-monitoring-workflow-node/27402/6 "2020-10-02T07:36:38Z")

</div>

While you wait for the model monitoring component (on its way),  
have you checked out the new [XAI View](https://kni.me/c/no0cArkQ94BnNpeb)?

![ezgif-3-43cacf89a4ba](https://forum-cdn.knime.com/uploads/default/original/3X/0/9/098274d843a3e87f11dd03934bf4cda28b2de81e.gif)

Workflows:

1. [https://kni.me/w/5xfwkuVsF6Uz8hMC](https://kni.me/w/5xfwkuVsF6Uz8hMC)
2. [https://kni.me/w/JZbuUdhGKZBEpdoK](https://kni.me/w/JZbuUdhGKZBEpdoK)

---

<div class="post-metadata">

**Author:** ![Wizard\_dk](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/w/e68b1a/32.png) [@Wizard\_dk](https://forum.knime.com/u/Wizard_dk)\
**Post date:** [October 3, 2020, 6:47pm UTC](https://forum.knime.com/t/model-monitoring-workflow-node/27402/7 "2020-10-03T18:47:15Z")

</div>

Hi @paolotamag,  
Good to hear that Model monitor is on its way !!🙂  
I was also able to implement something similar to your [suggestion](https://forum.knime.com/t/model-monitoring-workflow-node/27402/4) like scoring from captured score pipeline then do monitoring over it then showing graph for model performance over time … retraining the model on user wish from capture train pipeline and deploying new model (model selection) as per user wish 🙂

[XAI](https://hub.knime.com/knime/spaces/Examples/latest/00_Components/Model%20Interpretability/XAI%20View) looks cool. I have tried these things on python. Good to have them on KNIME. Will definitely going to use them in future!!

Regards,  
Wizard\_Dk

---

<div class="post-metadata">

**Author:** ![Daniel\_Weikert](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/d/e47c2d/32.png) [@Daniel\_Weikert](https://forum.knime.com/u/Daniel_Weikert)\
**Post date:** [October 4, 2020, 10:01am UTC](https://forum.knime.com/t/model-monitoring-workflow-node/27402/8 "2020-10-04T10:01:51Z")

</div>

Can you share your flow?

---

<div class="post-metadata">

**Author:** ![Wizard\_dk](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/w/e68b1a/32.png) [@Wizard\_dk](https://forum.knime.com/u/Wizard_dk)\
**Post date:** [October 9, 2020, 11:38am UTC](https://forum.knime.com/t/model-monitoring-workflow-node/27402/9 "2020-10-09T11:38:50Z")

</div>

Hi Daniel

Sorry, somehow i missed your post.  
That workflow was develop on office premise and I guess we can not share.

I just follow this [series](https://www.knime.com/integrated-deployment-knime-blog-series)  
and Paolo’s [comment](https://forum.knime.com/t/model-monitoring-workflow-node/27402/4) .

if you have some difficulty in some parts then please do share here.  
I will be happy to help.

Regards,  
Wizard Dk

---

<div class="post-metadata">

**Author:** ![system](https://forum-cdn.knime.com/uploads/default/original/1X/ab3ccf34482a0329361734a18199390177204f15.png) [@system](https://forum.knime.com/u/system)\
**Post date:** [October 16, 2020, 11:38am UTC](https://forum.knime.com/t/model-monitoring-workflow-node/27402/10 "2020-10-16T11:38:52Z")

</div>

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

---

<div class="post-metadata">

**Author:** ![paolotamag](https://forum-cdn.knime.com/user_avatar/forum.knime.com/paolotamag/32/1061_2.png) [@paolotamag](https://forum.knime.com/u/paolotamag)\
**Post date:** [February 2, 2021, 9:43am UTC](https://forum.knime.com/t/model-monitoring-workflow-node/27402/11 "2021-02-02T09:43:49Z")

</div>

Hello there, we just published new [Verified Components](https://www.knime.com/verified-components) for monitoring of a classification model. It works both for binary and multiclass classification but the model needs to be captured within the production workflow via [integrated deployment](https://www.knime.com/integrated-deployment).

Until we can load the external deployment workflow as an integrated deployment connection you have to recapture the deployment workflow within the same workflow using the components. If you used the [AutoML](https://kni.me/c/33fQGaQzuZByy6hE) component that should be super easy. More information in the component descriptions and example workflow. A blog post should also out soon.

- [Model Monitor View](https://kni.me/c/qoPsGhs0i5GjUiPu)
- [Model Monitor View (Compare)](https://kni.me/c/1OK0-S9S7PZFarzD)
- [Model Monitoring with AutoML and MLOps](https://kni.me/w/vMMSjNlCCI_4zGIJ)

Cheers  
Paolo

---

<div class="post-metadata">

**Author:** ![paolotamag](https://forum-cdn.knime.com/user_avatar/forum.knime.com/paolotamag/32/1061_2.png) [@paolotamag](https://forum.knime.com/u/paolotamag)\
**Post date:** [April 22, 2021, 10:07am UTC](https://forum.knime.com/t/model-monitoring-workflow-node/27402/12 "2021-04-22T10:07:26Z")

</div>

Hello there,  
a blog post on the topic was published today:

> **[Monitoring the Production Workflow: Model Monitoring in a Data Science Context...](https://www.knime.com/blog/model-monitoring-in-data-science-context)**
>
> Here we present model monitoring in the data science context and demonstrate a component to monitor the ML model and re-evaluate its performance during production.

Enjoy!
