Model Monitoring workflow node

KNIME Analytics Platform
4

Hi @Wizard_dk,
at the moment we do not have an available Component for the monitoring part yet.
We are working on it but I cannot guarantee any deadline.
Sorry about that. If you use Integrated Deployment you should be able to re-execute pieces of workflows on demand. Something like:

  1. Train your model on train set (generic Learner node)

  2. Score model on test set (generic Predictor node)

  3. Capture Scoring (generic Predictor node) with Integrated Deployment (node 1 and 2)

  4. Deploy Scoring as REST API on KNIME Server via Integrated Deployment (Deploy node)

  5. Capture from point 1 to point 4 (Learner node + Captured Predictor Node + Deploy node) with Integrated Deployment (node 1 and 2)

  6. On a separate workflow query for new data for which you have ground truth (maybe from a frequently updated database with some timestamp column)

  7. Call Deployed scoring model (point 4) via Call Workflow node

  8. Measure performance (and optionally plot it in a line plot)

  9. Check if performance is below a threshold you decide

  10. Execute previously captured workflow (point 5) to retrain workflow and also redeploying it

The cool part is that with point 10 by calling this main workflow you are executing a chain reaction to repeat from 1 to 10 and the whole process restarts.

It might be a bit tricky to wrap your head around this but it should do the job.
I gave for granted you have access to a KNIME Server. If you don’t it should still work locally.
Just replace the Deploy node with a Workflow Writer node.
Let me know if you have any questions.

Cheers
Paolo

5 Likes