Conformal predcition combine with hyperparameter optimization

[evert.homan_scilifelab.se]

Hi,

I would like to optimize the hyperparameters of the Random Forest Learner node within the Conformal Prediction workflow (Conformal prediction workshop by Redfield – KNIME Hub), but am unsure where to put the Parameter Optimization Loop End node, as the Conformal Calibration section has two out ports

[temesgen-dadi]

Hi @evert.homan_scilifelab.se

I am sorry but, I don’t understand your question. The Parameter Optimization Loop End node has only a flow variable input port.

Why is it important, that if the node you want to attach it to has two output port?

Kind regards,
Temesgen

[evert.homan_scilifelab.se]

Hi,

Let me rephrase then: where would you put the Parameter Optimization Loop End in the workflow?

Bets regards/Evert

[temesgen-dadi]

Thanks for the clarification,

Here is an example workflow how to user the parameter optimization loop node pairs. You may need to use a scorer node first so that the node gets the objective function it tries to optimize for.

Best,
Temesgen

[Redfield]

Hello @evert.homan_scilifelab.se

I think I got your idea right. I tried to re-implement the workflow from the webinar with parameter optimization for both model and calibration table.
In this case you will need to train n * p1 * p2 * … * pm models, and will get the same number of calibration tables. n - is the number of iterations of calibration loop, pm - range of the values of the m-th parameter of the model you are optimizing. I would also consider using the Brute Force mode in optimization loop.

In the settings of Conformal Prediction loop you need to use the external (optimization) loop iteration number to sync the calibration table with model. In the current case it is Iteration #1 column.

Please pay attention that I removed the data from the workflow to make it light-weight to upload here.
Conformal prediction with model optimization.knwf (253.7 KB)

Best regards,
Artem.