Selection of ROC curve after hyperparameter tuning

KNIME Analytics Platform
#1

Good evening,

I am training different models (RF, logistic regression -LR-, SVM) using 10-fold crossvalidation and tuning different hyperparameters (for example, the variance value in LR with Laplace regularization) and using AUC as the measure of performance.

After CV, the higher AUC is achieved with a given value of the hyperparameter (let’s say, the first out of ten different values tested), but the ROC curve shown corresponds to the AUC with the last value of the hyperparameter (in this case, the tenth). How can I generate the ROC curve corresponding to the highest AUC, that with the optimal value for the hyperparameter?

Thank you,
Marc

#2

You would need to save the ROC Curve image for every iteration of your optimization loop, e.g. use image to table and append it to the table holding your model scores.

1 Like
#3

Ok, thank you @Marten_Pfannenschmidt!

#4

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.