Cutoff threshold for binary classifier models

[tamaraa21]

Hi,

I am trying to optimize a binary classifier tree ensemble model. It correctly predicts one class, giving me many true negatives, and does not correctly predict the other class, giving me a lot of false positives. I believe it assumed a threshold value of 0.5 to be classified as a positive since the ROC Curve shows confusion matrices for all thresholds. Does anyone know of a way to change this threshold to minimize those false negatives?

[ScottF]

Hi @tamaraa21 -

Have you tried the Binary Classification Inspector node? It will allow you to adjust the threshold, and see how the confusion matrix and other metrics change on the fly.

[mlauber71]

In this article I have a sample workflow with a Metanode that would try to find an optimal cut-off point using two metrics. H2O nodes would find their best cutoff.

H2O.ai AutoML in KNIME for classification problems

H2O.ai Automl - a powerful auto-machine-learning framework wrapped with KNIME It features various models like Random Forest or XGBoost along with Deep Learning. It has wrappers for R and Python but also could be used from KNIME. The results will be written to a folder and the models will be stored in MOJO format to be used in KNIME (as well as on a Big Data cluster via Sparkling Water). One major parameter to set is the running time the model has to test various models and do some hyperparam…

In general a cut-off would very much depend on you business question and often your costs associated with making a wrong prediction.