Using a cost/revenue matrix to set a decision threshold

[alfroc]

Hi,

Is it possible to choose an optimal threshold based on a cost/revenue matrix?

Thanks

Alfredo

[Vincenzo]

Hi Alfredo,

Could you please describe a little bit more in details what you would like to achieve?

Would you like to find the optimal value of a specific function?

In this case I would suggest to have a look at the Parameter Optimization Loop nodes and the Math Formula node. The Parameter Optimization Loop nodes allow you to find the optimal value of a specific parameter, instead the Math Formula node allows you to build your custom function.

Hope that helps,

Best,

Vincenzo

[alfroc]

Consider the attached image. You can see a cost matrix and three confusion matrixes based on different threshold values, the optimal profit (0,0729) is for 0,75.
What I’d like to know is whether I can get an output like this:
Threshold TP TN FP FN Profit
…
0,40 311 56 94 39 0,0297
…
0,50 296 70 80 54 0,0472
…
0,75 247 100 50 103 0,0729
…

[alfroc]

Hi Vincenzo,
I did solve it by myself creating a metanode containing a java snippet node inside a loop that calculates, for each threshold value, its confusion matrix and, outside the loop, another java node that calculates the accuracy and kappa statistics and the per unit cost from the cost matrix (a simply table).
Best,
Alfredo