Individual P values/confidence values for predictions

KNIME Analytics Platform
1

Hi everybody,

I think I have seen in some mining models there is an option for individual confidence value for each predicted class attribute. However, is there a general way, let's say a node, to see these individual confidece values for predictions independent of the model options.

Bora

 

 

 

2

Anybody has any idea? I haven't found the answer of this question yet.

3

How would you get the confidence in a seperate node? If the model doesn't output the confidence then it won't be available downstream. 

Cheers

Sam

4

Hi Sam,

What I meant was the option might not have added to the each individual models (which seems the case) but like the Statistics node we may be able to get the p values for each individual prediction, probably with a complimentary node.

 

5

Thanks E.

I remember confidence values for predictions to be provided in SPSS Modeler (automatically, if the option selected), almost for all classification models. Actually, I had that in mind when I raised whether a similar approach is possible in Knime or not.

Bora

6

Typically, p-values hinge on assumptions about underlying probability distributions related to hypotheses, neither of which mining algorithms typically make - so there's no meaningful definition to begin with.

Your best bet would be bootstrapped confidence intervals for regression predictors (based on which you can derive p-values), or using the "Crosstab" node for scoring classification. Unlike the traditional scoring nodes, it gives you a a p-value for the "no association" null hypothesis based on the Chi-square statistic.

For warnings about p-values, read here: http://www.nature.com/news/scientific-method-statistical-errors-1.14700

Hope that helps.

-E