Decision Tree Predicting Only One Outcome?

It looks like you just use the % prediction and then the scorer sets a result and since the majority would remain the moment a leaf reaches 50+% remain it counts as A (Active).

It might be that your Decision Tree does not find small enough groups where T (Terminate) is the majority. You could still derive a score from the tree and rank the people which are most likely to go.

So the person most likely to leave might have a score of 0.42 or something - by the 0.5+ rule he would be classified as A but it might still be possible to find a cutoff to decide who is of a higher risk of leaving.

You would have to set the cutoff point yourself instead of using the 50%+ rule.

This example also allows you to extract rules from your tree so you can see what is driving its decision:

One other way to measure the quality of such a model is Gini/AUC - we had a discussion about that here. Also you might want to try other models as well.

Then you might have to adapt your business strategy if you cannot get better data or find a better model. You might not be able to make immediate decision but make intermediate ones. Like putting people on a watchlist or assigning some money to hire new people based on a probability some of them will leave.

Also you might want to be careful with the Accuracy claim. As we have discussed on several occasions. This is a good example why ‘Accuracy’ is not always helpful. 87% would sound impressive for some AI buzz but as everyone can easily see here in this case the number is pretty useless in itself.

4 Likes