Urgent - What is wrong with my decision tree predictor for new data

[mlauber71]

These things:

• you will have to do the whole preprocessing you did for your training also for your new data, otherwise the model will not have the same structure and cannot be applied
• then you are not splitting your data into test and training for the development of your model. So the very high score is not very useful since both deal with the same set of data
• and you will have to make sure that the answer is not somewhere encoded in the data (which will not be present in any future data you might want to score)

I will have a look and see if I can find a fix.

If you want to read about Yes/No models you can follow these links and will also find some example workflows you could use (the data preparation would still have to be yours)

Looking for options to evaluate a decision tree

One ‘brutal’ way could be to convert the decision tree rules into SQL and then use the SQL code. Or use the ruleset to make some kind of If then rules. Not very elegant I have to admit. [27] [35] kn_example_decision_tree.knwf (510.8 KB)

Understand metrics like AUC and Gini (and use H2O.ai)

Random Forest Model Not Working

I attached a working example of a Random Forest with H2O.ai nodes and a 0/1 Target from a Kaggle competition (please note the example only uses the first 1.000 lines, you should remove this restriction or use your own data for your own purpouses). In order to get an idea about the quality of you model you might want to read this article: I always advise not just to use a Scorer but rather a metric like Gini/AUC to see the relative quality of the model. In the end the ‘score’ is not a magic…

Help choosing analytics algorithm

I think you could do two things. You could treat your outlier group as label/target (1/0 or TRUE/FALSE) and the rest of the data as explaining variables. You would have to remove the time column since this ‘leaks’ the information you want to find. You could then use an algorithm like Random Forest Learner *1) that also gives you a list of the most important variables, the ones that make the outliers the outlier compared to the regular cases. You could also just use the time for each transaction…

Models for Multiclass Targets:

Any advice to improve the performance of a classification model ?

We had a similar discussion about metrics in multi class predictions here: Besides accuracy you could use a LogLoss metric. That means that not only the outcome of prediction will be taken into account but also the ‘level of certainty’. If you have two predictions from two models A) 1=51%, 2=39%, 3=10% TRUTH=>1 B) 1=90%, 2=5%, 3=4% TRUTH=>1 both come up with the correct result (1). But prediction B) would be better as it predicts the correct answer with higher confidence and might be mor…

Metrics in multiclass classification

I used this example to turn it into a workflow to predict multiple classes. And I also added the accuracy statistics. One with F1 in Python but also the KNIME Scorer which also has a “micro” F1 as “Accuracy” - so the Python is more there to test the logic. I am though a little bit reluctant to just use such multiple classifications as targets. Maybe someone could weight in who has more experience in that field. And of course you should be careful and test your results with real life data (and…