Bad correlation

[mlauber71]

you can do these things:

• check the importance of your variables in the model and see if the top ones might contain ‘leaks’ that somehow give away the correct answer
• see if you have variables like year not in a relative but an absolute way (like year=2016 which might change over time)
• check if your train, test, validate datasets are truly separated. Eg you have households that you split by person, and one ends up in test one in training. But they share pretty much the same data and target
• see if your new data contains all the variables in the same quality like your original set
• you might think about what you accuracy does mean in the context of you business question (cf. links below)

Then you might provide us with more details or even a sample workflow, you might use fake data if you cannot share the original data.

---

Models for 0/1 or Yes/No Targets

Looking for options to evaluate a decision tree

One ‘brutal’ way could be to convert the decision tree rules into SQL and then use the SQL code. Or use the ruleset to make some kind of If then rules. Not very elegant I have to admit. [27] [35] kn_example_decision_tree.knwf (510.8 KB)

Understand metrics like AUC and Gini (and use H2O.ai)

Random Forest Model Not Working

I attached a working example of a Random Forest with H2O.ai nodes and a 0/1 Target from a Kaggle competition (please note the example only uses the first 1.000 lines, you should remove this restriction or use your own data for your own purpouses). In order to get an idea about the quality of you model you might want to read this article: I always advise not just to use a Scorer but rather a metric like Gini/AUC to see the relative quality of the model. In the end the ‘score’ is not a magic…

Help choosing analytics algorithm

I think you could do two things. You could treat your outlier group as label/target (1/0 or TRUE/FALSE) and the rest of the data as explaining variables. You would have to remove the time column since this ‘leaks’ the information you want to find. You could then use an algorithm like Random Forest Learner *1) that also gives you a list of the most important variables, the ones that make the outliers the outlier compared to the regular cases. You could also just use the time for each transaction…