H2O AutoML Numeric Scorer and evaluation in python

[inirem]

Hey everyone.

I am currently using the extension H2O for some AutoML-Tasks. When trying to evaluate the model performance with the H2O numeric scorer node, I started wondering about the used metrics.

In particular I can’t find an in-depth documentation of the used formulas. For example R² could also be an adjusted R² - or some R² can only get to 0, while others can be negative. But there is no documentation about what specific R² is used. Is there any documentation about this and the other metrics?

I also noticed that the Python-Based H2O allows for much more indepth analysis. Is there a way to export my model and predictions and then use them in python code to manually assert more/other metrics?

Thanks in advance - I am new to ML and Python, so please excuse if some of the awnsers are obvious.

[mlauber71]

@inirem

export my model and predictions and then use them in python code to manually assert more/other metrics

You can export your H2O model as mojo file and then extract more information via a Python node:

H2O Generalized Linear Model Learner (Regression) - H2O Model Port to variable or table

@jichaohe @jichaohe @elliegreib @kevweath well actually there is a way to do this. One could employ a Python node and H2O.ai in the background to extract a buch of parameters and also the summary from the MOJO model file. Though admittedly not the most elegant way you could have the information in a file [image] kn_example_ml_regression_h2o_glm_parameter.knwf (222.2 KB) If you are interested you might also check out these article and workflows (I plan on having a walkthroug…

[inirem]

Thanks a lot. I assume you also don’t know about the exact formulas used for the metrics?