# How to evaluate multiple (classification) models and choose the best one? (without doing too much manual work) **URL:** \ **Category:** KNIME Analytics Platform\ **Created:** [November 26, 2013, 11:06pm UTC](https://forum.knime.com/t/how-to-evaluate-multiple-classification-models-and-choose-the-best-one-without-doing-too-much-manual-work/3277 "2013-11-26T23:06:56Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![gstoel](https://forum-cdn.knime.com/user_avatar/forum.knime.com/gstoel/32/141_2.png) [@gstoel](https://forum.knime.com/u/gstoel)\ **Post date:** [November 26, 2013, 11:06pm UTC](https://forum.knime.com/t/how-to-evaluate-multiple-classification-models-and-choose-the-best-one-without-doing-too-much-manual-work/3277/1 "2013-11-26T23:06:56Z")
Hello, I am looking into KNIME as an analytical workbench and data mining tool. Looks very promising and have built some models already. However I find it pretty hard to find an efficient way to optimize and choose amongst different models. So for an example, let's look at a classification example.  - I have three classification models (algorithms) - Decision tree - Logistic regression - SVM - I have a dataset split in train, validation and test (60/20/20) My questions are 1. How can I use train and validate to optimize the learner 1. How can I run different versions of the learners (eg. Dec.Tree with and without pruning, SVM with Poly, HT or RBF kernel) 2. with the validation I would like to optimize the parameters of the algorithms, like min. nr of records per node, gamma, kappa, etc. 2. How can I compare the performance of the different models on the different data sets and select my best model? Doing something like this in SAS EM would require 'just a couple of nodes' (and a lot of Euros :)), not sure how to do this in KNIME and if it is possible at all..  Any help would be appreciated. regards, Geoffrey ---
**Author:** ![swebb](https://forum-cdn.knime.com/user_avatar/forum.knime.com/swebb/32/369_2.png) [@swebb](https://forum.knime.com/u/swebb)\ **Post date:** [November 27, 2013, 11:04am UTC](https://forum.knime.com/t/how-to-evaluate-multiple-classification-models-and-choose-the-best-one-without-doing-too-much-manual-work/3277/2 "2013-11-27T11:04:41Z")
Do you want to automate this?  You could do some loops that iterate over different configurations to achieve your DT learning. Or you could manually configure the mutliple options and stick them in a meta node (this would be my preference if you only have a few different configs).  There are performance calculating nodes like "Scorer" which will produce a performance for classification models. I'm sure there is an equivalent for regression.  For optimising parameters look into the parameter optimisation loop nodes in the labs section. They should do what you are after. Cheers Sam ---
**Author:** ![rogerius1st](https://forum-cdn.knime.com/user_avatar/forum.knime.com/rogerius1st/32/38975_2.png) [@rogerius1st](https://forum.knime.com/u/rogerius1st)\ **Post date:** [October 19, 2022, 1:50pm UTC](https://forum.knime.com/t/how-to-evaluate-multiple-classification-models-and-choose-the-best-one-without-doing-too-much-manual-work/3277/3 "2022-10-19T13:50:59Z")
Dear @swebb, Sorry for reviving such an old post… But I hope you’re still here, and can receive (and, answer to) this post of mine. I have a similar situation of comparison between different algorithms’ performances, and I wish to apply some loops to achieve their best configurations. Can you help me with this task? I can easily share with you the anonymized data of my research… Thanks in advance for any help you can offer… B.R., Rogério. ---
**Author:** ![Daniel\_Weikert](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/d/e47c2d/32.png) [@Daniel\_Weikert](https://forum.knime.com/u/Daniel_Weikert)\ **Post date:** [October 19, 2022, 4:09pm UTC](https://forum.knime.com/t/how-to-evaluate-multiple-classification-models-and-choose-the-best-one-without-doing-too-much-manual-work/3277/4 "2022-10-19T16:09:09Z")
You could try out the AutoML component to avoid creating individual models yourself. br ---
**Author:** ![mlauber71](https://forum-cdn.knime.com/user_avatar/forum.knime.com/mlauber71/32/73590_2.png) [@mlauber71](https://forum.knime.com/u/mlauber71)\ **Post date:** [October 19, 2022, 4:49pm UTC](https://forum.knime.com/t/how-to-evaluate-multiple-classification-models-and-choose-the-best-one-without-doing-too-much-manual-work/3277/5 "2022-10-19T16:49:45Z")
@rogerius1st I see these approaches (besides it would be better to open a new thread and leave such old ones alone). First- AutoML Regression and Classification Examples. Like @Daniel_Weikert has mentioned these are the components currently being developed by KNIME: [AutoML Regression and Classification Examples – KNIME Community Hub](https://kni.me/w/ifIPg5Pr0hCk62GS) (example on the hub for classification and regression) [Guided Automation – KNIME Community Hub](https://kni.me/w/eAGfGtEAIr-1iYR-) (guided automation) [Compute and Visualize Global Feature Importance Metrics – KNIME Community Hub](https://kni.me/w/NyMrzvzPx35QN2Gi) (global feature importance) There is a blog collection describing the approach: > **[Blog | KNIME](https://www.knime.com/blog)** > > Free and open source with all your data analysis tools. Create data science solutions with the visual workflow builder, & put them into production in the enterprise. * * * Second - H2O.ai AutoML in KNIME for classification problems (my own little approach utilising H2O.ai AuoML) > [@H2O.ai AutoML in KNIME for classification problems](https://forum.knime.com/t/h2o-ai-automl-in-knime-for-classification-problems/20923): > > H2O.ai Automl - a powerful auto-machine-learning framework wrapped with KNIME It features various models like Random Forest or XGBoost along with Deep Learning. It has wrappers for R and Python but also could be used from KNIME. The results will be written to a folder and the models will be stored in MOJO format to be used in KNIME (as well as on a Big Data cluster via Sparkling Water). One major parameter to set is the running time the model has to test various models and do some hyperparam… H2O.ai AutoML (generic KNIME nodes) in KNIME for classification problems - a powerful auto-machine-learning framework > **[H2O.ai AutoML (generic KNIME nodes) in KNIME for classification problems - a...](https://hub.knime.com/mlauber71/spaces/Public/automl/kn_automl_h2o_classification~1Bfn1eP5d223-eHL/current-state)** > > H2O.ai AutoML (generic KNIME nodes) in KNIME for classification problems - a powerful auto-machine-learning framework (https://hub.knime.com/mlauber71/spaces/P… Sparkling predictions and encoded labels - “the poor man’s ML Ops” > **[s\_618 - H2O.ai AutoML (generic KNIME nodes) in KNIME for classification...](https://hub.knime.com/mlauber71/spaces/Public/kn_example_bigdata_h2o_automl_spark_46/s_618_h2o_automl_spark~1pwn4G2uUkYHdD68/current-state)** > > s\_618 - H2O.ai AutoML (generic KNIME nodes) in KNIME for classification problems - a powerful auto-machine-learning framework applied via Sparkling Water on a … Results get evaluated with R node collection ([Model Quality Classification - Graphics – KNIME Community Hub](https://kni.me/c/dSHRtdVTV0f4nE6L)) and are stored in sub-folders * * * Third - The KNIME Model Process Factory (2017) - an older approach by @Iris at collecting and evaluating models > **[The KNIME Model Process Factory | KNIME](https://www.knime.com/blog/the-knime-model-process-factory)** > > Read about the KNIME Model Factory, a flexible, extensible, and scalable application to efficiently run and monitor large numbers of model processes. > **[The KNIME Model Factory – knime](https://hub.knime.com/knime/spaces/Examples/50_Applications/26_Model_Process_Management/Factory/Model_Factory~qP89D5WIXurNiDg0/current-state)** > > This workflow is part of the model factory eco system. Please refer to our white paper for further details. This is the heart of the model factory. Everytime i… > **[Model\_Process\_Management\_20170404\_1.pdf](https://www.knime.com/sites/default/files/inline-images/Model_Process_Management_20170404_1.pdf)** ---
**Author:** ![system](https://forum-cdn.knime.com/uploads/default/original/1X/ab3ccf34482a0329361734a18199390177204f15.png) [@system](https://forum.knime.com/u/system)\ **Post date:** [June 23, 2023, 8:18pm UTC](https://forum.knime.com/t/how-to-evaluate-multiple-classification-models-and-choose-the-best-one-without-doing-too-much-manual-work/3277/6 "2023-06-23T20:18:59Z")