# xgboost regression hyperparameter optimization **URL:** \ **Category:** KNIME Analytics Platform\ **Created:** [May 9, 2021, 8:47am UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823 "2021-05-09T08:47:32Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![BerkayAkar](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/b/ed655f/32.png) [@BerkayAkar](https://forum.knime.com/u/BerkayAkar)\ **Post date:** [May 9, 2021, 8:47am UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/1 "2021-05-09T08:47:32Z")
hi dear form users ı m new to knime platform. ı have a quaestions. ı want to predict by using xgboost parameters={ ``` "learning_rate":[0.1,0.01,0.001], "n_estimators":range(100, 1200, 100), } ``` xgb\_model = xg.XGBRegressor(random\_state=0) this is my pyhton code and ı m searc by GridSearch to hyperparameter . how can ı do this on knime ? ---
**Author:** ![iperez](https://forum-cdn.knime.com/user_avatar/forum.knime.com/iperez/32/24881_2.png) [@iperez](https://forum.knime.com/u/iperez)\ **Post date:** [May 9, 2021, 3:09pm UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/2 "2021-05-09T15:09:50Z")
Hi @BerkayAkar The parameter optimization loop allows you to find the best values of the parameters you choose to explore. ![imagen](https://forum-cdn.knime.com/uploads/default/original/3X/7/7/779517f3a5924c40b15c1eb60d81822439c565b3.png) Hope this helps ---
**Author:** ![BerkayAkar](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/b/ed655f/32.png) [@BerkayAkar](https://forum.knime.com/u/BerkayAkar)\ **Post date:** [May 9, 2021, 3:46pm UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/3 "2021-05-09T15:46:58Z")
coldu you send me a work flow ? please beacuise ı have a lot of hyperparameter columb and this method using only one parameter is it true ? thanks for giving advice sir ---
**Author:** ![iperez](https://forum-cdn.knime.com/user_avatar/forum.knime.com/iperez/32/24881_2.png) [@iperez](https://forum.knime.com/u/iperez)\ **Post date:** [May 9, 2021, 8:21pm UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/4 "2021-05-09T20:21:50Z")
Sure. This might help you to get on the way.[XGBoost Optimized.knwf](https://forum.knime.com/uploads/short-url/uMYnhUIxTVDP8cQOZ3hkEeqODdU.knwf) (52.3 KB) ---
**Author:** ![BerkayAkar](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/b/ed655f/32.png) [@BerkayAkar](https://forum.knime.com/u/BerkayAkar)\ **Post date:** [May 9, 2021, 10:09pm UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/5 "2021-05-09T22:09:02Z")
thanks for ansver sir ı have one question ı want finish my learning rate value from 0.001 to 1 ![image](https://forum-cdn.knime.com/uploads/default/original/3X/5/6/565dc82d0359d0b8322a3bf58902387ec4078ce0.png) is it true configruation ? and another questions is why we use table row to verable node ? could you help me sir ? thanks for giving advice ex learning rate = [0.001,0.01,0.1,1] only this verable ---
**Author:** ![iperez](https://forum-cdn.knime.com/user_avatar/forum.knime.com/iperez/32/24881_2.png) [@iperez](https://forum.knime.com/u/iperez)\ **Post date:** [May 9, 2021, 10:32pm UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/6 "2021-05-09T22:32:00Z")
HI. No the workflow is to get you started, I suggest that you check the parametrization you have in Python and check the parametrization in KNIME’s node, then make the adjustments you need. The numeric socreer gives multiple numeric criteria that you may use to optmize you hyperpartameters, you have to be able to pass the parameter to the loop end, you’d need to tweak the last two nodes to get what you need. Hope I’m clear ---
**Author:** ![BerkayAkar](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/b/ed655f/32.png) [@BerkayAkar](https://forum.knime.com/u/BerkayAkar)\ **Post date:** [May 9, 2021, 10:34pm UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/7 "2021-05-09T22:34:06Z")
ı m sorry for my bad englsh so ı didnt understand ı want to check this parametres learning rate [0.1,0.001,1] n-estiminators[100,200,300,400…1200] it is posibble for this node ? ---
**Author:** ![BerkayAkar](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/b/ed655f/32.png) [@BerkayAkar](https://forum.knime.com/u/BerkayAkar)\ **Post date:** [May 9, 2021, 10:38pm UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/8 "2021-05-09T22:38:13Z")
hi thansk for advice but ı didnt understand ı m sorry for my bad english ı want to check this parametres learning rate [0.1,0.001,1] n-estiminators[100,200,300,400…1200] it is posibble for this node ? ---
**Author:** ![iperez](https://forum-cdn.knime.com/user_avatar/forum.knime.com/iperez/32/24881_2.png) [@iperez](https://forum.knime.com/u/iperez)\ **Post date:** [May 9, 2021, 10:52pm UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/9 "2021-05-09T22:52:57Z")
The learning rate in Knime is the eta parameter, I guess that you want tho check learning rates between 0.001 and 1 with a step increment of 0.1. What are the n-estimators? ---
**Author:** ![BerkayAkar](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/b/ed655f/32.png) [@BerkayAkar](https://forum.knime.com/u/BerkayAkar)\ **Post date:** [May 10, 2021, 7:42am UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/11 "2021-05-10T07:42:17Z")
xgboost n\_estimators is between 100 adn 1200 verable and increment vaue is 100. ex 100,200,300,400,500,600,700,800,900,1000,1100,1200,1300 ---
**Author:** ![iperez](https://forum-cdn.knime.com/user_avatar/forum.knime.com/iperez/32/24881_2.png) [@iperez](https://forum.knime.com/u/iperez)\ **Post date:** [May 10, 2021, 2:48pm UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/12 "2021-05-10T14:48:43Z")
Hi, check this workflow: [XGBoost Optimized.knwf](https://forum.knime.com/uploads/short-url/fWZUkLEpOu0KbcuEcd5fM41IDzj.knwf) (98.9 KB) The optimization loop minimizes the root mean square edrror by trying different learning rates between 0.001 and 1 with a 0.1 step and varying the number of trees between 100 and 1200 with a 100 step. ![imagen](https://forum-cdn.knime.com/uploads/default/original/3X/1/8/18fd7378724b8b1bd5a164b8aeac0900f6683a84.png) Hope this will get you on your way to tweak this flow to your needs ---
**Author:** ![BerkayAkar](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/b/ed655f/32.png) [@BerkayAkar](https://forum.knime.com/u/BerkayAkar)\ **Post date:** [May 10, 2021, 8:54pm UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/13 "2021-05-10T20:54:28Z")
> [@iperez](#): > > p. thanks for your ansver sir have a nice day 🙂 ---
**Author:** ![kienerj](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/k/ce7236/32.png) [@kienerj](https://forum.knime.com/u/kienerj)\ **Post date:** [May 11, 2021, 12:53pm UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/14 "2021-05-11T12:53:28Z")
@BerkayAkar Please do not use the workflow “as-is”. Any type of parameter optimization like this will lead to a completely overfit model because you are optimizing for one single train/test split. . The model needs to be evaluated by cross-validation for example by taking the median of your target metric over all cross-validation loops. So inside the parameter optimization loop you need a CV-loop. ---
**Author:** ![iperez](https://forum-cdn.knime.com/user_avatar/forum.knime.com/iperez/32/24881_2.png) [@iperez](https://forum.knime.com/u/iperez)\ **Post date:** [May 11, 2021, 1:30pm UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/15 "2021-05-11T13:30:39Z")
Absolutely!!. It was meant as a starting point. Regarding @kienerj suggestion you can check the EXAMPLES Server 04\_Analytics\>11\_Optimization\>07\_Cross\_Validation\_with\_SVM\_and\_Parameter\_Optimization ---
**Author:** ![BerkayAkar](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/b/ed655f/32.png) [@BerkayAkar](https://forum.knime.com/u/BerkayAkar)\ **Post date:** [May 13, 2021, 11:24am UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/16 "2021-05-13T11:24:42Z")
sir ı shuold use cross validation for partiton node is it true ? ---
**Author:** ![BerkayAkar](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/b/ed655f/32.png) [@BerkayAkar](https://forum.knime.com/u/BerkayAkar)\ **Post date:** [May 13, 2021, 11:28am UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/17 "2021-05-13T11:28:20Z")
and ı have a question sir can ı use table row to verabe loop start ? beacuise parameter optimization loop method is too long for best parameters so ı want to use this method and ı want to use cross validation in front of learner and predictor nodes is it posibble ? becauise i couldn’t it ![image](https://forum-cdn.knime.com/uploads/default/original/3X/d/5/d540c3537aa42be460421c35ee08f77610e36064.png) ![image](https://forum-cdn.knime.com/uploads/default/original/3X/6/2/62ea10c74b153146228191cd3f2615f7d6b66e5d.png) is it posibble ? and wht ı use table row to verable node ? why ? ---
**Author:** ![BerkayAkar](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/b/ed655f/32.png) [@BerkayAkar](https://forum.knime.com/u/BerkayAkar)\ **Post date:** [May 13, 2021, 11:39am UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/18 "2021-05-13T11:39:17Z")
how can ı use cross validation in front of the learning node ? ı can’t make it sir ---
**Author:** ![kienerj](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/k/ce7236/32.png) [@kienerj](https://forum.knime.com/u/kienerj)\ **Post date:** [May 13, 2021, 1:03pm UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/19 "2021-05-13T13:03:41Z")
As @iperez already answered beforehand, see the examples workflow: ![image](https://forum-cdn.knime.com/uploads/default/original/3X/9/9/994d5a18329588175efed5550c9bee5e7b092c1c.png) And here the content of the workflow: ![image](https://forum-cdn.knime.com/uploads/default/original/3X/1/e/1e3f9677d6d6656b1e3439fcf2cbfea853bb52a4.png) This of course needs to be further extend as all feature selections steps must also happen only on the training set. Advanced topic: I must also add that in case of parameter optimization I generate the cross-validation splits just once and apply them via reference row filtering and group start. The reason for that is that the X-Partitioner is very slow as in that it repeats the row splitting for every iteration which significantly impacts runtime. ---
**Author:** ![BerkayAkar](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/b/ed655f/32.png) [@BerkayAkar](https://forum.knime.com/u/BerkayAkar)\ **Post date:** [May 13, 2021, 1:08pm UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/20 "2021-05-13T13:08:44Z")
thanks for giving advice ı have a problem I am developing my project both at sklearn and knime but my values in knime are much higher than my values in sklearn What do you think is the reason ? pretreatments and data are the same ![image](https://forum-cdn.knime.com/uploads/default/original/3X/3/d/3d12b940098e2b765503017ed727355cd8778ea0.png) square is 0.35 but sklearn giving 0.22 cv = 5 and k = 5 ---
**Author:** ![Daniel\_Weikert](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/d/e47c2d/32.png) [@Daniel\_Weikert](https://forum.knime.com/u/Daniel_Weikert)\ **Post date:** [May 13, 2021, 1:22pm UTC](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823/21 "2021-05-13T13:22:03Z")
You probably do not use the same seeds (random states). Also there might be a slight difference on how default parameters are set in scikit vs knime. If a loop takes to long use random search and not grid search. br [Next page](https://forum.knime.com/t/xgboost-regression-hyperparameter-optimization/32823.md?page=2)