Hi,
I have a gradient boost regression model and i want to learn how can i overfitting contol for my model?
Best Regards,
gökhan
Hi,
I have a gradient boost regression model and i want to learn how can i overfitting contol for my model?
Best Regards,
gökhan
Hello @gokhan_sir,
I assume you are using the Gradient Boosted Trees Learner (Regression) node?
This node essentially provides you with three parameters that can help to control overfitting:
My recommendation is to keep the tree depth in the interval 3 to 5 and play with different configurations of learning rate and number of models. Note, however, that there are no general best settings (otherwise, they would be our defaults ) and you will have to try and see what works for your data.
Best,
Adrian
Hi Adrian,
Thank you for reply.
Tree Depth : I am using 4 tree depth as you recommend
Number of models: Default settings is 100. Is it ok ? or can i use different number ? I used 100 models and our model R^2 is 0,913 - than used 1000 models and our R^2 is 0,93 and also have some root mean square error.
Learning Rate : i am also using defaults value (0,1).
When i changed learning rate between 0,05 to 0,5 - R^2 changed between 0,94 to 0,90. This mean our data is normal ?
I am the new gut for this machine learning algortihms . If i ask very simple and dummy question, i will be very sorry
Best Regards,
Gökhan
Hello Gökhan,
I can’t tell you whether your data is normal as I am not sure how normal would be defined in this case.
However, a larger R^2 means that your model explains the variation in the data better, so a smaller learning rate seems to benefit your model perhaps because it might allow the algorithm to make finer distinctions.
Kind regards,
Adrian