Tuning of XGboost

KNIME Analytics Platform
1

Dear everyone
Applying XGboost to an imbalanced dataset, did anyone ever encounter a method for tuning the model and finding the sweetspots for the parameters :

max_depth =
min_child_weight
gamma =
subsample, colsample_bytree =
scale_pos_weight =

I would love to hear your ideas

2

Hi @stoffer88 -

Have you tried the Parameter Optimization Loop nodes in KNIME for this type of task? There are several examples of their use available on the Workflow Hub, but this is probably a good one to start with.

https://workflows.knime.com/knime/hub/workflows/04_Analytics%3A11_Optimization%3A06_Parameter_Optimization_two_examples

Since you’re interested in finding sweet spots for several different hyperparameters, I’d go with the Hillclimbing strategy to save time.

1 Like
3

Dear Scott…
Thank you for the good and correct solution !

I tried what you explained and I keep on getting this error: ERROR Gradient Boosted Trees Predictor (deprecated) 0:82 Execute failed: (“ArrayIndexOutOfBoundsException”): null

A lot of trial and error has been conducted tonight and i’m unsure why it states this. I tried both brute force scenario and hillclimb…

1 Like
4

Not sure why that’s happening off hand. Are you able to provide a minimal example workflow that reproduces the problem?

Also, is there a reason you’re using the older deprecated version of the node, instead of the more recent version?

1 Like
5

Hey Scott

I just changed the node and it’s still the same scenario. I’m unfortunately not in a position where I can upload data. However I just tried copying in the same ‘param loop start’ and ‘param loop ending’ into the example workflow and set the XGBoost up the same way. Here it doesn’t fail.

Can it be an issue in my dataset?

6

That’s my guess. Let me ask some folks internally about the error message and see what I can find out.

7

Hey Scott

Thank you so much. It really is quite difficult to figure out

8

Dear Scott

Any news?

9

Hey again… I actually managed to make a workflow and reproduce the problem in a dataset you are allowed to look at :slight_smile: would you be OK with me e-mailing this to you?

best, Christopher

10

Hello @stoffer88 -

Sorry, I had a bit of travel scheduled and have been away from the forum for a couple of days. Please feel free to email the workflow to me at scott.fincher@knime.com.

Thanks!