Tuning of XGboost

Dear everyone
Applying XGboost to an imbalanced dataset, did anyone ever encounter a method for tuning the model and finding the sweetspots for the parameters :

max_depth =
gamma =
subsample, colsample_bytree =
scale_pos_weight =

I would love to hear your ideas

Hi @stoffer88 -

Have you tried the Parameter Optimization Loop nodes in KNIME for this type of task? There are several examples of their use available on the Workflow Hub, but this is probably a good one to start with.


Since you’re interested in finding sweet spots for several different hyperparameters, I’d go with the Hillclimbing strategy to save time.

1 Like

Dear Scott…
Thank you for the good and correct solution !

I tried what you explained and I keep on getting this error: ERROR Gradient Boosted Trees Predictor (deprecated) 0:82 Execute failed: (“ArrayIndexOutOfBoundsException”): null

A lot of trial and error has been conducted tonight and i’m unsure why it states this. I tried both brute force scenario and hillclimb…

1 Like

Not sure why that’s happening off hand. Are you able to provide a minimal example workflow that reproduces the problem?

Also, is there a reason you’re using the older deprecated version of the node, instead of the more recent version?

1 Like

Hey Scott

I just changed the node and it’s still the same scenario. I’m unfortunately not in a position where I can upload data. However I just tried copying in the same ‘param loop start’ and ‘param loop ending’ into the example workflow and set the XGBoost up the same way. Here it doesn’t fail.

Can it be an issue in my dataset?

That’s my guess. Let me ask some folks internally about the error message and see what I can find out.

Hey Scott

Thank you so much. It really is quite difficult to figure out

Dear Scott

Any news?

Hey again… I actually managed to make a workflow and reproduce the problem in a dataset you are allowed to look at :slight_smile: would you be OK with me e-mailing this to you?

best, Christopher

Hello @stoffer88 -

Sorry, I had a bit of travel scheduled and have been away from the forum for a couple of days. Please feel free to email the workflow to me at scott.fincher@knime.com.