Applying optimization to a fitted machine learning model

Is it possible to do an optimizer in KNIME with ML models as objective function?

What tasks should it solve:
For example, we built an ML model.
Next, we want the model to give us the optimal parameters to maximize objective function.

Here’s an example of how it’s done in Python: a sklearn ML model is taken and put into the optimizer from the SciPy library:

Perhaps there is already something like this, or it can be easily wrap the same Python libraries and use them to make an optimizer, which can be used for knime projects.

Hi @sha367,

for this task you can use a parameter optimization loop.

Here you can find an example workflow on the KNIME Hub: 2 Examples for Parameter Optimization Loops – KNIME Hub

And here a short video about the parameter optimization loop: Parameter Optimization Loop - YouTube


This node (parameter optimization) usually used for best hyperparameter search using brute force or random search (like in Python sklearn library: grid search and random search). But when we have nonlinear continuous function like machine learning model (as a function of variables, not hyperparameters) we need more complex algorithm like genetic algorithm, BFGS, Generalized Reduced Gradient, etc

ah sorry I misunderstood your task. Just to double check that I understand your goal correctly :slight_smile:

For example you have trained a Regression Tree. Then your goal is to find the values for your input features that lead to the highest output.

Is that correct?


Yes, that is correct

hmm, as far as I know there is no node for that.

To solve this task in KNIME I would use a parameter optimization loop, with one parameter for each input feature of the model and a defined range. In the loop body I would convert the flow variables into a table, apply the model and use the predicted value as the objective value to maximize. Maybe that leads already to good results with one of the available optimizers (Bayesian Optimization (TPE), hill climbing or grid search (maybe to time consuming))

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.