Parameter optimisation bug

Hi,

When using knime 2.9.4 and the parameter optimisation loop start and end nodes there seems to be a bug when HillClimbing is selected. There appears to be a complete disregard for the defined step intervals (which I have defined by a flow variable) in which the algorithm seems to allow any possible step values, however the from and to ranges are obeyed. Switching to brute force, and the step intervals are now obeyed.

On a separate point, less a bug, but more a limitation with HillClimbing method, is that the algorithm completes as soon as no iteration improves the objective, wouldn't it be more ideal to have a user defined fail limit of how many consecutive attempts to make to improve the objective before it completes. I have had it fall woefully short sometimes of what the optimal objective value can be with brute force. I have typically been using 9 variables with three possible values each. 

Simon.

1 Like

I don't fully understand your problem. The stepsize passed in as a paramater is obeyed also with hillclimbing. Can easily be seen by specifying different step sizes (e.g. 0.01 and 0.1) and then the number of iterations should change significantly (which it does). Or did you mean something else?

On your second question: this is how hill climbing works. You go into the direction of largest increase until you don't find any better solution in the neighbourhood. This is faster than brute force but of course you may end up with a local optima. This is the drawback of hillclimbing.

I had a student working on additional search methods such as tabu search or particle swarm optimization. I need to look up that code at some point in time and see whether we can integrate it. This may help in your case when hillclimbing misses the global optimum.

Hi Thor,

I am finding that for me anyway, when using HillClimbing, the stepsize is not obeyed.

i.e. I have got the stepsize set using a variable which is 0.3.

However, when I a look at the iterations in each loop, it is not using steps of 0.3, it is doing things like 0.2945453, and 0.275255 etc.

If I use Bruteforce, then it is always using 0.3 steps!

Simon.

Then something else is wrong. It has to use at least some fixed step size, it certainly doesn't change it during the loop. Can you construct a simple workflow?

Seems to show a problem on just the simplest setup.

Just create a workflow with a Paramter Loop Start and End node connected to each other.

Add 3 parameters, which I called "New parameter1", "New parameter2", "New parameter3".

Run the loop start node, and look at variable output and it looks fine for BruteForce.

Change it to HillClimbing, and then I get :

0 0:88:280 New parameter3" (DOUBLE: 0.13578759831500553) 0.13578759831500553
0 0:88:280 New parameter2" (DOUBLE: 0.42926194288566333) 0.42926194288566333
0 0:88:280 New parameter1" (DOUBLE: 0.7030876687664309) 0.7030876687664309

 

Ah, I see. With hillclimbing the start position is assigned randomly. The parameters are within the bounds but not necessarily aligned to the "step size grid".