since three weeks I have been implementing an optimizer node which minimizes or maximizes an objective function. This optimizer provides different algorithms for optimization, e.g. the Nelder-Mead-Downhill algorithm. The output of my optimizer node is a vector of optimized parameters x and the input should be a vector of objective function values f(x). After every internal iteration by the algorithm I would like to request a new objective function value vector depending on the provided parameter set.
How can I notify the next connected node out of a working node about the change of the input data?
I have been working with KNIME since three weeks. So, I am not familiar with the inter node communication. It would be very nice, if someone had an idea how I can solve this problem.
I’m not sure if you can model this kind of explicit inter-node communication in KNIME - but even if it was technically possible I would suggest to move the loop outside of your node into the workflow. (Using the existing loop nodes within KNIME.)
Your node would perform only a single optimization step in each execution and generate a set of different x for which the objective function f(x) is then evaluated by another node. Your optimization node would then receive the set of (x, f(x)) values and generate a new set of x’.
This way, the user has the freedom to implement his own objective function in any node and has also the freedom to implement his own convergence criterion.
Does that make sense?
yes, I also thought about the possibility you mentioned. Relating on the concept of KNIME, it would be the best solution.
The problem is that during the execution of a single optimization step using nelder-mead-simplex, new simplex points have to be evaluated by the objective function. Normally you would allocate a pointer to the objective function wrapper, which returns the requested function value depending on the parameters. That's why I asked for a method actualizing the workflow, which generates the objective function values.
Do you know about other optimization algorithms, that can be realized like a black-box using a static input for generating an optimum?
Does someone else have experiences realizing parameter estimation in KNIME?
What about genetic algorithms or simulated annealing? One could implement a "cross-over node" and a "mutation node" that take a population of vectors as input and generate a new population of objects. Scoring and selection could probably be implemented using existing nodes.