I have a loop which worked fine when it was the last step in a long series of processes. I utilized the heaviest weighted variables output from the Random Forrest Regression and the R-Squared output from the Numeric Scorer to confirm some theories.
The problem occurs when I attach a multitude of steps trying to automatically use those outputs downstream. If I leave the first Math Formula node (for deriving the heaviest weighted variables) attached to the RFR node and the Numeric Scorer to the Row Filter, the loop fires every step down stream causing a train-wreck. If I run the loop and manually then manually attach the down stream nodes I get error messages about leaving the loop.
apologies for the late response.
One important thing to know here is that all downstream nodes that are connected to any node within the loop body (e.g., the Random Forest Learner or the Numeric Scorer) will be executed during every iteration of the loop. For example, the sequence of Math Formula nodes will be executed several times during the Cross Validation loop. However you would need a loop end node to collect the results of each iteration. Right now you would only receive the result of the last iteration (for everything up to the String Manipulation node). Same goes for the Numeric Scorer. You would only receive the value for the last iteration of the Feature Selection Loop. Connecting the two different loop executions by using the Column Appender breaks the workflow in the end, because the inner loop wants to run the Column Appender as well. However, this won’t work as the Numeric Scorer cannot be executed as the inner loop is not finished.
What I usually do in this cases - where I want to have intermediate results from within loops - is that I use a String Manipulation (Variable) node to generate a file name using the currentIteration flow variable and then write the results of each iteration into files and load them again after the loops have been finished.
I hope this helps. If there are any further questions feel free to reach out again.
