This might be a simple question but I am new to Knime and I am having problems with a nested loop. Here is a screenshot of my workflow:
I am creating some clusters with k-means and then I want to run the gradient boosted trees model for each cluster. I have the Table To Row Variable Loop to read a file that has the cluster labels so that I can filter the data for a particular cluster (if you know a better way to do this, please share). Then, I pass that data to the model, where I want to use Parameter Optimization. The error on the Learner node reads: Can’t merge FlowVariable Stacks! (likely a loop problem.)
That worked perfectly, thanks a lot!
I have a follow up question regarding this loop: right now with the setup that you suggested I can only see the collected results from the Parmeter Optimization, but I also want to see the Scorer results for the iteration with the best parameters. I tried connecting the scorer to a 2 ports Loop End but this was not succesful (I get the error: Branches are not permitted to leave loops!). Is there a way to do this?
there are two things that you could do
Retrain the model with the best set of parameters (if you set the static random seed in the learner node you will get the exact same model again)
Write the output table of the scorer node in each iteration with a table writer node and a changing file name. Then read only the table of the best parameter set.
If you want to continue with the second approach, please let me know and I can build a small workflow for you.
I would like to continue with the second option since this will also serve me later. I would like to store the stats for the model trained with the data from each machine in a cluster individually and then do an analysis of those results with the results of the model trained with the data of the entire cluster.
I would appreciate it if you can help me building a small workflow.
Thanks so much for your help,
attached is a little workflow. The idea is that we create a flow variable with the output location inside the loop. The file name contains the current parameter settings, in my case the number of models, and the current cluster, so that the output location is different in every iteration.
Example_Write_Table_Inside_A_Loop.knwf (435.5 KB)
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.