Nested ML with hyperparameters tuning

Hello,
I would like to run the attached nested cross-validation with hyperparameters tuning but I have an error to make it run.
Could you please help me to spot the issue?
Thanks,
Zied
ML2021.knwf (3.2 MB)

This appears to be because both the X-Aggregator and the Parameter Optimization Loop Start nodes inside your metanode both create a flow variable called iteration with an initial value of zero.

Is it possible to redesign your nested loop structure to avoid this problem? It seems fairly complex.

4 Likes

I was based on the solution proposed in the question below:

Ah. I think, in that case, @HansS has already posed the solution within that thread:

Basically, convert your metanode into a component (we don’t call them “wrapped metanodes” anymore). I just did a short test using your workflow posted above, and it seemed to work, or at least made more progress than it was before :slight_smile:

3 Likes

@ScottF @HansS @lisovyi
I got rid of the metanode and the component and I’ve put all my nodes in the same workflow.
I still have the error on the X-partitioner of the inner loop:
“can’t merge flowVariable Stacks! (likely a loop problem.)”
I attach here the updated workflow.
ML2021v2.knwf (3.2 MB)

Why did you put all your nodes on the same level of the workflow, instead of converting your previous metanode into a component as suggested before? As I mentioned, that seemed to work for me (although I didn’t do elaborate testing)…

1 Like

@ScottF , I tried what you suggested but I still get the same error , so I removed the component and I have the same error.
Do you see it running for a few iterations at least? In my case, it does not work from the first iteration.

Yes, I’ve attached a version with the folds and iterations dialed way back to show that the loops work.

I got some errors along the way with a few nodes because they were expecting a variable (target) that doesn’t exist in your dataset. (This is probably a domain issue arising from copy-pasting the workflow that originally used a different dataset.) But to fix those, all I had to do was open the configuration of the affected nodes and click OK, then it was fine.

See what you think:

ML2021_SF.knwf (3.2 MB)

You may be thinking about the component conversion, “why would that work?” It has something to do with how components treat flow variables, covered in more detail here: KNIME Components Guide

3 Likes

I don’t really get the idea of the inner CV loop. What exactly is it doing? Can you elaborate on that? Where do you optimize the rate?
THanks

@ScottF @Daniel_Weikert
The inner loop is doing the hyperparameters tuning using the sam algorithm with cross validation.
The outer loop will repeat the tuning for all fragments of the cross validation.
I used a workflow that was posted in the forum. People used the normalisation and the normalization (applu) node. In my test set I have some values that are out of range, so I got some warnings. Should I ignore that or I have to use another notmalisation node for the test set like the one in the training set?

Thanks,
Zied

Warnings are no errors. I have not seen the parameter optimization value actually applied in the inner cv as variable but maybe it was to late for me yesterday.
Thanks for answering
br

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.