Saving all models when using X-Partitioner

[DataDudes]

Hello,

I am trying to save all models from an X-Partitioner loop. What would be the best way to do that? the way I set it up saves only the last one.

Thanks!

[aworker]

Hi @DataDudes and welcome to the KNIME forum!

One way to save the models within a loop is to convert them at every iteration into an object using the “PMML to Cell” node:

PMML To Cell

or the “Model to Cell” node:

Model to Cell

Once a model is in cell format, you could add it into an extra column which is fed with a new model at every new loop iteration. Once all the iterations are finished, you should have a column with all the models saved. All these models can thus later be saved as a KNIME table using the table writer node:

Table Writer

You could at any time, read this table with your stored models using the Table Reader node:

Table Reader – scottF

And at this stage, using a new loop, convert them again into PMML format using the cell to PMML node:

Cell To PMML

or the Cell to Model node:

Cell To Model

and finally execute them with their associated predictor. This procedure works for most of the models, regardless of the algorithm you are using to generate the model (here XGBoost Tree Ensemble Learner).

Here below a possible implementation:

B.t.w. this is just a possible solution, but there are a few others …

Hope this helps.

Best

Ael

[aworker]

Hi @DataDudes

I’m adding a workflow as an example of what I described in my last post:

20210628 Pikairos Saving all models when using X-Partitioner.knwf (606.4 KB)

Best

Ael

[DataDudes]

Thanks so much for your detailed answer and for sharing the workflow! I will test it and let you know how it worked. I also wonder if there’s a way to run a loop that saves models using the original model writer by changing the path of the model file.

[aworker]

My pleasure.

@DataDudes

I also wonder if there’s a way to run a loop that saves models using the original model writer by changing the path of the model file.

Please find a modified version which implements too the writing of the current model at each iteration:

20210629 Pikairos Saving all models when using X-Partitioner.knwf (732.3 KB)

It adds too the local path with the name of the models to the final table with models, in case you would need the names too.

Hope this helps too

Best

Ael

[kienerj]

Create a new file name inside the loop with string manipulation (variable) or java edit variable node. Use the iteration flow variable as part of file name: “model_iteration_” + iteration.

Then you get as many files as iterations. Question is why do you need all the models? It doesn’t make that much sense?

[aworker]

Thanks @DataDudes for validating the solution !

All the best,

Ael

[Daniel_Weikert]

@kienerj might have a point. Maybe you have a particular reason. Do you want to elaborate on this? I am just curious.
Kudos to @aworker
Whenever I see X-Partitioner I still think of X-Aggregator as the only closing option so thanks for your workflow.