CDDS Transformation workflow is not generated

Hi KNIME team,

I am experimenting with CDDS based on the L3-DE course. I followed the instructions in the course by creating the Training subfolder and using Integrated Deployment to capture the transformation workflow. Afterwards I submitted it into the 01 - CDDS Development space.

If I understand the concept correctly, the captured transformation workflow should be generated under the Production subfolder automatically by CDDS in the 02 - CDDS Training space and the subsequent spaces (03 - CDDS Validation and 04 - CDDS Production space).

However, I am not able to find the subfolder and the workflow.

Can you please let me know, whether I miss any important step or requirement?

My workflow is stored here: Sign in to KNIME Business Hub for Education

Thank you very much in advance.

Hi hanantoprabowo,

great that you are experimenting with CDDS!

I’m judging from your screenshots that you looked in your private spaces—CDDS is not set up there, though, but in the L3-self-paced team:

There I can find your workflow here and here. Does that help? Is that in line with what you expect?

Kind regards,
Lukas

1 Like

Hi @LukasS

thank you very much for your reply :slight_smile:

My workflow name is CDDS Workflow hanantoprabowo

What I’m looking for is the Production subfolder in the 04 CDDS - Production space: Sign in to KNIME Business Hub for Education

But I’m not able to find one :frowning:

Best regards,

Hananto

Hi @hanantoprabowo,

ah yes, sorry, now I wrapped my head around things (thank you @Lada for helping!). Could you try to set the Parameter Name in the Workflow Service Output Node of your Training Workflow to prediction-workflow?

It seems like the documentation is missing this detail—if that works for you, I’ll fix that.

Kind regards,
Lukas

Hi @LukasS ,

Thanks both of you for your support

I did the change as you mentioned

Afterwards:

  • I updated the workflow in the 01 CDDS - Development space
  • submitted it for validation
  • approved it to production

Still, I’m not able to see the Production subfolder in the 04 CDDS - Production space

I tried to run the example mentioned in the link you posted (Level_2_Fraud Detection with ID) and it created the Production subfolder successfully. I’m wondering whether the captured workflow segment has to contain ML related nodes or can it be a normal ETL workflow? (in my workflow, I am NOT using any ML related node (e.g. Partitioning, Learner, Predictor, etc.)

Thank you very much for your help in advance.

Best regards,

Hananto

Hi @LukasS ,

I have a new finding:

  • If my folder only contains the Training subfolder, the Production subfolder is created after submitting it to validation

  • If my folder contains the Validation subfolder (for unit test), the Production subfolder is NOT created after submitting it to validation

So I assume, it has something to do with the Validation subfolder. I tried to run the unit test workflow under the Validation subfolder and it was executed successfully in Business Hub


Is it something I overlooked with regards to the Validation subfolder or its underlying Unit Test workflow? or is it not allowed to have the Validation subfolder at all?

Thank you very much in advance

Best regards,

Hananto

Hi @hanantoprabowo,

ok, one step closer with the correct parameter name :slight_smile:

  1. So long the training workflow contains the integrated deployment nodes and the Workflow Service Output at the end (with the parameter name now correctly set), it does not matter which kind of nodes are used in the rest of the workflow. A normal ETL workflow will work just fine.
  2. We are still investigating—the current working hypothesis is that the production workflow wasn’t copied in time to the location where the custom test can access it.

We’ll keep you posted!
Kind regards,
Lukas

2 Likes

Hi @hanantoprabowo,

hmm, the issue is still elusive: I used your example as a base for my experiment, and copied a folder Lukas test 1 (which contains the same data, Training, and Validation folder as you used in your project CDDS Workflow with validation hanantoprabowo) to 02 CDDS - Training > Queue.

After a little while (2 minutes or so) I see the folder structure including a Production folder in 03 CDDS - Validation > Pending.


I tried this a few times, and each time it did create the Production Folder with the prediction-workflow as expected.

Did I maybe miss an extra step you where taking?

Best regards,
Lukas

1 Like

Hi @LukasS ,

it is elusive indeed :sweat_smile:

What I did in the past was I published the artifact into 01 CDDS - Development folder as mentioned in the L3 course. I did the same few minutes ago and still experienced the same issue.

I tried to publish it to 02 CDDS - Training > Queue folder as mentioned in your post, but I received an error message: The selected folder is not writable. Choose a new location.

I’m wondering:

  • which folder the artifact has to be published into? 01 - CDDS Development or 02 CDDS - Training > Queue?
  • In case of 02 CDDS - Training > Queue, maybe I’m lacking of permission to be able to write into the folder?

Thank you very much in advance!

Best regards,

Hananto

Hi @hanantoprabowo,

we found something on our end, and we identified an issue with the User Data App, where your project is pushed directly to Validation, skipping the Training step, if there is the Validation folder in your project—thus not creating the Production folder. Nothing wrong on your end :slight_smile: I’ll see to fix it and will come back to you!

Best regards,
Lukas

2 Likes

Hi @LukasS,

what a good news before weekend :smiley:

Thank you very much and I’m looking forward to the solution!

Best regards,

Hananto

1 Like

Hi @LukasS,

Do you maybe any news whether the issue has already been fixed or still work in progress?

Thank you very much and have a great day!

Best regards,

Hananto