Dear Knimers,
I tried to copy the workflow from our TEST server to our PROD server instance using scp in order to keep the snapshot.
The workflow is copied in the PROD environment with the snapshots but the workflow is not visible in the workflow group even if I click on the refresh button in KNIME Analytics Platform.
What should I do in order to display this workflow on the KNIME server in the KNIME client.
Regards
Joel
Hi @joel,
The KNIME Server uses it’s own file and permission system on his own workflow repository. Therefor it only recognizes changes on and new created files if they are either done by a workflow running on the KNIME Executor or initiated via the KNIME Server mountpoint within the KNIME Explorer of the Analytics
Platform.
A KNIME Server restart would be needed to force the KNIME Server to recognize new and changed files that were initiated directly on the KNIME Servers file system. I will check with my colleagues internally if there is a better way to deploy workflows from TEST to PROD while keeping the snapshots.
Best,
Michael
Hi @joel . While I would still want to know what @MichaelRespondek comes back with (I’m guessing it may have to do with the metadata file), I was wondering why you were copying the workflow that way.
You could connect to the PROD server directly from the KNIME client and drag the workflow to the PROD server within the KNIME client itself.
Depending on your setting and firewall, you can either connect via your local KNIME client, in which case you would need to connect to both the TEST and PROD server within your local KNIME client and then drag the workflow from the TEST server to the PROD server, or you can connect via the TEST server’s KNIME client, in which case you would need to connect to the PROD server within that KNIME client and drag the workflow to the PROD server.
Alternatively, you could export the workflow from the TEST server, and then import the workflow into the PROD server (export and import via KNIME client to a .knwf file).
EDIT: Just to expand about what I said about connecting to the PROD server via Knime, take a look at your KNIME Explorer, there should be the EXAMPLES connection by default. When to create a connection to your PROD server, the PROD server will be added to your KNIME Explorer where you can copy workflows by just dragging the workflows.
This is how the KNIME Explorer of my local KNIME client looks like:
In my case, I’m also connected to my Knime Hub space, and I can drag my workflow directly within the KNIME client from my local to my Knime Hub space. The same thing can be done with the PROD server in your case.
The other thing with scp is that it only adds and overwrites. It does not delete. For instance, if you are updating an existing workflow and you happen to replace or remove some nodes, physically, all nodes are stored in sub folders, and deleting or replacing some nodes will result in their folders be deleted. With scp, you won’t be able to apply these changes and could create unexpected results (you would usually want to use rsync instead of scp in this case as rsync can sync by also removing files/folders - but I would still advise against copying that way).
Hi @bruno29a ,
I am not able to drag a workflow from the TEST mounpoint server to the PROD mountpoint server using KNIME Analytics platform.
I am copying the workflow that way to keep the .metadainfo folder which contains the permission and properties files but especially the snapshot folder.
Regards,
Joel
Do you think that I could use REST API ?
I mean put method of the following endpoint ​/rest​/v4​/repository​/{path}:data.
Regards,
Joel
Hi
I tried to use the REST API to upload the workflow from the TEST server to the PROD one.
Before to execute the API, I created a zip file related to the workflow containing the .metainfo folder.
I executed the API and the workflow has been created on the PROD server but the subfolder snapshot has not been uploaded on the PROD server whereas the .metainfoder was.
Regards,
Joel
@joel So I understood it right that you want to transfer one workflow from the TEST to the PROD server while keeping the snapshots (versioning) of it.
I checked this with our development team and this is only possible via copying the workflow folder as you already did. To recognize the objects and set the inherit KNIME Server permissions on it a KNIME Server restart is needed. You could also copy the files over after stopping the KNIME Server Prod and start after finishing the transfer.
All other ways would not transfer the snapshots, which is often also not wanted as these could contain credentials from the testing on the dev/test environment. If you want to keep them there is only the way to archive them using different workflow names and transfer them using a KNIME Analytics Platform. Beside that you can use REST to download the recent workflow or use the KNIME hub.
Best,
Michael
You assumption was correct and thanks for the explanations.
Now, I have to think what will be the best way to work because restarting the PROD server is something that I would like to avoid the most possible in order to not impact the web portal users.
Regards,
Joel
Note that I don’t know about which amount of workflows we are talking, this possible solution just came to my mind:
What you could do is using the snapshots on the TEST server to download the versions to your Analytics Platform you want to have available on the PROD also by choosing “Download Snapshot” in the context menu. Beginning with the oldest to keep you can then deploy the downloaded snapshots one by one to PROD using the same target name and using “Create Snapshot”.
Just take care that you don’t damage your workflow and it’s history on TEST by replacing it with a snapshot.
As the deployment on PROD is done using the KNIME Server itself a restart is not needed.
Cheers,
Michael
