despite enabling the setting to reset all workflows prior to uploading them, therefore not uploading the temp-data, attempting to sync the local workspace between two PCs using the private hub fails:
Upload of 'LOCAL:/' to 'My-KNIME-Hub' failed: complete size (582 MB) exceeds upload limit (50 MB). Please split upload of workflow groups into multiple smaller uploads.
I assumed this limit is per workflow but it rather seems to apply to the entire “upload job” because locally exporting the workspace results in an identical file.
Are there any recommended best practices? I have seen a few posts and have some ideas to automate the sync of a workspace or workflow across devices myself.
Hub limits each “upload job”. So for the time being, you would have to “chunk” your uploads as you suspect.
However, we are currently working on a new uploader (and downloader) which does not have the limit per “job”, so you should be able to transfer large workspaces to Hub soon (if your quota allows it, of course).
thanks for the intel. Are you considering parallel up-/downloads? So i.e. instead of compressing the entire workspace, do it workflow wise, which would give the option to skip workflows which are open, reduce the system strain, leverage multicore for compression and as well the network throughput.
It might even ease the strain on the Hub server because of smaller more incremental uploads. Ah, here is another idea … a queue … local to server or, if the server is “too busy” also from server to local (similar to 429 to many requests).
Upload should already happen in multiple chunks now if you select more than one workflow group. A single workflow group will result in a single upload.
