Current workflow attached. If I change the Copy/Move Files node to Transfer Files node, I’d like to be able to input to the node so I could use a loop of file names in the future. Thanks.File Transfer Test 1.knwf (60.4 KB)
Mark:
Have you had time to look at my problem?
Sry somehow lost track. Gonna look into it on Monday.
Enjoy your weekend,
Mark
1 Like
@rfeigel your workflow says you want to move the zipped files from the server/remote location to your local file system. Is this a KNIME server? Do you need the zipped files on your local FS? Do you want to delete the archives on the server?
Here you can find some documentation on the new framework. https://docs.knime.com/2020-12/analytics_platform_file_handling_guide/index.html
Mark
1 Like
This File Handling guide is a great addition to a quest in understanding how to work with KNIME. Maybe not always obvious from the start but once you really start working with KNIME it becomes essential to know where everything is stored and how to access it - I often see people puzzled where their stuff ends up; better to address this up front even if it is somewhat technical.
5 Likes
@Mark_Ortmann @rfeigel on another note:
The logic and handling of the new path variables in accordance with the KNIME protocol is not always straightforward. I have put together some (new) examples how to handle (local) files with them - not everything seems to work and some things are not consistent (eg the old Excel Reader access a URI as path variable while the new one does not).
You have to toy around with these things, and it would be further complicated if you add ZIP / UNZIP functions to the fold (I assume). I would take a look path them also.
4 Likes
The “Server” is not a Knime server. It could be a corporate server or simply another folder on my PC. My question is simple. How do I replace the current Copy/Move Files node with the new Transfer Files node in my current workflow?
here is the workflow File Transfer Test 1.knwf (18.1 KB).
I was asking those question in order to create a proper solution, but anyway here is one that works.
- Currently transfer files does not support table inputs, we are working on that.
- I removed the file compression. I assume you used it to save time when downloading, but actually in order to compress those files you have to download them. The compression is not being carried out on the remote machine.
Happy holidays,
Mark
4 Likes
Hi all - Happy New Year!
I’ve been playing with using the @Mark_Ortmann (thank you!) workflow to create a way of backing up my knime-workspace.
I realised that I obviously needed to take out the actual workflow itself (
) but I also need to remove the .metadata folder.
It appears to work OK but will removing the metadata folder cause me any problems when using the backed up folder to re-instate workflows I’ve messed up?
Cheers
Zx
Update: the workflow fails on certain files. I remove the folder (via the filter) containing the file and the loop continues then fails on another file.
First time its a String to Date_Time node, second time its a Joiner
Here’s the console:
Here’s the log, if anyone has any suggestions very much appreciated. It’d be great to get this working as until we have Knime server (this month I hope) all my work is on my hard drive and I use this workflow to transfer to a OneDrive folder for a backup.
2021-01-06 17:34:13,817 : ERROR : KNIME-Worker-216-Transfer Files 0:1 : : Node : Transfer Files : 0:1 : Execute failed: C:\Users\leXXXXXXXXX\00 Knime\MAGASIN workflows\Magasin_Orig_Dev\Joiner (#26)
java.nio.file.FileAlreadyExistsException: C:\Users\XXXXX\DOCUMENTS\000 Workspaces\00 Knime\MAGASIN workflows\Magasin_Orig_Dev\Joiner (#26)
at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:81)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:504)
at org.knime.filehandling.core.connections.local.LocalFileSystemProvider.createDirectory(LocalFileSystemProvider.java:233)
at java.nio.file.Files.createDirectory(Files.java:674)
at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781)
at java.nio.file.Files.createDirectories(Files.java:727)
at org.knime.filehandling.core.connections.FSFiles.createDirectories(FSFiles.java:165)
at org.knime.filehandling.utility.nodes.transfer.PathCopier.createDirectories(PathCopier.java:218)
at org.knime.filehandling.utility.nodes.transfer.PathCopier.copyDirectories(PathCopier.java:253)
at org.knime.filehandling.utility.nodes.transfer.PathCopier.copyPath(PathCopier.java:237)
at org.knime.filehandling.utility.nodes.transfer.TransferFilesNodeModel.copy(TransferFilesNodeModel.java:177)
at org.knime.filehandling.utility.nodes.transfer.TransferFilesNodeModel.execute(TransferFilesNodeModel.java:139)
at org.knime.core.node.NodeModel.executeModel(NodeModel.java:576)
at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1245)
at org.knime.core.node.Node.execute(Node.java:1025)
at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:558)
at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:95)
at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:201)
at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:117)
at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:334)
at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)@zedleb I created and example where the KNIME workflow (or workflow group) is being zipped into a file with a current timestamp. And can later be restored to a (separate) folder.
This restore can either be automated also or you could do it manually if you need it.
Please note you will have to configure the Path variables and workflow names to your needs. I use local paths that will be created dynamically.
3 Likes
While this is cool it would be great if KNIME offered such an option by default. At least the possibility to reset other workflows inside a knime workflow would help a lot. Backup with data can become a space/time problem specially if backup up over network
1 Like
Concerning Reset there is a node that would allow a reset at the start of a workflow. I agree a ‘remote’ reset would be good. The KNIME server offers the possibility to reset workflows before they are scheduled to run.
I tried combining @mlauber71 and @Mark_Ortmann ideas
Putting the end of the loop after the compression node.
It works really well!
Haven’t tried restoring yet though.
Restoring not working 
No indication what the error is.
I made sure to delete the workflow from my workspace before importing.
Are you trying to import from a .zip file or from a folder? Maybe you could put your idea of combining the approaches in a sample workflow (or workflow group) so we might have a look.
Here’s the workflow.
There are a mix of workflow groups and workflows in the backup. All zipped in the backup process - which appears to work well.
The workflow mentioned in the Import error above actually imported OK, but the other workflows and groups were not imported.
Sorry @mlauber71 I don’t know how to change the text associated with your workflow on the Hub (first time I’ve loaded anything there).
@zedleb I do not really unterstand the point. The uploaded workflow cannot run on its own and refers to a path that is not present.
My sample was intended on providing an idea how to set such a thing up. You would need to provide a list of workflows you would want to store.
In general my suggestion would be to familiarise yourself with KNIME and the KNIME Hub.
Then (as mentioned before) there is the:
about how KNIME handles paths and file systems.
In general this ‘make your own’ backup thing is more like a demonstration it could be done, but may also lead to a false sense of security since there is no real substitution for a solid backup strategy that would include versioning and complete distaste recovery (disk is damaged/lost).
Hello there!
With new KNIME version 4.4.0 original issue was addressed with Transfer Files (Table) node.
Br,
Ivan
1 Like
This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.