File Transfer Node fails with timestamps in file name which include ':'

I am using the file transfer node to copy files from a Linux server to S3. All files transfer except for those with ‘:’ in their name, such as test_Jan_20_17_17:34:30_2019.txt.

Error like : “Execute Failed: Illegal char <:> at index 18 …”

Transfer the files with timestamps without modifying the filename would be optimal, as changing names for a large # of files in a complicated Amazon S3 bucket structure is expected to be very tedious. Then again, I have only nominal experience using S3.

Any advice would be greatly appreciated.

Hi @AdamLee, and welcome to the Community!

As far as I can see, a : is not allowed in the filename on S3 (as opposed under Linux), so I wouldn’t see another way than renaming the files, unfortunately.

If you are using the newest KNIME AP 4.4 (released today), you can use the “Transfer Files (Table)” Node in combination with a “List Files/Folder” node and the “String Manipulation” Node and replace any : with, say, a dash -.

Personally, I would probably use the shell to change the filename on the Linux machine before copying. It’d also be a nice excersise to adapt this workflow to do so :slight_smile:

Let me know if that helps!

Cheers,
Lukas

3 Likes

Hi @AdamLee , welcome to the Community.

I totally agree with @LukasS. Having “:” in file name is not commonly legal. Only a very few systems, such as Linux, allow it. Windows does not allow it, neither does S3 it would seem.

For compatibility reasons, it’s very advisable to avoid using “:” in file naming.

3 Likes