File Transfer Issue: only bucket owner can access files ... How to enable bucket-owner-full-control (ACL) for file transfers to S3

[AdamLee]

I have created a simple workflow transferring data from a Linux platform to Amazon S3. The nodes consist of: SSH Connector, Amazon Authentication, Amazon S3 Connector, and the Transfer Files node. When files are written to S3, I need to be able to specify bucket ownership such that other groups (specified in AWS) within my organization can also access them. As a prerequisite in AWS, we have already changed the bucket parameter for object ownership to “Bucket owner preferred”.

With Bucket owner preferred, new objects uploaded to this bucket with bucket-owner-full-control canned ACL will be owned by the bucket owner…

What configurational changes do I need to make in the KNIME nodes to utilize the bucket-owner-full-control canned access control list (ACL)?

In the Amazon Authentication node I am using “Default Credential Provider Chain” and “Switch Role” is not enabled.

FileTransfer_Linux-to-S3.knwf (548.7 KB)

[bruno29a]

Hi @AdamLee , I think this is more of an S3 configuration. I don’t think you can manage access via Knime. Knime can only do the transfer of files from one connection to another.

You need to manage the access via the S3 console. If set properly, then the grantees should be able to access the any files uploaded in the bucket. You should not have to set the permissions each time you upload a file in the bucket.

[sascha.wolke]

Hi @AdamLee,

the KNIME S3 connector does not support the ACL-Specific Request Headers. You can use Roles to share data between different AWS accounts and that’s the preferred way to do this I guess.

Sascha