If you select the Default Credential Provider Chain local ENV variables or the credentials file should work. Looking into the aws-sdk-java, session tokens should be supported too, but i have not tested it so far. Do you use IAM roles?
This is only the representation inside of KNIME. What Node do you use access S3 after the S3 Connection node?
This should work with the Default Credentials Provider Chain. Can you test this with ENV variables (AWS_ACCESS_KEY_ID, AWS_SECRET_KEY and AWS_SESSION_TOKEN)?
Reviving this very old thread
I am trying to build an automated process to convert some files into Parquet files and then load them straight in an S3 bucket.
In order to do this, I have a few nodes that get the temporary access tokens:
aws_access_key_id, aws_secret_access_key and aws_session_token.
Unfortunately, I cannot pass these values automatically to the Amazon Authentication Node via variables, as there are no equivalent Variables defined in the node. I have tried assigning the values to the Credentials, username and session token fields, but no luck.
Is there any way around this as dumping the values in a file then manually copy pasting isn’t very … automated?
As stated above, the problem is that I can’t pass the data I receive from the Authentication nodes to the Amazon Authentication node.
At the end of the chain in the first picture, it dumps the current logon information into a CSV file from where I have to manually copy paste the values in the Amazon Authentication node because I can’t find a Access Key ID or a Secret Key mapping in the Flow Variables tab.
You can convert flow variables with two strings for username and password to a Credentials flow variable with the Variable to Credentials node (Variable to Credentials – KNIME Hub). Does that help?
That part is done, but what do I map it to?
Again, I have 3 values that I receive and need to set:
Access Key ID
So, the Metanode (details in original post, first pic) retrieves the 3 values.
As you can see, now I dump them in a CSV.
The Table Row to Variable Node takes care on the conversion to Flow Variables, BUT, and this is the key, WHAT do I map it to?
If I map to Credentials and Username, it doesn’t work and I don’t have Access Key ID or Secret Key mappings in the Node.
It does not work with the Table Row to Variable node, because credentials are protected in a special fashion. You will need the Variable to Credentials node to convert Access Key ID and Secret Key to a special credentials flow variable (Access Key ID = username, Secret Key = password). Then you can select the option “Credentials” in the dialog (currently grayed out for you because there is no credentials flow var). The session token you can map as-is in the Flow Variables tab.