We want to add some information to this....
The try with "default credentials provider chain" was already described
Whenever we trying the Access Key Id and Secret Key we are also failing because it seems, that Knime take the key id as part of the S3 bucket address...
I am also trying to connect to an S3 resource using my credentials that are protected by multi-factor authentication. It seems important that the Amazon S3 Connection node support MFA.
If you select the Default Credential Provider Chain local ENV variables or the credentials file should work. Looking into the aws-sdk-java, session tokens should be supported too, but i have not tested it so far. Do you use IAM roles?
This is only the representation inside of KNIME. What Node do you use access S3 after the S3 Connection node?
This should work with the Default Credentials Provider Chain. Can you test this with ENV variables (AWS_ACCESS_KEY_ID, AWS_SECRET_KEY and AWS_SESSION_TOKEN)?
Reviving this very old thread
I am trying to build an automated process to convert some files into Parquet files and then load them straight in an S3 bucket.
In order to do this, I have a few nodes that get the temporary access tokens:
aws_access_key_id, aws_secret_access_key and aws_session_token.
Unfortunately, I cannot pass these values automatically to the Amazon Authentication Node via variables, as there are no equivalent Variables defined in the node. I have tried assigning the values to the Credentials, username and session token fields, but no luck.
Is there any way around this as dumping the values in a file then manually copy pasting isn’t very … automated?
Thank you.
Did you select the Access Key ID and Secret Key option in the dialog and activated the use Session Token checkbox? If this does not help, maybe create a new thread with the error message.
As stated above, the problem is that I can’t pass the data I receive from the Authentication nodes to the Amazon Authentication node.
At the end of the chain in the first picture, it dumps the current logon information into a CSV file from where I have to manually copy paste the values in the Amazon Authentication node because I can’t find a Access Key ID or a Secret Key mapping in the Flow Variables tab.
Thank you.
Hi @MMarius,
You can convert flow variables with two strings for username and password to a Credentials flow variable with the Variable to Credentials node (Variable to Credentials – KNIME Hub). Does that help?
Kind regards,
Alexander
That part is done, but what do I map it to?
Again, I have 3 values that I receive and need to set:
Access Key ID
Secret Key
Session Token
So, the Metanode (details in original post, first pic) retrieves the 3 values.
As you can see, now I dump them in a CSV.
The Table Row to Variable Node takes care on the conversion to Flow Variables, BUT, and this is the key, WHAT do I map it to?
If I map to Credentials and Username, it doesn’t work and I don’t have Access Key ID or Secret Key mappings in the Node.
Thank you.
Hi,
It does not work with the Table Row to Variable node, because credentials are protected in a special fashion. You will need the Variable to Credentials node to convert Access Key ID and Secret Key to a special credentials flow variable (Access Key ID = username, Secret Key = password). Then you can select the option “Credentials” in the dialog (currently grayed out for you because there is no credentials flow var). The session token you can map as-is in the Flow Variables tab.
Kind regards,
Alexander
Hello,
we just release version 4.7 of the KNIME Analytics Platform which now supports AWS session tokens.
For an overview of all the new features have a look at the What’s new page. For all the details go to the changelog.
Bye
Tobias