Amazon Authentication node - how to use AWS profile from MFA (Okta) authentication

[rsherhod]

Hi,

I’m trying to figure out how to use the Amazon Authentication node to connect to one of our company’s Amazon accounts, in order to access our s3 buckets. I’m not having much luck though, mainly because I don’t know what I’m doing with this node.

I’m currently getting this 403 error:

My ~/.aws/credentials file looks something like this:

[default]
aws_access_key_id     = [ABC...]
aws_secret_access_key = [123...]

[my_profile]
aws_access_key_id        = [XYZ...]
aws_secret_access_key    = [456...]
aws_session_token        = [...]
aws_security_token       = [...]
x_principal_arn          = arn:aws:sts::[account_num]:assumed-role/Okta-TeamAccess/my.email@company.com
x_security_token_expires = 2021-06-16T16:35:36Z

[another_profile]
...

The credentials I need to use are in the my_profile AWS profile, but I don’t seem to be able to specify a profile in the node.

I’ve tried using both the “Default Credential Provider Chain” option, and the “Access Key ID and Secret Key” option with my aws_access_key_id in the Access Key ID box and aws_secret_access_key in the Secret Key box. Neither worked.

I’ve also tried the “Switch Role” option, with my 12-digit account_num from ARN string in the Account box and several variations of the stuff after assumed-role/ in the Role box.

Any idea what I’m doing wrong, or how I can get this to work?

Cheers,

Rich

[bruno29a]

Hi @rsherhod , I am not sure how you are linking this credential files to Knime. Can you please elaborate on what node you are using and how you are configuring it?

[rsherhod]

I’m using the Amazon Authentication node. The options and boxes I refer to in my post are those presented in the node dialog. I’m also not linking the file to KNIME myself; I know that if I use the “Default Credential Provider Chain” option, from the node dialog, ~/.aws/credentials is where KNIME looks. However, the node doesn’t seem able to use the right profile.

[rsherhod]

Can anyone from the KNIME and/or Big Data extension team help with this?

[sascha.wolke]

Hi @rsherhod,

the AWS authentication node does not support profiles right now. You can set the AWS_PROFILE environment variable to select a profile as a workaround and use the default credentials provider chain. As a starting point, I suggest to add your settings to the default profile as a test and ensure this works.

Cheers,
Sascha

[rsherhod]

Thanks for the reply Sascha.

Adding the profile settings into the default profile worked, thanks. So, how would I set the environment variable for my KNIME desktop client?

Longer term, we may need to access our s3 buckets from a KNIME Server instance. Is this feasible with our multi-account set up?

Also, are there plans to add profile support to this node. Using different profiles for production, staging, dev environments seems to be pretty common.

Cheers,

Richard

[sascha.wolke]

Hi @rsherhod,

setting the environment variable depends on the operating system, what OS are you running?

The environment variable works only on a KNIME instance level, not a workflow level. If you use only one profile on your KNIME client / server then this workaround might work. I have created an internal ticket to add this option to the Amazon authentication node, but we have do not have a planned version yet on this.

Cheers,
Sascha

[rsherhod]

I use macOS. I know how to set an environment variable for bash etc. I was wondering if there was a way to set it in some KNIME config, so that it uses my profile by default.

If/when we start using the KNIME Server, I guess we just set it in the container config.

Thanks,

Richard