New Amazon S3 connector chain does not pick up fresh access token

MeikelBrandmeyer · January 25, 2021, 9:50am

Hi,

I experience a problem with the new Amazon S3 nodes. I run the connection and everything is fine. Then at some point the access token expires. Now I retrieve a new token and reset the connection related nodes. However, they don’t pick up the new token! The connector node still says, that the token has expired. The now deprecated connector has no such issue. It happily picks up the fresh token and keeps running. The whole thing can only be fixed by restarting KNIME.

BTW, the connection is set to use the “Default credential providder chain”. I also tried to set the AWS id and secret key directly with credentials, but that doesn’t work neither. I’m not sure why.

Is this a bug? Or am I missing something?

sascha.wolke · January 26, 2021, 10:43am

Hi @MeikelBrandmeyer,

thanks for reporting this. Looks like a bug in the new S3 Connector node and there seems to be no simple workaround. I will have a look into this shortly.

sascha.wolke · February 1, 2021, 11:19am

Hi @MeikelBrandmeyer,

KNIME 4.3.1 was release on Friday and contains a fix to reload the default credentials provider chain if you reset the S3 connection node. Does this help?

MeikelBrandmeyer · February 2, 2021, 8:35am

It still does not work for me. Do I have to replace the node? Or should it work just opening the workflow with the new KNIME version?

sascha.wolke · February 2, 2021, 10:18am

Hi @MeikelBrandmeyer,

you have to update KNIME or download the new KNIME Version. You can verify the Version via:
Help > About KNIME… > Installation details. There should be a KNIME Amazon Cloud Connectors feature with Version 4.3.1.v202101261633.

The current node contains the fix and you don’t need to replace it. Restarting the node should be enough to reload the default credentials provider chain.

MeikelBrandmeyer · February 2, 2021, 1:49pm

No. Doesnt work for me. I have updated yesterday.

sascha.wolke · February 3, 2021, 2:29pm

Hmpf, thats strage. How do you set the credentials? I have tested this on Linux using a credentials file and the credentials are reloaded after restarting the S3 connector node.

MeikelBrandmeyer · February 5, 2021, 1:05pm

We have a custom node from our IT department, which talks to some REST service to obtain a new token. These credentials are then written to files in the ~/.aws directory. I presume, the credentials are picked-up from there. At least it worked like that until the 4.3 update. And the old, now-deprecated S3 connector continues to work like that.

Just checked again: Does not work for me with 4.3.1 neither.

vs.

Edit: I’m on Windows. Didn’t mention that before.

sascha.wolke · February 12, 2021, 12:21pm

I have tested this on windows now and it does not work with KNIME 4.3.0, but with 4.3.1 the credentials are reloaded on connector restart. Just to be secure, can you check that the content of the ~/.aws/credentials file changed (on Windows C:\Users\your-username\.aws\credentials) and that your workflow with the legacy S3 connector does not use any flow variables? Can you take a look in the logs (KNIME > View > Open KNIME log) and what error was reported after executing the S3 Connector node?