New 4.1.1 Amazon Authentication and Athena Connector Credentials Issue

Just tried the updated Amazon Authentication node to allow the Amazon Athena Connector in KNIME v4.1.1 on Windows 10. The Amazon Authentication node now allows the default credential chain and alternative the selection of credentials from flow-variables, and in both cases testing the connection succeeds. However, the downstream Athena Connector fails with:

DEBUG Amazon Athena Connector 0:580:627  New database session: DefaultDBSessionInformation(id=4710bd02-88c2-42f3-867b-26446ef8c576, dbType=DBType(id=athena, name=Amazon Athena, description=null), driverDefinition=DBDriverDefinition(id=Athena, name=Amazon Athena, version=2.0.0, driverClass=com.simba.athena.jdbc.Driver, dbType=DBType(id=athena, name=Amazon Athena, description=null), description=, origin=EXTENSION), connectionController=org.knime.cloud.aws.athena.connector.AthenaDBConnectionController=(url=jdbc:awsathena://athena.us-east-1.amazonaws.com:443/), dialectId=athena, attributeValues={knime.db.dialect.sql.delimiter.identifier.opening=, knime.db.connection.jdbc.fetch_size=100000, knime.db.dialect.sql.delimiter.identifier.closing=})
INFO  Amazon Athena Connector 0:580:627  Could not create connection to database using URL: jdbc:awsathena://athena.us-east-1.amazonaws.com:443/ and parameters: [UseResultsetStreaming, S3OutputLocation, AwsCredentialsProviderClass, AwsCredentialsProviderArguments]. Exception: [Simba][AthenaJDBC](100191) Failed to create AWS Credentials Provider class: org.knime.cloud.aws.athena.connector.KNIMEAWSCredentialsProvider.
DEBUG Amazon Athena Connector 0:580:627  reset
ERROR Amazon Athena Connector 0:580:627  Execute failed: [Simba][AthenaJDBC](100191) Failed to create AWS Credentials Provider class: org.knime.cloud.aws.athena.connector.KNIMEAWSCredentialsProvider.

If I choose the AWS Key ID and Secret Key authentication option in the Amazon Authentication node, the Athena Connection works fine. Is there anything I can do to work around this issue?

Hey @bfrutchey,
could you please have a look into the KNIME.Log (View -> Open KNIME Log) and see if the StackTrace of the error gives a more detailed error message?

best Mareike

Exceptions I found in the Knime log attached. The first exception is from a test when credentials created upstream are selected (these same credentials work in the S3 nodes). The second exception is from a test with the default credential chain. The role being used in the ARN is the same which works when using id/key login.

2020-02-07 22:53:09,720 : ERROR : KNIME-Worker-23-Amazon Athena Connector 0:580:627 :  : Node : Amazon Athena Connector : 0:580:627 : Execute failed: [Simba][AthenaJDBC](100191) Failed to create AWS Credentials Provider class: org.knime.cloud.aws.athena.connector.KNIMEAWSCredentialsProvider.
java.sql.SQLException: [Simba][AthenaJDBC](100191) Failed to create AWS Credentials Provider class: org.knime.cloud.aws.athena.connector.KNIMEAWSCredentialsProvider.
	at com.simba.athena.athena.utilities.AJUtilities.createAwsCredentialsProvider(Unknown Source)
	at com.simba.athena.athena.api.AJClient.<init>(Unknown Source)
	at com.simba.athena.athena.core.AJConnection.connect(Unknown Source)
	at com.simba.athena.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
	at com.simba.athena.jdbc.common.AbstractDriver.connect(Unknown Source)
	at org.knime.database.connection.UrlDBConnectionController$ControlledDriver.connect(UrlDBConnectionController.java:95)
	at org.knime.database.connection.UrlDBConnectionController.createConnection(UrlDBConnectionController.java:308)
	at org.knime.database.connection.AbstractConnectionProvider.createConnection(AbstractConnectionProvider.java:89)
	at org.knime.database.connection.impl.DBConnectionManager.lambda$2(DBConnectionManager.java:458)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Caused by: com.simba.athena.support.exceptions.GeneralException: [Simba][AthenaJDBC](100191) Failed to create AWS Credentials Provider class: org.knime.cloud.aws.athena.connector.KNIMEAWSCredentialsProvider.
	... 12 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.simba.athena.athena.utilities.AJUtilities.createAwsCredentialsProvider(Unknown Source)
	at com.simba.athena.athena.api.AJClient.<init>(Unknown Source)
	at com.simba.athena.athena.core.AJConnection.connect(Unknown Source)
	at com.simba.athena.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
	at com.simba.athena.jdbc.common.AbstractDriver.connect(Unknown Source)
	at org.knime.database.connection.UrlDBConnectionController$ControlledDriver.connect(UrlDBConnectionController.java:95)
	at org.knime.database.connection.UrlDBConnectionController.createConnection(UrlDBConnectionController.java:308)
	at org.knime.database.connection.AbstractConnectionProvider.createConnection(AbstractConnectionProvider.java:89)
	at org.knime.database.connection.impl.DBConnectionManager.lambda$2(DBConnectionManager.java:458)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: com.simba.athena.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException: 1 validation error detected: Value 'arn:aws:iam:::role/' at 'roleArn' failed to satisfy constraint: Member must have length greater than or equal to 20 (Service: AWSSecurityTokenService; Status Code: 400; Error Code: ValidationError; Request ID: 8567b3f2-4a26-11ea-b10e-8b4faa84720b)
	at org.knime.cloud.aws.athena.connector.KNIMEAWSCredentialsProvider.<init>(KNIMEAWSCredentialsProvider.java:84)
	... 17 more
Caused by: com.simba.athena.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException: 1 validation error detected: Value 'arn:aws:iam:::role/' at 'roleArn' failed to satisfy constraint: Member must have length greater than or equal to 20 (Service: AWSSecurityTokenService; Status Code: 400; Error Code: ValidationError; Request ID: 8567b3f2-4a26-11ea-b10e-8b4faa84720b)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1712)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1367)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)
	at com.simba.athena.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)
	at com.simba.athena.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)
	at com.simba.athena.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.doInvoke(AWSSecurityTokenServiceClient.java:1389)
	at com.simba.athena.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1356)
	at com.simba.athena.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1345)
	at com.simba.athena.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.executeAssumeRole(AWSSecurityTokenServiceClient.java:528)
	at com.simba.athena.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.assumeRole(AWSSecurityTokenServiceClient.java:500)
	at org.knime.cloud.aws.athena.connector.KNIMEAWSCredentialsProvider.getCredential(KNIMEAWSCredentialsProvider.java:123)
	at org.knime.cloud.aws.athena.connector.KNIMEAWSCredentialsProvider.<init>(KNIMEAWSCredentialsProvider.java:82)
	... 17 more
2020-02-07 23:01:00,893 : ERROR : KNIME-Worker-28-Amazon Athena Connector 0:580:627 :  : Node : Amazon Athena Connector : 0:580:627 : Execute failed: [Simba][AthenaJDBC](100191) Failed to create AWS Credentials Provider class: org.knime.cloud.aws.athena.connector.KNIMEAWSCredentialsProvider.
java.sql.SQLException: [Simba][AthenaJDBC](100191) Failed to create AWS Credentials Provider class: org.knime.cloud.aws.athena.connector.KNIMEAWSCredentialsProvider.
	at com.simba.athena.athena.utilities.AJUtilities.createAwsCredentialsProvider(Unknown Source)
	at com.simba.athena.athena.api.AJClient.<init>(Unknown Source)
	at com.simba.athena.athena.core.AJConnection.connect(Unknown Source)
	at com.simba.athena.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
	at com.simba.athena.jdbc.common.AbstractDriver.connect(Unknown Source)
	at org.knime.database.connection.UrlDBConnectionController$ControlledDriver.connect(UrlDBConnectionController.java:95)
	at org.knime.database.connection.UrlDBConnectionController.createConnection(UrlDBConnectionController.java:308)
	at org.knime.database.connection.AbstractConnectionProvider.createConnection(AbstractConnectionProvider.java:89)
	at org.knime.database.connection.impl.DBConnectionManager.lambda$2(DBConnectionManager.java:458)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Caused by: com.simba.athena.support.exceptions.GeneralException: [Simba][AthenaJDBC](100191) Failed to create AWS Credentials Provider class: org.knime.cloud.aws.athena.connector.KNIMEAWSCredentialsProvider.
	... 12 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.simba.athena.athena.utilities.AJUtilities.createAwsCredentialsProvider(Unknown Source)
	at com.simba.athena.athena.api.AJClient.<init>(Unknown Source)
	at com.simba.athena.athena.core.AJConnection.connect(Unknown Source)
	at com.simba.athena.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
	at com.simba.athena.jdbc.common.AbstractDriver.connect(Unknown Source)
	at org.knime.database.connection.UrlDBConnectionController$ControlledDriver.connect(UrlDBConnectionController.java:95)
	at org.knime.database.connection.UrlDBConnectionController.createConnection(UrlDBConnectionController.java:308)
	at org.knime.database.connection.AbstractConnectionProvider.createConnection(AbstractConnectionProvider.java:89)
	at org.knime.database.connection.impl.DBConnectionManager.lambda$2(DBConnectionManager.java:458)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: com.simba.athena.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException: User: arn:aws:iam::816300528458:user/vane_dev is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::816300528458:role/nuwave-vane (Service: AWSSecurityTokenService; Status Code: 403; Error Code: AccessDenied; Request ID: 9e414929-4a27-11ea-ace9-63387e48648e)
	at org.knime.cloud.aws.athena.connector.KNIMEAWSCredentialsProvider.<init>(KNIMEAWSCredentialsProvider.java:84)
	... 17 more
Caused by: com.simba.athena.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException: User: arn:aws:iam::816300528458:user/vane_dev is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::816300528458:role/nuwave-vane (Service: AWSSecurityTokenService; Status Code: 403; Error Code: AccessDenied; Request ID: 9e414929-4a27-11ea-ace9-63387e48648e)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1712)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1367)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)
	at com.simba.athena.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)
	at com.simba.athena.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)
	at com.simba.athena.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.doInvoke(AWSSecurityTokenServiceClient.java:1389)
	at com.simba.athena.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1356)
	at com.simba.athena.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1345)
	at com.simba.athena.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.executeAssumeRole(AWSSecurityTokenServiceClient.java:528)
	at com.simba.athena.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.assumeRole(AWSSecurityTokenServiceClient.java:500)
	at org.knime.cloud.aws.athena.connector.KNIMEAWSCredentialsProvider.getCredential(KNIMEAWSCredentialsProvider.java:123)
	at org.knime.cloud.aws.athena.connector.KNIMEAWSCredentialsProvider.<init>(KNIMEAWSCredentialsProvider.java:82)
	... 17 more

@bfrutchey,
are you using Temporary security credentials for Athena?
Do you use the role switch option?

best Mareike

Sorry, seems I failed to reply. We are not using temporary security credentials. We only need the role switch option when using the ID/Key authentication, which is the option that is working for us. The credentials option is using an account which does not need to switch roles.

Hi @bfrutchey,
this is unfortunately a bug in the Connector. If you use the default credentials provider chain the Athena Connector currently assumes role switching, even if it is not used. We have opened a ticket for the bug, the work around for now (as you mentioned) is to use ID/Key authentication instead of the provider chain.

best Mareike

2 Likes

Hello,
I’m happy to announce that the problem has been fixed with KNIME Analytics Platform version 4.2.0 which is available since Monday and the bug fix release 4.1.4 which will be available soon.
Bye
Tobias

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.