New Amazon S3 connector return 403 error

Hey there,

It’s my first post in the Knime Forum, I will try to explain my issue.

Node: new S3 connector

Context: we need to sync a folder with S3 bucket’s vendor. It gives me:

  1. The access key + access secret key id (I have used to connect through the Amazon authentication node)
  2. The bucket and the folder name

I guess I have permission only to explore folders/files belonging to the Bucket that the vendor has created for me.

Issue: when I try to configure my working directory using the Amazon s3 connector > Browse… it’s returning a 403 error.

See below my error message:point_down:

WARN NioFile Could not list files in ‘/’: Access Denied (Service: S3, Status Code: 403, Request ID: 4Q5T3K7…, Extended Request ID: GJkZzqbT6HZF9ZWmEQCY…)

Hi @Diego_Rodriguez

welcome to the KNIME forum!

I guess I have permission only to explore folders/files belonging to the Bucket that the vendor has created for me.

Well that explains it :slight_smile: When you start browsing at the root ("/"), you need permission to get the list of buckets which apparently you don’t have.

If all you want to do is browse inside that one bucket where you have access, then you need to manually set /bucketname as the working directory.

Also please note that strictly speaking you can just leave the working directory at the root ("/"), you don’t need to set it in order to read/write files in that bucket. The working directory is only a convenience feature that allows you to work with relative paths more easily. More information can be found in our documentation:
https://docs.knime.com/latest/analytics_platform_file_handling_guide/index.html#working-directory

4 Likes

Thanks for the warming welcome @bjoern.lohrmann :pray:

I have tried leaving the root ("/") path in the working directory string as you suggested (‘Normalize Paths’ flagged :white_check_mark:

But when I try to put my personal path (bucket and folder created for me) in the CSV writer-node connected to the S3-node (see below): :point_down:

and then inside the CSV node I fill the file path with my personal bucket and folder (the one that the vendor configure for me) is still giving a 403 error (see below 2): :point_down:

Instead, I was able to upload a file from my local machine to the S3 vendor bucket using AWS CLI command-line tool

But I prefer doing with Knime because I can integrate multiple data wrangler operations in one workflow :partying_face:

Hey @bjoern.lohrmann or @knime team any news on this issue?

It will be very helpful to be able to use the S3 connector :pray:

Hi @Diego_Rodriguez,

welcome to the forum :slight_smile:

Can you double check that you do not have any typos in the bucket name? In your screenshot, the bucket name contains dots and this is not recommended, but should work.

If this does not help, can you enable debug logging in KNIME and post the log lines here (go to the preferences KNIME > KNIME GUI):

Hola @sascha.wolke :wave:

Thanks again for your help and support.

But unfortunately, there is no typo in my bucket name :cry:

Below you can find :point_down: :

  1. Logs when I have reset the S3 connector node putting /bucketname in the ‘Workning directory’ field:

    DEBUG NodeContainerEditPart CSV Writer 3:5 (CONFIGURED)
    DEBUG NodeContainerEditPart Amazon S3 Connector 3:3 (EXECUTED)
    DEBUG MemoryAlertSystem Tenured gen heap space usage below threshold (87%) after GC, currently 4% (0.18GB/4.90GB)
    DEBUG MemoryAlertSystem Tenured gen heap space usage below threshold (66%) after GC, currently 4% (0.18GB/4.90GB)
    DEBUG CSV Writer 3:5 reset
    DEBUG CSV Writer 3:5 clean output ports.
    DEBUG Amazon S3 Connector 3:3 CSV Writer 3:5 has new state: IDLE
    DEBUG Amazon S3 Connector 3:3 reset
    DEBUG Amazon S3 Connector 3:3 clean output ports.
    DEBUG Amazon S3 Connector 3:3 Removing handler ec1f0a48-10b4-4e2f-9fa1-f2da8977a57c (Amazon S3 Connector 3:3: ) - 29 remaining
    DEBUG Amazon S3 Connector 3:3 Amazon S3 Connector 3:3 has new state: IDLE
    DEBUG Amazon S3 Connector 3:3 Configure succeeded. (Amazon S3 Connector)
    DEBUG Amazon S3 Connector 3:3 Amazon S3 Connector 3:3 has new state: CONFIGURED
    DEBUG PoolingHttpClientConnectionManager Connection manager is shutting down
    WARN CSV Writer 3:5 No connection available. Execute the connector node first.
    DEBUG PoolingHttpClientConnectionManager Connection manager shut down
    DEBUG CSV Writer 3:5 Configure succeeded. (CSV Writer)
    DEBUG Amazon S3 Connector 3:3 CSV Writer 3:5 has new state: CONFIGURED
    DEBUG PoolingHttpClientConnectionManager Connection manager is shutting down
    DEBUG PoolingHttpClientConnectionManager Connection manager shut down
    DEBUG WorkflowEditor Workflow event triggered: WorkflowEvent [type=NODE_SETTINGS_CHANGED;node=3:3;old=null;new=null;timestamp=08-Apr-2021 22:39:32]
    DEBUG ExecuteAction Creating execution job for 1 node(s)…
    DEBUG NodeContainer Amazon S3 Connector 3:3 has new state: CONFIGURED_MARKEDFOREXEC
    DEBUG NodeContainer Amazon S3 Connector 3:3 has new state: CONFIGURED_QUEUED
    DEBUG NodeContainer inject_data_botify 3 has new state: EXECUTING
    DEBUG NodeContainer ROOT has new state: EXECUTING
    DEBUG Amazon S3 Connector 3:3 Amazon S3 Connector 3:3 doBeforePreExecution
    DEBUG Amazon S3 Connector 3:3 Amazon S3 Connector 3:3 has new state: PREEXECUTE
    DEBUG Amazon S3 Connector 3:3 Adding handler 3a070b9d-7343-4499-9bd9-5ecc021abced (Amazon S3 Connector 3:3: ) - 30 in total
    DEBUG Amazon S3 Connector 3:3 Amazon S3 Connector 3:3 doBeforeExecution
    DEBUG Amazon S3 Connector 3:3 Amazon S3 Connector 3:3 has new state: EXECUTING
    DEBUG Amazon S3 Connector 3:3 Amazon S3 Connector 3:3 Start execute
    DEBUG PoolingHttpClientConnectionManager Connection request: [route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 0 of 50; total allocated: 0 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection leased: [id: 32][route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG DefaultHttpClientConnectionOperator Connecting to s3.amazonaws.com/52.217.46.198:443
    DEBUG DefaultHttpClientConnectionOperator Connection established 192.168.86.89:62684<->52.217.46.198:443
    DEBUG DefaultManagedHttpClientConnection http-outgoing-32: set socket timeout to 30000
    DEBUG PoolingHttpClientConnectionManager Connection [id: 32][route: {s}->https://s3.amazonaws.com:443] can be kept alive for 60.0 seconds
    DEBUG DefaultManagedHttpClientConnection http-outgoing-32: set socket timeout to 0
    DEBUG PoolingHttpClientConnectionManager Connection released: [id: 32][route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection request: [route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection leased: [id: 32][route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG DefaultManagedHttpClientConnection http-outgoing-32: set socket timeout to 30000
    DEBUG DefaultManagedHttpClientConnection http-outgoing-32: set socket timeout to 30000
    DEBUG PoolingHttpClientConnectionManager Connection [id: 32][route: {s}->https://s3.amazonaws.com:443] can be kept alive for 60.0 seconds
    DEBUG DefaultManagedHttpClientConnection http-outgoing-32: set socket timeout to 0
    DEBUG PoolingHttpClientConnectionManager Connection released: [id: 32][route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    WARN Amazon S3 Connector 3:3 The credentials provided have restricted permissions. File browsing might not work as expected.
    INFO Amazon S3 Connector 3:3 Amazon S3 Connector 3:3 End execute (0 secs)
    DEBUG Amazon S3 Connector 3:3 Amazon S3 Connector 3:3 doBeforePostExecution
    DEBUG Amazon S3 Connector 3:3 Amazon S3 Connector 3:3 has new state: POSTEXECUTE
    DEBUG Amazon S3 Connector 3:3 Amazon S3 Connector 3:3 doAfterExecute - success
    DEBUG Amazon S3 Connector 3:3 Amazon S3 Connector 3:3 has new state: EXECUTED
    DEBUG CSV Writer 3:5 Configure succeeded. (CSV Writer)
    DEBUG Amazon S3 Connector 3:3 inject_data_botify 3 has new state: IDLE
    DEBUG NodeContainer ROOT has new state: IDLE

  2. when I try to write a test .csv file in my bucket through the ‘CSV Writer’ node:

    DEBUG PoolingHttpClientConnectionManager Closing connections idle longer than 60000 MILLISECONDS
    DEBUG PoolingHttpClientConnectionManager Closing connections idle longer than 60000 MILLISECONDS
    DEBUG PoolingHttpClientConnectionManager Connection request: [route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG DefaultManagedHttpClientConnection http-outgoing-33: Close connection
    DEBUG PoolingHttpClientConnectionManager Connection leased: [id: 34][route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG DefaultHttpClientConnectionOperator Connecting to s3.amazonaws.com/52.217.169.0:443
    DEBUG DefaultHttpClientConnectionOperator Connection established 192.168.86.89:62714<->52.217.169.0:443
    DEBUG DefaultManagedHttpClientConnection http-outgoing-34: set socket timeout to 30000
    DEBUG PoolingHttpClientConnectionManager Connection [id: 34][route: {s}->https://s3.amazonaws.com:443] can be kept alive for 60.0 seconds
    DEBUG DefaultManagedHttpClientConnection http-outgoing-34: set socket timeout to 0
    DEBUG PoolingHttpClientConnectionManager Connection released: [id: 34][route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection request: [route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection leased: [id: 34][route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG DefaultManagedHttpClientConnection http-outgoing-34: set socket timeout to 30000
    DEBUG DefaultManagedHttpClientConnection http-outgoing-34: set socket timeout to 30000
    DEBUG DefaultManagedHttpClientConnection http-outgoing-34: Close connection
    DEBUG DefaultManagedHttpClientConnection http-outgoing-34: Shutdown connection
    DEBUG PoolingHttpClientConnectionManager Connection released: [id: 34][route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 0 of 50; total allocated: 0 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection request: [route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 0 of 50; total allocated: 0 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection leased: [id: 35][route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG DefaultHttpClientConnectionOperator Connecting to s3.amazonaws.com/52.217.169.0:443
    DEBUG DefaultHttpClientConnectionOperator Connection established 192.168.86.89:62715<->52.217.169.0:443
    DEBUG DefaultManagedHttpClientConnection http-outgoing-35: set socket timeout to 30000
    DEBUG PoolingHttpClientConnectionManager Connection [id: 35][route: {s}->https://s3.amazonaws.com:443] can be kept alive for 60.0 seconds
    DEBUG DefaultManagedHttpClientConnection http-outgoing-35: set socket timeout to 0
    DEBUG PoolingHttpClientConnectionManager Connection released: [id: 35][route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection request: [route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 0 of 50; total allocated: 0 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection request: [route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection leased: [id: 35][route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG DefaultManagedHttpClientConnection http-outgoing-35: set socket timeout to 30000
    DEBUG PoolingHttpClientConnectionManager Connection leased: [id: 36][route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG DefaultManagedHttpClientConnection http-outgoing-35: set socket timeout to 30000
    DEBUG DefaultHttpClientConnectionOperator Connecting to s3.amazonaws.com/52.217.169.0:443
    DEBUG PoolingHttpClientConnectionManager Connection [id: 35][route: {s}->https://s3.amazonaws.com:443] can be kept alive for 60.0 seconds
    DEBUG DefaultManagedHttpClientConnection http-outgoing-35: set socket timeout to 0
    DEBUG PoolingHttpClientConnectionManager Connection released: [id: 35][route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection request: [route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection leased: [id: 35][route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG DefaultManagedHttpClientConnection http-outgoing-35: set socket timeout to 30000
    DEBUG DefaultManagedHttpClientConnection http-outgoing-35: set socket timeout to 30000
    DEBUG PoolingHttpClientConnectionManager Connection [id: 35][route: {s}->https://s3.amazonaws.com:443] can be kept alive for 60.0 seconds
    DEBUG DefaultManagedHttpClientConnection http-outgoing-35: set socket timeout to 0
    DEBUG PoolingHttpClientConnectionManager Connection released: [id: 35][route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection request: [route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection leased: [id: 35][route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG DefaultManagedHttpClientConnection http-outgoing-35: set socket timeout to 30000
    DEBUG DefaultManagedHttpClientConnection http-outgoing-35: set socket timeout to 30000
    DEBUG DefaultHttpClientConnectionOperator Connection established 192.168.86.89:62716<->52.217.169.0:443
    DEBUG DefaultManagedHttpClientConnection http-outgoing-36: set socket timeout to 30000
    DEBUG PoolingHttpClientConnectionManager Connection [id: 35][route: {s}->https://s3.amazonaws.com:443] can be kept alive for 60.0 seconds
    DEBUG DefaultManagedHttpClientConnection http-outgoing-35: set socket timeout to 0
    DEBUG PoolingHttpClientConnectionManager Connection released: [id: 35][route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection request: [route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection leased: [id: 37][route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 2 of 50; total allocated: 2 of 50]
    DEBUG DefaultHttpClientConnectionOperator Connecting to s3.amazonaws.com/52.217.169.0:443
    DEBUG DefaultHttpClientConnectionOperator Connection established 192.168.86.89:62717<->52.217.169.0:443
    DEBUG DefaultManagedHttpClientConnection http-outgoing-37: set socket timeout to 30000
    DEBUG PoolingHttpClientConnectionManager Connection [id: 37][route: {s}->https://s3.amazonaws.com:443] can be kept alive for 60.0 seconds
    DEBUG DefaultManagedHttpClientConnection http-outgoing-37: set socket timeout to 0
    DEBUG PoolingHttpClientConnectionManager Connection released: [id: 37][route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 2 of 50; total allocated: 2 of 50]
    WARN NioFile Could not list files in ‘/’: Access Denied (Service: S3, Status Code: 403, Request ID: WAC7KZHCHBQ7N23M, Extended Request ID: 7Ra76CmlaIioDpOZ54Y8yCEdDmktW954WjHGL3ujSJgkExD8XWjt3oKHrHqBdzCtNPd5osH1iyQ=)
    DEBUG CSV Writer 3:5 reset
    DEBUG CSV Writer 3:5 clean output ports.
    DEBUG CSV Writer 3:5 CSV Writer 3:5 has new state: IDLE
    DEBUG CSV Writer 3:5 Configure succeeded. (CSV Writer)
    DEBUG CSV Writer 3:5 CSV Writer 3:5 has new state: CONFIGURED
    DEBUG WorkflowEditor Workflow event triggered: WorkflowEvent [type=NODE_SETTINGS_CHANGED;node=3:5;old=null;new=null;timestamp=08-Apr-2021 22:45:11]
    DEBUG ExecuteAction Creating execution job for 1 node(s)…
    DEBUG NodeContainer CSV Writer 3:5 has new state: CONFIGURED_MARKEDFOREXEC
    DEBUG NodeContainer CSV Writer 3:5 has new state: CONFIGURED_QUEUED
    DEBUG NodeContainer inject_data_botify 3 has new state: EXECUTING
    DEBUG CSV Writer 3:5 CSV Writer 3:5 doBeforePreExecution
    DEBUG CSV Writer 3:5 CSV Writer 3:5 has new state: PREEXECUTE
    DEBUG NodeContainer ROOT has new state: EXECUTING
    DEBUG CSV Writer 3:5 Adding handler ea61575c-9f51-4b64-8ba4-22f8abbdae89 (CSV Writer 3:5: ) - 31 in total
    DEBUG CSV Writer 3:5 CSV Writer 3:5 doBeforeExecution
    DEBUG CSV Writer 3:5 CSV Writer 3:5 has new state: EXECUTING
    DEBUG CSV Writer 3:5 CSV Writer 3:5 Start execute
    DEBUG CSV Writer 3:5 KNIME Buffer cache statistics:
    DEBUG CSV Writer 3:5 82 tables currently held in cache
    DEBUG CSV Writer 3:5 11071 distinct tables cached
    DEBUG CSV Writer 3:5 10953 tables invalidated successfully
    DEBUG CSV Writer 3:5 38 tables dropped by garbage collector
    DEBUG CSV Writer 3:5 18297 cache hits (hard-referenced)
    DEBUG CSV Writer 3:5 335 cache hits (softly referenced)
    DEBUG CSV Writer 3:5 0 cache hits (weakly referenced)
    DEBUG CSV Writer 3:5 8 cache misses
    DEBUG PoolingHttpClientConnectionManager Connection request: [route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG DefaultManagedHttpClientConnection http-outgoing-35: Close connection
    DEBUG PoolingHttpClientConnectionManager Connection leased: [id: 38][route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG DefaultHttpClientConnectionOperator Connecting to s3.amazonaws.com/52.217.79.150:443
    DEBUG DefaultHttpClientConnectionOperator Connection established 192.168.86.89:62719<->52.217.79.150:443
    DEBUG DefaultManagedHttpClientConnection http-outgoing-38: set socket timeout to 30000
    DEBUG PoolingHttpClientConnectionManager Connection [id: 38][route: {s}->https://s3.amazonaws.com:443] can be kept alive for 60.0 seconds
    DEBUG DefaultManagedHttpClientConnection http-outgoing-38: set socket timeout to 0
    DEBUG PoolingHttpClientConnectionManager Connection released: [id: 38][route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection request: [route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection leased: [id: 38][route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG DefaultManagedHttpClientConnection http-outgoing-38: set socket timeout to 30000
    DEBUG DefaultManagedHttpClientConnection http-outgoing-38: set socket timeout to 30000
    DEBUG PoolingHttpClientConnectionManager Connection [id: 38][route: {s}->https://s3.amazonaws.com:443] can be kept alive for 60.0 seconds
    DEBUG DefaultManagedHttpClientConnection http-outgoing-38: set socket timeout to 0
    DEBUG PoolingHttpClientConnectionManager Connection released: [id: 38][route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection request: [route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection leased: [id: 38][route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG DefaultManagedHttpClientConnection http-outgoing-38: set socket timeout to 30000
    DEBUG DefaultManagedHttpClientConnection http-outgoing-38: set socket timeout to 30000
    DEBUG PoolingHttpClientConnectionManager Connection [id: 38][route: {s}->https://s3.amazonaws.com:443] can be kept alive for 60.0 seconds
    DEBUG DefaultManagedHttpClientConnection http-outgoing-38: set socket timeout to 0
    DEBUG PoolingHttpClientConnectionManager Connection released: [id: 38][route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG MemoryAlertSystem Tenured gen heap space usage below threshold (87%) after GC, currently 4% (0.18GB/4.90GB)
    DEBUG MemoryAlertSystem Tenured gen heap space usage below threshold (66%) after GC, currently 4% (0.18GB/4.90GB)
    DEBUG MemoryAlertSystem Tenured gen heap space usage below threshold (87%) after GC, currently 4% (0.17GB/4.90GB)
    DEBUG MemoryAlertSystem Tenured gen heap space usage below threshold (66%) after GC, currently 4% (0.17GB/4.90GB)
    DEBUG CSV Writer 3:5 Closing input stream on “C:\Users\Administrator\AppData\Local\Temp\knime_inject_data_bot13772\knime_container_20210408_5336549546102917115.bin.snappy”, 1 remaining
    DEBUG PoolingHttpClientConnectionManager Connection request: [route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG PoolingHttpClientConnectionManager Connection leased: [id: 38][route: {s}->https://s3.amazonaws.com:443][total kept alive: 0; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG DefaultManagedHttpClientConnection http-outgoing-38: set socket timeout to 30000
    DEBUG DefaultManagedHttpClientConnection http-outgoing-38: set socket timeout to 30000
    DEBUG PoolingHttpClientConnectionManager Connection [id: 38][route: {s}->https://s3.amazonaws.com:443] can be kept alive for 60.0 seconds
    DEBUG DefaultManagedHttpClientConnection http-outgoing-38: set socket timeout to 0
    DEBUG PoolingHttpClientConnectionManager Connection released: [id: 38][route: {s}->https://s3.amazonaws.com:443][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
    DEBUG CSV Writer 3:5 reset
    ERROR CSV Writer 3:5 Execute failed: Unable to marshall request to JSON: Parameter ‘Key’ must not be null
    DEBUG CSV Writer 3:5 CSV Writer 3:5 doBeforePostExecution
    DEBUG CSV Writer 3:5 CSV Writer 3:5 has new state: POSTEXECUTE
    DEBUG CSV Writer 3:5 CSV Writer 3:5 doAfterExecute - failure
    DEBUG CSV Writer 3:5 reset
    DEBUG CSV Writer 3:5 clean output ports.
    DEBUG CSV Writer 3:5 Removing handler ea61575c-9f51-4b64-8ba4-22f8abbdae89 (CSV Writer 3:5: ) - 30 remaining
    DEBUG CSV Writer 3:5 CSV Writer 3:5 has new state: IDLE
    DEBUG CSV Writer 3:5 Configure succeeded. (CSV Writer)
    DEBUG CSV Writer 3:5 CSV Writer 3:5 has new state: CONFIGURED
    DEBUG CSV Writer 3:5 inject_data_botify 3 has new state: IDLE
    DEBUG NodeContainer ROOT has new state: IDLE
    DEBUG PoolingHttpClientConnectionManager Closing connections idle longer than 60000 MILLISECONDS
    DEBUG PoolingHttpClientConnectionManager Closing connections idle longer than 60000 MILLISECONDS
    DEBUG PoolingHttpClientConnectionManager Closing connections idle longer than 60000 MILLISECONDS
    DEBUG DefaultManagedHttpClientConnection http-outgoing-38: Close connection
    DEBUG PoolingHttpClientConnectionManager Closing connections idle longer than 60000 MILLISECONDS
    DEBUG DefaultManagedHttpClientConnection http-outgoing-37: Close connection

Hope I am sharing the correct info with you and it could be useful.

Thanks again! :palm_tree:

Hi @Diego_Rodriguez , could it be that you are initiating the S3 connection too early to the point that it’s idle long enough for the connection to close?

Can you try linking your Node 20 to your Node 1 with a flow variable link? (Notice the direction, from Node 20 to Node 1, meaning flow variable output of Node 20 (right side) to flow variable input of Node 1 (left side)), that way the workflow will ensure that Node 1 is executed only after Node 20 is done.

1 Like

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.