Hi there,
I’m trying to upload a file into AWS S3 bucket using Access key and Secret access key. IAM and permission/authentication have been checked and they work fine. I’ve looked through forums and other posts but couldn’t find anything similar to this issue.
Using the same S3 folder location, access key and secret key, I am able to upload and list the files from the S3 location using these legacy nodes, but the new ones are throwing errors.
Legacy nodes
This below one with newer Amazon S3 Connector nodes and CSV Reader/Writer and List Files/Folders nodes do not work. In both cases old and new, S3 Connector nodes have a Warning message. New S3 Connector warns “The credentials provided have restricted permissions. File browsing might not work as expected.” Legacy S3 Connector warns “The credentials provided have restricted permissions. File browsing in the Remote File nodes might not work as expected.
All buckets will be assumed existing, as they cannot be listed.”
Errors are “Execute failed: Unable to access /myBucketLocationInS3/” from CSV Reader, “Execute failed: Unable to access /myBucketLocationInS3/Test.csv” from CSV Writer and “Execute failed: Unable to access /myBucketLocationInS3” from List Files/Folders.
Hello alabamian2,
which KNIME version are you using? We had a problem with bucket listing and restrictive accounts in KNIME Analytics Platform which was fixed with version 4.3.0.
If you are using version 4.3.0 or newer can you please execute the nodes again and have a look into the KNIME log if it contains further information about the problem. To view the log file go to View → Open KNIME Log.
Bye
Tobias
Hi @tobias.koetter,
Sure, here is the information. Thanks for your time and support.
2021-03-16 09:33:49,514 : WARN : main : : Node : CSV Writer : 4:134 : No connection available. Execute the connector node first.
2021-03-16 09:33:49,515 : WARN : main : : Node : CSV Reader : 4:137 : No connection available. Execute the connector node first.
2021-03-16 09:33:49,516 : WARN : main : : Node : List Files/Folders : 4:132 : No connection available. Execute the connector node first.
2021-03-16 09:33:55,435 : WARN : KNIME-Worker-48-Amazon S3 Connector 4:129 : : Node : Amazon S3 Connector : 4:129 : The credentials provided have restricted permissions. File browsing might not work as expected.
2021-03-16 09:33:55,472 : ERROR : KNIME-Worker-49-CSV Reader 4:137 : : Node : CSV Reader : 4:137 : Execute failed: Unable to access /XXXXX/
org.knime.filehandling.core.defaultnodesettings.ExceptionUtil$FormattedAccessDeniedException: Unable to access /XXXXX/
at org.knime.filehandling.core.defaultnodesettings.ExceptionUtil.createAccessDeniedException(ExceptionUtil.java:196)
at org.knime.filehandling.core.connections.FSFiles.exists(FSFiles.java:299)
at org.knime.filehandling.core.defaultnodesettings.filechooser.FileChooserPathAccessor.getRootPath(FileChooserPathAccessor.java:289)
at org.knime.filehandling.core.defaultnodesettings.filechooser.FileChooserPathAccessor.getFSPaths(FileChooserPathAccessor.java:201)
at org.knime.filehandling.core.defaultnodesettings.filechooser.reader.ReadPathAccessor.getPaths(ReadPathAccessor.java:96)
at org.knime.filehandling.core.node.table.reader.TableReaderNodeModel.getPaths(TableReaderNodeModel.java:186)
at org.knime.filehandling.core.node.table.reader.TableReaderNodeModel.execute(TableReaderNodeModel.java:153)
at org.knime.core.node.NodeModel.executeModel(NodeModel.java:576)
at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1245)
at org.knime.core.node.Node.execute(Node.java:1025)
at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:558)
at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:95)
at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:201)
at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:117)
at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:334)
at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)
2021-03-16 09:33:58,688 : ERROR : KNIME-Worker-50-CSV Writer 4:134 : : Node : CSV Writer : 4:134 : Execute failed: Unable to access /XXXXX/Test.csv
org.knime.filehandling.core.defaultnodesettings.ExceptionUtil$FormattedAccessDeniedException: Unable to access /XXXXX/Test.csv
at org.knime.filehandling.core.defaultnodesettings.ExceptionUtil.wrapIOException(ExceptionUtil.java:169)
at org.knime.filehandling.core.connections.FSFiles.newOutputStream(FSFiles.java:135)
at org.knime.base.node.io.filehandling.csv.writer.CSVWriter2NodeModel.createOutputStream(CSVWriter2NodeModel.java:211)
at org.knime.base.node.io.filehandling.csv.writer.CSVWriter2NodeModel.writeToFile(CSVWriter2NodeModel.java:166)
at org.knime.base.node.io.filehandling.csv.writer.CSVWriter2NodeModel.execute(CSVWriter2NodeModel.java:146)
at org.knime.base.node.io.filehandling.csv.writer.CSVWriter2NodeModel.execute(CSVWriter2NodeModel.java:1)
at org.knime.core.node.NodeModel.executeModel(NodeModel.java:576)
at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1245)
at org.knime.core.node.Node.execute(Node.java:1025)
at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:558)
at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:95)
at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:201)
at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:117)
at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:334)
at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)
2021-03-16 09:34:04,366 : ERROR : KNIME-Worker-51-List Files/Folders 4:132 : : Node : List Files/Folders : 4:132 : Execute failed: Unable to access /XXXXX
org.knime.filehandling.core.defaultnodesettings.ExceptionUtil$FormattedAccessDeniedException: Unable to access /XXXXX
at org.knime.filehandling.core.defaultnodesettings.ExceptionUtil.createAccessDeniedException(ExceptionUtil.java:196)
at org.knime.filehandling.core.connections.FSFiles.exists(FSFiles.java:299)
at org.knime.filehandling.core.defaultnodesettings.filechooser.FileChooserPathAccessor.getRootPath(FileChooserPathAccessor.java:289)
at org.knime.filehandling.core.defaultnodesettings.filechooser.FileChooserPathAccessor.getFSPaths(FileChooserPathAccessor.java:201)
at org.knime.filehandling.utility.nodes.listpaths.ListFilesAndFoldersNodeModel.writeOutput(ListFilesAndFoldersNodeModel.java:191)
at org.knime.filehandling.utility.nodes.listpaths.ListFilesAndFoldersNodeModel.execute(ListFilesAndFoldersNodeModel.java:137)
at org.knime.core.node.NodeModel.executeModel(NodeModel.java:576)
at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1245)
at org.knime.core.node.Node.execute(Node.java:1025)
at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:558)
at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:95)
at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:201)
at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:117)
at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:334)
at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)
I’m testing out using legacy nodes for now. Legacy nodes allow me to upload and list. Delete is not working maybe due to my permission (checking on that now). I tried to create a directory/sub folder using the new Create Folder node, but the color of file system connection port is different and won’t let me connect them. Does that mean legacy nodes are not compatible with newer nodes?
Happy to report that Legacy Delete node works. Legacy Create Directory is throwing an error.
Not sure how I need to configure this node, but I tried different settings between Location, Name and Abort checkmark and it does not like it. Both Location and Name fields need to be populated in order to Execute but throws this error.
Execute failed: The directory s3://XXXXX@us-east-1/locationOfS3bucketPageSpeedAPIData/ does already exist
