Uploading a csv file to S3

Hello,

I have a simple task to do but I’m confused because I don’t find how to do it.
I’m using few nodes to create a new file from 2 csv files, list from a S3 bucket.
And I want to create a new csv files (or overwrite it) on this bucket.

But I can’t find how I can make it.
I’ve tried something like that, in order to add the URL path in the csv writer node, but I’m getting this error : ERROR CSV Writer 3:214 Execute failed: Error writing request body to server
image

Do you know how I can do it ?

Thanks,
Mathieu.

Hi,
you need to write the file locally and then use the Upload node to actually send the file to S3. A good approach is to use Create Temp Dir and write into the created directory, as this is automatically deleted when the corresponding node is reset. Please see here for an example of S3 node usage:


Kind regards
Alexander

2 Likes

Hi there @MathieuSF,

writing to CSV file on S3 from within KNIME still requires Upload node but with new file handling concept (for example Excel Writer already has this option, see here for more) this should work almost as you designed it in your workflow :wink:

Stay tuned!

Br,
Ivan

1 Like

Hi Alexander,

Thanks for your help.
I believe that I have understood how to manage my workflow with your answer.
But now, i’m getting this error when I want to run this workflow from our AWS Knime server

“RMI executor could not be started”

Do you have ever seen this error ?

Hi @ipazin

And thank you for your help.
But I think you are talking about a node existing since v4.1.x
And I had to downgrade my version, in order to use the node “table row to variable” :frowning:

Hi,
I think that might be related to this: Java update 8u242 breaks execution with RMI executors

You will need to update KNIME to Server 4.10.1 or later, including Executor 4.1.1.
Kind regards
Alexander

1 Like

Thanks @AlexanderFillbrunn

I’ll check that point with my tech team.
Actually, I’ve looked at our AWS Knime server, and we are on the 4.9.2, so if you’re right, that’s explaining our issue.
Thanks again !

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.