POST in HTTP-Retriever - including file upload (multiple body content)

In continuation of my other question, here my problem with POST:

Thanks to user qqilihq I managed the GET portion with token authentication to work with Palladium HTTP-Retriever.

I also managed a POST request, where I upload a JSON text to the API of the system.

For one particular case though, I have to upload a complete (text) file; the file (name) is outside of the JSON string, meaning you need a multiple body content (or whatever it is called).

I manage this in POSTMAN (but that seems to support dummies like me very well, thus doesn't really count?).

In another thread I  was also given the hint of it not really being possible with the POST node from Knime, but perhaps the Palladian could do it.

So I guess, the question is how to make a multiple body content to pass through the node?

Here are two examples for CURL and for RUBY:
curl -H X-CDD-Token:$TOKEN -F ‘file=@path/to/file.csv’ --form-string ‘json={"different stuff1", "different":"stuff2"}’ ‘https://app.collaborativedrug.com/api/v1/vaults/23/slurps’

RestClient.post 'https://app.collaborativedrug.com/api/v1/vaults/23/slurps', {:file => File.new("path/to/file.csv"), :json => '{"different stuff1", "different":"stuff2"}'}, {:"X-CDD-Token" => ‘<API_TOKEN>’ }

Postman makes for example (parts of) Ruby looks like this:

request = Net::HTTP::Post.new(url)
request["content-type"] = 'multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW'
request["x-cdd-token"] = 'API_TOKEN'
request["cache-control"] = 'no-cache'
request["postman-token"] = '12122ced-da0a-da7d-0770-37400e5effe7'
request.body = "------WebKitFormBoundary7MA4YWxkTrZu0gW\r\nContent-Disposition: form-data; name=\"file\"; filename=\"C:\\Users\\Alex\\AppData\\Roaming\\Postman\\IndexedDB\\file__0.indexeddb.blob\\1\\00\\72\"\r\nContent-Type: false\r\n\r\n\r\n------WebKitFormBoundary7MA4YWxkTrZu0gW\r\nContent-Disposition: form-data; name=\"json\"\r\n\r\n{\"different stuff1\", \"different\":\"stuff2\"}\r\n------WebKitFormBoundary7MA4YWxkTrZu0gW--

So, I guess I could try to pull the file (or the path?) together with my JSON into a single cell and use that for POST? Problem is how to create such a "string" and where do I get/create a token (here postman-token) from? Do I actually need something like that? I made an attempt, but probably don't have the filepath in the correct format (tried several ones) - I concatenate it with the JSON portion (comma separated), but that didn't work so far.

I enclose an example workflow, the HTTP part won't work because of the token, so perhaps it won't do much good? Sorry about that.

Appreciate any thoughts and inputs! Thank you for your time!

1 Like

Hi docminus2,

could you add the mentioned example workflow? This makes it easier for me to have a look and understand your current progress -- feel free to replace the token by some garbage string. (you can also email the workflow to: mail [ät] seleniumnodes [döt] com if you prefer)

Best,
Philipp

1 Like

Sorry, thought I had it attached. Should be available now. Thanks!

1 Like

Bad + good news --

when trying to compile a usage example for the multipart thingie, I discovered a bug in Palladian, which would probably make the request invalid in many cases. Good news: I've taken the opportunity to fix this bug today (and some other issues as well). The update will hit the software sites within the next few hours (restart KNIME, go to File -> Update KNIME and make sure that Palladian v1.7.0 shows up, select it and run the update)

After that (please, really make sure you're running v1.7.0), you can have a look at attached example workflow, which illustrates how to perform a multipart request. (I agree that this is still not 100% intuitive and probably no one would figure out the usage without this example, but this particular node was originally built for an internal use case, and I don't have much time nowadays for Palladian nodes maintenance.)

Still, I hope this helps.

Best,
Philipp

2 Likes

Thank you so much! Much appreciated.

I adapted this to my specific API, but need to test more (getting illegal parameters back ... this could be though due to my Json as such)  - but I think I get the overall idea.

Hopefully I can get back to you with an even more positive answer!

 

 

 

1 Like

I managed! It works also for my specific case.

You are awesome!

1 Like

Great to hear + thanks for the positive feedback!

-- Philipp

1 Like

Thank you both for this thread! I am trying to also post data in CDD. I got stuck in the same place as I was able to create the JSON but got stuck on how to create the connection to the file I’m trying to upload. I’m super excited to try this out.

Thanks!
Jason