# HTTP Retriever fails to fetch URL's containing a "?" outside of the context of query parameters.

**URL:** <https://forum.knime.com/t/http-retriever-fails-to-fetch-urls-containing-a-outside-of-the-context-of-query-parameters/28018>\
**Category:** Community Extensions\
**Tags:** palladian-selenium\
**Created:** [October 15, 2020, 10:20am UTC](https://forum.knime.com/t/http-retriever-fails-to-fetch-urls-containing-a-outside-of-the-context-of-query-parameters/28018 "2020-10-15T10:20:27Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nancyjay](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/n/c89c15/32.png) [@Nancyjay](https://forum.knime.com/u/Nancyjay)\
**Post date:** [October 15, 2020, 10:20am UTC](https://forum.knime.com/t/http-retriever-fails-to-fetch-urls-containing-a-outside-of-the-context-of-query-parameters/28018/1 "2020-10-15T10:20:27Z")

</div>

I am using the HTTP Retriever node to fetch multiple product page URL’s. 3 requests return 403 Error responses. These failed requests share the commonality of containing a “?” in the path component of the URL. These question marks do not appear to indicate the start of query parameters - they are possibly a mistake. I’ve never actually seen a URL like this before. Nonetheless, they are successful in the browser.

I’ve included a small demonstration workflow with 3 regular URL’s and 3 URL’s containing the mystery question mark.

Thanks for your time.

[Failed\_Requests\_HTTP.knwf](https://forum.knime.com/uploads/short-url/8s64RvJz0gtEcpkhoCtmGjOmD8s.knwf) (9.0 KB)

---

<div class="post-metadata">

**Author:** ![qqilihq](https://forum-cdn.knime.com/user_avatar/forum.knime.com/qqilihq/32/50_2.png) [@qqilihq](https://forum.knime.com/u/qqilihq)\
**Post date:** [October 15, 2020, 10:48am UTC](https://forum.knime.com/t/http-retriever-fails-to-fetch-urls-containing-a-outside-of-the-context-of-query-parameters/28018/2 "2020-10-15T10:48:18Z")

</div>

Hi Nacyjay,

Many thanks for this. We’ll have a look.

Would you be willing to give the pre-release build a try? This already fixes the issue you described [here](https://forum.knime.com/t/using-cookie-retrieved-with-http-retriever-node-doesnt-work-in-workflow-however-copy-pasting-exact-same-cookie-from-web-console-does/27646) and probably this one will be resolved as well. If not, we’ll try to get this fixed for the next release as well. If you would give the pre-release a try, please contact us at mail@palladian.ws

Thanks,  
Philipp

---

<div class="post-metadata">

**Author:** ![qqilihq](https://forum-cdn.knime.com/user_avatar/forum.knime.com/qqilihq/32/50_2.png) [@qqilihq](https://forum.knime.com/u/qqilihq)\
**Post date:** [October 16, 2020, 7:26am UTC](https://forum.knime.com/t/http-retriever-fails-to-fetch-urls-containing-a-outside-of-the-context-of-query-parameters/28018/3 "2020-10-16T07:26:47Z")

</div>

I am not sure if there’s anything to fix. All of these URLs (no matter whether there’s a ? or not) give me a 404 when I try to load them in the browser:

 ![image](https://forum.knime.com/uploads/default/original/3X/0/9/090040c79069dc568cbcd1702215b4160cbb6d4f.png)

---

<div class="post-metadata">

**Author:** ![Nancyjay](https://forum-cdn.knime.com/letter_avatar_proxy/v4/letter/n/c89c15/32.png) [@Nancyjay](https://forum.knime.com/u/Nancyjay)\
**Post date:** [October 21, 2020, 7:57am UTC](https://forum.knime.com/t/http-retriever-fails-to-fetch-urls-containing-a-outside-of-the-context-of-query-parameters/28018/4 "2020-10-21T07:57:08Z")

</div>

Hi,

that’s strange because all 6 URL’s included in the workflow I uploaded here do work in the browser, and then only 3 work (return 200 status code) in the workflow.

 ![Screenshot 2020-10-21 at 08.55.05](https://forum.knime.com/uploads/default/original/3X/4/6/46f142feda68a6b2ccb7c41e879cfe62d740680b.png)  
 ![Screenshot 2020-10-21 at 08.55.22](https://forum.knime.com/uploads/default/original/3X/b/d/bd15673f5e227b1f6d01294963daa0fbded15122.png)

and yes I will give the prerelease a go, thanks.

---

<div class="post-metadata">

**Author:** ![qqilihq](https://forum-cdn.knime.com/user_avatar/forum.knime.com/qqilihq/32/50_2.png) [@qqilihq](https://forum.knime.com/u/qqilihq)\
**Post date:** [October 23, 2020, 10:26am UTC](https://forum.knime.com/t/http-retriever-fails-to-fetch-urls-containing-a-outside-of-the-context-of-query-parameters/28018/5 "2020-10-23T10:26:04Z")

</div>

Hi Nancyjay,

received your email – thanks. I will be in touch regarding the prerelease for the next version, this will still take a bit, as we’re currently working on some outstanding issues.

Regarding what I wrote below – my apologies. I just realized that I’d need to click on “English” on the website, and only then the links work (that’s why you set the cookie at all, I totally overlooked this 😬 ). So I can confirm that all the links work fine in the browser, but the ones with ? return 404 in the workflow. I think we have addressed this issue in the recent Palladian lib update as well.

Thanks for the feedback, and I’ll be in touch about the pre-release (hopefully next week).

–Philipp

---

<div class="post-metadata">

**Author:** ![system](https://forum-cdn.knime.com/uploads/default/original/1X/ab3ccf34482a0329361734a18199390177204f15.png) [@system](https://forum.knime.com/u/system)\
**Post date:** [April 21, 2023, 9:38pm UTC](https://forum.knime.com/t/http-retriever-fails-to-fetch-urls-containing-a-outside-of-the-context-of-query-parameters/28018/6 "2023-04-21T21:38:15Z")

</div>

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.
