HTTP Retriever failing with a specific URL

Community Extensions Palladian & Selenium
#1

Hello,

When requesting the url “https://www.iciparisxl.nl” with the “HTTP Retriever” node, I get an exception:

Error retrieving https://www.iciparisxl.nl: Exception java.net.SocketTimeoutException: Read timed out for URL “https://www.iciparisxl.nl”: Read timed out

However, it works with the “Webpage Retriever” node.

Please find hereby an example workflow.
iciparis.knwf (10.7 KB)

Best regards,
Sébastien

#2

Hi Sébastien,

I just tried running your workflow, and actually both nodes fail for me. Also it fails when doing a plain curl on the command line or requesting it with a REST tool. In contrast to that, it’ll load fine in a normal web browser. I assume they have some measures in place for detecting bots.

Have you tried if switching the user agent string makes any difference?

–Philipp