Selenium nodes: too many requests on internet page - how to bypass?


I have a table with a lot of URLs (roughly 2800) referring to notices on TED portal (Place of performance (Map) - TED Tenders Electronic Daily).

I want to save the source code of each URL. However, after 500 I receive an error since too many requests were sent and I can try in 24 hours again.

I noticed that having a different IP-address for the next 500 URLs can bypass this problem, but I am not sure how and if this can be implemented and if there are other possible ways to bypass this error.

Thx and kind regards

Please see here:

HTH - Philipp


This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.