Proxy settings of Webdriver ignored

Hi,

I am trying to use selenium nodes with proxy settings and they are ignored.
These are the options I set
image

And this is the result when I access https://whatismyipaddress.com/ with Start Web driver
image
I also tried passing the settings as JSON input, but the result is the same

Instead, when I set the proxy settings in the Firefox browser, not the Webdriver,
the result is this one:
image

I have also tried setting the proxy in KNIME settings, but they are also ignored by Webdriver.
Any ideas?

I’d expect httpProxy to only work for non-HTTPS, but the example page you mention uses HTTPS.

– Philipp

1 Like

I have also tried with ssl,but got the same result

Is there any way to make it work with https?

Ok, I found a way

Setting it like this seems to work

Thanks for the feedback, @dasturiol

To clarify for me: You were not able to get it to work using the included “Snippets”, neither with Firefox nor Chrome?

To clarify for you: The proxy settings in KNIME are not considered by the Selenium Nodes. This is a conscious decision, as for crawling, scraping, it sometimes makes sense to use a dedicated proxy setting which differs from the default, system-wide settings.

– Philipp

1 Like

Hi Philipp,

Yes, I was not able to get it to work using the “templates” included in the WebDriver node. Neither Firefox nor Chrome.

Then, googleing, I found out the option I posted above. Tried and it worked.

I think what happened is that, at least chrome, changed the way to set the proxy settings in some version (I am currently using v83) and I think this is the reason why it was not working.

Regarding the KNIME proxy settings not being considered for Selenium, it makes perfect sense. Actually I was hoping it was like this. Mainly because it gives the possibility to change the proxy depending on where you are scraping or crawling.

cheers.

1 Like

Many thanks for the feedback! We’ll definitely investigate this in detail! I’ll keep this thread updated.

– Philipp

2 Likes

Again, thanks for noticing this @dasturiol! There were indeed two bugs which we fixed, available now in the 4.4.3 release!

Changelog:

2 Likes