KNIME's proxy settings/authentification details ignored by WebSearcher & UrlResolver node ?

Hi

Without the firewall of my company all palladian nodes work without problems. But if I am sitting behind the company's firewall, I have to change settings in  FILE>PREFERENCES>GENERAL>NETWORK CONNECTIONS to get data from external ressources. The value of "Active Provider" is changed to "Manual"and for HTTP and HTTPS connection I provide a hostname and a port for a proxy as well as the login credentials (user / password).

 

These settings work for nodes like the FileReader node, etc.  (with the same http address generated by the WebSearcher, e.g. http://de.wikipedia.org/w/api.php?action=query&list=search&format=json&srlimit=50&srsearch=Germany ?)

 

If I use a workflow like TableCreator (with search term) > WebSearcher, then it seems that the proxy is okay, but the WebSearcher ignores the authentification details of the KNIME settings and comes up with an error.

 

A similar behaviour has a workflow like  TableCreator > UrlExtractor > UrlNormalizer > UrlResolver. Each nodes uses the output of the its predecessor. Everything works fine except the UrlResolver that generates all the times empty cells even if the URL is valid.

For a workflows like  TableCreator (with feed address) > HttpRetriever > FeedParser everything works fine without an error. The only things is that the FeedParser will not come back if the address of the feed is wrong.

Best Regards, Frank

 

Hi Frank,

indeed, we do currenlty not support authenticated proxies (the log should give an output like "Proxy authentication is currently not supported!"). I have added this feature request and try to support proxies with login with the next update.

I will keep you updated on the topic here in this thread.

Best,
Philipp

Hi,

 

I got similar message "Proxy authentication is currently not supported!" with HTTPRetriever and wondering if there is a way to fix it or things have been changed ?

 

Best,

Abdul

Hi

It seem that there is no workaround until you are behind a firewall with authentification. It is a basic problem with the current palladian nodes, but the developers wrote that they will be working on this within the next version.

 

Frank

 

 

Exactly. Please have a little patience, as I just returned from holidays. But I will have a fix ready during the next weeks.

Best,
Philipp

Looking forward to having this as well - the earlier the merrier. ;-)

 

E

First, my apologies for the long delay. This nightly build will finally support proxies with authentication.

As we do not have a proxy server which I could have used for extensive testing, I had to refrain to some locally set up squid server to do my tests. Authentication worked well, but I did not have any time for excessive testing. If you should still encounter any issues, please let me know!

(only the LocationViewer node might still have issues, but there is restructuring/improvement going on in that direction anyways).

Best,
Philipp

Sounds great! I will test the new feature extensively after my holiday.

 

Frank

Hi Philipp,

 

Palladian now works behind a fire wall. I tried to retrieve some external feeds and webpages - and it works!

 

Thanks!  

Frank

 

Hi Frank,

glad to hear and thank you for reporting back!

Best,
Philipp

Hi Philipp,

 

do you plan to implement this new feature in the next stable release? When will you release the next "stable" version?

 

Frank

 

Hi Frank,

yes, definitely (the stable branch currently lags long behind unfortunately). Expect an update within the next month.

Regards,
Philipp