Struggling with proxies

#1

Hi all.

I am trying to download a file from an URL and need to set the proxy programatically. I read in Use Proxy Configuartion in custom node that the proxy settings are used automatically but failed to reproduce it. This is a five years old post so the code base may have changed.

For my test I added a download of the KNIME logo and debugged it.

private static boolean urlTest() {
    try (InputStream stream = FileUtil.openStreamWithTimeout(
        new URL("https://www.knime.com/themes/custom/bootstrap_knime/logo.svg"))) {
      return true;
    } catch (final Exception e) {
      return false;
    }
}

Here I am using FileUtil#openStreamWithTimeOut which produces a timeout exception behind a proxy. In the source code I did not find any proxy configuration.

public static InputStream openStreamWithTimeout(final URL url, final int timeout)
        throws IOException {
    URLConnection conn = url.openConnection();
    conn.setConnectTimeout(timeout);
    conn.setReadTimeout(timeout);
    return conn.getInputStream();
}

To actually make it work I needed to set the proxy manually in my client code.

// TODO: From KNIME FileUtil. To be deleted.
public static InputStream openStream(final URL url) throws IOException {

    final Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(some_host, some_port));

    final URLConnection conn = url.openConnection(proxy);
    conn.setConnectTimeout(1000);
    conn.setReadTimeout(1000);
    return conn.getInputStream();
}

This is far from desirable. I should set the proxy for every client with one. Then I tried to retrieve the proxy configuration from the Eclipse configuration in Network connections from eclipse.core.ui.net. Here I am taking the proxies entered by the user in that dialog.

final IProxyService proxyService = ProxyManager.getProxyManager();

final IProxyData[] pd = proxyService.getProxyData();

if (proxyService.isProxiesEnabled()) {
    final IProxyData httpProxy = proxyService.getProxyData(IProxyData.HTTP_PROXY_TYPE);
    if (httpProxy != null) {
        if (httpProxy.getHost() != null && httpProxy.getPort() != -1) {
            System.setProperty("http.proxyHost", httpProxy.getHost());
            System.setProperty("http.proxyPort", Integer.toString(httpProxy.getPort()));
        }
    }

    final IProxyData httpsProxy = proxyService.getProxyData(IProxyData.HTTPS_PROXY_TYPE);
    if (httpsProxy != null) {
        if (httpsProxy.getHost() != null && httpsProxy.getPort() != -1) {
            System.setProperty("https.proxyHost", httpsProxy.getHost());
            System.setProperty("https.proxyPort", Integer.toString(httpsProxy.getPort()));
        }
    }
}

Unfortunately the proxy service is returning empty proxies unlike the ones in my KNIME workspace. I am debugging my RCP application against this KNIME workspace and when I start standalone KNIME all the proxies are there. Maybe I have to query the preference store instead.

Anyway, as you can see it is pretty troublesome to get this configuration and it would be a great help if you could point me to the KNIME API that downloads files with the user-given configuration or any tips with the Eclipse code I pasted.

Lots of thanks,
Miguel

0 Likes

#2

Hi Miguel,

I tried to reproduce your issue, but was unable to do so, with KNIME 4.0.1 I could define a proxy server in my preferences, and then ran your logo download test code during a nodes execute() The first time I ran the node it asked me for the proxy credentials, but the node ran through fine.

Can you tell me more about your setup? What kind of proxy are you using and how have you defined it in the preferences? I

best,
Gabriel

2 Likes

#3

Hi Gabriel,

thanks. I made a quick node around it using the KNIME 3 and 4 target platforms and failed to reproduce it. I think something in my RCP application must be wrong. The test is a fork of the SDK setup project https://github.com/miguelalba/knime-sdk-setup/tree/releases/2019-06.

I am using a webproxy on port 8080. It has no authentication and I have it configured under Network connections as HTTP and HTTPS proxies for the manual provider.

Thanks a lot,
Miguel

0 Likes