[solved] Download node not working

KNIME Extensions
1

Hi,

I have a general problem downloading a file using some of the new File Handling node. As a simple reproduceable example, I have the following configuration:

FTP Connection -> Download

Where:

FTP Connection Node Settings:

Host: ftp.ncbi.nih.gov

Port: 21

Authentication: None

Test connection: Succeeds

and:

Download Node Settings:

Source file or folder: ftp://ftp.ncbi.nih.gov/pubchem/data_spec/pubchem_fingerprints.txt

Path handling: Use full path

Target folder: file:/C:/

If exists: Overwrite

 

Running the Download node causes a NullPointerException with the following stack trace:

 

2013-01-04 12:28:34,164 DEBUG KNIME-Worker-8 Download : Execute failed: ("NullPointerException"): null
java.lang.NullPointerException
    at org.knime.base.filehandling.remote.connectioninformation.port.ConnectionInformation.fitsToURI(ConnectionInformation.java:195)
    at org.knime.base.filehandling.remote.files.RemoteFileFactory.createRemoteFile(RemoteFileFactory.java:87)
    at org.knime.base.filehandling.download.DownloadNodeModel.execute(DownloadNodeModel.java:123)
    at org.knime.core.node.NodeModel.executeModel(NodeModel.java:536)
    at org.knime.core.node.Node.invokeNodeModelExecute(Node.java:1000)
    at org.knime.core.node.Node.execute(Node.java:894)
    at org.knime.core.node.workflow.SingleNodeContainer.performExecuteNode(SingleNodeContainer.java:895)
    at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:100)
    at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:166)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:124)
    at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:239)

 

Can anybody download this file? I've tried out different FTP sites / files / configuration without success.

 

The node Download / Upload from List doesn't work either. Here I get an exception "Login failed", even though the FTP Connection was successful. Stack trace:

 

java.io.IOException: Login failed
    at org.knime.base.filehandling.remote.files.FTPRemoteFile$FTPConnection.open(FTPRemoteFile.java:484)
    at org.knime.base.filehandling.remote.files.RemoteFile.open(RemoteFile.java:118)
    at org.knime.base.filehandling.remote.files.RemoteFileFactory.createRemoteFile(RemoteFileFactory.java:129)
    at org.knime.base.filehandling.downloaduploadfromlist.DownloadUploadFromListNodeModel.execute(DownloadUploadFromListNodeModel.java:136)
    at org.knime.core.node.NodeModel.executeModel(NodeModel.java:536)
    at org.knime.core.node.Node.invokeNodeModelExecute(Node.java:1000)
    at org.knime.core.node.Node.execute(Node.java:894)
    at org.knime.core.node.workflow.SingleNodeContainer.performExecuteNode(SingleNodeContainer.java:895)
    at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:100)
    at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:166)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:124)
    at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:239)

 

Best

Gideon

2

Hi Gideon,

the source and target settings are only meant for the path to the file/folder, without the protocol and host info. Does that solve your first problem?

 

Example:
Source file or folder: /pubchem/data_spec/pubchem_fingerprints.txt
Target folder: C:/

 

I could not replicate the second problem. Can you upload your workflow or describe it in more detail?

 

Thanks,
Patrick

3

Hi Patrick,

 

thanks for the tip. I followed your suggestion, but the first problem still exists, although with a different exception:

 

2013-01-07 18:23:41,154 DEBUG KNIME-Worker-10 Download : Execute failed: Permission denied: recv failed
java.net.SocketException: Permission denied: recv failed
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:150)
    at java.net.SocketInputStream.read(SocketInputStream.java:121)
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
    at java.io.InputStreamReader.read(InputStreamReader.java:184)
    at java.io.BufferedReader.fill(BufferedReader.java:154)
    at java.io.BufferedReader.read(BufferedReader.java:175)
    at org.apache.commons.net.io.CRLFLineReader.readLine(CRLFLineReader.java:58)
    at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:310)
    at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:290)
    at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:479)
    at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:552)
    at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:601)
    at org.apache.commons.net.ftp.FTP.pasv(FTP.java:952)
    at org.apache.commons.net.ftp.FTPClient._openDataConnection_(FTPClient.java:755)
    at org.apache.commons.net.ftp.FTPClient._retrieveFileStream(FTPClient.java:1747)
    at org.apache.commons.net.ftp.FTPClient.retrieveFileStream(FTPClient.java:1739)
...

 

I've attached a workflow (knime_ftp_test1.zip) to reproduce the error (I'm using KNIME 2.7.0).

 

For the second problem I originally mentioned, I've also produced and attached a workflow (knime_ftp_test2.zip). The second example produces a NullPointerException:

2013-01-07 18:51:54,310 DEBUG KNIME-Worker-15 Download / Upload from List : Execute failed: ("NullPointerException"): null
java.lang.NullPointerException
    at org.knime.base.filehandling.remote.files.FileRemoteFile.getParent(FileRemoteFile.java:217)
    at org.knime.base.filehandling.remote.files.RemoteFile.mkDirs(RemoteFile.java:425)
    at org.knime.base.filehandling.downloaduploadfromlist.DownloadUploadFromListNodeModel.execute(DownloadUploadFromListNodeModel.java:152)
    at org.knime.core.node.NodeModel.executeModel(NodeModel.java:536)
    at org.knime.core.node.Node.invokeNodeModelExecute(Node.java:1000)
    at org.knime.core.node.Node.execute(Node.java:894)
    at org.knime.core.node.workflow.SingleNodeContainer.performExecuteNode(SingleNodeContainer.java:895)

 

Thanks

Gideon

4

 

Hi Gideon,
 
the first exception seems to be a problem with the windows firewall.
There seem to be three possible solutions:
1. Hotfix from Microsoft:
2. Executing the following command:

netsh advfirewall set global StatefulFTP disable

3. Adding the following argument to the JavaVM:

-Djava.net.preferIPv4Stack=true

This has to be put in the knime.ini
 
I hope this fixes the issue for you.
 
The Download node currently has the issue of not creating the folderstructure in the case a single file is selected instead of a folder. This will be resolved in 2.7.1. Until then it is advised to use the option use only filename.
 
In the second workflow there are two problems.
1. The URI to a local file usually starts with file:/ and then the local path
2. The target has to point to the file you want to write.
In your case file:/C:/pubchem_fingerprints.txt should do
 
Regards,
Patrick

 

1 Like
5

Excellent! That sorted it out for me, thanks Patrick.