Download files from list of URLs

Hi,

I have a list of URLs pointing to subsets of the ZINC database. On Linux these can be downloaded one after another by issuing

wget -i <file_with_urls>

Is there a Knime node or workflow that can do something similar?

Thanks/Evert

The 'Download'-node, e.g., combined with the 'HTTP Connection'-node, is the one you're looking for? (both are part of the 'KNIME File Handling Nodes' plugin)

Maybe a File Reader node to read the file with the list of URLs, and then if they are text files (as opposed to e.g. HTML), then:

If you want each URL to be loaded into a single cell...

Use the Vernalis 'Load text-based files' node which should do the job

If you want the files to be loaded with one line of the file per row of table...

You need a 'Table Row to Variable Loop Start' --> 'File Reader' --> 'Loop End' sequence of nodes

If the files are e.g. SDF files, then replace the File reader in the loop in the second option with an SDF reader.

Hope this helps

Steve

Just what I was looking for!

Much appreciated/Evert

By the way, the 'Load text-based files' is the reason I always have the vernalis plugin installed, great idea! Very useful :-)

1 Like