check for new zip file , download and extract ftp

KNIME Analytics Platform
1

Hi there,

My data aggregator platform will feed csv file in zipped format every week to my server host (ftp)

I'd like KNIME toperform these task:

  1. connect to ftp
  2. check if there is new zip file that has not yet been downloaded in the remote folder
  3. download those zip files
  4. extract the csv from downloaded zip files

for no.1 I can find ftp connector node, but I'm stuck on the no.2 -4 at the moment.

Your advice will be greatly appreciated,

cheers,

Kris.

2

Hi Kris,

nice project!

So you did setup the FTP connector node?

Than I would use a list file node on the local directory and a remote list files node on the remote directory. Afterwards, you need the URL to File Path node, (on both list files tables)  to extract the file name from the path. Now I would use a reference row splitter to get only the files from remote which are not in the local. And finally those can be downloaded with the download node again.

Does this makes sense?

Best, Iris 

3

Hi Iris,

Thanks a million!! will try this out and let you know. :)

4

Hi Iris,

I'm trying the remote list file node and it never finish processing as picture attached.

I tried different path, even public_html still no success.Any light?

Here are the log

***                Copyright by KNIME GmbH, Konstanz, Germany                ***

********************************************************************************

Log file is located at: /Users/apichakriskalambasuta/knime-workspace/.metadata/knime/knime.log

WARN  Row Filter           0:17       No row filter specified

WARN  FTP Connection       0:25       No configuration available

WARN  List Remote Files    0:26       No settings available

WARN  List Remote Files    0:26       No settings available

WARN  List Remote Files    0:26       Directory missing

WARN  List Remote Files    0:26       Directory missing

5

Hi,

what is the message when you mouse over the progress?

Maybe there are a lot of files in the folder?