cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: How to download only new files from FTP server

From: Guenter <lists_at_gknw.net>
Date: Tue, 18 Aug 2009 12:03:10 +0200

Hi,
Daniel Stenberg schrieb:
> On Mon, 17 Aug 2009, Ankur Saxena wrote:
>
>> I want to poll an FTP server at regular intervals, and download only
>> newly added files. I looked at CURLOPT_FILETIME / curl_easy_setopt()
>> and CURLINFO_FILETIME / curl_easy_getinfo() functionality, but I am
>> not sure if this is the way to go.
>>
>> Could someone tell me how to achieve this?
>
> In short: it's a pain. libcurl offers the ability get a file
> conditionally based on its date over FTP (with CURLOPT_TIMECONDITION),
> but you need to specify exactly that file in the URL then.
>
> If you want to list a directory to find out which files that are new,
> you're really left to either to LIST or NLST and then parse the results
> in your app, but none of the directory listing outputs are standardized
> so you'd have to either make a very clever parser or you handicraft it
> specificly for the server(s) you know you'll work with.
I thought something similar, and if the above usage is all the app needs
to do I wouldnt use C / libcurl at all - instead you can do this with
Perl with only few lines; few weeks ago I just coded exactly such script
for a friend; also the huge benefit if a scripting language here is that
it works on almost every platform unchanged, and that you can modify
things without need for a development environment. If you're interested
you can contact me directly, and I clean up the script a bit for general
purpose and send you.

Günter.
Received on 2009-08-18