You might consider using a web spidering tool such as this rudimentary
one I wrote in PHP (but there are probably better solutions that do
use curl).

http://code.google.com/p/phpspider/

----
Sent from my mobile computing device. Please excuse my brevity and  
misspellings.
On Sep 21, 2009, at 4:04 PM, David Romano <romanod_at_math.grinnell.edu>  
wrote:
> Hi,
>
> I want to dowload a small directory of .pdf documents using http,  
> and a colleague suggested I use curl.  From the man page and FAQ it  
> seems that curl cannot get these recursivey (say, by asking for the  
> parent directory and its contents), but I was hoping to at least use  
> curl to recreate the directory structure locally using --create- 
> dirs.  The trouble is I can't seem to get the sytax correct.  The  
> directory structure is very simple:
>
> top directory
> www.pdf.org/subject
>
> children directories
> www.pdf.org/subject/amy
> www.pdf.org/subject/birgid
>
> where amy and birgid contain only pdf files, named according to  
> content (no serially or sequentially).
>
> Any help suggestions would be greatly apreciated!
>
> Thank you,
> David Romano
>
> -------------------------------------------------------------------
> List admin: http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users
> FAQ:        http://curl.haxx.se/docs/faq.html
> Etiquette:  http://curl.haxx.se/mail/etiquette.html
-------------------------------------------------------------------
List admin: http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html