Hello together,

A question about failover and reliability tuning.

Location A has a server with a cronjob that every 15 minutes asks
Location B whether or not there's any data to download. 'data' is
directories with 1-100 files of the size 3-8 MB.

In case there's data, a download is started utilizing curl within php.
We download 1 file at a time for possible concurrent connections, and
have MD5SUM checking in the end. In case a file was transferred
incorrectly, it's deleted and the next cronjob run will download it.

We grab a mutex for every cronjob to avoid having more than one cronjob
started at a time.

Now Location A is in a place in the world where internet connections are
slow, unreliable and expensive--thus leaving it to break down for 1-10
minutes quite often.

When such a breakdown happens, when the connection returns, our curl
download is not resuming. It just hangs, leaving downloads to not
proceed since the job hanging still holds the lock.

Had the curl part that does the actual download simply exited with an
exit code or just ended, or cleanup code after the download routine
would delete all files that wasn't correctly transferred for the next
cronjob start to pick up and transfer once more.

So what to do? Is there a curl option to tune it to time out? Do I have
to monitor the progress from the outside and send it a signal when it
seems dead? Should I install a signal handler in php to invoke from the
outside?

-- 
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                 -- A. P. J.
_______________________________________________
http://cool.haxx.se/cgi-bin/mailman/listinfo/curl-and-php