cURL / Mailing Lists / curl-users / Single Mail

curl-users

Implement "Safe Put" with curl

From: Yair Lenga <yair.lenga_at_citigroup.com>
Date: Tue, 20 Feb 2007 10:04:55 -0500

Hi,

I'm currently using a small wrapper around the FTP command line to
implement "safe put" from production scripts (bash scripts). The "Safe"
version is used to upload data to a busy site, where users are pulling
data all the time. Before implementing the safe method I had instances
where a reader pull a file in the middle of an upload, or that two
concurrent upload into the same file resulting in "mixed" content.

I would like to switch to using "curl" upload function - it will allow
me to use the same code for HTTPS and FTP uploads. Does anyone know if
there is any plan to add those functions to cURL "-T". For example:

   --upload-suffix=suffix # Will upload into /path/to/file.suffix,
then rename /path/to/file.suffix -> /path/to/file
   --upload-unique # to upload using STOU, then rename the
result file (if necessary) to the target name

My current scripts look like:

Using program generated temporary file - FTP command line

    ftp ...
    user USER PASSWORD
    cd /path/to/dir
    put newfile newfile.tmp
    rename newfile.tmp newfile
    quit

The second alterntive is using the "Store Unique" feature - using a perl
wrapper. I've found that the implementation of unique feature is usaully
working, but sometime it does not work when running a cross proxies, etc.

    my $target = "/path/to/file" ;
    my $ftp = new Net::FTP(...)
    $ftp->login(...)
    my $unique_name = $ftp->put_unique("file", $target)
    $ftp->rename($unique_name, $target) if $target ne $unique_name ;
    $ftp->quit
Received on 2007-02-20