cURL / Mailing Lists / curl-users / Single Mail

curl-users

Re: curl and fallocate

From: Dan Fandrich <dan_at_coneharvesters.com>
Date: Wed, 5 Feb 2014 09:30:31 +0100

On Tue, Feb 04, 2014 at 11:18:46PM +0100, Manfred Schwarb wrote:
> Another idea:
> instead of doing the fallocate(2) inside curl, one could add an option which
> omits the O_TRUNC on open and does a ftruncate(2) at program termination time.
>
> Then the user would be responsible in preallocating the file if desired.

The problem here is that in most cases the user won't know the size to
preallocate until curl gets it from the server.

> This could also be interesting when you mirror some files at a high rate which
> do change in contents, but not in length. You could then simply write into
> the already existing file. Perhaps there are even filesystems / OS's which
> do optimize away the write operation if to-be-written and pre-existing hunks are the same.

I don't know if that's the case or not. It would make sense for SSD drives to
avoid needless flash write cycles.

>>> Dan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ: http://curl.haxx.se/docs/faq.html
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2014-02-05