cURL / Mailing Lists / curl-library / Single Mail

curl-library

Re: fflush and fsync

From: Jamie Lokier <jamie_at_shareable.org>
Date: Tue, 22 Apr 2008 20:10:24 +0100

Dan Fandrich wrote:
> On Tue, Apr 22, 2008 at 05:48:51PM +0100, Jamie Lokier wrote:
> > Seems fine to me if both programs are on the same machine.
> >
> > If they are on different machines, 'file A written' and 'file B
> > created' can be seen out of order on the other machine. Is that
> > relevant to the remote testing?
>
> It's very relevant. When I was getting remote testing working a few months
> ago I tried it on two machine with a shared NFS directory; the file syncing
> problems were very evident in this configuration. I "solved" it by simply
> adding a delay of several seconds to the test harness after running each
> test step before checking the output files. Even that wasn't completely
> reliable, but I didn't attempt to improve it any further.
>
> I don't know whether there's a way to guarantee the file synchronization
> semantics we need in all possible remote filesystems (Coda would be an
> extreme example). Since the remote case currently requires some customizing
> of runtests.pl anyway, it's probably not worth putting too much effort into
> this case, but concentrate on making the single machine case reliable.

The following is a rather general description.

The problems typically arise from clients caching file contents and
stat() results of remote files, and contents of directories. Often a
time-based cache is used: after reading something, the client assumes
it can report the same data for X seconds (e.g. 1 minute is not
unheard of), and doesn't check reliably on every read. Also, even if
it does check, it might check the file's mtime, so if that didn't
change (1 second resolution), the client assumes the file or directory
contents haven't changed.

The following should be quite reliable however:

   - When machine 1 creates a new file with a new name (e.g. random),
     writes it, closes it, then signals it's done (e.g. by the remote
     program exiting), then machine 2 opens the file for reading, and
     machine 2 has not tried to open a file with that name
     previously, then machine 2 will see the whole contents.

-- Jamie

>
> >>> Dan
> --
> http://www.MoveAnnouncer.com The web change of address service
> Let webmasters know that your web site has moved
Received on 2008-04-22