Hi Tim,

On Wed, Jun 29, 2016 at 10:24 AM, tim Rowledge <tim at rowledge.org> wrote:

>
> > On 29-06-2016, at 7:08 AM, Levente Uzonyi <leves at caesar.elte.hu> wrote:
> >
> > This seems to be a missing #flush after changes are written to the file.
> > Without #flush both processes (unix) will maintain their own version of
> the file in memory.
>
> Pretty much exactly what I was about to type. We just had part of this
> discussion wrt Scratch project files on the Pi - adding flush/sync etc.
>
> In many cases letting an OS buffer the buffering of the buffer’s buffer
> buffer is tolerable - though insane, and wasteful, and a symptom of the
> lack of careful analysis that seems to pervade the world of software these
> days - because nothing goes horribly wrong in most cases. Everything
> eventually gets pushed to actual hardware, the system doesn’t crash,
> evaporate, get zapped by Zargon DeathRay(™) emissions, the power doesn’t
> get ripped out etc. Evidently, on a Pi in a classroom we can’t assign quite
> such a low probability to the Zargon problem.
>
> However, the changes file is supposed to be a transaction log and as such
> I claim the data ought to hit hardware as soon as possible and in a way
> that as near as dammit guarantees correct results. So the mega-layer
> buffering is An Issue so far as I’m concerned.
>
> We also, still, and for decades now, have the behaviour I consider stupid
> beyond all reason whereby a file write is done by
> a) tell the file pointer to move to a certain location
> b) think about it
> c) oh, finally write some text to the file.
> With the obvious possibility that the file pointer can be changed in b)
> Then if you can open-for-write a file multiple times, how much confusion
> can that actually cause? What about a forked process with nominally the
> same file objects? Are we at all sure any OS properly deals with it? Are we
> sure that what is purportedly ‘proper’ makes any sense for our requirements?
>
> The most obvious place where this is an issue is where two images are
> using the same changes file and think they’re appending. Image A seeks to
> the end of the file, ‘writes’ stuff. Image B near-simultaneously does the
> same. Eventually each process gets around to pushing data to hardware.
> Oops! And let’s not dwell too much on the problems possible if either
> process causes a truncation of the file. Oh, wait, I think we actually had
> a problem with that some years ago.
>

The thing is that this problem bites even if we have a unitary primitive
that both positions and writes if that primitive is written above a
substrate that, as unix and stdio streams do, separates positioning from
writing.  The primitive is neat but it simply drives the problem further
underground.

A more robust solution might be to position, write, reposition, read, and
compare, shortening on corruption, and retrying, using exponential back-off
like ethernet packet transmission.  Most of the time this adds only the
overhead of reading what's written.

_,,,^..^,,,_
best, Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20160629/adada25f/attachment.htm