> On 29-06-2016, at 10:35 AM, Eliot Miranda <eliot.miranda at gmail.com> wrote:
> 
{snip much rant}

> The most obvious place where this is an issue is where two images are using the same changes file and think they’re appending. Image A seeks to the end of the file, ‘writes’ stuff. Image B near-simultaneously does the same. Eventually each process gets around to pushing data to hardware. Oops! And let’s not dwell too much on the problems possible if either process causes a truncation of the file. Oh, wait, I think we actually had a problem with that some years ago.
> 
> The thing is that this problem bites even if we have a unitary primitive that both positions and writes if that primitive is written above a substrate that, as unix and stdio streams do, separates positioning from writing.  The primitive is neat but it simply drives the problem further underground.


Oh absolutely - we only have real control over a small part of it. It would probably be worth making use of that where we can.

> 
> A more robust solution might be to position, write, reposition, read, and compare, shortening on corruption, and retrying, using exponential back-off like ethernet packet transmission.  Most of the time this adds only the overhead of reading what's written.  

Yes, for anything we want reliable that’s probably a good way. A limit on the number of retries would probably be smart to stop infinite recursion. Imagine the fun of an error causing infinite retries of writing an error log about an infinite recursion. On an infinitely large Beowulf cluster! 

It’s all yet another example of where software meeting reality leads to nightmares.


tim
--
tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
If it was easy, the hardware people would take care of it.