EndOfStream unused

nicolas cellier ncellier at ifrance.com
Wed Nov 7 03:41:59 UTC 2007


Andreas Raab a écrit :
> nicolas cellier wrote:
>> Wrong.
> 
> What exactly is wrong? That it does stack searches? That it evaluates 
> handler blocks? That it will cause pain due to unforeseen interactions? 
> That it is slow? Let's start there. How about a little benchmark:
> 
> ReadStream subclass: #ReadStreamWithNil
> 
> ReadStreamWithNil>>next
>     <primitive: 65>
>     position >= readLimit
>         ifTrue: [^EndOfStream signal]
>         ifFalse: [^collection at: (position := position + 1)]
> 
> 
> And now:
> 
> streamClass := ReadStream. "vs. ReadStreamWithNil"
> data := (1 to: 5) asArray.
> [1 to: 100000 do:[:i|
>     stream := streamClass on: data.
>     [stream next == nil] whileFalse.
> ]] timeToRun.
> 
> If you run this trivial little benchmark, the results should be 
> enlightening: With ReadStream it completes (on my box) within 280 msecs. 
> With ReadStreamWithNil it completes in 1617 msecs. That is a factor of 
> 6x in speed. Even if you change it to 100 elements in data you are 
> *still* at half of the speed (1645 vs. 3172 ms).
> 

Common, I could expect better from you.
This test is totally biased.
You could as well write

streamClass := ReadStream. "vs. ReadStreamWithNil"
data := (1 to: 100000) asArray.
[1 to: 5 do:[:i|
     stream := streamClass on: data.
     [stream next == nil] whileFalse.
]] timeToRun.

And obtain different results (260 original, 278 with EndOfStream)
Now, I'am not spoiling so much.

And then:

streamClass := ReadStreamWithNil.
data := (1 to: 100000) asArray.
[1 to: 5 do:[:i|
     [| aStream |
	aStream := streamClass on: data.
     [aStream next. true] whileTrue] on: EndOfStream do: [:exc | exc 
return].
]] timeToRun.

255 fine, now I am as fast if not faster than == nil.

And far better than:

streamClass := ReadStream. "vs. ReadStreamWithNil"
data := (1 to: 100000) asArray.
[1 to: 5 do:[:i|
     stream := streamClass on: data.
     [stream atEnd] whileFalse: [stream next].
]] timeToRun.

433 too bad if ever you have to handle collections with nils...


OK, I loose with nicer code:

streamClass := ReadStreamWithNil.
data := (1 to: 100000) asArray.
[1 to: 5 do:[:i|
     stream := streamClass on: data.
     [stream next] repeatUntil: EndOfStream.
]] timeToRun.

  477 simply because repeat not inlined by compiler... nothing to do 
with exception handling...

Enough with these bench...


>> VW does support it, and concerning efficiency, they are not that crazy.
> 
> VW isn't Squeak. If you think that VWs and Squeaks exception handling 
> have comparable performance, allow me to laugh heartily. Besides which 
> VW is a *lot* faster to begin with so the overhead is less noticable in 
> real applications (though I'm sure the overhead is measurable).
> 
>> True, there is a stack scan, but ONLY ONCE at the end of the stream.
>> If the stream is long enough, it will save A LOT of atEnd tests.
> 
> Err, only if the stream *has* atEnd tests. Most code that is concerned 
> with efficient stream reads today goes like this:
> 
>   [(value := stream next) == nil] whileFalse:[].
> 

There are simply 267 senders of atEnd in my 3.10, most in while loops.

And once again, == nil is not general enough but for some character 
streams and the like.


> No atEnd tests are saved in that situation. But even if we change our 
> benchmark to, e.g.,
> 
> data := (1 to: 5) asArray.
> [1 to: 100000 do:[:i|
>     stream := ReadStream on: data.
>     [stream atEnd] whileFalse:[stream next].
> ]] timeToRun.
> 
> It comes in at 325 msecs on my box with is only 30% worse than the 
> "naked" stream next == nil test and about 4x *faster* than using 
> EndOfStream. And if you extend data to 100 elements it still comes in 
> right in the middle of the other variants (2500 msecs).
> 
>> This is called optimistic programming.
> 
> And what I do is called "measuring" ;-)
>

Sure it's not called unfair biased argumentation?

Of course there is a trade-off in optimistic programming. It relies on 
Exception being rare. Your example put the cursor at opposite.

Now, I admit that i can degrade some short stream created in tight loops 
and optimized with == nil tests. But you have to show some real example, 
something more serious than above tricks.


>> Imagine that I ask you at each step "Are we arrived?"; you would not 
>> bear a very long walk, would you? That's what the atEnd test is doing.
> 
> True, when it's there. But unless you change exception handling it's 
> often (in particular for internal streams) still a *lot* faster since EH 
> is expensive in tight loops.
> 

I know that.
My aim is not to use it in tight loops!
Only once at end of stream.

>> In following mail and at http://bugs.squeak.org/view.php?id=6755 I 
>> already noticed possible exception handling problem that caused Marcus 
>> to retract this change. This is because EndOfStream were declared an 
>> Error instead of a Notification.
> 
> Have you actually *tried* it? Because you may be in for a nasty little 
> surprise. I'm not sure if this problem is going to bite you or not but 
> from the code it looks like it should so try it - it is just about 
> *exactly* the kind of thing that goes wrong for "no good reason" when 
> you change something as fundamental as this.
> 

That is wise. Of course it deserve testing! who is saying the contrary?
I would not like myself that some guy do impose such potentially 
dangerous changes in my image. He has to prove first for sure.
I do not want to impose it now. I want it to be discussed.

I passed some of the tests in my image (not all, because they hang my 
3.10 without the change).
No problem so far.

Why? because I'am not that crazy, I turned the Error into a 
Notification. Period.

Now it's still dangerous if a fool is catching Notification or even 
better Exception!
Or if some Exception mechanism use streaming themselves...
Who knows...

>> And testing next == nil is a ugly hack i reject because i have some 
>> collections with some nil.
> 
> Then use atEnd. That's what it's for.
> 
>> I want for example to use
>>     aCollection lazily
>>         collect: [:e | e color];
>>         select: [:e | e notNil]
>> using LazyStreams iterating only once.
>> Absence of EndOfStream notification is just spoiling the game.
> 
> I don't know. I find it hard to imagine an atEnd test that could 
> possibly be as costly as running the EH machinery. It's certainly worth 
> measuring before conjecturing about it.
> 

It is, because a SelectStream doing a select: operation is duplicating 
the job calling the block once in atEnd test and another in next, AND 
because I cannot use == nil trick in above example.


>> But nevermind, I will just publish on VW public store where I will 
>> find crazy guys interested.
> 
> As you'd like. If you are ever interested in having a serious discussion 
> about the topic I'll be waiting here.
> 

This was an answer to crazy.
4 AM. For a really serious discussion, I now need to rest a little.

> Cheers,
>   - Andreas
> 
> 

Agree that your arguments are not all wrong. But the way you push them 
is more than unfriendly.
These points deserve discussion. No use to turn it into bashing!

Cheers




More information about the Squeak-dev mailing list