> I think that Bolot's new PWS implementation is all stream-based > rather than passing strings around -- both for speed and for memory
> hits.  

streamContents rules!  Indeed, avoding unnecessary concatenations and non-linear appends is one of the easiest and most direct ways to speed up string-intensive code.  Moreover, I have discovered that doing this properly has a tendency to make code clearer and more readable (although it does have the unfortunate tendency to look more like traditional imperative code).

> (Try to serve a 25M QuickTime Star Wars Trailer in the current > PWS :-)  This was one of the tests that Bolot did of the current > system.

You will be more likely to get to the front of a line to see the film before the file is served.  Moreover, LucasArts is a notoriously agressive intellectual property plaintiff.

> Hmm, I just wrote a tiny-and-still-incomplete HTML tag scanner for my > class as a demonstration

> A hand-built scanner will probably be faster than a regular > expression system.

In the case of NFA-style re systems, this is almost certainly correct.  The re-based scanner might be more readily malleable, however.  A linear, DFA-based table scanner can be breathtakingly fast and is hard to beat, but like the hand-based scanner is unlikely to be easily malleable by general users of the code.

And, of course, depends upon whose hands are building the hand-built scanner.  :-)

Some thought might be given, however, to consider where inner loop speedups will result in substantial improvement to the system as a whole, whether reducing these byte-pushing syntactic operations to a plugin primitive that can be interpreted or compiled as needed, might not be the best compromise between hard-coded complexity-for-speed and ease-of-reuse.

Perhaps a first step would be to isolate a general byte-pushing syntactic operations strategies class that admits a variety of implementations -- some tight and hand-coded for speed, others less speedy, but clear, concise and readily capable of being reused.  (Pluggability rules).