[apologies if this is late in the discussion.  I have spent the last couple of hours battling with various verschluggener mail profiles for reestablishing a setup that works for me to post on squeak-dev [my problem and new ISP ;-]]

Michael van der Gulik <mikevdg@gmail.com>  wrote...

Is the policy of the VM makers (whoever they currently are) to prevent the VM from crashing, particularly when given malicious bytecodes?

This is a general question, mostly related to http://bugs.squeak.org/view.php?id=1395 which is now closed. Is it considered a bug if I can crash the VM with a maliciously crafted method?
Which direction would the Squeak community want to go in? Should we aim to have a VM that would never seg fault and dump core (or blue screen under Windows), regardless of what rubbish is fed to it? Doing extra sanity checks and bounds checking would possibly have a performance penalty.

This is an interesting topic, and at this point I can't presume to answer where the community wants to go with it.  I can say that, as first designed, the VM was considered to be perfect if it would execute every valid method perfectly, and ideal if it did so at maximum speed.

To deal with garbage would require a lot of checks to be made, all of them in frequent bytecodes, and many of them more expensive than the actual operations themselves.  For instance, it would appear that you would want to do range checks on loads, stores and jumps, which is just about all there is.   I suppose you could do less than this and still make it uncrashable, but if one is going to pay this kind of attention to integrity, one would want to tell the user at the first sign of error, rather than just keep making errors but not crash.  This would mean checking load bounds rather than waiting until something weird happens later because you loaded garbage from an invalid location.

Therefore, I suspect that things would be much more complicated, and would run something like 2-3 times slower than before.  And what, exactly, is improved over VMs of the last 10 years?  My opinion, FWIW, is that execution of valid methods well compiled is still a good success criterion for the VM.  If someone wants to feed the VM from a different source, then they should provide appropriate integrity checks on their bytecodes.  And if security is the real agenda here, then it's probably best to acknowledge that up front and look at the bigger system issues involved (http://www.ERights.org  being a good place to start ;-), before worrying about malicious bytecodes.

        - Dan