Safety.  One can no longer overwrite literals or methods (or anything else whose state you want to protect) accidentally.

Expressive power.  One can use immutability as a write barrier in orthogonal persistence or distribution schemes (and other things) .  One marks an object as immutable to indicate that it is persistent or distributed and its changes should be exported.  The immutability scheme allows one to associate managers for specific objects that handle NoModificationErrors on a per-object basis.  So if an object is immutable and has an appropriate manager that manager can temporarily enable immutablity for the object, allow the modification, communicate the modification (e.g. to a database or remote image) and then re-enable immutability.  The framework allows an object to have multiple managers.  (This is all standard Squeak and very similar to the code I wrote for VW.  The VM code is also available from Cadence).

No.  As you point out Symbols are unique and so immutable strings don't replace them.  But it is nice to have immutable Symbols.  e.g. try to do this in your Squeak image:

    #class pvtAt: 1 put: 0 asCharacter

Scary :)