I've been thinking about immutability in Squeak too! And hell yeah it's bigger than databases.

- makes concurrency painless - improves testability - allows certain compiler optimizations that aren't possible when there's mutable state - makes certain things formally provable that aren't when there's mutable state

I think these are all great reasons to experiment with. I'd like to assert though: immutability should be introduced in a way that's optional, and doesn't introduce any new syntax.

I'd like to ask an object #isMutable and be able to do certain things differently if it answers false. Mutable objects are ones that have state that can change.

Randal: FWIW, I think the Newspeak guys are playing with this stuff.

On Wednesday, March 17, 2010, Stephen Pair stephen@pairhome.net wrote:

On Wed, Mar 17, 2010 at 12:27 PM, Bert Freudenberg bert@freudenbergs.de wrote:

On 17.03.2010, at 17:12, Chris Cunnington wrote:

...

Databases are for peasants. We are unique and occupy a unique position. Let's embrace that. Stop trying to turn us into shadows of Gemstone, Cimcom, or other merchants of tedium. If Kent Beck isn't happy with Squeak, I don't see it as our job to cater.

There is nothing else on earth like us and what we do. Squeak is a never ending game of capture the flag.

Chris

And even if it *was* only for databases it wouldn't hurt to support it in Squeak. It takes nothing away from the fun.

And it's not only about databases.  I would be interested in immutability implementations that go beyond what VW does.  I would like to see some thought given to deep immutability and irrevocable immutability (irrevocable at least at some language layer).  These facilities would be immensely useful for safely and efficiently sharing and replicating state between processes.

• Stephen

On 18 March 2010 03:35, Chris Muller asqueaker@gmail.com wrote:

This is where I start to get confused.  If "immutable" is a global attribute of each object, wouldn't there need to tend to need to be only one universal user of the bit at a time?  Otherwise, the database might be setting the bit on an object for it's purposes, but for which the [your other favorite framework] in having its own purposes for the bit, might not want it set at that time...?

Good point. Really, what you would do, if you having two frameworks, sharing same object, and both want to use (set/reset) immutable bit for own purposes? This creates a conflict.

I am of course, wondering about my own context.  In thinking about the purposes of an ODBMS, one of the chief beneciary's of this type of change, it could help Magma immensely.  But then Magma would have to impose on the user/application that, "the immutable bit belongs to Magma, don't use it or mess with it, ok?  I need it..."  But the feature is not there today, so at least legacy applications would enjoy the performance boost..

On Wed, Mar 17, 2010 at 1:26 PM, Casey Ransberger ron.spengler@gmail.com wrote:

...

I've been thinking about immutability in Squeak too! And hell yeah it's bigger than databases.

- makes concurrency painless  - improves testability  - allows certain compiler optimizations that aren't possible when there's mutable state  - makes certain things formally provable that aren't when there's mutable state

I think these are all great reasons to experiment with. I'd like to assert though: immutability should be introduced in a way that's optional, and doesn't introduce any new syntax.

I'd like to ask an object #isMutable and be able to do certain things differently if it answers false. Mutable objects are ones that have state that can change.

Randal: FWIW, I think the Newspeak guys are playing with this stuff.

On Wednesday, March 17, 2010, Stephen Pair stephen@pairhome.net wrote:

...

On Wed, Mar 17, 2010 at 12:27 PM, Bert Freudenberg bert@freudenbergs.de wrote:

On 17.03.2010, at 17:12, Chris Cunnington wrote:

...

Databases are for peasants. We are unique and occupy a unique position. Let's embrace that. Stop trying to turn us into shadows of Gemstone, Cimcom, or other merchants of tedium. If Kent Beck isn't happy with Squeak, I don't see it as our job to cater.

There is nothing else on earth like us and what we do. Squeak is a never ending game of capture the flag.

Chris

And even if it *was* only for databases it wouldn't hurt to support it in Squeak. It takes nothing away from the fun.

And it's not only about databases.  I would be interested in immutability implementations that go beyond what VW does.  I would like to see some thought given to deep immutability and irrevocable immutability (irrevocable at least at some language layer).  These facilities would be immensely useful for safely and efficiently sharing and replicating state between processes.

• Stephen

-- Casey Ransberger

On 18 March 2010 07:26, James Foster Smalltalk@jgfoster.net wrote:

On Mar 17, 2010, at 8:45 PM, Igor Stasenko wrote:

...

On 18 March 2010 03:35, Chris Muller asqueaker@gmail.com wrote:

...

This is where I start to get confused.  If "immutable" is a global attribute of each object, wouldn't there need to tend to need to be only one universal user of the bit at a time?  Otherwise, the database might be setting the bit on an object for it's purposes, but for which the [your other favorite framework] in having its own purposes for the bit, might not want it set at that time...?

Good point. Really, what you would do, if you having two frameworks, sharing same object, and both want to use (set/reset) immutable bit for own purposes? This creates a conflict.

Yes it does, so you need to make a decision about which of the two frameworks you will use, or you need to look at using them in such a way that they don't interfere with each other. Consider the two most likely candidates for use of immutability (in my experience): compiler and database proxy.  - With immutability, the compiler can share literals (strings, arrays, etc).  - With immutability, a database framework can recognize modifications to objects copied from a remote system.

Immutability is not the only way how one can address the immutability of literals. Take a symbols, for instance: an implementation inherently maintains a set of symbols and makes sure they are not mutable and unique. This means, that we could do the same for any other kinds of literals: arrays, strings, characters, and share them without the need of introducing an immutable bit.

For the most part, these two uses will not conflict. The potential conflict would be where a compiler literal is passed to the database framework and written to a remote store. By default (say) the database framework would mark the object immutable and if a modification was attempted it would log it as 'dirty' (needing to be written in the next commit) and then change it to be mutable. This would corrupt the compiler usage.

The solution is to have the database framework notice on its first write that the object is immutable and keep it in its cache with a 'immutable' flag so that it would not accept an attempt to modify it.

do you mean, like this:

object beImmutable. database commit: [ object ]. object beMutable. object setFoo: bar. object beImmutable. database commit: [ object ].

the above is an example when object, recorded as an immutable one, then mutated outside a DB transaction. So db can't capture the attempt to modify it. What GemStone doing to handle this?

Another thing, i think, is how to allow nesting:

object beImmutable. [ [ object setFoo: bar ] on: AttemptToModifyImmutable do: [:ex | framework1 handleException: ex. ] ] on: AttemptToModifyImmutable do: [:ex | framework2 handleException: ex. ]

in this way, if framework1 were one who set immutable bit, it should handle it and continue as normally. but, if framework1 seen this object before as an immutable , then it should pass the exception to outer layer, who may handle it - framework2. But then it could be problematical to have changes made by framework2 be seen by framework1.

The general rule is that one can only make mutable something that one has earlier made immutable. With that, the risk of conflicts is much reduced.

In any case, the potential for a conflict is not a reason to deny the feature. One simply has to use it carefully and with knowledge of what the frameworks expect.

I'd say that while immutability is helping to make databases more fast, it actually has a much wider usability. All that OODBs need is to track the objects which were modified during transaction. Immutability is not the only way how to achieve that.

James

On Thu, Mar 18, 2010 at 8:17 AM, James Foster Smalltalk@jgfoster.netwrote:

On Mar 17, 2010, at 11:28 PM, Igor Stasenko wrote:

...

the above is an example when object, recorded as an immutable one, then mutated outside a DB transaction. So db can't capture the attempt to modify it. What GemStone doing to handle this?

Actually, GemStone traps object modification at the byte-code level in the VM so does not rely on immutability to be informed of a modification.

Not preferrentially. It used to do that, but it is a problematic approach. The preferred implementation is above VM-supported per-object immutability. We've recently discussed this topic, I think on the Pharo list. Martin McClure of GemStone wrote excellent posts describing the implementations, their trade offs, and the higher levels of the framework which support installing managers for the immutability exception that supports multiple frameworks using immutability concurrently.

James

On 18.03.2010, at 06:26, James Foster wrote:

On Mar 17, 2010, at 8:45 PM, Igor Stasenko wrote:

...

On 18 March 2010 03:35, Chris Muller asqueaker@gmail.com wrote:

...

This is where I start to get confused. If "immutable" is a global attribute of each object, wouldn't there need to tend to need to be only one universal user of the bit at a time? Otherwise, the database might be setting the bit on an object for it's purposes, but for which the [your other favorite framework] in having its own purposes for the bit, might not want it set at that time...?

Good point. Really, what you would do, if you having two frameworks, sharing same object, and both want to use (set/reset) immutable bit for own purposes? This creates a conflict.

Yes it does, so you need to make a decision about which of the two frameworks you will use, or you need to look at using them in such a way that they don't interfere with each other. Consider the two most likely candidates for use of immutability (in my experience): compiler and database proxy.

• With immutability, the compiler can share literals (strings, arrays, etc).
• With immutability, a database framework can recognize modifications to objects copied from a remote system.

For the most part, these two uses will not conflict. The potential conflict would be where a compiler literal is passed to the database framework and written to a remote store. By default (say) the database framework would mark the object immutable and if a modification was attempted it would log it as 'dirty' (needing to be written in the next commit) and then change it to be mutable. This would corrupt the compiler usage.

The solution is to have the database framework notice on its first write that the object is immutable and keep it in its cache with a 'immutable' flag so that it would not accept an attempt to modify it.

The general rule is that one can only make mutable something that one has earlier made immutable. With that, the risk of conflicts is much reduced.

In any case, the potential for a conflict is not a reason to deny the feature. One simply has to use it carefully and with knowledge of what the frameworks expect.

James

IMHO there should be no way to reset the immutable flag. You can implement "soft" immutability in the image, but "hard" VM-level immutability needs to be permanent, no fiddling possible. Once set, the object stays immutable. Only a copy of an immutable object will be mutable again.

- Bert -

On 18.03.2010, at 15:51, Colin Putney wrote:

On 2010-03-18, at 5:39 AM, Bert Freudenberg wrote:

...

HO there should be no way to reset the immutable flag. You can implement "soft" immutability in the image, but "hard" VM-level immutability needs to be permanent, no fiddling possible. Once set, the object stays immutable. Only a copy of an immutable object will be mutable again.

This would be good for some use cases - literals, for example, but not so good for others. One of the common uses of "immutability" is not so much to *prevent* mutation, as to control it: databases, concurrency etc. Do you have another feature in mind for these cases, or some overriding interest in immutable immutability that you want to pursue?

Colin

Seems to me like two separate issues - one being informed of modifications, and one preventing modifications. Immutability to me means the latter. The former is more like a write barrier.

With true immutables you can start developing in a mixed object/functional style, which would allow interesting optimizations, e.g. for concurrency, memoization etc.

- Bert -

On 18.03.2010, at 16:30, Ralph Johnson wrote:

On 3/18/10, Bert Freudenberg bert@freudenbergs.de wrote:

...

With true immutables you can start developing in a mixed object/functional style, which would allow interesting optimizations, e.g. for concurrency, memoization etc.

I've developed in a mixed object/functional style for years, and I think a lot of other people do, too. You don't need language support for this, though I expect it would be helpful. It makes concurrency and distributed programming easier, and is important when you are dealing with databases. Eric Evans calls this "Value Object" in his book "Domain Driven Design".

-Ralph Johnson

Right, the style is possible without VM support in your own subsystem. But being able to *guarantee* immutability would allow to generalize this beyond your own special-purpose code. And possibly it would enable further VM-level optimizations.

- Bert -

On 30.03.2010, at 11:45, Bryce Kampjes wrote:

On Thu, 2010-03-18 at 16:40 +0100, Bert Freudenberg wrote:

...

On 18.03.2010, at 16:30, Ralph Johnson wrote:

...

On 3/18/10, Bert Freudenberg bert@freudenbergs.de wrote:

...

With true immutables you can start developing in a mixed object/functional style, which would allow interesting optimizations, e.g. for concurrency, memoization etc.

I've developed in a mixed object/functional style for years, and I think a lot of other people do, too. You don't need language support for this, though I expect it would be helpful. It makes concurrency and distributed programming easier, and is important when you are dealing with databases. Eric Evans calls this "Value Object" in his book "Domain Driven Design".

-Ralph Johnson

Right, the style is possible without VM support in your own subsystem. But being able to *guarantee* immutability would allow to generalize this beyond your own special-purpose code. And possibly it would enable further VM-level optimizations.

Even perfectly VM enforced immutability isn't enough to allow optimisation if you still allow become:. It's always possible to swap out the immutable object with a different one via become:.

It's fine to swap immutable objects, unless the object holding the reference is itself unmutable. No instance variable of an immutable object can be stored into, not even by become:. Obviously, become: would have to honor the immutability flag:. For references inside immutable objects, become: becomes a forward-become (or a no-op if both were immutable):

a := Array with: A new. b := Array with: B new beImmutable. c := (Array with: C new) beImmutable. a first become: b first. "a points to the immutable B instance, b to the A instance". a first become: c first. "both a and c now point to the immutable C instance"

- Bert -

On 31.03.2010, at 15:03, Igor Stasenko wrote:

On 30 March 2010 14:09, Bert Freudenberg bert@freudenbergs.de wrote:

...

On 30.03.2010, at 11:45, Bryce Kampjes wrote:

...

On Thu, 2010-03-18 at 16:40 +0100, Bert Freudenberg wrote:

...

On 18.03.2010, at 16:30, Ralph Johnson wrote:

...

On 3/18/10, Bert Freudenberg bert@freudenbergs.de wrote:

...

With true immutables you can start developing in a mixed object/functional style, which would allow interesting optimizations, e.g. for concurrency, memoization etc.

I've developed in a mixed object/functional style for years, and I think a lot of other people do, too. You don't need language support for this, though I expect it would be helpful. It makes concurrency and distributed programming easier, and is important when you are dealing with databases. Eric Evans calls this "Value Object" in his book "Domain Driven Design".

-Ralph Johnson

Right, the style is possible without VM support in your own subsystem. But being able to *guarantee* immutability would allow to generalize this beyond your own special-purpose code. And possibly it would enable further VM-level optimizations.

Even perfectly VM enforced immutability isn't enough to allow optimisation if you still allow become:. It's always possible to swap out the immutable object with a different one via become:.

It's fine to swap immutable objects, unless the object holding the reference is itself unmutable. No instance variable of an immutable object can be stored into, not even by become:. Obviously, become: would have to honor the immutability flag:. For references inside immutable objects, become: becomes a forward-become (or a no-op if both were immutable):

a := Array with: A new. b := Array with: B new beImmutable. c := (Array with: C new) beImmutable. a first become: b first. "a points to the immutable B instance, b to the A instance". a first become: c first. "both a and c now point to the immutable C instance"

one of the problem with #become:/becomeForward: that it should swap all of existing references atomically. Now imagine that while walking the heap, it replaced a 50% of references to given object, and then discoreved that it needs to swap a reference held by immutable object. Now, what you suppose to do? Revert all changes and fail primitive? Ignore this object (do not alter it references) and continue? Or, before doing the real swapping, scan the heap and check all object to make sure that none of them are immutable and so, #become will work ok?

Just ignore all writes into immutable objects (that is, in the become code, do not insert a forwarding block), otherwise proceed as usual.

- Bert -

On 31.03.2010, at 16:10, Igor Stasenko wrote:

On 31 March 2010 16:07, Bert Freudenberg bert@freudenbergs.de wrote:

...

On 31.03.2010, at 15:03, Igor Stasenko wrote:

...

On 30 March 2010 14:09, Bert Freudenberg bert@freudenbergs.de wrote:

...

On 30.03.2010, at 11:45, Bryce Kampjes wrote:

...

On Thu, 2010-03-18 at 16:40 +0100, Bert Freudenberg wrote:

...

On 18.03.2010, at 16:30, Ralph Johnson wrote: > > On 3/18/10, Bert Freudenberg bert@freudenbergs.de wrote: > >> With true immutables you can start developing in a mixed object/functional style, which would allow interesting optimizations, e.g. for concurrency, memoization etc. > > I've developed in a mixed object/functional style for years, and I > think a lot of other people do, too. You don't need language support > for this, though I expect it would be helpful. It makes concurrency > and distributed programming easier, and is important when you are > dealing with databases. Eric Evans calls this "Value Object" in his > book "Domain Driven Design". > > -Ralph Johnson

Right, the style is possible without VM support in your own subsystem. But being able to *guarantee* immutability would allow to generalize this beyond your own special-purpose code. And possibly it would enable further VM-level optimizations.

Even perfectly VM enforced immutability isn't enough to allow optimisation if you still allow become:. It's always possible to swap out the immutable object with a different one via become:.

It's fine to swap immutable objects, unless the object holding the reference is itself unmutable. No instance variable of an immutable object can be stored into, not even by become:. Obviously, become: would have to honor the immutability flag:. For references inside immutable objects, become: becomes a forward-become (or a no-op if both were immutable):

a := Array with: A new. b := Array with: B new beImmutable. c := (Array with: C new) beImmutable. a first become: b first. "a points to the immutable B instance, b to the A instance". a first become: c first. "both a and c now point to the immutable C instance"

one of the problem with #become:/becomeForward: that it should swap all of existing references atomically. Now imagine that while walking the heap, it replaced a 50% of references to given object, and then discoreved that it needs to swap a reference held by immutable object. Now, what you suppose to do? Revert all changes and fail primitive? Ignore this object (do not alter it references) and continue? Or, before doing the real swapping, scan the heap and check all object to make sure that none of them are immutable and so, #become will work ok?

Just ignore all writes into immutable objects (that is, in the become code, do not insert a forwarding block), otherwise proceed as usual.

But such semantics is no longer can be treated as #become (or #becomeForward), because you leaving a references to old object instead of uniformly replacing them with swapped one.

I don't see a problem with that. Can you give an example where it would be problematic?

- Bert -

On 31.03.2010, at 17:01, Igor Stasenko wrote:

On 31 March 2010 17:14, Bert Freudenberg bert@freudenbergs.de wrote:

...

On 31.03.2010, at 16:10, Igor Stasenko wrote:

...

On 31 March 2010 16:07, Bert Freudenberg bert@freudenbergs.de wrote:

...

On 31.03.2010, at 15:03, Igor Stasenko wrote:

...

On 30 March 2010 14:09, Bert Freudenberg bert@freudenbergs.de wrote:

...

On 30.03.2010, at 11:45, Bryce Kampjes wrote: > > On Thu, 2010-03-18 at 16:40 +0100, Bert Freudenberg wrote: >> On 18.03.2010, at 16:30, Ralph Johnson wrote: >>> >>> On 3/18/10, Bert Freudenberg bert@freudenbergs.de wrote: >>> >>>> With true immutables you can start developing in a mixed object/functional style, which would allow interesting optimizations, e.g. for concurrency, memoization etc. >>> >>> I've developed in a mixed object/functional style for years, and I >>> think a lot of other people do, too. You don't need language support >>> for this, though I expect it would be helpful. It makes concurrency >>> and distributed programming easier, and is important when you are >>> dealing with databases. Eric Evans calls this "Value Object" in his >>> book "Domain Driven Design". >>> >>> -Ralph Johnson >> >> Right, the style is possible without VM support in your own subsystem. But being able to *guarantee* immutability would allow to generalize this beyond your own special-purpose code. And possibly it would enable further VM-level optimizations. > > Even perfectly VM enforced immutability isn't enough to allow > optimisation if you still allow become:. It's always possible to swap > out the immutable object with a different one via become:.

It's fine to swap immutable objects, unless the object holding the reference is itself unmutable. No instance variable of an immutable object can be stored into, not even by become:. Obviously, become: would have to honor the immutability flag:. For references inside immutable objects, become: becomes a forward-become (or a no-op if both were immutable):

a := Array with: A new. b := Array with: B new beImmutable. c := (Array with: C new) beImmutable. a first become: b first. "a points to the immutable B instance, b to the A instance". a first become: c first. "both a and c now point to the immutable C instance"

one of the problem with #become:/becomeForward: that it should swap all of existing references atomically. Now imagine that while walking the heap, it replaced a 50% of references to given object, and then discoreved that it needs to swap a reference held by immutable object. Now, what you suppose to do? Revert all changes and fail primitive? Ignore this object (do not alter it references) and continue? Or, before doing the real swapping, scan the heap and check all object to make sure that none of them are immutable and so, #become will work ok?

Just ignore all writes into immutable objects (that is, in the become code, do not insert a forwarding block), otherwise proceed as usual.

But such semantics is no longer can be treated as #become (or #becomeForward), because you leaving a references to old object instead of uniformly replacing them with swapped one.

I don't see a problem with that. Can you give an example where it would be problematic?

It defeats a purpose of #become. If you think about objects as a nodes in directed graph, then become is a way to replace a single node with another one, such that all nodes pointing to node A, now start pointing to node B. If you make any exceptions, you'll end up with having part of nodes in graph pointing to B, while another part still pointing to node A. Many things in system relying on such semantics, and i don't even want to guess, how many places will break if you change that, because in all cases where one using #become, he expects that all references is replaced, but now he should keep in mind that some references keep unchanged and moreover, you don't have a reliable control over this process. So, instead of simple and predictable behavior, #become will be something unreliable and brittle, which cannot be fully trusted.

self call: Shenanigans ;)

Show me a single place that would break in a real-world use-case.

In fact, #become: is rarely used in Squeak because of its inherent slowness caused by our direct-pointer object model. And those rare places I can think of would work just fine.

- Bert -

On 31.03.2010, at 17:23, Andreas Raab wrote:

On 3/31/2010 8:13 AM, Bert Freudenberg wrote:

...

Show me a single place that would break in a real-world use-case.

In fact, #become: is rarely used in Squeak because of its inherent slowness caused by our direct-pointer object model. And those rare places I can think of would work just fine.

I think there is one, and only one, place where #become: is intrinsically required: Changing class shape.

Thought about that. Should be fine.

- Bert -

On 31.03.2010, at 18:15, Igor Stasenko wrote:

On 31 March 2010 18:24, Bert Freudenberg bert@freudenbergs.de wrote:

...

On 31.03.2010, at 17:23, Andreas Raab wrote:

...

On 3/31/2010 8:13 AM, Bert Freudenberg wrote:

...

Show me a single place that would break in a real-world use-case.

In fact, #become: is rarely used in Squeak because of its inherent slowness caused by our direct-pointer object model. And those rare places I can think of would work just fine.

I think there is one, and only one, place where #become: is intrinsically required: Changing class shape.

Thought about that. Should be fine.

Magma using a forwarding proxies to indicate a not-yet reified objects in memory.

Suppose that you having multiple different objects, which referencing such proxy. Proxy is not reified before a first message sent to it and used as a simple reference. Nothing prevents you from having many other objects, which pointing to that proxy without reifying a real object. And now, think about that with introduction of immutability, some of these objects is immutable. This means that it is now possible to have an immutable object, which having a reference to forwarding proxy. (even more than that, this is more than likely, because Magma will use an immutability for own purposes, to track changes to objects which belong to DB graph). At some stage, your model sending a message to forwarding proxy, and therefore magma needs to reify a real object in memory and replace all references to proxy with it. This is easy to implement by using a #becomeForward:, but if you change the #becomeForward: semantics to not replace the references in immutable objects, you will efficiently break this scheme , and it can't be used reliably anymore.

Not at all. Of course you should not make an object immutable that you want to mutate later. That's the whole point really.

So, with such change, we have to invent a different way, how OODB could reliably replace a forwarding proxy with real object, which is read from remote storage.

No, you just cannot use VM-level immutability for detecting this case.

It would still be perfectly fine to put immutable objects into Magma. You just couldn't modify them. Again, that's the point of immutability.

It'd also be fine to have a proxy for an immutable object. But when the proxied object is brought in, it will have to bring in all the objects it referenced, because these are immutable by definition, too.

- Bert -

2010/3/31 Bert Freudenberg bert@freudenbergs.de

On 31.03.2010, at 18:15, Igor Stasenko wrote:

...

On 31 March 2010 18:24, Bert Freudenberg bert@freudenbergs.de wrote:

...

On 31.03.2010, at 17:23, Andreas Raab wrote:

...

On 3/31/2010 8:13 AM, Bert Freudenberg wrote:

...

Show me a single place that would break in a real-world use-case.

In fact, #become: is rarely used in Squeak because of its inherent

slowness caused by our direct-pointer object model. And those rare places I can think of would work just fine.

...

...

...

I think there is one, and only one, place where #become: is

intrinsically required: Changing class shape.

...

...

Thought about that. Should be fine.

Magma using a forwarding proxies to indicate a not-yet reified objects in memory.

Suppose that you having multiple different objects, which referencing such proxy. Proxy is not reified before a first message sent to it and used as a simple reference. Nothing prevents you from having many other objects, which pointing to that proxy without reifying a real object. And now, think about that with introduction of immutability, some of these objects is immutable. This means that it is now possible to have an immutable object, which having a reference to forwarding proxy. (even more than that, this is more than likely, because Magma will use an immutability for own purposes, to track changes to objects which belong to DB graph). At some stage, your model sending a message to forwarding proxy, and therefore magma needs to reify a real object in memory and replace all references to proxy with it. This is easy to implement by using a #becomeForward:, but if you change the #becomeForward: semantics to not replace the references in immutable objects, you will efficiently break this scheme , and it can't be used reliably anymore.

Not at all. Of course you should not make an object immutable that you want to mutate later. That's the whole point really.

With your "become logic" reference to proxy (like proxy in magma and glorp) will never materialized if immutability used for object modification tracking. And it will break this frameworks (magma, glorp)

On 1 April 2010 02:31, Bert Freudenberg bert@freudenbergs.de wrote:

On 31.03.2010, at 20:45, Denis Kudriashov wrote:

2010/3/31 Bert Freudenberg bert@freudenbergs.de

...

On 31.03.2010, at 18:15, Igor Stasenko wrote:

...

On 31 March 2010 18:24, Bert Freudenberg bert@freudenbergs.de wrote:

...

On 31.03.2010, at 17:23, Andreas Raab wrote:

...

On 3/31/2010 8:13 AM, Bert Freudenberg wrote:

...

Show me a single place that would break in a real-world use-case.

In fact, #become: is rarely used in Squeak because of its inherent slowness caused by our direct-pointer object model. And those rare places I can think of would work just fine.

I think there is one, and only one, place where #become: is intrinsically required: Changing class shape.

Thought about that. Should be fine.

Magma using a forwarding proxies to indicate a not-yet reified objects in memory.

Suppose that you having multiple different objects, which referencing such proxy. Proxy is not reified before a first message sent to it and used as a simple reference. Nothing prevents you from having many other objects, which pointing to that proxy without reifying a real object. And now, think about that with introduction of immutability, some of these objects is immutable. This means that it is now possible to have an immutable object, which having a reference to forwarding proxy. (even more than that, this is more than likely, because Magma will use an immutability for own purposes, to track changes to objects which belong to DB graph). At some stage, your model sending a message to forwarding proxy, and therefore magma needs to reify a real object in memory and replace all references to proxy with it. This is easy to implement by using a #becomeForward:, but if you change the  #becomeForward: semantics to not replace the references in immutable objects, you will efficiently break this scheme , and it can't be used reliably anymore.

Not at all. Of course you should not make an object immutable that you want to mutate later. That's the whole point really.

With your "become logic" reference to proxy (like proxy in magma and glorp) will never materialized if immutability used for object modification tracking.

You are talking about a write barrier. I'm talking about immutability. As mentioned previously in this thread, these two are related, but not identical. A write barrier might be more useful for current applications indeed. I simply wouldn't call that immutability.

And it will break this frameworks (magma, glorp)

Well if you make objects immutable you should be aware of what you are doing ;) IMHO hard immutability would be a valid and consistent model. Whether it would actually be practical and useful in Squeak I'm not sure. There seems to be no  compelling use case for it yet. OTOH, soft "almost-immutability" can be implemented in the image, so what would be the compelling reason to make this a VM feature, which almost certainly will come at a price?

Yep. IMO we should think more about models, which could help us to improve a real, existing use cases (like Magma or GLORP). IMO, an immutability as a separate model seems not very useful, especially in 'hard' form, when it propagating through references. So far, Squeak system is built without the need of immutability, and it works well. And hence, it raising a logical question: how much we really need an immutability as a core concept in object model? In what areas it would simplify implementation and/or make it more robust?

• Bert -

On 1 April 2010 06:06, Colin Putney cputney@wiresong.ca wrote:

On 2010-03-31, at 8:24 AM, Bert Freudenberg wrote:

...

On 31.03.2010, at 17:23, Andreas Raab wrote:

...

On 3/31/2010 8:13 AM, Bert Freudenberg wrote:

...

Show me a single place that would break in a real-world use-case.

In fact, #become: is rarely used in Squeak because of its inherent slowness caused by our direct-pointer object model. And those rare places I can think of would work just fine.

I think there is one, and only one, place where #become: is intrinsically required: Changing class shape.

Thought about that. Should be fine.

So you're saying you think it's OK that immutable objects don't get migrated to the new version of the class? I suppose that conforms to a strict interpretation of the of the term "immutable" - neither state nor behaviour may be change.

At some point though, such a strict interpretation isn't very useful. An object that can't change state, can't change behaviour, can't refer to mutable objects, and can't become mutable again is certainly worth of the term, but it's also useless from a practical point of view. Has anyone proposed a use case for this sort of immutability? If not, why insist on defining the term so tightly?

As far as I'm aware, the use cases for immutability, loosely defined, are as follows:

(a) Literals. If the compiler makes objects immutable before putting them in the literal frame of a method, we can be sure that the state of the object when the method is activated is the same as what appears in the source code. This is certainly nice. Mutability of literals hasn't been a problem so far, though, so it's not crucial.

(b) Write Barrier. Immutability as implemented in, for example VW, allows object mutation to be efficiently tracked, which is handy for persisting objects outside the image, or synchronizing their state between images.

(c) Concurrency. The tricky thing about concurrency is shared mutable state. If it's possible to make objects immutable, it's then possible to share them safely between units of concurrency - Processes, Islands, images etc. When sending state between units of concurrency, immutable objects can often be transmitted more efficiently.

(d) Garbage Collection. If there are enough immutable objects in the image, it might be possible to improve the efficiency of the garbage collector. Have an "immutable space" and cache all the pointers back into mutable space. Or something. Or the "Shared Perm Space" thing that VW used to have.

Any other ideas?

For me, (b) and (c) are the interesting cases. Seaside apps almost always need to persist data in some way, and (b) would make this a lot easier and more efficient. (c) is maybe less clear cut, but in a lot of ways it's more interesting. I like the general trend in Squeak toward event loop concurrency, rather than classical Processes & Semaphores. Immutability would enable more experimentation in this area.

Perhaps if we focused on these use cases, we'd have a more productive discussion. It may be that VM-level immutability isn't useful enough to be worth the effort to implement and support. But if it *is* worthwhile, it'll be because of what it let's us do, rather than how semantically correct our definition of immutability is.

For concurrency.. perhaps. But i'd say that most problems of concurrency is sharing a mutable state, not immutable one, because in most cases its trivial to define a shared static state in your application and make sure it doesn't mutating unexpectably. Yes, it would help detecting a *trivial* problems at initial stages of development, and exchange the immutable data more efficiently, but again, most of computational tasks anyways dealing with mutable state, and i don't see how immutability could help in this regard. What i mean that once you detect immutable object - you can simply pass a pointer (or copy data, whatever). Cool! But once you detect that you wanna share a mutable object between islands/images/threads - you still having the very same concurrency problems as before. And so, a real problem still is yours :)

Perm space. I see it as a form of generational GC specialization. Yes, it could improve GC performance. But why only for immutables? I think that you could achive nearly same effect by simply adding 1 more generation to GC model without need of separating objects on ones, which can mutate and ones which can't.

So, for me, there is only (a) and (b) seem useful with an easily foreseen impact.

Colin

I think usage of immutability for object modification tracking is not very well solution and not simple solution. With that you must put every code with your objects in block that catch ModificationException. You can't track arbitrarily object modifications in any other places of system.

I think more general approach is registering arbitrary modification handler on object mutation like

object handleMutationBy: [:mutation | ] and object resetMutationTracking

And maybe it can be implemented like immutability approach. with single bit in object header. VM check this bit for #at:put: like primitives and send message #handleMutation: to target object if It set.

And now immutability logic with ModificationException has very simple implementation in language side

Object>>handleMutation: aMutation self raise: (ModificationException new mutation: aMutation)

Implementation of my messages:

Object>>handleMutation: aMutation self trackMutation. ^mutationHandler value: aMutation

And VM needs two primitives: #trackMutation and #resetMutationTracking

On 1 April 2010 08:02, Colin Putney cputney@wiresong.ca wrote:

On 2010-03-31, at 9:31 PM, Igor Stasenko wrote:

...

For concurrency.. perhaps. But i'd say that most problems of concurrency is sharing a mutable state, not immutable one, because in most cases its trivial to define a shared static state in your application and make sure it doesn't mutating unexpectably. Yes, it would help detecting a *trivial* problems at initial stages of development, and exchange the immutable data more efficiently, but again, most of computational tasks anyways dealing with mutable state, and i don't see how immutability could help in this regard. What i mean that once you detect immutable object - you can simply pass a pointer (or copy data, whatever). Cool! But once you detect that you wanna share a mutable object between islands/images/threads - you still having the very same concurrency problems as before. And so, a real problem still is yours :)

IMO the interesting possibilities here are all about concurrency *without* shared mutable state. If objects can be made immutable, the code that manages communication between islands, images, vats etc, can be intelligent about how it handles them. Application-level code still has to manage communications between units of isolation, but its interface with the runtime can get simpler if the runtime can tell what's mutable and what's not. It's about making it easier to implement Islands, etc. in Squeak, not (directly) about making it easier to write an app with such an implementation.

Admittedly, it's not as easy to predict how much of an impact immutability would have in this realm, especially when compared with (b). But I do think it's one of the basic building blocks that would enable experimentation here.

Maybe. I still thinking, that OO approach is much more powerful and flexible. As easy as, object replicateIn: island lets an object to decide how to replicate itself in given island/environment, whether create a copy or pass as a reference (or any other form), without a need of introducing any magic from VM side. This gives you a direct control over what happens and when. Bug again, we don't have a model, already implemented and widely used which employs an immutability in a way you suggested. So, this still lies in experimental area. While with DB replication schemes, we already having a real use examples. So, logically would be to focus on this ones instead.

...

Perm space. I see it as a form of generational GC specialization. Yes, it could improve GC performance. But why only for immutables? I think that you could achive nearly same effect by simply adding 1 more generation to GC model without need of separating objects on ones, which can mutate and ones which can't.

<shrug> Yeah, I was scraping the bottom of the barrel with this one. It's a stretch no matter how you look at it.

...

So, for me,  there is only (a) and (b) seem useful with an easily foreseen impact.

Fair enough, though (c) may have unforeseen impact.

Colin

On 18.03.2010, at 16:45, Igor Stasenko wrote:

On 18 March 2010 17:23, Bert Freudenberg bert@freudenbergs.de wrote:

...

On 18.03.2010, at 15:51, Colin Putney wrote:

...

On 2010-03-18, at 5:39 AM, Bert Freudenberg wrote:

...

HO there should be no way to reset the immutable flag. You can implement "soft" immutability in the image, but "hard" VM-level immutability needs to be permanent, no fiddling possible. Once set, the object stays immutable. Only a copy of an immutable object will be mutable again.

This would be good for some use cases - literals, for example, but not so good for others. One of the common uses of "immutability" is not so much to *prevent* mutation, as to control it: databases, concurrency etc. Do you have another feature in mind for these cases, or some overriding interest in immutable immutability that you want to pursue?

Colin

Seems to me like two separate issues - one being informed of modifications, and one preventing modifications. Immutability to me means the latter. The former is more like a write barrier.

Right! And for OODBs first is sufficient.

...

With true immutables you can start developing in a mixed object/functional style, which would allow interesting optimizations, e.g. for concurrency, memoization etc.

I don't sharing your optimism, based on immutability. You can't run away from the fact that computing is the way of transforming data from one form into another. So, somewhere, either in language or at VM level, you have to mutate the state of data, otherwise you can't do much.

Free advice of the day: don't ever get near a convention of functional programmers ;)

- Bert -

On 18 March 2010 17:55, Bert Freudenberg bert@freudenbergs.de wrote:

On 18.03.2010, at 16:45, Igor Stasenko wrote:

...

On 18 March 2010 17:23, Bert Freudenberg bert@freudenbergs.de wrote:

...

On 18.03.2010, at 15:51, Colin Putney wrote:

...

On 2010-03-18, at 5:39 AM, Bert Freudenberg wrote:

...

HO there should be no way to reset the immutable flag. You can implement "soft" immutability in the image, but "hard" VM-level immutability needs to be permanent, no fiddling possible. Once set, the object stays immutable. Only a copy of an immutable object will be mutable again.

This would be good for some use cases - literals, for example, but not so good for others. One of the common uses of "immutability" is not so much to *prevent* mutation, as to control it: databases, concurrency etc. Do you have another feature in mind for these cases, or some overriding interest in immutable immutability that you want to pursue?

Colin

Seems to me like two separate issues - one being informed of modifications, and one preventing modifications. Immutability to me means the latter. The former is more like a write barrier.

Right! And for OODBs first is sufficient.

...

With true immutables you can start developing in a mixed object/functional style, which would allow interesting optimizations, e.g. for concurrency, memoization etc.

I don't sharing your optimism, based on immutability. You can't run away from the fact that computing is the way of transforming data from one form into another. So, somewhere, either in language or at VM level, you have to mutate the state of data, otherwise you can't do much.

Free advice of the day: don't ever get near a convention of functional programmers ;)

It is good to use a universal approach in modelling the universe. And as to me, 'everything is an object' is more universal than 'everything is a function' ;)

• Bert -

On Thu, Mar 18, 2010 at 10:11 AM, Brent Pinkney brent@zamail.co.za wrote:

Ahoy,

I would like to contribute to this thread by explaining a problem that occurs often in telecommunications and logistics.

It is common to have planning tools that allow the user to 'fork' the master view of the 'domain model' and make local changes to explore "what ifs". Alas. the domain model is a directed graph of many thousands of instances, born of hundreds of classes, so duplicating all of this is not feasible.

I have always wanted an object mechanism to detect 'copy of write', so that I might slay this pesky problem. If I had "OCOW" (Object Copy on Write), I could let the happy users work on the master object graph and manage the small portion that was changed by an such user experimentation. Obviously, there would need to be a nice exceptions or dependencies to wrap all this.

My question (I really do not know) then is: does immutability, as proposed in this thread, help ?

Yes. For example take Alan Knight's GLORP framework which does object-relaitonal mapping. How does one maintain the set of changed objects and the changes? Mark the set of objects uploaded from the DB as immutable. As each is first written to, trap the exception, take a shallow copy of the object and enter it into a dictionary mapping modified object to unmodified copy. At commit time the modified objects are the keys in the dictionary and what slots have been modified can be found by comparing the unmodified copy with the modified current object. The use of immutability here is an extremely beneficial optimization. It avoids work, depending on what the base implementation is, e.g.

- avoids having to compare the entire object graph with the state of the database to discover differences - avoids having to take a copy of the data set uploaded from the database (makes copying lazy)

cheers Eliot

Brent

On Thu, Mar 18, 2010 at 8:23 AM, Bert Freudenberg bert@freudenbergs.dewrote:

On 18.03.2010, at 15:51, Colin Putney wrote:

...

On 2010-03-18, at 5:39 AM, Bert Freudenberg wrote:

...

HO there should be no way to reset the immutable flag. You can implement

"soft" immutability in the image, but "hard" VM-level immutability needs to be permanent, no fiddling possible. Once set, the object stays immutable. Only a copy of an immutable object will be mutable again.

...

This would be good for some use cases - literals, for example, but not so

good for others. One of the common uses of "immutability" is not so much to *prevent* mutation, as to control it: databases, concurrency etc. Do you have another feature in mind for these cases, or some overriding interest in immutable immutability that you want to pursue?

...

Colin

Seems to me like two separate issues - one being informed of modifications, and one preventing modifications. Immutability to me means the latter. The former is more like a write barrier.

With true immutables you can start developing in a mixed object/functional style, which would allow interesting optimizations, e.g. for concurrency, memoization etc.

But the same VM support can allow both styles. The VM support is a per-object bit that marks an object as immutable along with modifications to inst var assign and at:put: primitives to check for the immutability bit, and a pair of primitives to fetch and set the per-object isImmutable flag. If the primitive to set immutablity is removed and replaced with a primitive that can only enable immutability, or if use of the primitive is suitably restricted then one can prevent clearing of the flag. The VM is the same in oth cases; it simply has to check for the immtability flag on each write, but above that one can use this as a write-barrier or to implement "true" immutability.

Note that the cost of checking immutability on each write is quite small in the at:put: primitives because these must fetch the object header containing an object's size to do bounds checking, and testing a bit in a register containng the header merely adds a register operation not a more expensive read. It does add to the cost of inst var write, but again the overhead is not that large because inst var write also involves the GC's write barrier. So overall the performance impact is in my experience (VW & Newspeak) < 5% overall speed decrease for macro benchmarks.

• Bert -