Hi Jason,
>> However, having just worked on a very large multi-threaded commercial
>> application in live production it is very clear that even single native
>> threaded Smalltalk applications have very nasty concurrency problems.
>>     
>
> Yes, and these problems are exaggerated by (what I would call) the old
> way of doing threaded programming, i.e. shared state, fine-grained
> locking.
>   

That may be the case from your - and others - perspective - and I have 
empathy for it -, however they are still valid techniques and others, 
such as myself, don't share your perspective.

Smalltalk should let people - the (educated) users - choose the 
mechanism of concurrency, not dictate it. In my humble opinion.


>> It's important that concurrency be taken into account at all levels of
>> an application's design, from the lowest levels of the virtual machine
>> through the end user experience (which is where concurrency on multiple
>> cores can really make a significant paradigm adjusting difference if
>> done well).
>>     
>
> But if we truly to the n-core (n being 100 and above) world to improve
> computation speed, and it looks as though we must, then this simply
> isn't realistic for most programmers.  No more then manual memory
> management was realistic for large applications.
>   

The reality of processor designs like the Tile 64 require us to have all 
available techniques at our disposal.


>> Of the lessons learned from this complex real world application was that
>> almost ALL of the concurrency problems you have with multiple cores
>> running N native threads you have with a single core running one native
>> thread.
>>     
>
> Depending on when execution can be interrupted, you have exactly the
> same issues.
>   

Exactly my point. Thus the solutions proposed as being "simplier" are 
just an illusion. They might be simplier in some cases but when you 
really need complex concurrency controls sometimes you need the other 
"dirtier" techniques at your disposal. Smalltalk is supposed to be a 
computer language with general power to control the computer and access 
it's true power and potential. Limiting the solution space by only 
implementing a limited set of concurrency primitives makes no sense. 
You'll just give the market to other lesser systems like Erlang and Java 
type systems.


>   
>> When you have a single native thread running, say, ten to twenty
>> Smalltalk green threads (aka Smalltalk processes) the concurrency
>> problems can be a real nightmare to contemplate. Comprehension of what
>> is happening is exasperated by the limited debugging information
>> captured at runtime crash dumps.
>>     
>
> But this depends largely on the model.  If you go away from the old,
> tried and untrue method of fine-grained locking then debugging gets
> much easier.  It's no problem at all, for example, in Erlang.
> Sometimes when something is really really hard to do, it is a sign
> that we are going about it the wrong way.
>   

Yes, the model is always important.

Yes, sometimes that  is a sign for improvement. Maybe I simply need 
detailed specifics of what you are talking about, however, if you really 
want Smalltalk to be general purpose - as I do - then it needs to cover 
the full domain of techniques for concurrency!


>> It is for the above reasons that I support many approaches be
>> implemented so that we can find out the best one(s) for various
>> application domains.
>>     
>
> We know from long experience what fine-grained is like.  At least one
> STM implementation is out there to try, and I believe the actor model
> Erlang uses is either out there, or easy to set up.
>   

Yes, we do. Sometimes it is what is needed.

For example, when building hard core operating systems.


>   
>> It's unlikely that there is a one solution fits all needs type of paradigm.
>>     
>
> No, but we can get the 99% like Garbage collection has.
>   

Not really.



>> 2) It's been mentioned that it would be straightforward to have squeak
>> start up multiple copies of the image (or even multiple different
>> images) in one process (task) memory space with each image having it's
>> own native thread and keeping it's object table and memory separate
>> within the larger memory space. This sounds like a very nice approach.
>> This is very likely practical for multi-core cpus such as the N-core
>> (where N is 2, 4, 8, 64) cpus from AMD, Intel, and Tilera.
>>
>> 3) A single image running on N-cores with M-native threads (M may be
>> larger than N) is the full generalization of course.
>>
>> This may be the best way to take advantage of paradigm shaking chips
>> such as the Tile64 processor from Tilera.
>>     
>
> If you mean by this a form of shared-state fine-grained programming
> then I disagree whole heartedly.  We have long experience with
> fine-grained in C++, Java, now C#, Smalltalk, on and on.  It just
> can't be the path to the future.
>   

There are many paths. I'm excited about the path that you are forging. 
All I ask is that you don't make that the only path to travel for people 
using Smalltalk.


> Smalltalk needs to keep inventing the future.  Chasing this primitive
> form of threading would put us firmly behind languages like C++ that
> have been doing it this way for decades.
>   

While I support Smalltalk inventing the future, keeping it from 
supporting valid concurrency techniques is ignoring the future (and the 
past) of what works!


>> However, we may need to rethink the entire architecture of the Smalltalk
>> virtual machine notions since the Tile 64 chip has capabilities that
>> radically alter the paradigm. Messages between processor nodes take less
>> time to pass between nodes then the same amount of data takes to be
>> written into memory. Think about that. It offers a new paradigm
>> unavailable to other N-Core processors (at this current time).
>>     
>
> I wonder how Erlang will run on these machines.
>   

I do as well.

All the best,

Peter




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20071019/1f986939/attachment.htm