[squeak-dev] Prepare for Thousands of Cores --- oh my Chip -
it's full of cores!
Peter William Lount
peter at smalltalk.org
Mon Jul 7 03:45:32 UTC 2008
Benoit St-Jean wrote:
> Peter, betting "All-in" is easy. Winning is something else.
> Criticizing is easy. Let's see the cards now! I've seen plently of
> Igor's code/papers/discussions/ideas, i.e REAL stuff... Now, it's
> your turn to go "all-in" ! What about Zoku Peter??? Will it support
> multi-core, native, "non-green" threads/processes ??? Any roadmap?
> Any ideas, plans, whatever? Any demo? Beyond theorical ideas,
> anything to refute Igor's ideas?
> I don't mind "ideas bashing." when the opponents have something to
> prove/show/discuss... I wouldn't mind arguing with Vassili Bykov,
> Elliott Miranda, Dan Ingalls, Matthew Fullmer, Stéhane Ducasse and a
> lot of others (sorry guys if you're not in the enumeration) but
> sometimes, being silent is the best solution when you've got nothing
> to add/prove to the discussion.
> I don't mind aurguing about ideas... But please guys, back your
> arguments with *something* !
> Benoit St-Jean
> Yahoo! Messenger: bstjean
> Blog: lamneth.wordpress.com
> A standpoint is an intellectual horizon of radius zero.
> (Albert Einstein)&
How's it going Benoit? It's been a long while. How's the family? It
would be good to talk with you again. You know my person email and phone
I think you misread my intentions; I was not intending to criticize the
hard work of others. I am a big supporter of all versions of Smalltalk
and encourage all developers of these systems to push their state of the
art forward. As for your "all in" comments I don't know what to say,
other than to say they are not conducive to a sense of community. I
agree that theory and practical results are both very important.
I think Igor is doing amazing work! I'm glad that he produced the
HydraVM. I was mistaken about it being able to run multiple native
threads since I've not had a chance to really study it yet. I like his
ideas for writing low level virtual machine code in a limited form of
Smalltalk (aka Igor slang), it is similar to an approach that I'm using
for ZokuScript in which the contents of blocks can be viewed, parsed,
interpreted or compiled in different ways. I'm looking forward to
looking at his work closer.
I simply desire Smalltalk to move to the general solution of one or more
native threads per image in memory. Smalltalk MT is one REAL example
that leads shining the way showing that M native threads per image in a
Smalltalk is not only doable but quite nice indeed. Smalltalk MT has
been out there for a long time now, fourteen years, proving that it can
be done. As you said the REAL stuff of proof it is in the pudding known
as Smalltalk MT.
"Smalltalk MT is fully multithreaded and all primitives are reentrant.
Threads are very lightweight and share a common address space, which is
consistent with the way threads are used on Win32. *The system is
scalable and can benefit from multiprocessor architectures. *...
Smalltalk MT is *multithreaded from the ground up*, with garbage
collection running in a separate thread. A process can have any number
of threads. This enables the development of scalable applications on
Windows NT, and simplifies considerably the design and implementation of
applications that use blocking I/O. There are no restrictions on sharing
data between threads (other than application-specific synchronization
issues). In addition, all Win32 synchronization mechanisms can be used,
which enables a non-Smalltalk thread or process to synchronize with a
Smalltalk thread. Starting with version 2.5, fibers and asynchronous
procedure calls (APC) are also supported."
"Smalltalk MT provides extensive multithreading capabilities based on
the Win32 API. For example, multithreading can be used to spawn several
unrelated applications that respond independently to user interaction.
An application can create any number of threads, and use Win32
synchronization objects to control their execution. The internal system
architecture is reentrant and multiprocessing safe. The automatic memory
management runs in a separate thread and preempts garbage collected
threads to scan their memory. ... *Unlike traditional implementations*
[of Smalltalk]*, Smalltalk MT does not have it's own processing model.
Instead, a Smalltalk process (we call it a thread) runs always as a
native thread.* Also there is no restrictions on what can be done in a
thread." - Page 172 of the Smalltalk MT manual cira 1995.
I do really like the N (small, medium, large) images in memory at the
same time model as it's excellent for things like web application
serving and many other applications.
However, fixing the number of native threads to one per image simply
limits solutions by forcing a developer/user to split their program
across multiple images just so they can run multiple native threads to
take advantage of their hardware. I think it not a good thing to impose
that upon developers or users, in part because it will often impose a
solution refactoring and in part because it only works for a subset of
problems that can be easily partitioned across images. A refactoring
might be worthwhile but may be cost prohibitive. Also moving all the
objects around between images (in one memory space or across memory
spaces) involves other problems just as complex as any multi-threading
issues solved. So all you've done it punt the ball aways down the field
but it's still there... lurking... ready to strike...
Sure it's easier for developing virtual machines, but it has been done
(Smalltalk MT) before so why can't it be done again? The only limit
seems to be the thinking of those who would need to do the work.
I am a big supporter of using separate images collaborating with each
other on the same processor box and across networks with many processing
nodes each with one to many images running. However, sometimes it is
easier, not to mention cheaper development time wise, to simply fork off
M native threads and get the work done in one image maximizing the
processors cores on one box.
I and my clients have Smalltalk applications that require M threads per
memory image. It's disappointing to me - and my clients - that the main
developers behind the squeak smalltalk (and the other commercial
smalltalk vendors) seemingly are not interested in moving in the general
purpose multi threading direction - which is fine as that's their
choice. I think it's a crucial mistake in direction for Smalltalk. I've
had clients tell me so and tell me to choose another environment that
does support full native multi-threading to take advantage of the huge
numbers of cores that are coming.
As for Zoku it is in development. There is no public road map at this
time. I have written a number of postings and articles about Zoku,
ZokuScript, and ZokuTalk. You can look them up if you wish. Zoku
supports M native threads per image (object space) and it can have N
images (object spaces) in memory at one time. It also supports multiple
independent or connected images (object spaces) on one or more processor
boxes. Zoku is not yet ready for commercial work. Thus, my need for
It seems that there are a number of different approaches to solving
concurrency issues and that the current developers working on squeak and
commercial smalltalks have chosen the easier path with minimal work to
utilize multi-core cpus. That is fine. It's not what I want or many of
my clients need but it is fine. It simply puts constraints upon the sets
of solutions that can be built with Smalltalk implementations or that
will be built.
If the Smalltalk/Squeak/Commercial community doesn't want to move
forward to take maximal general purpose advantage of the multi-core cpus
that would be a big disappointment indeed. The other languages will
leave Smalltalk in their dust. That is what clients are telling me and I
happen to agree with them.
I ask the Squeak Smalltalk (and Commerical Smalltalk) virtual machine
developers and communities to please reconsider the general purpose many
native threads per image solution for the next versions of your virtual
I think it's best to let each developer-user choose how many native
threads per image and how many images are loaded and which images are
loaded. After all it's their program. Isn't our job as language and
systems designers to provide them with the tools that they need to build
the systems that they want to build? I think so. What do you think?
All the best,
Peter William Lount
[ | peter at smalltalk dot org ]
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Squeak-dev