Complexity and starting over on the JVM (was Re: Traits or not...)

Paul D. Fernhout pdfernhout at kurtz-fernhout.com
Mon Feb 4 14:43:27 UTC 2008


stephane ducasse wrote:
> Now if people believe that traits are the problems, they are of course.
> I can tell you that I will ***never** maintain Squeak anymore and I'm
> really thinking about what I will do.
> Because I think that I did a lot for smalltalk in general the last 10
> years and may be this is the time
> to do something else to get a large breath of fresh air.

Stéphane-

I sympathize. I think the biggest issue of Squeak is issues with modularity
and managing complexity. These issues translate to frustration for
maintainers (and users :-).

Anyway, I had related frustrations to yours many years ago and they are why
I ended up doing a lot in Python and Jython on the JVM in the last decade,
"Open source VW (was Re: Why FUD?)"
http://groups.google.com/group/comp.lang.smalltalk/browse_thread/thread/3b3a67ab0ab756e0/6d08b70aa9f6aea1?lnk=st&q=&rnum=1&hl=en#6d08b70aa9f6aea1
  "[Edu-sig] Python & Smalltalk (was Re: OLPC related: pyGTK)"
  http://mail.python.org/pipermail/edu-sig/2006-December/007476.html
  "[Edu-sig] Freedom: some Smalltalk history and Python implications"
  http://www.mail-archive.com/edu-sig@python.org/msg02717.html
even to the point of working on PataPata.
    http://patapata.sourceforge.net/
That was a Squeak-ish thing on Python, now stalled as I hit fundamental
Python limits related to "Edit and continue" and also see the less desirable
aspects of prototype-based systems.
  " PataPata critique: the good, the bad, the ugly"
  http://patapata.sourceforge.net/critique.html

Now I am looking at mixing Jython, Scala, and the JVM. But I still miss the
Squeak ideals and the Smalltalk syntax. And Jython/Python as a community is
never (or not for a long while it seems) going to focus on stuff like
coding-in-the-debugger or restarting from where an exception is thrown.

Spoon is a good Squeak-ish try in the direction of managing complexity,
although I think building the image from source, like GNU Smalltalk does, is
ultimately a better idea.
  "[Help-smalltalk] Bootstrapping a Smalltalk image"
  http://lists.gnu.org/archive/html/help-smalltalk/2008-01/msg00129.html
But otherwise I think Spoon really has a lot of great ideas like remote
development from one image to another.
  http://www.netjam.org/spoon/

I think the most important single issue in maintaining any large system is
managing complexity (documenting intent maybe comes next, including
well-named variables and methods and functions). This has never been a
priority for Squeak IMHO.

There are several ways to manage complexity, which include:
* modularity (namespaces, packages like Java or GNU Smalltalk or Debian,
letting someone else do that hard work by leveraging libraries or VMs or
languages, like Squeak does by using a C compiler to generate the VM)
* cleverness (brilliant redesign, like traits was hopefully going to be)
* laissez faire, and also to each his or her own image (that is what we have
now, and it is not that bad an idea, if the *core* is small and well thought
out, like Spoon, so the *image* instance becomes the *module*. But alas, it
is not, witness how confusing Morphic is to unravel).

Modularity is the one way to manage complexity which seems to work best in
practice, although the others have their role. However, if Squeak images
could easily talk to each other and share some state, and we had Spoon-like
remote debugging and development, then we could have just one application
per image, and that would be easier to maintain (it would be modular to a
degree but in an unusual way). But I would still suggest such a system built
on well-though out (clever) modules would be more powerful and easier to use
than a mess of spaghetti code, even if we had only one application per image.

Personally, I think a rebuild of Squeak piece by piece, starting from
nothing, but drawing from clearly licensed code (like GST or parts of Squeak
or other systems), using a free license (MIT/BSD or LGPL) on top of the JVM
(not necessarily Java) as a baseline (but also being able to generate VM
with C as a less-supported alternative), perhaps using an intermediate
type-inferencing functional language like Scala (instead of C)
  http://www.scala-lang.org/
to define the VM, would be a great system. Fast. Flexible. Modular.
Extensible. Cross-platform everywhere Java runs (Mac, PC, GNU/Linux,
cellphones, supercomputers) while still open via generating C VMs to
everywhere else (OLPC). Leveraging a world of Java libraries across lots of
platforms, but still with a private VM in C for those times when people want
to run Squeak without the JVM or maybe want a little extra speed. Maybe use
the core GNU Smalltalk code instead of Squeak to get around licensing
worries, but then modify Squeak packages to run on top of that. Use Java
reflection to allow selectively sending either a *message* or a *call* to a
class depending whether it is Squeak-ish (implements processMessage:) or
Java-ish (see code below).

Anyway, that's a vision, but I don't have enough energy and time to pull it
off just by myself. The Squeak community could do it working together -- the
community knows enough and has enough person-power.  I don't see why it
would not take the community (say ten part-time developers or so) more than
a month or two to do, at least to the point where everyone else would want
to jump onboard. :-)

Yes I know there have been a few Java/JVM Smalltalks (including at least two
or three derived from Squeak). But maybe one with a clearer license might
help to push beyond that continuing contentious issue. Personally I can live
with the LGPL or even GPL for the VM, like GNU Smalltalk does, meaning I
personally would have no problems drawing from GNU Smalltalk for the core
(respecting the license). If the system is modular, GPL for the VM and LGPL
for the libraries really should not effect people very much in practice.
LGPL jar libraries for the JVM rarely pose a major problem with using the
JVM for proprietary applications if people worry about that. Personally, I
see such licensing in the *spirit* of the original Squeak license, to share
VM changes and share changes to the core libraries. And all the work done on
Squeak relicensing would pay off in bits and pieces as it could now be
clearer what parts could be added into a new Squeak.

I really would like it if people could see an Alan Kay demo and go and try
it themselves and keep going, rather than what I have seen happen in
practice which is things start blowing up pretty quickly for the average
newbie -- for mostly trivial-seeming (but really deep) reasons related to
complexity (like loading incompatible versions of various software or the VM).

For more on complexity, see for example my comments here seven or so years ago:
  "Belling the cat of complexity" Thu Jun 29 16:17:47 CEST 2000
http://lists.squeakfoundation.org/pipermail/squeak-dev/2000-June/001371.html
"Squeak complexity in 2.8 has become a complex cat from the simple kitten
complexity of 1.13(?) in 1996. Back then, Dan Ingalls wrote on 10 Nov
1996 those prescient words: "The Squeak team has an interest in doing
the world's simplest application construction framework, but I suspect
that we will get sucked up with enough other things that this won't
happen in the next two months (but who knows...)." "

For me, the biggest change since I wrote that is Java and the JVM becoming
truly free (well, almost, version seven will be the full change it is said)
and after a decade getting a lot of the bugs out of the core Java libraries
and the JVM. So it is finally a stable-enough and free-enough platform IMHO
to do some good work on. And to let someone else worry about a lot of low
level details and deployment testing (unless you really *want* to worry
about them by generating your own C VM, which you still could).

Or more recently I spoke to this concern here:
  http://mail.python.org/pipermail/edu-sig/2007-March/007822.html
"There is once again the common criticism leveled at Smalltalk of being
too self-contained. Compare this proposal with one that suggested making
tools that could be used like  telescope or a microscope for relating to
code packages in other languages -- to use them as best possible on
their own terms (or perhaps virtualized internally). Consider how the
proposal suggests scripting all the way down -- yet how are the
scripting tools built in Squeak? Certainly not with the scripting
language. And consider there are always barriers to any system -- where
you hit the OS, or CPU microcode, or proprietary hardware
specifications, or even unknowns in quantum physics, and so on. :-) So
every system has limits. But by pretending this one will not, this
project may miss out on the whole issue of interfacing to systems beyond
those limits in a coherent way. For example -- OpenGL works, and
represents an API that has matured over a quarter century, and is linked
with lots of hardware and software implementations; why not build on it
instead of "bitblt"?. Or consider how Jython can use Java Swing
libraries and other such Java libraries easily. Consider this example of
proposed failure: the proposal's decision to punt on printing -- no
focus there in difficulties of interfacing with a diversity of OS
services as a guest -- of relating to prior art. This is where Python
often shines as a system (language plus libraries plus community) -- and
Python is based on a different design philosophy or perhaps different
design ethic. Python has also prioritized "modularity" from the
beginning, which has made a lot of these issues more manageable; Kay's
proposal talks a lot about internal integration, but the word
"modularity" is not even in the document. In this sense, I think Kay's
group are repeating mistakes of the past and also dodging another very
difficult issue -- one that Python somehow has more-or-less gotten right
in its own way."

Some sample Java code to implement this selective dispatch, inspired in part
by Alan Kay's and Ian Piumarta's et al's new efforts:
  http://piumarta.com/software/cola/
  http://lambda-the-ultimate.org/node/2483
(hereby released into the Public Domain :-).

This code defines a "JMessageProcessorInterface", two classes (only one of
which implements the interface), and then a JMessages class which, based on
characteristics of the Java instance, either calls
"processMessage(JMessage)"  or uses reflection to dispatch the message as a
function call. There are probably better ways to do this at the byte-code
level or using a fancier functional intermediate language (like Scala).

public interface JMessageProcessorInterface {
	public Object processMessage(JMessage message);
}

==============

public class JMessageProcessorTest implements JMessageProcessorInterface {
	public Object processMessage(JMessage message) {
		System.out.println("Got a message");
		System.out.println("The message name was: ");
		System.out.println(message.messageName);
		return this;
	}

}

==================

public class JMessageProcessorTest2 {
	public Object foo() {
		System.out.println("foo: You sent no data. ");
		return this;
	}
	
	public Object bar(String data) {
		System.out.println("bar: You sent: " + data);
		return this;
	}
	
	public Object foo(String data) {
		System.out.println("foo: You sent some data: " + data);
		return this;
	}

}

============

public class JMessage {
	String messageName;
	Object sender;
	Object receiver;
	Object context;
	int parameterCount;
	Object parameters[];
	
	JMessage(String messageName, Object[] args) {
		this.messageName = messageName;
		this.parameters = args;
	}
}

====================

import java.lang.reflect.Method;
import java.lang.reflect.InvocationTargetException;

// References:
// http://java.sun.com/j2se/1.4.2/docs/api/java/lang/reflect/Method.html
// http://java.sun.com/docs/books/tutorial/reflect/member/methodInvocation.html

public class JMessages {

	/**
	 * @param args
	 */
	public static void main(String[] commandLineArgs) {
		System.out.println("JMessages starting...\n");
		
		// Dispatch to an object implementing a JMessageProcessorInterface interface
		System.out.println("Trying to dispatch to JMessageProcessorInterface");
		JMessageProcessorInterface messageProcessor = new JMessageProcessorTest();
		Object[] args = {"Hello world", };
		JMessage testMessage = new JMessage("foo", args);
		Object result;
		// next line is a more direct dispatching
		// result = messageProcessor.processMessage(testMessage);
		result = dispatchMessage(messageProcessor, testMessage);
		System.out.println("The result was: ");
		System.out.println(result);
		
		// now try to dispatch to a an object which does nto implement a
JMessageProcessorInterface interface
		System.out.println("\nTrying to dispatch to JMessageProcessorInterface");
		Object messageProcessor2 = new JMessageProcessorTest2();
		JMessage testMessage2 = new JMessage("foo", args);
		Object result2;
		// The following would result in a compile error of
processMessage(JMessage) undefined for Object
		// result2 = messageProcessor2.processMessage(testMessage2);
		result2 = dispatchMessage(messageProcessor2, testMessage2);
		System.out.println("The result was: ");
		System.out.println(result2);
		
	}
	
	public static Object dispatchMessage(Object receiver, JMessage theMessage) {
		Object result = null;
		if (receiver instanceof JMessageProcessorInterface) {
			JMessageProcessorInterface messageProcessor =
(JMessageProcessorInterface)receiver;
			result = messageProcessor.processMessage(theMessage);
		}
		else {
			System.out.println("Cannot directly dispatch message");
			// now do magic with Java reflection to see if can match message name
against a function name in the class
			Class c = receiver.getClass();
			Method[] allMethods = c.getDeclaredMethods();
			
			Method matchingMethod = null;
			
			// only finds first match with same name and same number of arguments
			// does not look at argument types but should eventually
			for (int i = 0; i < allMethods.length; i++) {
				Method m = allMethods[i];
				String methodName = m.getName();
				System.out.println("The methodName was: " + methodName);
				if (methodName == theMessage.messageName) {
					Class[] parameterTypes = m.getParameterTypes();
					for (int j = 0; j < parameterTypes.length; j++) {
						Class parameterClass = parameterTypes[j];
						System.out.println("  The paramter type was: " + parameterClass);
					}
					if (parameterTypes.length == theMessage.parameters.length) {
						matchingMethod = m;
						break;
					}
				}
			}
			
			System.out.println("invoking " +  matchingMethod.getName());
			try {
				matchingMethod.setAccessible(true);
				
				try {
					// does not look at argument types or try casts, but should eventually...
					result = matchingMethod.invoke(receiver, theMessage.parameters);
				} catch (IllegalAccessException x) {
				    x.printStackTrace();
				}
				
			    System.out.println("returned: " + result);

			// Handle any exceptions thrown by method to be invoked.
			} catch (InvocationTargetException x) {
			    Throwable cause = x.getCause();
			    System.out.println("invocation failed: " + matchingMethod.getName() +
" Cause:" +  cause.getMessage());
			}
			
		}
		return result;
	}

}

Anyway, probably back to lurking for a few more years, unless I hear a
groundswell of grassroots interest to "Burn Our Disk Packs". :-)
  http://www.smalltalk.org/smalltalk/TheEarlyHistoryOfSmalltalk_V.html

--Paul Fernhout



More information about the Squeak-dev mailing list