Hi,

There are a number of different issues going on in the discussion about the
creation of a Smalltalk Image that get intermixed with each other. I ask
for your concentration in advance of your reading the following, as it's
kind of long.

agree at carltonfields.com wrote:
> I'm quite sure I don't understand the nuances of this discussion, 
> but I'll venture the following: In what significant manner is an
> initial image, and workspace script not a concise and effective 
> specification of the new image.  
> (Remember that SystemTracer permits the specification of Sets of classes
> to be included/excluded from the new image, and also provides 
> methods to "clean up" an existing image).

An initial (existing) Smalltalk image may be corrupt in various ways or
contain objects that are not needed in the new image. These 'corrupted or
unneeded' objects can be transferred (without your knowledge) to the new
image via object references. So you can have situations where the new image
is potentially corrupted. You can have situations where the new image has
objects that it does not need (thus it's larger than required and cluttered
up). You can even have situations where the new image is corrupt because it
didn't get objects that it needed because the references in the prior image
were corrupted so the needed objects didn't get moved/copied over.

Individual object could have corrupted data, object headers could be
corrupt, references to other objects could be corrupted, important objects
could be missing, there could be extra objects whose presence would cause
problems. A detailed application and required runtime object instance
specification has definite advantages and can be used to create a new image
from scratch.

The key question that I am getting at is "what is the quality of the new
image that is created from an existing image?". Can I trust the new image?
Will it be reliable? 

There does seem to be a long history with many successive generations of
images being created successfully with the System Tracer. You can as you
say "clean up the image".  Maybe I'm concerned about nothing. The quality
of my software is important to me and I want to ensure that it's going to
work. 

> Given that most practical applications can be reduced to the
> desired content via a change set (including all the declarative 
> code to "set up" the classes), why, is it necessary to have a 
> complete declarative specification from the "ground up" of the 
> build of the entire underlying system in practice?  I have 
> made builds of significant commercial applications and given arms-length 
> entities source code "deliverables" and a makefile for constructing them,
> together with appropriate documentation.  On the other hand, I have never
> provided or considered providing source codes of the compiler libraries, 
> together with code and makefiles for the same (and often 
> didn't have them), except to the extent they were modified in order to 
> produce the application. [As a practical matter, I do typically deliver
> them in object code at least to assure they can perform a rebuild in
> the future.]  Nor did I routinely specify the underlying system
> facilities (compiler, linker, system rom and OS code) 
> upon which the application relies for build and execution, other than to 
> identify the name and version when relevant.

Writing applications in Smalltalk is different that writing applications in
C, C++, Java and other complied languages where the development tools are
clearly separated from the application. In Smalltalk the development tools
(compiler, browsers, debuggers, linkers, etc...) are ALL in the same memory
space (image) as the application program. This changes things because
Smalltalk is an integrated system.

The closest example from the C based world that I can think of is how GNU
tools are included with Linux (and NextStep 5.0/MaxOSX Server ;--). 
But in these situations there is still a clear distinction between the
C compiler, the debugger, and the application programs.

A significant number of questions arise when delivering a Smalltalk
application to end users. Do you include the development tools in the
image? If you application does not need the compiler will removing it in a
runtime image break the application? How can you ensure that it won't? 

Since there is not a 100% clear or clean separation between tools and
application classes (in many cases) you have to dig to find the answers.
(An automated tool would be nice in squeak to help with this 
separating operation to create a runtime). 

What if you've made extensive additions and modifications to the 'system'
classes? You must ensure that they get transferred over. 

The easy thing is to just include everything but just disable access to the
tools. 

What happens to your application when it gets a run time bug? Does a
debugger panel come up (or as one of my clients staff calls it, "we've got
a red panel").

One company has about 20 users of a Smalltalk application and the
simplest solution to this dilemma was to just install full development
copies of the application on their desks. They only actually use the
application except when a red panel comes up in which case the 
application can be debugged. This has been a tremendous boon 
and the client has eliminated bugs very fast. But it requires an 
on site developer (in this case that was easy). A good part of their 
system has evolved in the debugger! 

Including everything seems to be in the spirit that Smalltalk was designed.
It was designed for kids to be able to use so why can't adults handle a red
debugger panel? The answer of course is that you definitely want to limit
who makes and propagates changes to applications used in production
environments. However, rapid debugging and 'field patches' can be a
tremendous advantage.

The experience at this client confirms what Dianna-Merry Shipiro, a member
of the very earliest Smalltalk team, used to tell me "a large portion of
Smalltalk was developed in the debugger".

A side issue: This leads to a very important discussion which is the
availability of source code to people using applications. (See
'Meta programming and Free Availability of Sources' by Francois-Rene Rideau
and Dang-Vu Ban at http://www.tunes.org/~fare and
http://www.tunes.org/~fare/articles/ll99/index.en.html). Basically I
think that the people using applications should have at least read access
to the internal logic of the application they are using. This is especially
true of financial, engineering, medical, government, business and any 
applications being used in situations that could effect peoples lives in
any significant way. While just showing the source code may help I 
think that we need to go beyond simple source code to representations 
of the application that make it easy to see what the application is 
doing. A simulated application view showing the user what the 
application is doing and how it makes it's "seemingly mysterious" 
decisions might help users when the application does not do what 
they want it to.

So a simple decision to deliver the application without the development
tools eliminates a whole side of capabilities that are normally available
from Smalltalk. This changes the nature of the application significantly.

Currently I am working on an application in the discontinued Visual
Smalltalk (from Digitalk) environment. In this Smalltalk you can package
classes and objects into "Smalltalk link libraries(SLL)" (just like DLLs)
that get loaded statically or dynamically by the application at runtime.
The problem with this approach is that Digitalk went wild and recreated
literally hundreds of these packaged SLLs that it's a nightmare figuring
out what you need to include. It's so complicated that I just include
everything because it better than having the application crash when
attempting to load a missing "SLL". This makes for an application size on
the order of 15 megabytes when it could be 2 megs or less. I also just
don't know how this application will function in the field because of all
this extra junk that gets included. What really aggravates this situation
is that it's very difficult to know what's in one of these SLLs and it's
sometimes very difficult to control what VSE puts into a SLL that you
create to hold your own code. Just a big time waster.

The real issue is identifying the dependencies that the new image will have
and giving it just what it needs - no more and no less - and to do so in a
time efficient manner that ensures reliability of the image produced.

A side issue: 
In a few years the application mentioned above will need to be ported to
another Smalltalk and significant changes will need to be made because the
GUI code will most likely be significantly different. The client has
already made the decision to only choose a Smalltalk which has ALL the
source code available. Being dependent upon the Smalltalk vendor leads to
repeated situations of having to port your code base when the vendor makes
unpredictable and unreliable marketing and technical decisions without
concern for their customers development investment. Squeak is being
considered by this client however they currently feel that the GUI's in
squeak are not up to the commercial standards that they need (MSWindows, 
Mac, Nextstep quality and style of look and feel).

If what you are creating is a development environment then you would want
to make sure that it has everything in it and you only quality concern is
that it is not corrupted in any way that will prevent further use and
development.

> Have I failed to provide my clients with an adequate specification of the
> application, as the term is used below?  (Perhaps I was right to have
> changed  professions, after all :-) )  In what sense is it 
> meaningful or important to 
> have so detailed a specification?

No I don't think that you failed to provide adequate specifications. It' s
just that Smalltalk applications are a little harder to separate from their
environment than C based applications. It's all a question of dependencies.
What does the application need to run? When you give your source code to
your client to make a build there are some assumptions of what dependencies
are required in order to successfully build the application. Years (or
months) later when their C based environment or OS changes (Mac OS Carbon
for instance or Win16 to Win32 for example) your application may no longer
build without changes. The same applies to Smalltalk. You could deliver the
Smalltalk source code to your client and specify that it requires Squeak
2.3 and they will be fine. However, when Squeak 3.0 comes out your
application could break because some essential component in the system no
longer functions as you thought when you wrote your code to work with
Squeak 2.3. Thus legacy support issues are born.

I've heard about "Pocket Smalltalk" that can create a tiny application
executables (for example a simple hello world program)  that compares with
the small size of similar C programs. Anyone know more?

It is important to have a detailed specification of a Smalltalk application
and the Smalltalk system that supports it when you need to deliver finished
applications to end customers. This detailed specification enables you to
improve and ensure the quality of the end product. It of course won't
remove your programming bugs :--(.

> I wrote because it seemed to me that this discussion was going around in 
> circles: 

I agree.

> The suggestion was made that a scratch rebuild facility without use of an
> existing image was desired.

I still desire such a facility, but the SystemTracer may suffice. In the
beginning of Smalltalk the first image must have been created somehow.
Little Smalltalk comes with such a facility and does not have a system
tracer. Squeak Smalltalk comes with a SystemTracer class but does not have
the ability to create a image from scratch. I wonder what the difference
really is? There might not be one in practice but the SytemTracer does
require that you have a working and running image! This does not seem to be
a problem though.

> In response, It was suggested that the SystemTracer would suit most needs
> for a scratch rebuild facility. 

Since the existing image works and it's much easier to write Smalltalk code
that to muck around in the C basement world I would agree. A run of the
SystemTracer with appropriate auditing of the objects transferred to the
new
image may meet my quality audit requirements.

> In rebuttal, it was suggested that the SystemTracer is inadequate, for it
> doesn't produce a specification sufficient for the system to be built
from 
> scratch without an existing image.

I suppose it depends upon what you mean by a specification: 
	a) human readable source code that specifies what's in the application and
what the system dependencies the application relies on (possibly in a
'declarative' format);
	b) an image plus some source code that specifies what object to carry
over; or,
	c) some other definition of specification.

> As understood, the last point seems to beg the question.  As I noted
> earlier, I am fairly certain I am missing the point, and write here in
> the selfish hope 
> that expressing my confusion will lead someone to straighten me out and 
> clarify for me the issues.

I don't know if I've covered all the issues in a way that will clarify
them for you or not. They are intermingled with multiple valid solutions to
the same problem. I suppose it comes down to which is the easiest to
implement in Squeak. Since we've got the SystemTracer....

All the best,

Peter

----------------------------------------------------------------------------
---------
Copyright 1999 by Peter William Lount. All rights Reserved.
Portions of this text may be copyright by others.
Trademarks belong to their respective trademark owners.