ExternalByteArray and structures

Edward P Luwish eluwish at uswest.com
Thu May 20 22:48:09 UTC 1999



Stephen Pair wrote:

> > I am not sure whether this concept fits in with the [much] earlier
> > discussion about calling out of or into Smalltalk, or how it compares with
> > pluggable entities, since I am new to Squeak, but it would be an excellent
> > way of "grafting" functionality into Squeak before completing its native
> > implementation.  I also do not know how much surgery would have to be
> > performed on ObjectMemory in order to accomplish it.  It just looks like
> > a good idea we can borrow if it hasn't been patented.
>
> Do you mean taking stuff from QKS and porting it to Squeak?

Not at all - as far as I know, there is nothing written in QKS
Smalltalk that has not been done in Squeak.  However, there are
lots of programs written in imperative programming languages that do
things that Squeak will probably not do for a while.  For some of
these programs or library routines, I only have the interface
definitions (.h files), not the sources.  I would like a way to add
their functionality to Squeak, without recoding these programs or
adding Squeak primitives.  If there exists a chunk of code written in
any compiled language, and it has the desired functionality,
there should be a way to use Squeak as a front end to it.

One way to do this is to have a shared region of memory.
The compiled code should be able to read and write to the
shared memory using its existing means for defining and
accessing data, such as a C assignment statement to a member
element of a struct.  The Smalltalk method should be able to
send messages to an object which implements appropriate access
to the pieces of the structure.  QKS implemented a way of
mapping C data types to classes such as CSignedInteger, CCharacter,
etc. (I'm making up the names, since it's been a while) as well
as mapping C aggregate types (arrays, structs, unions) to subclasses
of Structure, which they defined as a packed array of varying length
sequences of bytes.  The big leap was to then allow any object to
optionally contain (or point to) a contiguous region of raw memory
which could be accessed via a subclass of Structure.  The external
program could access it directly or any other way it felt like.  You
could substitute ExternalByteArray to manage raw memory, rather than
making it part of an object.  The suggestion I make, which would
simplify the interface to existing external code, is to introduce a
Structure class which places an overlay or template on the raw data,
so that individual data items can be accessed by sending messages to
an instance of the appropriate subclass of Structure (there would be
one subclass for each different struct or union - union classes would
have their own subclasses for each variant).

The goal is to interface with existing code without having to jacket
it in any special way.  If I find a neat PostScript font rasterizer, I
don't want to have to force it to look like a primitive.

> I don't think a
> Squeak implementation of a call-in call-out mechanism is that far off.
> Also, one would probably be required to purchase QKS in order to get that
> functionality.  There are really two components involved in a generic
> call-in call-out mechanism:  1) the marshalling (which you have described
> the QKS solution) and 2) the thunking.  I believe that both should be
> independent and very pluggable.
>

It's not exactly marshalling in the strict sense of the word,
since we are not talking about calling a subroutine with parameters,
but more like being able to access structured global and dynamically
allocated variables, which the average C program uses to a
nauseating degree.  I haven't thought about how you would deal with
the thunking or worse, how you synchronize memory allocation and
deallocation between a Smalltalk program and a C program so you
don't leak or crash.

>
> Also, I think the same functionality can be achieved without changing the
> structure of ObjectMemory in any way.  I'm not familiar with the QKS
> implementation, but embedding raw binary formats in any object seems a bit
> extreme.
>
> - Stephen

I agree.  I like ExternalByteArray much better than embedding.  I still
advocate Structure classes as a way of allowing Smalltalk to deal with
structured data ala C structs and Pascal records, without compromising
its purity.





More information about the Squeak-dev mailing list