callbacks

Andreas Raab Andreas.Raab at gmx.de
Sat Jun 22 19:12:47 UTC 2002


Lex,

You are basically correct with your judgement but you are missing a few
points. First of all let's say that your worrying about "opaque C
pointers" is no issue at all. All you need to do is to pass in some
pointer and coerce it from Smalltalk by using "FooStructure fromHandle:
fooPointer" and if FooStructure is an external structure things will
work well.

The more interesting stuff is elsewhere. First of all, the issue (as you
pointed out correctly) where exactly callbacks are invoked from. Most
systems don't do true async callbacks (e.g., callbacks from interrupts)
so what's needed is to check all places within Squeak which might invoke
system calls that might in turn invoke callbacks. This includes stuff
like ioMicroMSecs which might well invoke a timer callback. However, I
actually believe that Squeak is mostly safe in this respect (details may
vary per platform).

Second, and this is where it gets *really* interesting is what is the
"callback pointer" you are passing to the C side. E.g., since you want
to signal a semaphore from the callback it would need to look something
like:
	void sqCallback(int semaIndex);
But what do you do if your callback is declared as
	DWORD wndProc(DWORD, DWORD, DWORD, DWORD);
??? E.g., C will call you with the argument *it* wants to pass which are
not the ones you actually need. In order to handle this problem you
almost certainly have to generate native code (there is a solution
without but it's incredibly awkward and has some severe limits).

Another interesting issue that you'll have to bind callbacks to
Processes to begin with. That's because you cannot easily interrupt the
running process and insert some "callback context" which will evaluate
the block (or message) you associate with the callback. This means that
callbacks will *always* be handled from different threads.

Last but not least you need to handle returns appropriately. And that's
not only an issue of deadlock situations (e.g., when some callback tries
to return through another one). In many cases, it is expected that a
callback completes before the invoking function returns. Now consider
this:
* process Foo calls primitiveBar
* primitiveBar invokes callback mumble (run in process Mumble)
* for "some reason" there is a process switch from Mumble to Foo
which would mean that Foo returns from primitiveBar before it even has
completed. Oh boy.

In short, what you need to do is something like:
* figure out if the VM is reentrant from all places
* implement the callback stubs which need to
  - save the arguments passed in from C
  - signal the associated semaphore with the callback [*1]
  - call interpret
* define a primitive that returns the "raw" callback data (e.g., just a
portion of the C stack which was used for the callback)
* figure out how to avoid deadlocks (when a callback is trying to return
"through" another callback)

[*1] A simpler and more safe way might be to suspend the active process
and pass it to the callback process. This would make sure that a process
is truly "interrupted" when a callback occurs. It would also avoid the
nested callback return problem to a large amount.

Well, good luck with it ;-)

Cheers,
  - Andreas

> -----Original Message-----
> From: squeak-dev-admin at lists.squeakfoundation.org 
> [mailto:squeak-dev-admin at lists.squeakfoundation.org] On 
> Behalf Of Lex Spoon
> Sent: Saturday, June 22, 2002 5:59 PM
> To: squeak-dev at lists.squeakfoundation.org
> Subject: callbacks
> 
> 
> Markus Gaelli <gaelli at emergent.de> wrote:
> 
> > > Ned Konz wrote:
> > > It's too bad, really; if you look at the Ruby bindings 
> for Gtk, you'll
> > > see that the callback code is very small.
> > >
> > How hard would it be to change the current 
> VM-Implementation(s?) to make
> > callback-services possible? Impossible? If Ruby can, why can't we?
> > 
> 
> Most of the problem simply seems very fiddly.  There are a lot of
> details.
> 
> Let's step through it and see what is needed.
> 
> When the C callback gets invoked, it will want to fire up 
> some Smalltalk
> code.  The simplest way given the current implementation would be to
> trigger a semaphore, re-invoke interpret(), and wait for the image to
> signal that it has finished processing the request.  If there is a
> return value from the callback, then this needs to be handled 
> somewhere.
>  The only new primitive needed by this discription is some sort of
> "return:" primitive, which is allowed to supply a return value.
> 
> The hardest part seems to be converting between C data and Smalltalk
> data.  In particular, what do you do with opaque pointers?
> 
> Finally, there is a complication if multiple callbacks are active at
> once and they return in the wrong order.  This can happen if the
> callbacks are handled in different threads!  It suggests that each
> callback might be given a code, and return: should supply that code as
> an argument.  Then there are two new tweaks to the system:
> 
> 	1. return: doesn't actually return unless the code 
> matches the expected
> code.
> 
> 	2. Somehow, "wrong" return's need to happen eventually. 
>  Perhaps if a
> return: fails, the process trying to return should be put on 
> a separate
> queue somewhere.  Then, the return: primitive can resume all processes
> on that queue before it actually returns from interpret().  Then, the
> next time interpret() is fired, all of these processes will try to
> "return" again.
> 
> One little detail is to decide what places a callback may 
> legally spring
> from.  For example, the processIO primitive would be a logical placess
> that it's allowed, and maybe any non-small primitive may do it.  The
> main issue is that all interpreter state must be saved before the
> interpreter is re-entered, and it must be restored afterwards.
> 
> Once all of this is in place, the VM should be sufficient.  
> Then, there
> "simply" remains the problem of translating Squeak data to C data and
> vice versa.  I mean, it would be especially nice if you could 
> supply the
> callback as a block and have Squeak handle all the conversions for you
> automatically.  But I don't know the details of how to do this.  In
> particular, we need to handle pointers to opaque C structures.
> 
> 
> -Lex
> 




More information about the Squeak-dev mailing list