[Vm-dev] Slang newbie question

Tue Sep 15 03:22:11 UTC 2009

The comment is mainly a note to myself, but by way of explanation,
if you allocate a new object (such as a String that you intend to
use within the plugin), the act of allocating that object may cause
Squeak to run its garbage collector to make space for the new string
object, which moves the objects in object memory, potentially making
OOP references in your plugin invalid. Thus my intent was that
#transientCStringFromString: should be used only if the null-terminated
string buffer is going to be used while the primitive is run, and
the comment is a reminder to use #pushRemappableOop: and #popRemappableOop
to protect against the garbage collector moving OOPs in the plugin.

Setting that issue aside for the moment, what you are probably trying
to do here is just convert the Squeak string into a (char *) that will
be valid for use in the external C library. For this you can malloc
a buffer for the null-terminated string (because the buffer will stay
allocated after your primitive exits, even if it is being used elsewhere
by an external library), or you can use a statically allocated buffer
in your primitive if you know in advance the maximum length of the string.

I recall that Eliot Miranda suggested using alloca() rather than
malloc(), as this avoids the need to free allocated memory later.
This is probably the best approach overall if the library function
runs to completion as part of your primitive (in other words, as
long as the function does not start some new thread that makes use
of the string after your primitive has completed and returned control
to Squeak).

To summarize: If you want to pass a Smalltalk string of length 10
to a primitive, copy the 10 characters in the string into a new
buffer of length 11 with the 11th byte set to zero, then pass that
buffer to the C function.

Dave

On Mon, Sep 14, 2009 at 07:36:41PM -0700, Ronald Spengler wrote:
> 
> Thanks Dave! That helps a lot. This snippet scares me a little bit though:
> "in a section of code in which the garbage collector is guaranteed not to run"
> 
> I realize now that:
>  - I don't know how to guarantee that the gc won't run
>  - My C library will take it's sweet time running, and it's runtime is
> a function of it's input, could be forever in the extreme case.
> 
> So, to be safe, on the C side of things, should I copy the string
> ASAP? Or does C code escape the garbage collector? Is it safe to
> malloc()?
> 
> Thanks again for your help, and please forgive my ignorance.
> 
> 
> On Sun, Sep 13, 2009 at 6:56 PM, David T. Lewis <lewis at mail.msen.com> wrote:
> >
> > On Sun, Sep 13, 2009 at 01:23:32PM -0700, Ronald Spengler wrote:
> > >
> > > Hello everyone.
> > > I have a named primitive, and I need to send a ByteString to it, to be
> > > processed and returned by an external library. To get a string into Slang,
> > > should I send it #asByteArray, and would that let me treat the bytes as
> > > integers on the stack? I'm basically trying to get a char* on the other
> > > side.
> >
> > You can use the ByteString as a parameter to the primitive, no problem.
> > The only tricky bit is that C expects null terminated strings, so you need
> > to copy the contents of the ByteString into a null terminated array before
> > you can let it be used by the C library as a char *.
> >
> > I'm sure there are lots of examples, but you can look at
> > OSProcessPlugin>>cStringFromString: and OSProcess>>transientCStringFromString:
> > for examples of how to copy the string buffer into a null terminated buffer
> > for use in C. Look at senders of these two methods for examples of primitives
> > that pass strings as parameters. (OSProcessPlugin is on SqueakSource if you
> > do not have it).
> >
> > Following are a couple of examples taken from OSPP. In both cases, a buffer
> > is allocated with size one greater than the string length, and the contents
> > of the Smalltalk string are copied into the buffer space with a trailing
> > null terminator. The #primitiveChdir example allocates a new Smalltalk
> > string to use for the buffer, and #primitivePutEnv uses malloc to allocate
> > the new buffer (because in this case the buffer must be "permanently" valid
> > after the primitive exits).
> >
> > primitiveChdir
> > ? ? ? ?"Call chdir(2) to change current working directory to the specified path string. Answer
> > ? ? ? ?nil for success, or errno on failure."
> >
> > ? ? ? ?| path errno |
> > ? ? ? ?self export: true.
> > ? ? ? ?self var: 'path' type: 'char *'.
> > ? ? ? ?self var: 'errno' type: 'extern int'.
> > ? ? ? ?path := self transientCStringFromString: (interpreterProxy stackObjectValue: 0).
> > ? ? ? ?(self chdir: path)
> > ? ? ? ? ? ? ? ?ifTrue: [interpreterProxy pop: 2; push: interpreterProxy nilObject]
> > ? ? ? ? ? ? ? ?ifFalse: [interpreterProxy pop: 2; pushInteger: errno].
> >
> > transientCStringFromString: aString
> > ? ? ? ?"Answer a new null-terminated C string copied from aString.
> > ? ? ? ?The string is allocated in object memory, and will be moved
> > ? ? ? ?without warning by the garbage collector. Any C pointer
> > ? ? ? ?reference the the result is valid only until the garbage
> > ? ? ? ?collector next runs. Therefore, this method should only be used
> > ? ? ? ?within a single primitive in a section of code in which the
> > ? ? ? ?garbage collector is guaranteed not to run. Note also that
> > ? ? ? ?this method may itself invoke the garbage collector prior
> > ? ? ? ?to allocating the new C string.
> >
> > ? ? ? ?Warning: The result of this method will be invalidated by the
> > ? ? ? ?next garbage collection, including a GC triggered by creation
> > ? ? ? ?of a new object within a primitive. Do not call this method
> > ? ? ? ?twice to obtain two string pointers."
> >
> > ? ? ? ?| len stringPtr newString cString |
> > ? ? ? ?self returnTypeC: 'char *'.
> > ? ? ? ?self var: 'stringPtr' declareC: 'char *stringPtr'.
> > ? ? ? ?self var: 'cString' declareC: 'char *cString'.
> > ? ? ? ?len := interpreterProxy sizeOfSTArrayFromCPrimitive: (interpreterProxy arrayValueOf: aString).
> > ? ? ? ?"Allocate space for a null terminated C string."
> > ? ? ? ?interpreterProxy pushRemappableOop: aString.
> > ? ? ? ?newString := interpreterProxy
> > ? ? ? ? ? ? ? ?instantiateClass: interpreterProxy classString
> > ? ? ? ? ? ? ? ?indexableSize: len + 1.
> > ? ? ? ?stringPtr := interpreterProxy arrayValueOf: interpreterProxy popRemappableOop.
> > ? ? ? ?cString := interpreterProxy arrayValueOf: newString. ? ? ? ? ? ?"Point to the actual C string."
> > ? ? ? ?self cCode: '(char *)strncpy(cString, stringPtr, len)'. ? ? ? ? "Make a copy of the string."
> > ? ? ? ?cString at: (len) put: 0. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "Null terminate the C string."
> > ? ? ? ?^ cString
> >
> > primitivePutEnv
> > ? ? ? ?"Set an environment variable using a string of the form 'KEY=value'. This
> > ? ? ? ?implementation allocates a C string using malloc to allocate from the C heap
> > ? ? ? ?(using cStringFromString rather than transientCStringFromString). This
> > ? ? ? ?is necessary because the C runtime library does not make a copy of the
> > ? ? ? ?string into separately allocated environment memory."
> >
> > ? ? ? ?| cStringPtr keyValueString |
> > ? ? ? ?self export: true.
> > ? ? ? ?self var: 'cStringPtr' declareC: 'char *cStringPtr'.
> > ? ? ? ?keyValueString := interpreterProxy stackObjectValue: 0.
> > ? ? ? ?cStringPtr := self cStringFromString: keyValueString.
> > ? ? ? ?((self putenv: cStringPtr) == 0) ? ? ? ?"Set environment variable."
> > ? ? ? ? ? ? ? ?ifTrue: [interpreterProxy pop: 2; push: keyValueString]
> > ? ? ? ? ? ? ? ?ifFalse: [^ interpreterProxy primitiveFail]
> >
> > cStringFromString: aString
> > ? ? ? ?"Answer a new null-terminated C string copied from aString. The C string
> > ? ? ? ?is allocated from the C runtime heap. See transientCStringFromString for
> > ? ? ? ?a version which allocates from object memory.
> > ? ? ? ?Caution: This may invoke the garbage collector."
> >
> > ? ? ? ?| len sPtr cString |
> > ? ? ? ?self returnTypeC: 'char *'.
> > ? ? ? ?self var: 'sPtr' declareC: 'char *sPtr'.
> > ? ? ? ?self var: 'cString' declareC: 'char *cString'.
> > ? ? ? ?sPtr := interpreterProxy arrayValueOf: aString.
> > ? ? ? ?len := interpreterProxy sizeOfSTArrayFromCPrimitive: sPtr.
> > ? ? ? ?cString := self callocWrapper: len + 1 size: 1. ? ? ? ? "Space for a null terminated C string."
> > ? ? ? ?self cCode: '(char *) strncpy (cString, sPtr, len)'. ? ?"Copy the string."
> > ? ? ? ?^ cString
> >
> > callocWrapper: count size: objectSize
> > ? ? ? ?"Using malloc() and calloc() is something I would like to avoid, since it is
> > ? ? ? ?likely to cause problems some time in the future if somebody redesigns
> > ? ? ? ?object memory allocation. This wrapper just makes it easy to find senders
> > ? ? ? ?of calloc() in my code. -dtl"
> >
> > ? ? ? ?self returnTypeC: 'void *'.
> > ? ? ? ?^ self cCode: 'calloc(count, objectSize)'
> >
> > Dave
> >
> 
> 
> 
> --
> Ron