[squeak-dev] Passing string to FFI function expecting array of strings (char**)

Michał Olszewski m.olszewski at nexat.pl
Sun Apr 17 12:07:07 UTC 2022


Hi,

Thanks for the answers. I'd like to ask one more question, out of 
curiosity.

Does allocating (and pinning them) a large (possibly very large) 
instances of RawBitsArray (Float64, ByteArray etc. for storing, say, 
textures) puts significant pressure on GC during garbage collection? Or 
should I not worry about that it will perform full GC (thus Image 
freeze) more frequently?

Michał

W dniu 06.04.2022 o 08:31, Marcel Taeumel pisze:
> Hi Michał --
>
> > Have you considered simplifying FFI interface, where callout (the 
> primitive) would only handle raw bytes (along with argument sizes), 
> without type checking?
>
> An approach without type checking would be the Alien FFI. Here is 
> Alien: http://www.squeaksource.com/Alien 
> <http://www.squeaksource.com/Alien>The required IA32ABI plugin is also 
> part of the OpenSmalltalk VM.
>
> Squeak FFI wants to make FFI programming more robust and better 
> integrated into the object-oriented world with its tools. Besides 
> these particular inconveniences you just experienced, the goal is to 
> make that FFI interface with ExternalType very simple and 
> straightforward. And robust. :-)
>
> > Maybe some support for auto-pinning objects that are passed to FFI?
>
> Maybe there already is around FFI callbacks. Not sure.
>
> > It's not like sending #pin is a big deal.
>
> It sure is. It takes time. If you, for example, want to program a 
> graphics back-end through FFI you need all the performance you can 
> get. And too much pinning might annoy the GC compaction phase. :-)
>
> Best,
> Marcel
>>
>> Am 05.04.2022 18:50:59 schrieb Michał Olszewski <m.olszewski at nexat.pl>:
>>
>> Hi,
>>
>>> The type model for pointer types with arity > 1 is still 
>>> rudimentary. You can always go the unsafe way to code against 
>>> 'void*'. In your case, the type 'char*[]' gives you an array type 
>>> with a 'char*' content type. Not sure whether type coercing will 
>>> work during the FFI call. You can also use the 'string[]' type for 
>>> that matter. Again, if you get a type check error on the call, you 
>>> might have to resort to 'void*' instead and pass on that 
>>> ExternalAddress your have at hand.
>> Yes, I did it, sorta. I allocated external bytes, copied the string 
>> to them (along with null terminator), put the handle's bytes into 
>> intptr_t type (a := ExternalType intptr_t allocate. a at: 1 put: h 
>> getHandle asInteger), then passed that one to the call.
>>
>> But I thought I could get away with only using in-squeak-memory 
>> objects (i.e. no external memory allocations since it's a bit of a 
>> pain to handle them across the methods).
>>
>>
>> I have a suggestion, it may not matter much, after all you're much 
>> more familiar with Squeak internals (and far more experienced in 
>> programming overall).
>>
>> Have you considered simplifying FFI interface, where callout (the 
>> primitive) would only handle raw bytes (along with argument sizes), 
>> without type checking? Type checking & marshalling would then be 
>> moved to the Image side, along with some "extras" for example 
>> retrieving memory address of a RawBitsArray instance. This could, for 
>> example, ease implementing n-dim pointers, since all the neccessary 
>> info is easily accessible & no more type juggling.
>>
>> Again, just some random stupid thought, please don't take it 
>> seriously :D.
>>
>>> You have to do it manually if you really need to pass on object 
>>> memory to a call and that callee holds on to the address after the 
>>> return. No need to pin objects that are just read during the call. 
>>> That is, for example, fill your RawBitsArray and pass it into the 
>>> function as is. I think. Just check via #isPinned. Watch out for 
>>> segfaults after GC. Might indicate that you do have to pin that 
>>> object. :-D
>> OK, I pin them all anyway, just in case:). Maybe some support for 
>> auto-pinning objects that are passed to FFI? It's not like sending 
>> #pin is a big deal. Just really easy to forget to insert 10th time 
>> the same method call (and this time it would actually be the one that 
>> is mandatory...).
>>
>> Michał
>>
>> W dniu 2022-04-05 o 11:54, Marcel Taeumel pisze:
>>> Hi Michał --
>>>
>>> > As in the title, what's the way to pass single string to External
>>> > Function that expects array of strings (char**)?
>>>
>>> The type model for pointer types with arity > 1 is still 
>>> rudimentary. You can always go the unsafe way to code against 
>>> 'void*'. In your case, the type 'char*[]' gives you an array type 
>>> with a 'char*' content type. Not sure whether type coercing will 
>>> work during the FFI call. You can also use the 'string[]' type for 
>>> that matter. Again, if you get a type check error on the call, you 
>>> might have to resort to 'void*' instead and pass on that 
>>> ExternalAddress your have at hand.
>>>
>>> (ExternalType typeNamed: 'char*[]') explore.
>>> (ExternalType typeNamed: 'string[]') explore.
>>>
>>> > I assume the new FFI interface will be released along with Squeak 
>>> 6.0? :)
>>>
>>> Well, I will make a tag so that everybody knows what to load in 
>>> Squeak 6.0. Yet, active development will continue after that release 
>>> against Trunk (e.g. 6.1alpha).
>>>
>>> > are objects passed to FFI (the ones allocated
>>> > on the Squeak heap) auto-pinned or do I need pin them manually?
>>>
>>> You have to do it manually if you really need to pass on object 
>>> memory to a call and that callee holds on to the address after the 
>>> return. No need to pin objects that are just read during the call. 
>>> That is, for example, fill your RawBitsArray and pass it into the 
>>> function as is. I think. Just check via #isPinned. Watch out for 
>>> segfaults after GC. Might indicate that you do have to pin that 
>>> object. :-D
>>>
>>> Best,
>>> Marcel
>>>>
>>>> Am 02.04.2022 03:34:38 schrieb Michał Olszewski <m.olszewski at nexat.pl>:
>>>>
>>>> Hello,
>>>>
>>>> As in the title, what's the way to pass single string to External
>>>> Function that expects array of strings (char**)? I know that the 
>>>> way to
>>>> do that in plain C would be something like this: &myString but I have
>>>> not idea how to manipulate Squeak objects such that I can pass their
>>>> addresses manually to FFI. I guess I have to somehow wrap stuff into
>>>> ExternalData but how?
>>>>
>>>> The image used is: Squeak 6.0 alpha 21520.
>>>>
>>>> I assume the new FFI interface will be released along with Squeak 
>>>> 6.0? :)
>>>>
>>>> At last, quick question: are objects passed to FFI (the ones allocated
>>>> on the Squeak heap) auto-pinned or do I need pin them manually?
>>>>
>>>> Michał
>>>>
>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20220417/10897710/attachment.html>


More information about the Squeak-dev mailing list