[Vm-dev] [squeak-dev] FFI ExternalTypeAlias

Nicolas Cellier nicolas.cellier.aka.nice at gmail.com
Fri Jun 19 14:02:50 UTC 2020


Err, I move this thread to Opensmalltalk VM dev, because it is not Squeak
specific!



Le ven. 19 juin 2020 à 15:10, Nicolas Cellier <
nicolas.cellier.aka.nice at gmail.com> a écrit :

> Hi Marcel,
> thanks for your vote. Anyone else?
>
> Le ven. 19 juin 2020 à 09:30, Marcel Taeumel <marcel.taeumel at hpi.de> a
> écrit :
>
>> Hi Nicolas, hi all! :-)
>>
>> > Type alias are subclasses of ExternalStructure.
>>
>> And with it, they get their own instance of ExternalType, which is
>> managed in the classVar StructTypes. Let's call them struct type.
>>
>> > Now we have another mean by adding a class side method in ExternalType.
>>
>> That kind of aliasing is for compile-time type referencing only -- such
>> as in FFI calls (i.e. <apicall:...> pragmas) or struct-field definitions
>> (i.e. #fields). (And soon <callback: ...> if I have the first version of
>> FFI-Callback to show you :)
>>
>> Because those aliases become fully transparent after compiling the method
>> for the FFI call into an instance of ExternalLibraryFunction and after
>> compiling the field specs of external structures into compiledSpec for
>> struct types.
>>
>> For example, on my machine in 32-bit Squeak, a 'size_t' becomes a 'long',
>> which is Squeak FFI term for 'int'. :-) See #ffiLongVsInt. And 'int32_t'
>> also becomes a 'long'. So, it just translates C vocabulary into Squeak FFI
>> vocabulary. Nothing more. Nothing less.
>>
>> Having that, please do not use class-side methods in ExternalType to
>> alias a (complex) struct type. Use them only for renaming atomic types.
>> Please. We may want to add checks to enforce that.
>>
>> I suppose that this discussion focuses on type aliases that are
>> represented as subclasses of ExternalStructure (or ExternalTypeAlias) and
>> thus get their own struct type. So, let's ignore this other mechanism for
>> now.
>>
>> > This has one advantage: type safety. You cannot pass a Foo to a
>> function expecting a Bar, even if they both alias int...
>>
>> Yes! So, we are now talking about type aliases to atomic types. The
>> struct type you get when aliasing a 'char' as Foo, for example, looks like
>> this:
>>
>> (...forgive my "creative" representation of WordArray here...)
>>
>> compiledSpec: 0A 04 00 01
>> referentClass: Foo
>>
>> How does the atomic type for 'char' look like?
>>
>> compiledSpec: 0A 04 00 01
>> referentClass: nil
>>
>> Consequently, argument coercing will fail if you pass a 'char' when a
>> 'Foo' is expected. Because the referentClass does not match --- even if the
>> compiledSpec does match.
>>
>> > But this has one major drawback: with current FFI, you cannot pass a
>> Smalltalk Integer, to a method expecting a Bar, even if Bar is an alias for
>> an integer type...
>>
>> That's correct. If the origin of such an argument is from within the
>> image, you have to wrap it. But when returned from another call ... well
>> ... that wrapping should have happened in the plugin ... But read on! :-)
>>
>> > FFI plugin is clever enough to recognize an atomic type alias and
>> simply return a Smalltalk Integer...
>> > So it's not Bar all the way down, I have to manually wrap Bar...
>> > This is not consistent.
>>
>> Uhh! That's a bug. The FFI Plugin must take a look at referentClass
>> because it does so for argument coercing.
>>
>> Now I understand why the code generation of struct-field accessors was so
>> apparently broken for me. I changed that so that having an alias type as
>> part of a larger struct will automatically wrap that for you into the alias
>> structure. :-) Even if you are just aliasing a plain 'int' ... or 'long'
>> ... Ha ha. ;-) #ffiLongVsInt.
>>
>> It should be "Bar all the way down". Definitely.
>>
>> > If we return an int, we must accept an int, otherwise, we cannot chain
>> FFI calls...
>>
>> Well, not for type aliases. I would like keep the path of type safety
>> here, you mentioned above.
>>
>> Just fix the FFI plugin to respect referentClass when packaging the
>> return value.
>>
>> > I could use the lightweight alias instead, [...]
>>
>> Please don't. See above. Those "lightweight alias" are for renaming
>> atomic types only. Let's keep it simple and try to address you concern here
>> with struct types and ExternalTypeAlias. :-)
>>
>> > there was another usage where it was convenient to use
>> ExternalTypeAlias: enum.
>> > Indeed, I simply defined all enum symbols as class side method.
>>
>> I like that! It reads like a next step for ExternalPool. Once you have
>> extracted the constant values from header files, and once you have those
>> values for your platform in classVars in your external pool, use that pool
>> to define enums through ExternalStructure.
>>
>> Like this:
>>
>> ExternalTypeAlias subclass: #MyEnumBar
>> instanceVariableNames: ''
>> classVariableNames: ''
>> poolDictionaries: 'HDF5Pool'
>> category: 'HDF5'
>>
>> MyEnumBar class >> #poolFields
>>    "
>>    self defineFields.
>>    "
>>    ^ #(
>>       (beg HDF5_BEG)
>>       (mid HDF5_MID)
>>       (end HDF5_END)
>>    )
>>
>> Note that HDF5_BEG etc. are from the HDF5Pool, which is managed via
>> external-pool definitions. See class comment in ExternalPool to get started.
>>
>> Note that #poolFields would translate to class-side methods as you
>> suggested.
>>
>> > If I use a simpler alias, where are those ids going to?
>>
>> Please don't. See above. :-)
>>
>> > IMO we cannot keep status quo in the plugin, we gotta to make the
>> inputs/outputs behave consistently.
>>
>> Absolutely!
>>
>> > - should a function expecting an ExternalTypeAlias of atomic type
>> accept an immediate value of compatible type?
>>
>> Considering type safety, I think not.
>>
>> Well ... since this is a convenience issue and thus not about saving
>> run-time cost ... What about adding a callback into the image to react on
>> FFIErrorCoercionFailed? Maybe the selector for such a callback could be
>> stored in ExternalLibraryFunction since this is accessible to the plugin
>> anyway?
>>
>> <apicall: void 'foo' (MyInt YourInt in) ifFailCoerceVia: #wrapInt: >
>>
>> ... would it be possible? Like that #doesNotUnderstand: callback?
>>
>> I would love to do ad-hoc packaging of arguments. (Also thinking about
>> FFI-Callback. But ignore me here :)
>>
>> > - should a function returning an ExternalTypeAlias of atomic type
>> instantiate the Alias?
>>
>> Yes, please! See above.
>>
>> If you want to trade type safety in for a performance gain, just use a
>> "lightweight alias" as explained above. And package that return value
>> manually.
>>
>> Best,
>> Marcel
>>
>> Am 18.06.2020 21:03:53 schrieb Nicolas Cellier <
>> nicolas.cellier.aka.nice at gmail.com>:
>> Hi all,
>> following the question of Marcel about usage of ExternalTypeAlias:
>>
>> Type alias are subclasses of ExternalStructure.
>> I used them in HDF5 because there was no other means to define an alias
>> at the time.
>> Now we have another mean by adding a class side method in ExternalType (I
>> would have rather imagined the usage of some annotation to get a
>> declarative style)
>>
>> This has one advantage: type safety. You cannot pass a Foo to a function
>> expecting a Bar, even if they both alias int...
>>
>> But this has one major drawback: with current FFI, you cannot pass a
>> Smalltalk Integer, to a method expecting a Bar, even if Bar is an alias for
>> an integer type...
>>
>> Thus, you have to wrap every Bar argument into a Bar instance...
>> Gasp, this is going too far...
>>
>> What about functions returning such alias?
>> Function returning struct by value allocate a struct, but it's not the
>> case here, FFI plugin is clever enough to recognize an atomic type alias
>> and simply return a Smalltalk Integer...
>> So it's not Bar all the way down, I have to manually wrap Bar...
>> This is not consistent.
>> If we return an int, we must accept an int, otherwise, we cannot chain
>> FFI calls...
>>
>> I could use the lightweight alias instead, but there was another usage
>> where it was convenient to use ExternalTypeAlias: enum.
>> Indeed, I simply defined all enum symbols as class side method. For
>> example for enum bar { beg=0, mid=1, end=2 }; I simply create a type Bar
>> aliasing int and having class side methods beg ^0, mid ^1, end ^2.
>> This way, I normally can pass a Bar mid to an external function.
>> (currently, we must defne mid ^Bar new value: 1; yourself)
>>
>> If I use a simpler alias, where are those ids going to?
>>
>> IMO we cannot keep statu quo in the plugin, we gotta to make the
>> inputs/outputs behave consistently.
>>  The choice is this one:
>> - should a function expecting an ExternalTypeAlias of atomic type accept
>> an immediate value of compatible type? (the permissive implementation)
>> - should a function returning an ExternalTypeAlias of atomic type
>> instantiate the Alias? (the typesafe implementation, with higher runtime
>> cost due to pressure on GC)
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20200619/f53314d3/attachment.html>


More information about the Vm-dev mailing list