[squeak-dev] Usability of Squeak FFI for anything serious...

Nicolas Cellier nicolas.cellier.aka.nice at gmail.com
Thu Apr 12 18:14:29 UTC 2018


2018-04-12 15:24 GMT+02:00 Ben Coman <btc at openinworld.com>:

>
>
> On 12 April 2018 at 16:00, Nicolas Cellier <nicolas.cellier.aka.nice@
> gmail.com> wrote:
>
>>
>>
>> 2018-04-12 3:10 GMT+02:00 Eliot Miranda <eliot.miranda at gmail.com>:
>>
>>> Hi Nicolas,
>>>
>>> On Wed, Apr 11, 2018 at 2:23 PM, Nicolas Cellier <
>>> nicolas.cellier.aka.nice at gmail.com> wrote:
>>>
>>>> I'm trying to see how hard it is to port my HDF5 work from VW to Squeak.
>>>> It was not a pleasure in VW, because that already raised some questions
>>>> despite that there is a somehow detailed DLLCC manual:
>>>>
>>>> https://stackoverflow.com/questions/49544642/why-cant-i-pass
>>>> -an-uninterpretedbytes-to-a-void-thru-dll-c-connect
>>>> https://stackoverflow.com/questions/49564024/isnt-pointer-ty
>>>> pe-checking-disabled-in-dll-c-connect-and-is-that-ok
>>>>
>>>
>>> What are you actually trying to do?  Can you post the C definitions?
>>> I'm very familiar with both the SqueakFFI and DLLCC (as of 7.4.1) and the
>>> basic differences are to do with DLLCC's better cross-platform support, in
>>> that it auto-redefines typedefs on load.  But other than that I think the
>>> SqueakFFI is either just as good, or in some cases better (e.g. DLLCC has a
>>> bug in that it is the actual parameter that carries detailed type
>>> information for a struct, whereas in Andreas' SqueakFFI it is, correctly,
>>> the formal parameter).
>>>
>>> Hi Eliot,
>> the library is huge (HDF5).
>> All the interfacing code is available at cincom public store (HDF5 bundle)
>> To get an idea, you can inspect a clone of HDF5 library at
>> https://github.com/live-clones/hdf5/tree/develop/src
>> The main structures/enum/typedefs/prototypes are in the H5*public.h files
>>
>> The problems are all exposed in SO questions, but I can try and
>> priotitize.
>>
>> The essential barrier is that those headers use almost exclusively
>> typedef'ed types.
>> Library is too huge for manual subsitution; this must be mechanized.
>> I would prefer some kind of support in FFI rather than having to hack own
>> fragile ad-hoc pre-processor.
>>
>
> Maybe more of a distraction than a help, but browsing around because its
> an interesting topic,
> some discoveries...
> * https://eli.thegreenplace.net/2012/12/17/dumping-a-c-
> objects-memory-layout-with-clang
> * https://github.com/rpav/c2ffi
> * http://blog.llvm.org/2018/03/dragonffi-ffijit-for-c-language-using.html
> * https://github.com/marianopeck/FFICHeaderExtractor
>
> cheers -ben
>
>
>
Hi Ben,
no time to read now, but I keep your pointers preciously :)
In the interim, I found how to deal with aliases (typedef) and answerd in SO
https://stackoverflow.com/questions/49783882/how-one-deals-with-typedefs-in-squeak-ffi
There is then a problem of initialization order which is not yet solved
though...

That will partly solve my problem of portability 32/64 bits.
https://stackoverflow.com/questions/49784522/how-one-supports-both-32-and-64-bits-target-in-squeak-ffi
But maybe not the problem of LLP64 vs ILP64, some native unsigned long
being used...
(I'll have to redefine some ExternalStructure when detecting a change of
platform...)



>
>> Then I have a problem because of alignment of struct fields which is
>> compact in Squeak FFI rather than conforming to platform conventions.
>> Same here, manually hacking with field size is a no go, this must be
>> mechanized.
>>
>> Then some unions are used too and there is no support at all in FFI.
>>
>> Then definitions like size_t are dependent on machine word size (32 or 64
>> bits), and there is no support for that in FFI.
>> I'll have to duplicate some work for supporting both 32 and 64 bits
>> Squeak, or explore new hacks...
>> But there is no support and no documentation (how to).
>>
>> Then while digging in source coden I did not see any double pointer
>> support in struct fields.
>> The example I provide on SO shows what I would qualify as a bug.
>> I don't know yet whether I'll need those, but it's not good.
>>
>> Maybe I can shoot each problem with more or less ugly workarounds.
>> At the end i can declare void * and deal with offset directly, but that's
>> both:
>> - a lot of work
>> - a good recipe for generating very hard to find bugs - crashes if lucky,
>> silent corruption if not
>>  (dealing with offsets is like writing in assembler instead of C)
>>
>> Given the size of the library, the probability of not creating such fault
>> in manual translation is 0%
>> Plus, there are a few evolutions from one version of the library to the
>> other...
>>
>> That's my definition of hard++
>>
>>
>>>
>>>>
>>>>
>>>> But in Squeak, it sounds like hard++:
>>>>
>>>> https://stackoverflow.com/questions/49782651/how-one-aligns-
>>>> structure-fields-in-squeak-ffi
>>>> https://stackoverflow.com/questions/49783126/how-one-defines
>>>> -a-union-type-in-squeak-ffi
>>>> https://stackoverflow.com/questions/49783443/how-one-defines
>>>> -a-fixed-size-array-member-in-a-struct-in-squeak-ffi
>>>> https://stackoverflow.com/questions/49783882/how-one-deals-w
>>>> ith-typedefs-in-squeak-ffi
>>>> https://stackoverflow.com/questions/49784253/how-one-deals-w
>>>> ith-multiple-pointer-level-like-char-in-squeak-ffi
>>>> https://stackoverflow.com/questions/49784522/how-one-support
>>>> s-both-32-and-64-bits-target-in-squeak-ffi
>>>>
>>>> Either I'm blind, or the very light documentation that I found lacks
>>>> those details
>>>> http://wiki.squeak.org/squeak/2426
>>>>
>>>> I think that I could throw a few more questions (dealing with enums...)
>>>> but I don't want to excede some unknown quota and start irritating a touchy
>>>> SO moderator ;)
>>>>
>>>> I'm afraid that there is no answer but "you're on your own"...
>>>> I'm open to studying Alien or UnifiedFFI of Pharo.
>>>> Or I can try and improve FFI.
>>>> At this stage, my trajectory has already been quite perturbed, and
>>>> despite my knowledge of rocket science, I can see that I'm not going to put
>>>> HDF5 on the intended orbit :(
>>>>
>>>> Thoughts?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> _,,,^..^,,,_
>>> best, Eliot
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20180412/41f01e17/attachment-0001.html>


More information about the Squeak-dev mailing list