[squeak-dev] Usability of Squeak FFI for anything serious...

Nicolas Cellier nicolas.cellier.aka.nice at gmail.com
Thu Apr 12 08:00:49 UTC 2018


2018-04-12 3:10 GMT+02:00 Eliot Miranda <eliot.miranda at gmail.com>:

> Hi Nicolas,
>
> On Wed, Apr 11, 2018 at 2:23 PM, Nicolas Cellier <
> nicolas.cellier.aka.nice at gmail.com> wrote:
>
>> I'm trying to see how hard it is to port my HDF5 work from VW to Squeak.
>> It was not a pleasure in VW, because that already raised some questions
>> despite that there is a somehow detailed DLLCC manual:
>>
>> https://stackoverflow.com/questions/49544642/why-cant-i-pass
>> -an-uninterpretedbytes-to-a-void-thru-dll-c-connect
>> https://stackoverflow.com/questions/49564024/isnt-pointer-
>> type-checking-disabled-in-dll-c-connect-and-is-that-ok
>>
>
> What are you actually trying to do?  Can you post the C definitions?  I'm
> very familiar with both the SqueakFFI and DLLCC (as of 7.4.1) and the basic
> differences are to do with DLLCC's better cross-platform support, in that
> it auto-redefines typedefs on load.  But other than that I think the
> SqueakFFI is either just as good, or in some cases better (e.g. DLLCC has a
> bug in that it is the actual parameter that carries detailed type
> information for a struct, whereas in Andreas' SqueakFFI it is, correctly,
> the formal parameter).
>
> Hi Eliot,
the library is huge (HDF5).
All the interfacing code is available at cincom public store (HDF5 bundle)
To get an idea, you can inspect a clone of HDF5 library at
https://github.com/live-clones/hdf5/tree/develop/src
The main structures/enum/typedefs/prototypes are in the H5*public.h files

The problems are all exposed in SO questions, but I can try and priotitize.

The essential barrier is that those headers use almost exclusively
typedef'ed types.
Library is too huge for manual subsitution; this must be mechanized.
I would prefer some kind of support in FFI rather than having to hack own
fragile ad-hoc pre-processor.

Then I have a problem because of alignment of struct fields which is
compact in Squeak FFI rather than conforming to platform conventions.
Same here, manually hacking with field size is a no go, this must be
mechanized.

Then some unions are used too and there is no support at all in FFI.

Then definitions like size_t are dependent on machine word size (32 or 64
bits), and there is no support for that in FFI.
I'll have to duplicate some work for supporting both 32 and 64 bits Squeak,
or explore new hacks...
But there is no support and no documentation (how to).

Then while digging in source coden I did not see any double pointer support
in struct fields.
The example I provide on SO shows what I would qualify as a bug.
I don't know yet whether I'll need those, but it's not good.

Maybe I can shoot each problem with more or less ugly workarounds.
At the end i can declare void * and deal with offset directly, but that's
both:
- a lot of work
- a good recipe for generating very hard to find bugs - crashes if lucky,
silent corruption if not
 (dealing with offsets is like writing in assembler instead of C)

Given the size of the library, the probability of not creating such fault
in manual translation is 0%
Plus, there are a few evolutions from one version of the library to the
other...

That's my definition of hard++


>
>>
>>
>> But in Squeak, it sounds like hard++:
>>
>> https://stackoverflow.com/questions/49782651/how-one-aligns-
>> structure-fields-in-squeak-ffi
>> https://stackoverflow.com/questions/49783126/how-one-defines
>> -a-union-type-in-squeak-ffi
>> https://stackoverflow.com/questions/49783443/how-one-defines
>> -a-fixed-size-array-member-in-a-struct-in-squeak-ffi
>> https://stackoverflow.com/questions/49783882/how-one-deals-
>> with-typedefs-in-squeak-ffi
>> https://stackoverflow.com/questions/49784253/how-one-deals-
>> with-multiple-pointer-level-like-char-in-squeak-ffi
>> https://stackoverflow.com/questions/49784522/how-one-support
>> s-both-32-and-64-bits-target-in-squeak-ffi
>>
>> Either I'm blind, or the very light documentation that I found lacks
>> those details
>> http://wiki.squeak.org/squeak/2426
>>
>> I think that I could throw a few more questions (dealing with enums...)
>> but I don't want to excede some unknown quota and start irritating a touchy
>> SO moderator ;)
>>
>> I'm afraid that there is no answer but "you're on your own"...
>> I'm open to studying Alien or UnifiedFFI of Pharo.
>> Or I can try and improve FFI.
>> At this stage, my trajectory has already been quite perturbed, and
>> despite my knowledge of rocket science, I can see that I'm not going to put
>> HDF5 on the intended orbit :(
>>
>> Thoughts?
>>
>>
>>
>>
>>
>>
>
>
> --
> _,,,^..^,,,_
> best, Eliot
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20180412/75de1578/attachment.html>


More information about the Squeak-dev mailing list