[Vm-dev] nested structures with FFI?

[Nicolas Cellier]

2015-08-21 11:53 GMT+02:00 Esteban Lorenzano <estebanlm at gmail.com>:

>
>
> On 21 Aug 2015, at 11:45, Nicolas Cellier <
> nicolas.cellier.aka.nice at gmail.com> wrote:
>
>
>
> 2015-08-21 5:57 GMT+02:00 Esteban Lorenzano <estebanlm at gmail.com>:
>
>>
>>
>> On 21 Aug 2015, at 04:02, Esteban Lorenzano <estebanlm at gmail.com> wrote:
>>
>>
>> On 20 Aug 2015, at 22:47, Esteban Lorenzano <estebanlm at gmail.com> wrote:
>>
>>
>> On 20 Aug 2015, at 21:48, Max Leske <maxleske at gmail.com> wrote:
>>
>>
>> On 20 Aug 2015, at 21:28, vm-dev-request at lists.squeakfoundation.org
>> wrote:
>>
>> Message: 1
>> Date: Thu, 20 Aug 2015 17:51:43 +0200
>> From: Esteban Lorenzano <estebanlm at gmail.com>
>> Subject: [Vm-dev] nested structures with FFI?
>> To: Squeak Virtual Machine Development Discussion
>> <vm-dev at lists.squeakfoundation.org>
>> Message-ID: <4F31ED4B-6246-43D6-B85B-7215D8570D28 at gmail.com>
>> Content-Type: text/plain; charset=utf-8
>>
>> Hi,
>>
>> I’m doing some tests with nested structures (to handle the conversion of
>> NativeBoost-FFI to regular FFI) and I’m having some problems when accessing
>> fields in the nested structure.
>> For example, this tests:
>>
>> FFITestNestingStructure class>>#fieldsDesc
>> ^ #(
>> byte one;
>> FFITestStructure nested;
>> )
>>
>> FFITestStructure class>>#fieldsDesc
>> ^ #(
>> byte byte;
>> short short;
>> long long;
>> float float;
>> double double;
>> int64 int64;
>> )
>>
>> (definition is NB-compatible, so do not take the differences with regular
>> FFI into account)
>>
>> testNestedStructure
>> | s1 |
>>
>> s1 := FFITestNestingStructure new.
>> s1 nested byte: 42.
>> self assert: s1 nested byte = 42
>>
>> testExternallyAllocatedNestedStructure
>> | s1 |
>>
>> s1 := FFITestNestingStructure externalNew.
>> self assert: s1 getHandle class = ExternalAddress.
>> s1 nested byte: 42.
>> self assert: s1 nested byte = 42
>>
>>
>> fails always because
>>
>> s1 nested byte = 0.
>>
>> (which is obviously bad).
>>
>> I remember Eliot saying that structures with FFI was not being optimal,
>> and I wonder (before start digging more in deep) if this could have
>> something to do?
>>
>>
>> I use nested structs in NB for the libgit2 bindings and they work as
>> advertised. There can be problems if some structure is packed (i.e. uses
>> non-4-byte alignment for some field) but that shouldn’t apply in your test
>> case.
>>
>> Not sure if this helps you in any way but I thought I’d let you know :)
>>
>>
>> I do not think so :)
>> I’m quite sure this is not going to be a problem when using the
>> ThreadedFFI backend instead ASMJIT… now my problem is with this backend,
>> for the moment :)
>>
>>
>> I’m just realised that this behaviour is correct according current
>> ExternalStructure implementation, because:
>>
>> nested
>> "This method was automatically generated"
>> ^FFITestStructure fromHandle: (handle structAt: 2 length: 27)
>>
>> and
>>
>> ByteArray>>#structAt: byteOffset length: length
>> "Return a structure of the given length starting at the indicated byte
>> offset."
>> | value |
>> value := ByteArray new: length.
>> 1 to: length do:[:i|
>> value unsignedByteAt: i put: (self unsignedByteAt: byteOffset+i-1)].
>> ^value
>>
>> So it is giving me always a copy…
>> So we need a better solution (a non copy way to access)? I’m thinking on
>> passing a “MappedByteArray” something that can work with the same original
>> ByteArray (or ExternalAddress, is the same), to affect the referenced data…
>> what do you think?
>>
>>
>> So yes, introducing a "referenced handle” (who applies offsets to same
>> memory address or byte array) makes everything work… and looks like the
>> correct way to handle this.
>> Do you want I commit this changes to FFI package or I keep them in my own
>> stuff?
>>
>> Esteban
>>
>>
> Yes!
>
> years ago when I ported Smallapack to Squeak, that just was the missing
> sugar that would have enabled implementing all using Smalltalk memory...
>
> Instead of that, I had to copy data to external heap in order to perform
> certain operations.
> For example, for extracting the imaginary part of a complex matrix
> interlaced representation { re[0] , im[0] , re[1] , im[1], ... } one trick
> is to cast a DoubleComplexMatrix * into a DoubleMatrix *, and start the
> copy at doubleMatrixPointer+1, with increment=2…
>
>
> yes, but you are talking of something more complex than what I did, I
> think (what I did just works for nested structures)… what you want is a
> functionality I also believe I will need to add anyway :)
> so I will keep playing with this and will commit when ready.
>
> cheers!
> Esteban
>
>
If we have possibility to pass a composed object (ByteArray + byte_offset)
to FFI and have it transformed in a (char *) ByteArray_start_of_data +
byte_offset, then it solves all the problems.
That's the only feature I need, the complex part in my example (casting
etc...) is completely handled in image code.




> Nicolas
>
>>
>> cheers!
>> Esteban
>>
>>
>>
>> Esteban
>>
>>
>> Cheers,
>> Max
>>
>>
>> cheers,
>> Esteban
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20150821/7ef1656f/attachment-0001.htm