[Vm-dev] nested structures with FFI?

[Esteban Lorenzano]

> On 21 Aug 2015, at 04:02, Esteban Lorenzano <estebanlm at gmail.com> wrote:
> 
>> 
>> On 20 Aug 2015, at 22:47, Esteban Lorenzano <estebanlm at gmail.com <mailto:estebanlm at gmail.com>> wrote:
>> 
>>> 
>>> On 20 Aug 2015, at 21:48, Max Leske <maxleske at gmail.com <mailto:maxleske at gmail.com>> wrote:
>>> 
>>> 
>>>> On 20 Aug 2015, at 21:28, vm-dev-request at lists.squeakfoundation.org <mailto:vm-dev-request at lists.squeakfoundation.org> wrote:
>>>> 
>>>> Message: 1
>>>> Date: Thu, 20 Aug 2015 17:51:43 +0200
>>>> From: Esteban Lorenzano <estebanlm at gmail.com <mailto:estebanlm at gmail.com>>
>>>> Subject: [Vm-dev] nested structures with FFI?
>>>> To: Squeak Virtual Machine Development Discussion
>>>> 	<vm-dev at lists.squeakfoundation.org <mailto:vm-dev at lists.squeakfoundation.org>>
>>>> Message-ID: <4F31ED4B-6246-43D6-B85B-7215D8570D28 at gmail.com <mailto:4F31ED4B-6246-43D6-B85B-7215D8570D28 at gmail.com>>
>>>> Content-Type: text/plain; charset=utf-8
>>>> 
>>>> Hi,
>>>> 
>>>> I’m doing some tests with nested structures (to handle the conversion of NativeBoost-FFI to regular FFI) and I’m having some problems when accessing fields in the nested structure. 
>>>> For example, this tests: 
>>>> 
>>>> FFITestNestingStructure class>>#fieldsDesc 
>>>> 	^ #(
>>>> 		byte one;
>>>> 		FFITestStructure nested;
>>>> 		)
>>>> 
>>>> FFITestStructure class>>#fieldsDesc
>>>> 	^ #(
>>>> 		byte byte;
>>>> 		short short;
>>>> 		long long;
>>>> 		float float;
>>>> 		double double;
>>>> 		int64 int64;
>>>> 		)
>>>> 
>>>> (definition is NB-compatible, so do not take the differences with regular FFI into account)
>>>> 
>>>> testNestedStructure
>>>> 	| s1 |
>>>> 	
>>>> 	s1 := FFITestNestingStructure new.
>>>> 	s1 nested byte: 42.	
>>>> 	self assert: s1 nested byte = 42
>>>> 
>>>> testExternallyAllocatedNestedStructure
>>>> 	| s1 |
>>>> 	
>>>> 	s1 := FFITestNestingStructure externalNew.
>>>> 	self assert: s1 getHandle class = ExternalAddress.
>>>> 	s1 nested byte: 42.	
>>>> 	self assert: s1 nested byte = 42
>>>> 
>>>> 
>>>> fails always because 
>>>> 
>>>> s1 nested byte = 0. 
>>>> 
>>>> (which is obviously bad). 
>>>> 
>>>> I remember Eliot saying that structures with FFI was not being optimal, and I wonder (before start digging more in deep) if this could have something to do?
>>> 
>>> I use nested structs in NB for the libgit2 bindings and they work as advertised. There can be problems if some structure is packed (i.e. uses non-4-byte alignment for some field) but that shouldn’t apply in your test case.
>>> 
>>> Not sure if this helps you in any way but I thought I’d let you know :)
>> 
>> I do not think so :)
>> I’m quite sure this is not going to be a problem when using the ThreadedFFI backend instead ASMJIT… now my problem is with this backend, for the moment :) 
> 
> I’m just realised that this behaviour is correct according current ExternalStructure implementation, because:
> 
> nested
> 	"This method was automatically generated"
> 	^FFITestStructure fromHandle: (handle structAt: 2 length: 27)
> 
> and 
> 
> ByteArray>>#structAt: byteOffset length: length
> 	"Return a structure of the given length starting at the indicated byte offset."
> 	| value |
> 	value := ByteArray new: length.
> 	1 to: length do:[:i|
> 		value unsignedByteAt: i put: (self unsignedByteAt: byteOffset+i-1)].
> 	^value
> 
> So it is giving me always a copy… 
> So we need a better solution (a non copy way to access)? I’m thinking on passing a “MappedByteArray” something that can work with the same original ByteArray (or ExternalAddress, is the same), to affect the referenced data… what do you think?

So yes, introducing a "referenced handle” (who applies offsets to same memory address or byte array) makes everything work… and looks like the correct way to handle this. 
Do you want I commit this changes to FFI package or I keep them in my own stuff?

Esteban

> 
> cheers!
> Esteban
>  
> 
>> 
>> Esteban
>> 
>>> 
>>> Cheers,
>>> Max
>>> 
>>>> 
>>>> cheers, 
>>>> Esteban

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20150821/7092af52/attachment-0001.htm