On Jun 9, 2020, at 5:10 AM, Marcel Taeumel marcel.taeumel@hpi.de wrote:
Hi Nicolas, hi Eliot.
If you want a type that can hold a pointer on both 32 and 64bits arch, then it's doable with a FFI type alias
You can just make an alias to a void*. It will have the right size. The only drawback would be that you might get an issue with argument coercion.
#(nil 'void*')
Your "type alias struct" will wrap around ExternalData<void*>. Why bother with ulong/ulonglong. Only if you want to work with integers in the image, you might want to use ulong or ulonglong depending on the #wordSize.
I think I will add 'word' as a type name in ExternalType's class-side "type constants" in Squeak FFI. Then you can use it in FFI call defs and struct-field defs:
ExternalType class >> #word "Use this atomic-type alias if the external library works with opaque handles that are different on 32bit and 64bit. On a platform change, all FFI calls will be re-compiled and this method is called again by the parser."
^ FFIPlatformDescription current wordSize = 4 ifTrue: [self unsignedLong] ifFalse: [self unsignedLongLong]
... maybe parts in the FFI plugin are faster with ulong/ulonglong compared to void* compared to type alias to void*... I don't know.
It might be better to use a C name such as intptr_t or uintptr_t.
Best, Marcel
Am 30.05.2020 18:00:33 schrieb Marcel Taeumel marcel.taeumel@hpi.de:
So, if the atomic types and structure types are initialized, they make use of ExternalType class >> #pointerSpec to store the size of the pointer in the #headerWord, which is the first element of an external type's 'compiledSpec'. For atomic types, this happens when the FFI package is loaded.
For structure types, this happens when a an external structure is mentioned for the first time in an FFI call method or in during compilation of another external structure's fields. See ExternalType class >> #newTypeNamed:force:. Squeak's Smalltalk parser or the compile logic in ExternalStructure invokes that type creation respectively..
Currently, if a platform change at system startup is detected, all atomic types and structure types are updated --- which implies a call to #pointerSpec in ExternalType class >> #initializeStructureTypes.
Now, what could be the situation where a struct-pointer type had the wrong pointer size? During development, your platform does not magically change the #wordSize. When loading code, methods in the external struct might be wrong (bc. Monticello does not trigger that field re-definition) BUT any external struct-pointer type that is already in the image must have the right size. And for new external struct-pointer types, it should work as explained above.
So, I think we should make the following changes:
- Remove ExternalType >> #asPointerType: (! note the colon !)
- Remove ExternalType >> #pointerSize:
- Change ExternalType >> #pointerSize to just read the compiledSpec
- Update all sends to #asPointerType: to #asPointerType and assume that the current pointer-type instances have the right size, as described above.
Best, Marcel
Am 30.05.2020 15:59:31 schrieb Marcel Taeumel marcel.taeumel@hpi.de:
But if it is obsolete now, nuke it.
I cannot tell. That information is stored in the external type's compiledSpec to be used from within the FFI plugin I suppose.
Best, Marcel
Am 29.05.2020 00:15:25 schrieb Eliot Miranda eliot.miranda@gmail.com:
On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier <> nicolas.cellier.aka.nice@gmail.com> wrote:
Hi Marcel, no idea... Look at timestamp: eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set FFI-Kernel-EstebanLorenzano.45 · So presumably Eliot added that.
Perhaps. I don't remember. But if it is obsolete now, nuke it.
Le jeu. 28 mai 2020 à 17:00, marcel.taeumel a écrit :
Hi Nicolas,
what is the role of ExternalStructure class >> #pointerSize then?
Best, Marcel
Nicolas Cellier wrote > Hi all, > beware of FFI 'long' and 'ulong'. > They are currently meaning (32 bits) int and unsigned int! > This dates back from the 80s when there were still odd OS with segmented > memory and 16-bits int. > long was thought as a safe int32_t at that time... and still means just > that in current FFI. > > I'd like to replace them with 'int' and 'uint', but we need to ensure > backward compatibility etc... > > FFI has ExternalType signedLongLong or just 'longlong' and ExternalType > unsignedLongLong or 'ulonglong' which are 64 bits on all architectures. > See implementors of #initializeAtomicTypes #initializeFFIConstants > > If you want a type that can hold a pointer on both 32 and 64bits arch, > then > it's doable with a FFI type alias > For example I have things like this in HDF5 module > Notice the single #(fieldName fieldType) which defines an alias, while > ordinary FFI struct gets an array of #(fieldName fieldType) > (and replace Size_t with Intptr_t if the purpose is hodling a pointer, > otherwise the ayatollahs of C standard will lash you) > > ExternalStructure subclass: #'Size_t' > instanceVariableNames: '' > classVariableNames: '' > poolDictionaries: '' > category: 'HDF5-External-atomic' > > Size_t class >>fields > "Size_t defineFields" > ^Smalltalk wordSize = 4 > ifTrue: [#( value 'ulong')] > ifFalse: [#( value 'ulonglong')] > > However the "fields" must be re-generated when platform changes... > I tried to automate that sometimes ago, see: > > ExternalStructure class>>install > "Resuming the image on another architecture may require a > re-compilation of structure layout." > | newPlatform | > newPlatform := Smalltalk platformName. > PreviousPlatform = newPlatform > ifFalse: > [self recompileStructures. > PreviousPlatform := newPlatform] > > It works if you switch from Win64 to MacOS64 for example... > However, this mechanism is not used yet at time of code loading, so > importing code generated on a different platform won't automatically > update > the structures (or alias) - the FFI package has no mechanism for that > (maybe we could add a hook in MC?). > > Le mer. 11 mars 2020 à 23:58, Bert Freudenberg <
> bert@
> > a > écrit : > >> I'd suggest to get OpenGL working outside of Croquet first: >> >> http://www.squeaksource.com/CroquetGL.html >> >> Step 1: Verify this works in 32 bits. (assuming you are doing this on >> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one) >> Step 2: Make it work in 64 bits. >> >> The second step requires that you understand how FFI works, and how it >> handles e.g. pointers and integer sizes. >> I am assuming we do have a working 64 bit FFI, at least for x86_64 >> machines. >> >> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a 32 >> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit >> system, >> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I >> don't know how many bits 'ulong' has in our 64 bit FFI, but that >> declaration may have to change. Etc. pp. >> >> If you have questions about FFI then those are best directed at the >> vm-dev >> list since it is dealing with VM-level interfaces. CC'ing, please follow >> up >> there. >> >> - Bert - >> >> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <> >>
> squeak-dev@.squeakfoundation
>> wrote: >> >>> Okey dokey, >>> >>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName after >>> >>> openGLLibraryName >>> ^Smalltalk osVersion = 'linux' >>> ifTrue: ['libGL.so.1'] >>> ifFalse: ['GL'] >>> >>> I removed it in my install and got past that error. >>> >>> Working exclusively with Croquet(Master)... >>> >>> >>> >>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode: >>> >>> glMatrixMode: mode >>> "This method was automatically generated." >>> "void glMatrixMode(GLenum mode);" >>> > >>> ^self externalCallFailed >>> >>> The > > fails >>> >>> How to think about this? >>> >>> Is Croquet behind OpenGL latest? >>> Would teaching myself OpenGL programming be of use to the Croquet >>> project? >>> >>> cheers, >>> >>> tty >>> >>> >>> >>
-- Sent from: http://forum.world.st/Squeak-VM-f104410.html
-- _,,,^..^,,,_ best, Eliot
On Thu, May 28, 2020 at 11:33 AM Nicolas Cellier nicolas.cellier.aka.nice@gmail.com wrote:
Hi Marcel, no idea... Look at timestamp: eem 2/16/2016 12:42 · accessing · 2 implementors · only in change set FFI-Kernel-EstebanLorenzano.45 · So presumably Eliot added that.
Perhaps. I don't remember. But if it is obsolete now, nuke it.
Le jeu. 28 mai 2020 à 17:00, marcel.taeumel Marcel.Taeumel@hpi.de a écrit :
Hi Nicolas,
what is the role of ExternalStructure class >> #pointerSize then?
Best,
Marcel
Nicolas Cellier wrote
> Hi all,
> beware of FFI 'long' and 'ulong'.
> They are currently meaning (32 bits) int and unsigned int!
> This dates back from the 80s when there were still odd OS with segmented
> memory and 16-bits int.
> long was thought as a safe int32_t at that time... and still means just
> that in current FFI.
>
> I'd like to replace them with 'int' and 'uint', but we need to ensure
> backward compatibility etc...
>
> FFI has ExternalType signedLongLong or just 'longlong' and ExternalType
> unsignedLongLong or 'ulonglong' which are 64 bits on all architectures.
> See implementors of #initializeAtomicTypes #initializeFFIConstants
>
> If you want a type that can hold a pointer on both 32 and 64bits arch,
> then
> it's doable with a FFI type alias
> For example I have things like this in HDF5 module
> Notice the single #(fieldName fieldType) which defines an alias, while
> ordinary FFI struct gets an array of #(fieldName fieldType)
> (and replace Size_t with Intptr_t if the purpose is hodling a pointer,
> otherwise the ayatollahs of C standard will lash you)
>
> ExternalStructure subclass: #'Size_t'
> instanceVariableNames: ''
> classVariableNames: ''
> poolDictionaries: ''
> category: 'HDF5-External-atomic'
>
> Size_t class >>fields
> "Size_t defineFields"
> ^Smalltalk wordSize = 4
> ifTrue: [#( value 'ulong')]
> ifFalse: [#( value 'ulonglong')]
>
> However the "fields" must be re-generated when platform changes...
> I tried to automate that sometimes ago, see:
>
> ExternalStructure class>>install
> "Resuming the image on another architecture may require a
> re-compilation of structure layout."
> | newPlatform |
> newPlatform := Smalltalk platformName.
> PreviousPlatform = newPlatform
> ifFalse:
> [self recompileStructures.
> PreviousPlatform := newPlatform]
>
> It works if you switch from Win64 to MacOS64 for example...
> However, this mechanism is not used yet at time of code loading, so
> importing code generated on a different platform won't automatically
> update
> the structures (or alias) - the FFI package has no mechanism for that
> (maybe we could add a hook in MC?).
>
> Le mer. 11 mars 2020 à 23:58, Bert Freudenberg <
> bert@
> > a
> écrit :
>
>> I'd suggest to get OpenGL working outside of Croquet first:
>>
>> http://www.squeaksource.com/CroquetGL.html
>>
>> Step 1: Verify this works in 32 bits. (assuming you are doing this on
>> Linux, you can run 32 bit Squeak side-by-side with the 64 bit one)
>> Step 2: Make it work in 64 bits.
>>
>> The second step requires that you understand how FFI works, and how it
>> handles e.g. pointers and integer sizes.
>> I am assuming we do have a working 64 bit FFI, at least for x86_64
>> machines.
>>
>> E.g. the OGLUnix>>glExtGetProcAddress: method returns a pointer. On a 32
>> bit system, that fits into a 'ulong' which is 32 bits. On a 64 bit
>> system,
>> a pointer is 64 bits wide so it would not fit into a 32 bit word. Now I
>> don't know how many bits 'ulong' has in our 64 bit FFI, but that
>> declaration may have to change. Etc. pp.
>>
>> If you have questions about FFI then those are best directed at the
>> vm-dev
>> list since it is dealing with VM-level interfaces. CC'ing, please follow
>> up
>> there.
>>
>> - Bert -
>>
>> On Wed, Mar 11, 2020 at 2:54 PM gettimothy via Squeak-dev <
>>
> squeak-dev@.squeakfoundation
>> wrote:
>>
>>> Okey dokey,
>>>
>>> Poking along, there is a stray glyph in OGLUnix openGLLibraryName after
>>>
>>> openGLLibraryName
>>> ^Smalltalk osVersion = 'linux'
>>> ifTrue: ['libGL.so.1']
>>> ifFalse: ['GL']
>>>
>>> I removed it in my install and got past that error.
>>>
>>> Working exclusively with Croquet(Master)...
>>>
>>>
>>>
>>> My next error is in OGLUnixX11LE(OpenGL)>>glMatrixMode:
>>>
>>> glMatrixMode: mode
>>> "This method was automatically generated."
>>> "void glMatrixMode(GLenum mode);"
>>>
> <apicall: void 'glMatrixMode' (ulong) module: '#openGLLibraryName'>
>>> ^self externalCallFailed
>>>
>>> The
> fails
>>>
>>> How to think about this?
>>>
>>> Is Croquet behind OpenGL latest?
>>> Would teaching myself OpenGL programming be of use to the Croquet
>>> project?
>>>
>>> cheers,
>>>
>>> tty
>>>
>>>
>>>
>>
--
Sent from: http://forum.world.st/Squeak-VM-f104410.html
-- _,,,^..^,,,_ best, Eliot