Marcel Taeumel uploaded a new version of FFI-Kernel to project FFI:
http://source.squeak.org/FFI/FFI-Kernel-mt.219.mcz
==================== Summary ====================
Name: FFI-Kernel-mt.219
Author: mt
Time: 23 August 2021, 9:50:31.066089 am
UUID: 1b8fdd1b-0e40-9e4b-90ef-52be520fc1e9
Ancestors: FFI-Kernel-mt.218
Fixes init bug where atomic-type accessors (i.e., ExternalType class >> #int32_t) got lost after FFI-Kernel-mt.215.
=============== Diff against FFI-Kernel-mt.218 ===============
Item was added:
+ ----- Method: ExternalAtomicType class>>initializeAtomicTypeAccessors (in category 'class initialization') -----
+ initializeAtomicTypeAccessors
+ "Call this to (re-)initialize accessors for atomic types, even if there are no substantial changes to the types themselves."
+
+ self assert: [AtomicTypes notNil].
+ self assert: [AtomicTypes notEmpty].
+
+ self atomicTypesDo: [:type | type generateTypeAccessor].!
Item was changed:
----- Method: ExternalType class>>initializeAtomicTypes (in category 'class initialization') -----
initializeAtomicTypes
"Private. (Re-)initialize all atomic types. Preserves object identity of all involved types. If the amount of types changed, use #resetAllAtomicTypes instead. Needs atomic-type codes to be initialized already."
self assert: [AtomicTypeCodes notNil].
self assert: [AtomicTypeCodes notEmpty].
self basicInitializeAtomicTypes.
self invalidateAtomicTypes.
ExternalAtomicType initializeAtomicTypes.
+ ExternalAtomicType initializeAtomicTypeCompanions.
+ ExternalAtomicType initializeAtomicTypeAccessors.
- ExternalAtomicType initializeAtomicTypeCompanions.
self initializeAtomicSends.!
Item was added:
+ ----- Method: ExternalType class>>initializeStructureTypeAccessors (in category 'class initialization') -----
+ initializeStructureTypeAccessors
+ "Call this to (re-)initialize accessors for types that have a referentClass, which generates/updates the class-side #externalType method according to the #useTypePool preference."
+
+ self allTypesDo: [:type |
+ type referentClass ifNotNil: [type generateTypeAccessor]].!
Item was changed:
----- Method: ExternalType class>>initializeStructureTypes (in category 'class initialization') -----
initializeStructureTypes
"(Re-)initialize all structure types and all array types. Preserves object identity of all involved types. If the amount of types changed, use #resetAllStructureTypes instead."
self basicInitializeStructureTypes.
self invalidateStructureTypes.
+ self initializeStructureTypeAccessors.
+
"Trigger #noticeModificationOf: callback to actually initialize all structure types. Since we invalidated all types, we cannot use #defineAllChangedFields. We rely on the #noticeModificationOf: callback."
ExternalStructure defineAllFields.!
Item was changed:
(PackageInfo named: 'FFI-Kernel') postscript: '"Update for array classes."
+ ExternalType initialize..'!
- ExternalType initialize.'!
Eliot Miranda uploaded a new version of Traits to project The Trunk:
http://source.squeak.org/trunk/Traits-eem.313.mcz
==================== Summary ====================
Name: Traits-eem.313
Author: eem
Time: 22 August 2021, 7:57:20.929755 am
UUID: 11b5e1e0-abf9-4352-b50a-eba7698082c3
Ancestors: Traits-pre.312
Avoid materializing method properties in methods that don't have them when computing originalTraitMethod. Good for a -5% speedup in compilation times.
=============== Diff against Traits-pre.312 ===============
Item was changed:
----- Method: CompiledMethod>>originalTraitMethod (in category '*Traits-NanoKernel') -----
originalTraitMethod
+ "Answer the original trait method for the receiver, or nil if none."
+ | propertiesOrSelector |
+ ^(propertiesOrSelector := self penultimateLiteral) isMethodProperties ifTrue:
+ [propertiesOrSelector originalTraitMethod]!
- "Remember the original trait method for the receiver."
- ^self properties originalTraitMethod!
Marcel Taeumel uploaded a new version of FFI-Kernel to project FFI:
http://source.squeak.org/FFI/FFI-Kernel-mt.217.mcz
==================== Summary ====================
Name: FFI-Kernel-mt.217
Author: mt
Time: 19 August 2021, 1:19:47.069627 pm
UUID: b50f6f6d-fcf9-9e46-8d36-d8aea7f5b4c3
Ancestors: FFI-Kernel-mt.216
Manage array classes as instVar in external type instead of classVar to further speed up type-based allocation.
Having #useArrayClasses enabled, we are now very close to the direct appraoch, which is infavorable due to exposing implementation details:
[Float32Array new: 10] bench.
GOAL: '55,200,000 per second. 18.1 nanoseconds per run. 8.45662 % GC time.'
[ExternalType float allocate: 10] bench
NOW: '33,200,000 per second. 30.1 nanoseconds per run. 5.33893 % GC time.'
=============== Diff against FFI-Kernel-mt.216 ===============
Item was changed:
Object subclass: #ExternalType
+ instanceVariableNames: 'compiledSpec referentClass referencedType byteAlignment arrayClass'
+ classVariableNames: 'ArrayTypes AtomicTypeCodes AtomicTypes ExtraTypeChecks StructTypes UseArrayClasses UseTypePool'
- instanceVariableNames: 'compiledSpec referentClass referencedType byteAlignment'
- classVariableNames: 'ArrayClasses ArrayTypes AtomicTypeCodes AtomicTypes ExtraTypeChecks StructTypes UseArrayClasses UseTypePool'
poolDictionaries: 'ExternalTypePool FFIConstants'
category: 'FFI-Kernel'!
!ExternalType commentStamp: 'mt 6/5/2020 18:25' prior: 0!
An external type represents the type of external objects.
Instance variables:
compiledSpec <WordArray> Compiled specification of the external type
referentClass <Behavior | nil> Class type of argument required
referencedType <ExternalType> Associated (non)pointer type with the receiver
byteAlignment <Integer | nil> The desired alignment for a field of the external type within a structure. If nil it has yet to be computed.
Compiled Spec:
The compiled spec defines the type in terms which are understood by the VM. Each word is defined as:
bits 0...15 - byte size of the entity
bit 16 - structure flag (FFIFlagStructure)
This flag is set if the following words define a structure
bit 17 - pointer flag (FFIFlagPointer)
This flag is set if the entity represents a pointer to another object
bit 18 - atomic flag (FFIFlagAtomic)
This flag is set if the entity represents an atomic type.
If the flag is set the atomic type bits are valid.
bits 19...23 - unused
bits 24...27 - atomic type (FFITypeVoid ... FFITypeDoubleFloat)
bits 28...31 - unused
Note that all combinations of the flags FFIFlagPointer, FFIFlagAtomic, and FFIFlagStructure are invalid, EXCEPT from the following:
FFIFlagPointer + FFIFlagAtomic:
This defines a pointer to an atomic type (e.g., 'char*', 'int*').
The actual atomic type is represented in the atomic type bits.
FFIFlagPointer + FFIFlagStructure:
This defines a structure which is a typedef of a pointer type as in
typedef void* VoidPointer;
typedef Pixmap* PixmapPtr;
It requires a byte size of four or eight (e.g. a 32-bit or 64-bit pointer) to work correctly.
[Note: Other combinations may be allowed in the future]
!
Item was changed:
----- Method: ExternalType class>>initializeArrayClasses (in category 'class initialization') -----
initializeArrayClasses
"
ExternalType initializeArrayClasses.
"
- ArrayClasses ifNil: [
- ArrayClasses := IdentityDictionary new].
-
RawBitsArray allSubclasses collect: [:arrayClass |
+ (arrayClass includesSelector: #contentType) ifTrue: [
+ [arrayClass new contentType setArrayClass: arrayClass]
+ on: Error do: [ "Ignore." ]]].
- [ArrayClasses at: arrayClass new contentType ifAbsentPut: arrayClass]
- on: Error do: [ "Ignore." ]].
String allSubclasses collect: [:stringClass | | contentType |
[ contentType := stringClass new contentType.
+ contentType asUnsigned setArrayClass: stringClass.
+ contentType asSigned setArrayClass: stringClass ]
- ArrayClasses at: contentType asUnsigned ifAbsentPut: stringClass.
- ArrayClasses at: contentType asSigned ifAbsentPut: stringClass ]
on: Error do: [ "Ignore."]].!
Item was changed:
----- Method: ExternalType class>>resetAllTypes (in category 'housekeeping') -----
resetAllTypes
"DANGEROUS!! Only do this if you think that your image is in an inconsistent state. You must not actively use FFI (i.e., have instances of external structures or external data) while calling this."
ExternalTypePool nukeAll.
AtomicTypeCodes := nil.
AtomicTypes := nil.
StructTypes := nil.
ArrayTypes := nil.
- ArrayClasses := nil.
self initialize.
self recompileAllLibraryFunctions.!
Item was changed:
----- Method: ExternalType>>allocateArrayClass: (in category 'external data') -----
allocateArrayClass: numElements
"Allocate space for containing an array of numElements of this dataType. Try to use an array class. Answer 'nil' if there is no such class for the receiver."
+ ^ arrayClass ifNotNil: [arrayClass new: numElements]!
- ^ ArrayClasses
- at: self
- ifPresent: [:arrayClass | arrayClass new: numElements]
- ifAbsent: [nil]
- !
Item was added:
+ ----- Method: ExternalType>>arrayClass (in category 'accessing') -----
+ arrayClass
+
+ ^ arrayClass!
Item was added:
+ ----- Method: ExternalType>>setArrayClass: (in category 'private') -----
+ setArrayClass: class
+
+ arrayClass := class.!
Item was changed:
+ (PackageInfo named: 'FFI-Kernel') postscript: '"Update for array classes."
+ ExternalType initialize.'!
- (PackageInfo named: 'FFI-Kernel') postscript: '"Update for generated type accessors."
- ExternalType initialize..'!
Marcel Taeumel uploaded a new version of FFI-Kernel to project FFI:
http://source.squeak.org/FFI/FFI-Kernel-mt.216.mcz
==================== Summary ====================
Name: FFI-Kernel-mt.216
Author: mt
Time: 19 August 2021, 11:20:10.194627 am
UUID: b88ec868-fec0-8249-a555-64edf5533ffa
Ancestors: FFI-Kernel-mt.215
Fixes minor slip in previous commit. Sorry for the noise.
=============== Diff against FFI-Kernel-mt.215 ===============
Item was added:
+ ----- Method: ExternalPointerType>>generateTypeAccessor (in category 'external structure') -----
+ generateTypeAccessor
+ "Overwritten to only generate accessor for alias-to-pointer types. The check for a non-nil referentClass is not enough when somebody is enumerating #allTypes."
+
+ self isTypeAlias ifTrue: [super generateTypeAccessor].!
Marcel Taeumel uploaded a new version of FFI-Tests to project FFI:
http://source.squeak.org/FFI/FFI-Tests-mt.63.mcz
==================== Summary ====================
Name: FFI-Tests-mt.63
Author: mt
Time: 19 August 2021, 11:13:58.859627 am
UUID: cb1940fa-b6fa-de43-976a-7f20aa39b120
Ancestors: FFI-Tests-mt.62
Complements FFI-Kernel-mt.215.
=============== Diff against FFI-Tests-mt.62 ===============
Item was added:
+ ----- Method: FFIImageTests>>test06AllTypes (in category 'tests') -----
+ test06AllTypes
+ "Check uniqueness of all known types."
+
+ | set |
+ set := Set new.
+ ExternalType allTypesDo: [:type |
+ self assert: (set ifAbsentAdd: type)].!
Item was added:
+ ----- Method: FFIImageTests>>test07AllTypeNames (in category 'tests') -----
+ test07AllTypeNames
+ "Check uniqueness of all known type names."
+
+ | set |
+ set := Set new.
+ ExternalType allTypesDo: [:type |
+ self assert: (set ifAbsentAdd: type typeName)].!
Marcel Taeumel uploaded a new version of FFI-Kernel to project FFI:
http://source.squeak.org/FFI/FFI-Kernel-mt.215.mcz
==================== Summary ====================
Name: FFI-Kernel-mt.215
Author: mt
Time: 19 August 2021, 11:12:47.424627 am
UUID: ca73c747-91ab-a749-8c8d-9ac0db1ca9bf
Ancestors: FFI-Kernel-mt.214
Generate type accessors for atomic types in ExternalType class. No need to hard-code them again. Names are already defined in #initializeAtomicTypeCodes and #initializeAtomicTypes.
Use type pool to generate quick-return accessors for atomic types in ExternalType and other struct-types in the type's class-side #externalType method. (Note that the use of the type pool can be disabled via #useTypePool preference.)
Adds enumerator to quickly access all known types.
=============== Diff against FFI-Kernel-mt.214 ===============
Item was changed:
----- Method: ExternalAtomicType class>>newTypeForAtomicNamed: (in category 'instance creation') -----
newTypeForAtomicNamed: atomicTypeName
| type pointerType |
type := ExternalAtomicType basicNew.
pointerType := ExternalPointerType basicNew.
type setReferencedType: pointerType.
pointerType setReferencedType: type.
type setAtomicTypeName: atomicTypeName.
AtomicTypes at: atomicTypeName put: type.
+ type generateTypeAccessor.
^ type!
Item was added:
+ ----- Method: ExternalAtomicType>>generateTypeAccessor (in category 'external structure') -----
+ generateTypeAccessor
+ "Overwritten to generate accessors for atomic types directly in ExternalType class. Note that alias-to-atomic types have their referentClass; see super implementation."
+
+ | theClass selector category expression source |
+ referentClass ifNotNil: [^ super generateTypeAccessor].
+
+ theClass := ExternalType.
+ selector := self typeName.
+ category := '*autogenerated - type constants'.
+ expression := ExternalType useTypePool
+ ifTrue: [ExternalTypePool assuredPoolVarNameFor: self]
+ ifFalse: ['AtomicTypes at: ''{1}''' format: {self typeName}].
+
+ source := '{1}\ {2}\ <generated>\ ^ {3}' withCRs
+ format: {
+ selector.
+ '"This method was automatically generated. See {1}>>{2}"'
+ format: {thisContext methodClass. thisContext selector}.
+ expression }.
+
+ theClass class
+ compileSilently: source
+ classified: category.!
Item was changed:
----- Method: ExternalStructureType class>>newTypeForStructureClass: (in category 'instance creation') -----
newTypeForStructureClass: anExternalStructureClass
| type pointerType referentClass |
referentClass := anExternalStructureClass.
self
assert: [referentClass includesBehavior: ExternalStructure]
description: 'Wrong base class for structure'.
type := self newTypeForUnknownNamed: referentClass name.
pointerType := type asPointerType.
referentClass compiledSpec
ifNil: [ "First time. The referent class' fields are probably just compiled for the first time."
type setReferentClass: referentClass.
pointerType setReferentClass: referentClass]
ifNotNil: [
type newReferentClass: referentClass.
pointerType newReferentClass: referentClass].
+ type becomeKnownTypeSafely.
+ type generateTypeAccessor.
+ ^ type!
- ^ type becomeKnownTypeSafely!
Item was changed:
Object subclass: #ExternalType
instanceVariableNames: 'compiledSpec referentClass referencedType byteAlignment'
classVariableNames: 'ArrayClasses ArrayTypes AtomicTypeCodes AtomicTypes ExtraTypeChecks StructTypes UseArrayClasses UseTypePool'
+ poolDictionaries: 'ExternalTypePool FFIConstants'
- poolDictionaries: 'FFIConstants'
category: 'FFI-Kernel'!
!ExternalType commentStamp: 'mt 6/5/2020 18:25' prior: 0!
An external type represents the type of external objects.
Instance variables:
compiledSpec <WordArray> Compiled specification of the external type
referentClass <Behavior | nil> Class type of argument required
referencedType <ExternalType> Associated (non)pointer type with the receiver
byteAlignment <Integer | nil> The desired alignment for a field of the external type within a structure. If nil it has yet to be computed.
Compiled Spec:
The compiled spec defines the type in terms which are understood by the VM. Each word is defined as:
bits 0...15 - byte size of the entity
bit 16 - structure flag (FFIFlagStructure)
This flag is set if the following words define a structure
bit 17 - pointer flag (FFIFlagPointer)
This flag is set if the entity represents a pointer to another object
bit 18 - atomic flag (FFIFlagAtomic)
This flag is set if the entity represents an atomic type.
If the flag is set the atomic type bits are valid.
bits 19...23 - unused
bits 24...27 - atomic type (FFITypeVoid ... FFITypeDoubleFloat)
bits 28...31 - unused
Note that all combinations of the flags FFIFlagPointer, FFIFlagAtomic, and FFIFlagStructure are invalid, EXCEPT from the following:
FFIFlagPointer + FFIFlagAtomic:
This defines a pointer to an atomic type (e.g., 'char*', 'int*').
The actual atomic type is represented in the atomic type bits.
FFIFlagPointer + FFIFlagStructure:
This defines a structure which is a typedef of a pointer type as in
typedef void* VoidPointer;
typedef Pixmap* PixmapPtr;
It requires a byte size of four or eight (e.g. a 32-bit or 64-bit pointer) to work correctly.
[Note: Other combinations may be allowed in the future]
!
Item was added:
+ ----- Method: ExternalType class>>allTypeNames (in category 'instance list') -----
+ allTypeNames
+ "Answers the names of the currently known types."
+
+ ^ self allTypes collect: [:each | each typeName]!
Item was added:
+ ----- Method: ExternalType class>>allTypes (in category 'instance list') -----
+ allTypes
+ "Answers the currently known types."
+
+ ^ Array streamContents: [:stream |
+ self allTypesDo: [:type | stream nextPut: type]]!
Item was added:
+ ----- Method: ExternalType class>>allTypesDo: (in category 'instance list') -----
+ allTypesDo: block
+
+ self allAtomicTypesDo: block.
+ self allStructTypesDo: block.
+ self allPointerTypesDo: block.
+ self allArrayTypesDo: block.!
Item was removed:
- ----- Method: ExternalType class>>bool (in category 'type constants') -----
- bool
- ^AtomicTypes at: 'bool'!
Item was removed:
- ----- Method: ExternalType class>>char16_t (in category 'type constants - stdint.h') -----
- char16_t
-
- ^ AtomicTypes at: 'char16_t'!
Item was removed:
- ----- Method: ExternalType class>>char32_t (in category 'type constants - stdint.h') -----
- char32_t
-
- ^ AtomicTypes at: 'char32_t'!
Item was removed:
- ----- Method: ExternalType class>>char8_t (in category 'type constants - stdint.h') -----
- char8_t
-
- ^ AtomicTypes at: 'char8_t'!
Item was removed:
- ----- Method: ExternalType class>>double (in category 'type constants') -----
- double
- ^AtomicTypes at: 'double'!
Item was removed:
- ----- Method: ExternalType class>>float (in category 'type constants') -----
- float
- ^AtomicTypes at: 'float'!
Item was removed:
- ----- Method: ExternalType class>>int16_t (in category 'type constants - stdint.h') -----
- int16_t
-
- ^ AtomicTypes at: 'int16_t'!
Item was removed:
- ----- Method: ExternalType class>>int32_t (in category 'type constants - stdint.h') -----
- int32_t
-
- ^ AtomicTypes at: 'int32_t'!
Item was removed:
- ----- Method: ExternalType class>>int64_t (in category 'type constants - stdint.h') -----
- int64_t
-
- ^ AtomicTypes at: 'int64_t'!
Item was removed:
- ----- Method: ExternalType class>>int8_t (in category 'type constants - stdint.h') -----
- int8_t
-
- ^ AtomicTypes at: 'int8_t'!
Item was changed:
----- Method: ExternalType class>>noticeModificationOf: (in category 'housekeeping') -----
noticeModificationOf: aClass
"A subclass of ExternalStructure has been redefined."
aClass withAllSubclassesDo: [:cls | | typeName type |
typeName := cls name.
(type := StructTypes at: typeName ifAbsent: [])
+ ifNil: ["Give unused types a chance to establish a hard reference via type accessor."
+ type := self structTypeNamed: typeName]
ifNotNil: [
type newReferentClass: cls.
type asPointerType newReferentClass: cls.
type newTypeAlias].
ArrayTypes at: typeName ifPresent: [:sizes |
sizes do: [:arrayType | arrayType ifNotNil: [
arrayType newContentType: type]]].
"Alias-to-array types, which are stored in StructTypes, will not update via #newContentType:. We scan StructTypes for #isArrayType to find such aliases to then call #newContentType:."
self flag: #performance.
StructTypes do: [:each |
(each notNil and: [each isArrayType and: [each contentType == type]])
ifTrue: [each newContentType: type]]].!
Item was removed:
- ----- Method: ExternalType class>>uchar16_t (in category 'type constants - stdint.h') -----
- uchar16_t
-
- ^ AtomicTypes at: 'uchar16_t'!
Item was removed:
- ----- Method: ExternalType class>>uchar32_t (in category 'type constants - stdint.h') -----
- uchar32_t
-
- ^ AtomicTypes at: 'uchar32_t'!
Item was removed:
- ----- Method: ExternalType class>>uchar8_t (in category 'type constants - stdint.h') -----
- uchar8_t
-
- ^ AtomicTypes at: 'uchar8_t'!
Item was removed:
- ----- Method: ExternalType class>>uint16_t (in category 'type constants - stdint.h') -----
- uint16_t
-
- ^ AtomicTypes at: 'uint16_t'!
Item was removed:
- ----- Method: ExternalType class>>uint32_t (in category 'type constants - stdint.h') -----
- uint32_t
-
- ^ AtomicTypes at: 'uint32_t'!
Item was removed:
- ----- Method: ExternalType class>>uint64_t (in category 'type constants - stdint.h') -----
- uint64_t
-
- ^ AtomicTypes at: 'uint64_t'!
Item was removed:
- ----- Method: ExternalType class>>uint8_t (in category 'type constants - stdint.h') -----
- uint8_t
-
- ^ AtomicTypes at: 'uint8_t'!
Item was changed:
----- Method: ExternalType class>>useTypePool (in category 'preferences') -----
useTypePool
+ <preference: 'Use type pool in generated accessors'
- <preference: 'Use type pool in structure fields'
categoryList: #('FFI Kernel')
+ description: 'When true, fill a pool of external types to be used in type accessors and struct-field accessors, which makes type access much faster. See ExternalTypePool.'
- description: 'When true, fill a pool of external types to be used in struct-field accessors. See ExternalTypePool.'
type: #Boolean>
^UseTypePool ifNil: [true]!
Item was changed:
----- Method: ExternalType class>>useTypePool: (in category 'preferences') -----
useTypePool: aBoolean
UseTypePool = aBoolean ifTrue: [^ self].
-
UseTypePool := aBoolean.
+ Cursor wait showWhile: [ExternalTypePool update].!
-
- Cursor wait showWhile: [
- "Either fill or clean out the type pool."
- ExternalTypePool reset.
- ExternalStructure defineAllFields.
- ExternalTypePool cleanUp].!
Item was removed:
- ----- Method: ExternalType class>>void (in category 'type constants') -----
- void
- ^AtomicTypes at: 'void'!
Item was added:
+ ----- Method: ExternalType>>generateTypeAccessor (in category 'external structure') -----
+ generateTypeAccessor
+ "Generate a fast accessor for the receiver, usually implemented on the class-side of the referentClass."
+
+ | poolVarName theClass selector source category |
+ referentClass ifNil: [^ self "See ExternalAtomicType >> #generateTypeAccessor"].
+
+ theClass := referentClass.
+ selector := #externalType.
+
+ self
+ assert: [theClass ~~ ExternalStructure]
+ description: 'You must not overwrite the generic type accessor'.
+
+ ExternalType useTypePool ifFalse: [
+ SystemChangeNotifier uniqueInstance doSilently: [theClass class removeSelector: selector].
+ ^ self "See ExternalStructure class >> #externalType"].
+
+ poolVarName := ExternalTypePool assuredPoolVarNameFor: self.
+
+ source := '{1}\ {2}\ <generated>\ ^ {3}' withCRs
+ format: {
+ selector.
+ '"This method was automatically generated. See {1}>>{2}"'
+ format: {thisContext methodClass. thisContext selector}.
+ poolVarName }.
+
+ category := '*autogenerated - external type'.
+ theClass class compileSilently: source classified: category.!
Item was changed:
+ (PackageInfo named: 'FFI-Kernel') postscript: '"Update for generated type accessors."
+ ExternalType initialize..'!
- (PackageInfo named: 'FFI-Kernel') postscript: '"Update for new 16-bit and 32-bit character types. Do not use #resetAllTypes to keep existing FFI context functional such as allocated executable pages in FFICallback."
- ExternalType initialize.'!