Marcel Taeumel uploaded a new version of FFI-Kernel to project FFI:
http://source.squeak.org/FFI/FFI-Kernel-mt.188.mcz
==================== Summary ====================
Name: FFI-Kernel-mt.188
Author: mt
Time: 6 August 2021, 6:30:54.160096 pm
UUID: 26659a26-1cfc-2540-8e26-30392b0f99b1
Ancestors: FFI-Kernel-mt.187
Adds #atAllPut:, which is common to raw-bits arrays and other sequenceable collections.
=============== Diff against FFI-Kernel-mt.187 ===============
Item was added:
+ ----- Method: ExternalData>>atAllPut: (in category 'accessing') -----
+ atAllPut: anObject
+
+ self sizeCheck.
+ 1 to: self size do:
+ [:index | self at: index put: anObject].!
Eliot Miranda uploaded a new version of SystemReporter to project The Trunk:
http://source.squeak.org/trunk/SystemReporter-eem.51.mcz
==================== Summary ====================
Name: SystemReporter-eem.51
Author: eem
Time: 3 August 2021, 5:27:47.183363 pm
UUID: d0bece24-3b56-4758-a9ee-cd422e598dc5
Ancestors: SystemReporter-codefrau.50
Include VM patrameter #76 'minimum unused bytes of headroom on all stack pages' in the system report now that VMs are starting to use the FastCPrimitive convention for running certain primitives on the Smalltalk stack.
=============== Diff against SystemReporter-codefrau.50 ===============
Item was changed:
----- Method: SystemReporter>>reportVMParameters: (in category 'reporting') -----
reportVMParameters: aStream
| vmParameters isStack isCog isSpur |
self header: 'Virtual Machine Parameters' on: aStream.
vmParameters := Smalltalk vm getVMParameters.
isStack := (vmParameters at: 42 ifAbsent: [0]) ~= 0. "42 = number of stack pages available"
isCog := isStack and: [(vmParameters at: 46) ~= 0]. "46 is machine code zone size"
isSpur := isStack and: [(vmParameters at: 41) anyMask: 2r10000]. "41 is imageFormatVersion for the VM; bit 16 is the Spur bit"
(isSpur
ifFalse:
[#( 1 'size of old space'
2 'size of young+old space'
3 'size of memory'
4 'allocationCount'
5 'allocations between GCs'
6 'survivor count tenuring threshold')]
ifTrue:
[#( 1 'size of old space'
2 'used bytes in new space (used eden + used past space)'
3 'size of heap')]),
#( 7 'full GCs since startup'
8 'total milliseconds in full GCs since startup'),
(isSpur
ifFalse: [#( 9 'incremental GCs since startup'
10 'total milliseconds in incremental GCs since startup'
11 'tenures of surving objects since startup'),
{12 to: 19. 'specific to the translating VM'}]
ifTrue: [#( 9 'scavenging GCs since startup'
10 'total milliseconds in scavenging GCs since startup'
11 'tenures of surving objects since startup'
12 'event trace mask (for debugging input events)'
13 'VM ticker start microseconds (Croquet/QwaqVM)'
14 'VM ticker count (Croquet/QwaqVM)'
15 'VM ticker call count (Croquet/QwaqVM)'
16 'total microseconds in idle since startup'
17 'proportion of code zone available for use (Sista VMs only; read-write)'
18 'total milliseconds in full GC compaction since startup (a portion of parameter 8)'
19 'scavenge threshold; the effective size of eden')]),
#( 20 'utc microseconds at startup (if non-zero)'
21 'root/remembered table size (occupancy)'
22 'root/remembered table overflows since startup'
23 'bytes of extra memory to reserve for VM buffers, plugins, etc.'
24 'free memory threshold above which object memory will be shrunk'
25 'memory headroom when growing object memory'),
(isStack
ifFalse:
[#( 26 'interruptChecksEveryNms - force an ioProcessEvents every N milliseconds, in case the image is not calling getNextEvent often')]
ifTrue:
[#( 26 'heartbeat period (ms; see #58)')]),
(isSpur
ifFalse:
[#( 27 'number of times mark loop iterated for current IGC/FGC includes ALL marking'
28 'number of times sweep loop iterated for current IGC/FGC'
29 'number of times make forward loop iterated for current IGC/FGC'
30 'number of times compact move loop iterated for current IGC/FGC')]
ifTrue: [#()]),
#( 31 'number of grow memory requests'
32 'number of shrink memory requests'),
(isSpur
ifFalse:
[#( 33 'number of root table entries used for current IGC/FGC'
34 'number of allocations done before current IGC/FGC'
35 'number of survivor objects after current IGC/FGC'
36 'millisecond clock when current IGC/FGC completed'
37 'number of marked objects for Roots of the world, not including Root Table entries for current IGC/FGC'
38 'milliseconds taken by current IGC'
39 'Number of finalization signals for Weak Objects pending when current IGC/FGC completed')]
ifTrue:
[#( 33 'number of root table entries at last scavenge'
35 'number of survivor objects at last scavenge (if non-zero)'
36 'millisecond clock when current scavenge completed'
38 'milliseconds taken by current scavenge'
39 'Number of finalization signals for Weak Objects pending when current SGC/FGC completed')]),
#( 40 'VM word size - 4 or 8'),
(isStack
ifTrue:
[#(
41 'imageFormatVersion for the VM'
42 'number of stack pages available'
43 'desired number of stack pages (stored in image file header, max 65535)'
44 'size of eden, in bytes'
45 'desired size of eden, in bytes (stored in image file header)'
46 'machine code zone size, in bytes (0 in Stack VM)'
47 'desired machine code zone size (0 => default 1Mb to 2Mb depending on processor)'),
{ 48. 'Persistent image header flags\ bit 0: implies Process has threadId as its 4th inst var\ bit 1: if set, methods that are interpreted will have the flag bit set in their header\ bit 2: if set, implies preempting a process does not put it to the back of its run queue\ bit 3: if set, implies the GUI should run on the first thread and event queues should not be accessed from other threads\ bit 4: if set, implies the new finalization scheme where WeakArrays are queued\ bit 5: if set, implies wheel events will be delivered as such and not mapped to arrow key events\ bit 6: if set, implies arithmetic primitives will fail if given arguments of different types (float vs int)\ bit 7: if set, causes times delivered from file primitives to be in UTC rather than local time.' withCRs },
#( 49 'max size the image promises to grow the external semaphore table to'),
(isSpur
ifFalse:
[{ 50 to: 51. 'reserved for VM parameters that persist in the image (such as size of eden above)'.
52 to: 56. 'specific to Spur' }]
ifTrue:
[{ 50 to: 51. 'reserved for VM parameters that persist in the image (such as size of eden above)' },
#( 52 'root/remembered table capacity'
53 'number of old space segments'
54 'total free old space'
55 'ratio of growth and image size at or above which a GC will be performed post scavenge')]),
#( 56 'number of process switches since startup'
57 'number of ioProcessEvents calls since startup'
58 'number of forceInterruptCheck calls since startup'
59 'number of check event calls since startup'
60 'number of stack page overflows since startup'
61 'number of stack page divorces since startup'
62 'compiled code compactions since startup'),
(isCog
ifFalse:
[#()]
ifTrue:
[#( 63 'total milliseconds in compiled code compactions since startup'
64 'the number of methods that currently have jitted machine-code')]),
{ 65. 'Cog feature flags\ bit 0: set if the VM supports MULTIPLE_BYTECODE_SETS.\ bit 1: set if the VM supports read-only objects (IMMUTABILITY).\ bit 2: set if the VM has an ITIMER_HEARTBEAT\ bit 3: set if the VM supports cross-platform BIT_IDENTICAL_FLOATING_POINT arithmetic' withCRs.
66. 'the byte size of a stack page'.},
(isSpur
ifFalse:
[{ 67 to: 69. 'reserved for more Cog-related info' }]
ifTrue:
[#( 67 'the maximum allowed size of old space (if zero there is no limit)'
68 'the average number of live stack pages when scanned by scavenge/gc/become'
69 'the maximum number of live stack pages when scanned by scavenge/gc/become')]),
#( 70 'the vmProxyMajorVersion (the interpreterProxy VM_MAJOR_VERSION)'
71 'the vmProxyMinorVersion (the interpreterProxy VM_MINOR_VERSION)'),
(isSpur
ifFalse: [#()]
ifTrue:
[#( 72 'milliseconds spent marking since startup'
73 'reserved for more Spur-related info'
74 'reserved for more Spur-related info'
+ 75 'do mixed arithmetic; if false binary arithmetic primitives will fail unless receiver and argument are of the same type')]),
+ (isCog
+ ifFalse: [#()]
+ ifTrue: [#(76 'minimum unused bytes of headroom on all stack pages')])]
- 75 'do mixed arithmetic; if false binary arithmetic primitives will fail unless receiver and argument are of the same type')])]
ifFalse:
[#()])
pairsDo: [:idx :desc | | value values |
+ (idx isInteger ifTrue: [idx] ifFalse: [idx last]) <= vmParameters size ifTrue:
+ [aStream nextPut: $#.
+ idx isInteger
+ ifTrue:
+ [value := vmParameters at: idx.
+ aStream
+ print: idx; tab: (idx < 10 ifTrue: [2] ifFalse: [1]);
+ nextPutAll: ((value isInteger and: [idx ~= 41])
+ ifTrue: [(desc includesSubstring: 'bit 0:')
+ ifTrue: [value printStringBase: 2 nDigits: value highBit]
+ ifFalse: [value asStringWithCommas]]
+ ifFalse: [value printString])]
+ ifFalse:
+ [value := vmParameters at: idx first.
+ aStream print: idx first; next: 2 put: $.; print: idx last; tab.
+ values := idx collect: [:i| vmParameters at: i].
+ values asSet size = 1
+ ifTrue: [aStream print: value]
+ ifFalse: [values do: [:v| aStream print: v] separatedBy: [aStream nextPutAll: ', ']]].
+ aStream tab; nextPutAll: desc; cr]]!
- aStream nextPut: $#.
- idx isInteger
- ifTrue:
- [value := vmParameters at: idx.
- aStream
- print: idx; tab: (idx < 10 ifTrue: [2] ifFalse: [1]);
- nextPutAll: ((value isInteger and: [idx ~= 41])
- ifTrue: [(desc includesSubstring: 'bit 0:')
- ifTrue: [value printStringBase: 2 nDigits: value highBit]
- ifFalse: [value asStringWithCommas]]
- ifFalse: [value printString])]
- ifFalse:
- [value := vmParameters at: idx first.
- aStream print: idx first; next: 2 put: $.; print: idx last; tab.
- values := idx collect: [:i| vmParameters at: i].
- values asSet size = 1
- ifTrue: [aStream print: value]
- ifFalse: [values do: [:v| aStream print: v] separatedBy: [aStream nextPutAll: ', ']]].
- aStream tab; nextPutAll: desc; cr]!
Eliot Miranda uploaded a new version of FFI-Kernel to project FFI:
http://source.squeak.org/FFI/FFI-Kernel.terf-eem.187.mcz
==================== Summary ====================
Name: FFI-Kernel.terf-eem.187
Author: eem
Time: 2 August 2021, 11:15:30.36735 am
UUID: 54737d19-448d-4377-859b-be958a51dec6
Ancestors: FFI-Kernel-eem.186
Performance:
- cache an ExternalStructure class's externalType in a class inst var.
- directly allocate an ExternalStructure[Type] using self byteSize rather than deferring to the externalType.
Minor: avoid variable shadowing warnings in compileStructureSpec:withAccessors:, compileTypeAliasSpec:withAccessors:, & maybeCompileAccessor:withSelector:.
=============== Diff against FFI-Kernel-eem.186 ===============
Item was changed:
----- Method: ExternalArrayType>>contentType (in category 'external data') -----
contentType "^ <ExternalType>"
+ "We are an array of things. Our content type is encoded in the compiledSpec's headerWord. The super implementation of #typeName can figure that out."
+ self flag: #contentVsContainer. "mt: For n-dimensional containers, we might have to adapt this."
+ ^ contentType ifNil: [contentType := ExternalType typeNamed: super typeName ]!
- ^ contentType!
Item was changed:
ExternalObject subclass: #ExternalStructure
instanceVariableNames: ''
classVariableNames: ''
poolDictionaries: 'ExternalTypePool FFIConstants'
category: 'FFI-Kernel'!
ExternalStructure class
+ instanceVariableNames: 'compiledSpec byteAlignment externalType'!
- instanceVariableNames: 'compiledSpec byteAlignment'!
!ExternalStructure commentStamp: 'eem 6/26/2019 15:26' prior: 0!
An ExternalStructure is for representing external data that is
- either a structure composed of different fields (a struct of C language)
- or an alias for another type (like a typedef of C language)
It reserves enough bytes of data for representing all the fields.
The data is stored into the handle instance variable which can be of two different types:
- ExternalAddress
If the handle is an external address then the object described does not reside in the Smalltalk object memory.
- ByteArray
If the handle is a byte array then the object described resides in Smalltalk memory.
Instance Variables (class side)
byteAlignment: <Integer>
compiledSpec: <WordArray>
byteAlignment
- the required alignment for the structure
compiledSpec
- the bit-field definiton of the structure in the ExternalType encoding understood by the VM's FFI call marshalling machinery.
A specific structure is defined by subclassing ExternalStructure and specifying its #fields via a class side method.
For example if we define a subclass:
ExternalStructure subclass: #StructExample
instanceVariableNames: ''
classVariableNames: ''
poolDictionaries: ''
category: 'garbage'.
Then declare the fields like this:
StructExample class compile: 'fields ^#( (name ''char*'') (color ''ulong'') )' classified: 'garbage'.
It means that this type is composed of two different fields:
- a string (accessed thru the field #name)
- and an unsigned 32bit integer (accessed thru the field #color).
It represents the following C type:
struct StructExample {char *name; uint32_t color; };
The accessors for those fields can be generated automatically like this:
StructExample defineFields.
As can be verified in a Browser:
StructExample browse.
We see that name and color fields are stored sequentially in different zones of data.
The total size of the structure can be verified with:
StructExample byteSize = (Smalltalk wordSize + 4).
An ExternalStructure can also be used for defining an alias.
The fields definition must contain only 2 elements: an eventual accessor (or nil) and the type.
For example, We can define a machine dependent 'unsigned long' like this:
ExternalStructure subclass: #UnsignedLong
instanceVariableNames: ''
classVariableNames: ''
poolDictionaries: ''
category: 'garbage'.
Then set the fields like this:
UnsignedLong class compile: 'fields ^(Smalltalk wordSize=4 or: [Smalltalk platformName=''Win64''])
ifTrue: [#(nil ''ulong'')] ifFalse: [#(nil ''ulonglong'')]' classified: 'garbage'.
And verify the size on current platform:
UnsignedLong byteSize.
Then, the class names 'UnsignedLong' and 'StructExamples' acts as a type specification.
They can be used for composing other types, and for defining prototype of external functions:
LibraryExample>>initMyStruct: aStructExample name: name color: anInteger
<cdecl: void 'init_my_struct'( StructExample * char * UnsignedLong )>
self externalCallFailed
!
ExternalStructure class
+ instanceVariableNames: 'compiledSpec byteAlignment externalType'!
- instanceVariableNames: 'compiledSpec byteAlignment'!
Item was changed:
----- Method: ExternalStructure class>>allocate (in category 'instance creation') -----
allocate
+ ^ self fromHandle: (ByteArray new: self byteSize)!
-
- ^self externalType allocate!
Item was changed:
----- Method: ExternalStructure class>>compileStructureSpec:withAccessors: (in category 'field definition - support') -----
compileStructureSpec: specArray withAccessors: aSymbol
"Compile a type specification for the FFI calls.
Return the newly compiled spec.
Eventually generate the field accessors according to following rules:
- aSymbol = #always always generate the accessors
- aSymbol = #never never generate the accessors
- aSymbol = #generated only generate the auto-generated accessors
- aSymbol = #absent only generate the absent accessors"
| newByteAlignment byteOffset typeSpec newCompiledSpec |
byteOffset := 0.
newByteAlignment := self minStructureAlignment.
typeSpec := WriteStream on: (WordArray new: 10).
typeSpec nextPut: FFIFlagStructure.
+ specArray do:
+ [:spec | | fieldName fieldTypeName fieldType typeSize fieldAlignment |
- specArray do: [:spec |
- | fieldName fieldTypeName externalType typeSize fieldAlignment |
fieldName := spec first.
fieldTypeName := spec second.
+ fieldType := (ExternalType typeNamed: fieldTypeName) ifNil: [self errorTypeNotFound: spec second].
+ typeSize := fieldType byteSize.
+ fieldAlignment := (fieldType byteAlignment max: self minFieldAlignment) min: self maxFieldAlignment.
- externalType := (ExternalType typeNamed: fieldTypeName)
- ifNil: [self errorTypeNotFound: spec second].
- typeSize := externalType byteSize.
- fieldAlignment := (externalType byteAlignment
- max: self minFieldAlignment)
- min: self maxFieldAlignment.
byteOffset := byteOffset alignedTo: fieldAlignment.
newByteAlignment := newByteAlignment max: fieldAlignment.
+ spec size > 2 ifTrue: "extra size"
+ [spec third < typeSize
- spec size > 2 ifTrue: ["extra size"
- spec third < typeSize
ifTrue: [^ self error: 'Explicit type size is less than expected'].
+ typeSize := spec third].
+ (fieldName notNil and: [self shouldGenerate: fieldName policy: aSymbol]) ifTrue:
+ [self generateStructureFieldAccessorsFor: fieldName startingAt: byteOffset + 1 type: fieldType].
+ typeSpec nextPutAll: (fieldType embeddedSpecWithSize: typeSize).
+ byteOffset := byteOffset + typeSize].
- typeSize := spec third.
- ].
- (fieldName notNil and: [self shouldGenerate: fieldName policy: aSymbol]) ifTrue: [
- self generateStructureFieldAccessorsFor: fieldName startingAt: byteOffset + 1 type: externalType.
- ].
- typeSpec nextPutAll: (externalType embeddedSpecWithSize: typeSize).
- byteOffset := byteOffset + typeSize.
- ].
newByteAlignment := newByteAlignment min: self maxStructureAlignment.
byteOffset := byteOffset alignedTo: newByteAlignment.
newCompiledSpec := typeSpec contents.
newCompiledSpec at: 1 put: (byteOffset bitOr: FFIFlagStructure).
self
setCompiledSpec: newCompiledSpec
+ byteAlignment: newByteAlignment!
- byteAlignment: newByteAlignment.!
Item was changed:
----- Method: ExternalStructure class>>compileTypeAliasSpec:withAccessors: (in category 'field definition - support') -----
compileTypeAliasSpec: spec withAccessors: aSymbol
"Define all the fields in the receiver.
Return the newly compiled spec."
+ | fieldName fieldTypeName fieldType |
- | fieldName fieldTypeName externalType |
fieldName := spec first.
fieldTypeName := spec second.
+ fieldType := (ExternalType typeNamed: fieldTypeName) ifNil: [self errorTypeNotFound: spec second].
+ (fieldName notNil and: [self shouldGenerate: fieldName policy: aSymbol]) ifTrue:
+ [self generateTypeAliasAccessorsFor: fieldName type: fieldType].
+ fieldType isPointerType
+ ifTrue: "Special case. Typedef for a pointer type, e.g., typedef char* LPSTR in Win32 API. Mark it as both structure and pointer. Note that #isPointerType will only answer true for the in-image pointer type, not the non-pointer alias for the pointer."
+ [self
- externalType := (ExternalType typeNamed: fieldTypeName)
- ifNil: [self errorTypeNotFound: spec second].
- (fieldName notNil and:[self shouldGenerate: fieldName policy: aSymbol]) ifTrue:[
- self generateTypeAliasAccessorsFor: fieldName type: externalType].
- externalType isPointerType
- ifTrue: ["Special case. Typedef for a pointer type, e.g., typedef char* LPSTR in Win32 API. Mark it as both structure and pointer. Note that #isPointerType will only answer true for the in-image pointer type, not the non-pointer alias for the pointer."
- self
setCompiledSpec: (WordArray with: ExternalType pointerAliasSpec)
byteAlignment: ExternalType pointerAliasAlignment]
+ ifFalse: "Usual case. Typedef for another struct type or atomic type. Just re-use compiled spec and extras from the aliased type."
+ [self
- ifFalse: ["Usual case. Typedef for another struct type or atomic type. Just re-use compiled spec and extras from the aliased type."
- self
flag: #isTypeAlias;
+ setCompiledSpec: fieldType compiledSpec
+ byteAlignment: fieldType byteAlignment]!
- setCompiledSpec: externalType compiledSpec
- byteAlignment: externalType byteAlignment].!
Item was changed:
----- Method: ExternalStructure class>>externalType (in category 'external type') -----
externalType
"Return an external type describing the receiver as a structure"
+ ^externalType ifNil: [externalType := ExternalType structTypeNamed: self name]!
- ^ExternalType structTypeNamed: self name!
Item was changed:
----- Method: ExternalStructure class>>maybeCompileAccessor:withSelector: (in category 'field definition - support') -----
maybeCompileAccessor: aString withSelector: selector
"Only compile if category or source changed."
+ | theCategory ref |
+ theCategory := #'*autogenerated - accessing'.
- | category ref |
- category := #'*autogenerated - accessing'.
((ref := MethodReference class: self selector: selector) isValid
+ and: [ref sourceString = aString]) ifTrue:
+ [ref category ~= theCategory ifTrue:
+ [self organization classify: selector under: theCategory].
+ ^self].
+ self compile: aString classified: theCategory!
- and: [ref category = category]
- and: [ref sourceString = aString])
- ifTrue: [^ self].
- self compile: aString classified: category.!
Item was added:
+ ----- Method: ExternalStructureType>>allocate (in category 'external data') -----
+ allocate
+ ^ referentClass fromHandle: (ByteArray new: self byteSize)!