[Vm-dev] VM Maker: VMMaker.oscog-eem.1866.mcz

[commits at source.squeak.org]

Eliot Miranda uploaded a new version of VMMaker to project VM Maker:
http://source.squeak.org/VMMaker/VMMaker.oscog-eem.1866.mcz

==================== Summary ====================

Name: VMMaker.oscog-eem.1866
Author: eem
Time: 25 May 2016, 5:58:11.196163 pm
UUID: 7755faa2-afdc-4d3d-9530-47599e2eaf14
Ancestors: VMMaker.oscog-eem.1865

Fixes to ephemeron scavenging.

Both fireEphemeronsInRememberedSet and fireEphemeronsOnEphemeronList must check for an ephemeron being fireable.  An as-yet-firable ephemeron misgt get added to the ephemeron list and later its key may get tenured, rendering the ephemeron unfirable in that cycle.

Fix marking of obj stack contents (markAndTraceObjStack:andContents:).  Again tenuring could cause contents to be forwarded and hewnce a read barrier is needed.

Fix asserts that check for ephemerons so that the change in an ephemeron's format once it gets added to the mournQueue doesn't break the asserts; isMaybeFiredEphemeron[Format]: answer true for ephemerons and ephemerons in the mournQueue.

Make pushLiteralVariable: store unfollowed literals back into the literal frame.  Do so in a function off to the side, not inlined into the interpreter loop.

Slang:
Fix computation of StackInterpreter's requiredMethodNames:; it must use objectMemoryClass, not assume NewObjectMemory.  And SpurMemoryManager's api includes api methods in the scavenger.  Henc3e printRememberedTable is now generated.

With these changes Guille's test case still crashes but gets much further :
| e |
Smalltalk supportsQueueingFinalization: true.
e := (1 to: 200000) collect: [ :i | Ephemeron key: (ObjectFinalizer receiver: 'test', 'asd' selector: #logCr) value: Object new ].
Smalltalk garbageCollect!

=============== Diff against VMMaker.oscog-eem.1865 ===============

Item was changed:
  ----- Method: SpurGenerationScavenger>>fireEphemeronsInRememberedSet (in category 'weakness and ephemerality') -----
  fireEphemeronsInRememberedSet
  	"There are ephemerons to be fired in the remembered set.
  	 Fire them and scavenge their keys.  Leave it to scavengeLoop
  	 to remove any scavenged ephemerons that no longer have
  	 new referents."
  	| i |
  	self assert: self noUnfiredEphemeronsAtEndOfRememberedSet.
  
  	i := 0.
  	[i < numRememberedEphemerons] whileTrue:
+ 		[ | ephemeron key |
- 		[ | ephemeron |
  		 ephemeron := rememberedSet at: i.
  		 self assert: (manager isEphemeron: ephemeron).
+ 		 key := manager keyOfEphemeron: ephemeron.
+ 		 (self isScavengeSurvivor: key) ifFalse:
+ 			[coInterpreter fireEphemeron: ephemeron.
+ 			 manager
+ 				storePointerUnchecked: 0
+ 				ofObject: ephemeron
+ 				withValue: (self copyAndForward: key)].
+ 		 "Fired ephemerons should have had their format changed."
+ 		 self deny: ((self isScavengeSurvivor: key) and: [manager isEphemeron: ephemeron]).
- 		 self assert: (self isScavengeSurvivor: (manager keyOfEphemeron: ephemeron)) not.
- 		 coInterpreter fireEphemeron: ephemeron.
- 		 manager
- 			storePointerUnchecked: 0
- 			ofObject: ephemeron
- 			withValue: (self copyAndForward: (manager keyOfEphemeron: ephemeron)).
  		 (self scavengeReferentsOf: ephemeron)
  			ifTrue: "keep in set"
  				[i := i + 1]
  			ifFalse:
  				[manager setIsRememberedOf: ephemeron to: false.
  				"remove from set by overwriting with next-to-be scanned"
  				 numRememberedEphemerons := numRememberedEphemerons - 1.
  				 previousRememberedSetSize := previousRememberedSetSize - 1.
  				 rememberedSetSize := rememberedSetSize - 1.
  				 "First overwrite with last firable ephemeron (could be a noop if this is the last one).
  				  Then overwrite last firable entry with next unscanned rememberedSet entry (could also be a noop).
  				  Then overwrite next unscanned entry with last unscanned rememberedSet entry (could also be a noop)."
  				 rememberedSet
  					at: i
  						put: (rememberedSet at: numRememberedEphemerons);
  					at: numRememberedEphemerons
  						put: (rememberedSet at: previousRememberedSetSize);
  					at: previousRememberedSetSize
  						put: (rememberedSet at: rememberedSetSize)]].
  
  	"no more firable ephemerons in this cycle.
  	 scavengeRememberedSetStartingAt: may find new ones."
  	numRememberedEphemerons := 0!

Item was changed:
  ----- Method: SpurGenerationScavenger>>fireEphemeronsOnEphemeronList (in category 'weakness and ephemerality') -----
  fireEphemeronsOnEphemeronList
  	"There are ephemerons to be fired in the remembered set.
  	 Fire them and scavenge their keys.  Be careful since copyAndForward:
  	 can remember ephemerons (ephemerons pointing to ephemerons)."
+ 	| ephemeron ephemeronCorpse key oldList oldCorpse | "old ones for debugging"
- 	| ephemeron ephemeronCorpse |
  	ephemeronList ifNil:
  		[^self].
+ 	oldCorpse := nil.
  	ephemeronCorpse := self firstCorpse: ephemeronList.
  	"Reset the list head so that new ephemerons will get added
  	 to a new list, not concatenated on the one we are scanning."
+ 	oldList := ephemeronList.
  	ephemeronList := nil.
+ 	[self assert: (manager isYoung: ephemeronCorpse).
+ 	 ephemeronCorpse notNil] whileTrue:
- 	[ephemeronCorpse notNil] whileTrue:
  		[self assert: (manager isForwarded: ephemeronCorpse).
  		 ephemeron := manager followForwarded: ephemeronCorpse.
+ 		 key := manager keyOfMaybeFiredEphemeron: ephemeron.
+ 		 (self isScavengeSurvivor: key) ifFalse:
+ 			[coInterpreter fireEphemeron: ephemeron.
+ 			 manager
+ 				storePointerUnchecked: 0
+ 				ofObject: ephemeron
+ 				withValue: (self copyAndForward: key)].
+ 		 "Fired ephemerons should have had their format changed."
+ 		 self deny: ((self isScavengeSurvivor: key) and: [manager isEphemeron: ephemeron]).
- 		 self assert: (self isScavengeSurvivor: (manager keyOfEphemeron: ephemeron)) not.
- 		 coInterpreter fireEphemeron: ephemeron.
- 		 manager
- 			storePointerUnchecked: 0
- 			ofObject: ephemeron
- 			withValue: (self copyAndForward: (manager keyOfEphemeron: ephemeron)).
  		 self cCoerceSimple: (self scavengeReferentsOf: ephemeron) to: #void.
+ 		 oldCorpse := ephemeronCorpse.
  		 ephemeronCorpse := self nextCorpseOrNil: ephemeronCorpse]!

Item was added:
+ ----- Method: SpurMemoryManager class>>requiredMethodNames: (in category 'translation') -----
+ requiredMethodNames: options
+ 	"return the list of method names that should be retained for export or other support reasons"
+ 	^(self exportAPISelectors: options),
+ 	  (SpurGenerationScavenger exportAPISelectors: options)!

Item was added:
+ ----- Method: SpurMemoryManager>>isMaybeFiredEphemeron: (in category 'object testing') -----
+ isMaybeFiredEphemeron: objOop
+ 	^self isMaybeFiredEphemeronFormat: (self formatOf: objOop)!

Item was added:
+ ----- Method: SpurMemoryManager>>isMaybeFiredEphemeronFormat: (in category 'header formats') -----
+ isMaybeFiredEphemeronFormat: format
+ 	"Answer if an object's format could be that of an ephemeron.  When ephemerons are born
+ 	 their format is ephemeronFormat, but when they fire their format is changed to 3 (inst vars
+ 	 plus indexable fields) or 1 (objects with inst vars), we haven't decided which yet."
+ 	^format <= self lastPointerFormat and: [format odd]!

Item was added:
+ ----- Method: SpurMemoryManager>>isObjEphemeron: (in category 'object testing') -----
+ isObjEphemeron: objOop
+ 	^self isEphemeronFormat: (self formatOf: objOop)!

Item was changed:
  ----- Method: SpurMemoryManager>>keyOfEphemeron: (in category 'object access') -----
  keyOfEphemeron: objOop
  	"Answer the object the ephemeron guards.  This is its first element."
+ 	self assert: ((self isNonImmediate: objOop) and: [self isObjEphemeron: objOop]).
- 	self assert: ((self isNonImmediate: objOop) and: [self isEphemeron: objOop]).
  	^self fetchPointer: 0 ofObject: objOop!

Item was added:
+ ----- Method: SpurMemoryManager>>keyOfMaybeFiredEphemeron: (in category 'object access') -----
+ keyOfMaybeFiredEphemeron: objOop
+ 	"Answer the object the ephemeron guards.  This is its first element."
+ 	self assert: ((self isNonImmediate: objOop) and: [self isMaybeFiredEphemeron: objOop]).
+ 	^self fetchPointer: 0 ofObject: objOop!

Item was changed:
  ----- Method: SpurMemoryManager>>markAndTraceObjStack:andContents: (in category 'obj stacks') -----
  markAndTraceObjStack: stackOrNil andContents: markAndTraceContents
  	"An obj stack is a stack of objects stored in a hidden root slot, such
  	 as the markStack or the ephemeronQueue.  It is a linked list of
  	 segments, with the hot end at the head of the list.  It is a word object.
  	 The stack pointer is in ObjStackTopx and 0 means empty."
  	<inline: false>
  	| index field |
  	stackOrNil = nilObj ifTrue:
  		[^self].
  	self setIsMarkedOf: stackOrNil to: true.
  	self assert: (self numSlotsOfAny: stackOrNil) = ObjStackPageSlots.
  	field := self fetchPointer: ObjStackNextx ofObject: stackOrNil.
  	field ~= 0 ifTrue:
  		[self markAndTraceObjStack: field andContents: markAndTraceContents].
  	field := stackOrNil.
  	[field := self fetchPointer: ObjStackFreex ofObject: field.
  	 field ~= 0] whileTrue:
  		[self setIsMarkedOf: field to: true].
  	markAndTraceContents ifFalse:
  		[^self].
  	"There are four fixed slots in an obj stack, and a Topx of 0 indicates empty, so
  	  if there were 6 slots in an oop stack, full would be 2, and the last 0-rel index is 5."
  	index := (self fetchPointer: ObjStackTopx ofObject: stackOrNil) + ObjStackNextx.
  	[index >= ObjStackFixedSlots] whileTrue:
+ 		[field := self followObjField: index ofObject: stackOrNil.
- 		[field := self fetchPointer: index ofObject: stackOrNil.
  		 (self isImmediate: field) ifFalse:
  			[self markAndTrace: field].
  		 index := index - 1]!

Item was changed:
  ----- Method: StackInterpreter class>>preambleCCode (in category 'translation') -----
  preambleCCode
  	^	
  '/* Disable Intel compiler inlining of warning which is used for breakpoints */
  #pragma auto_inline off
  sqInt warnpid, erroronwarn;
  void
  warning(char *s) { /* Print an error message but don''t necessarily exit. */
  	if (erroronwarn) error(s);
  	if (warnpid)
  		printf("\n%s pid %ld\n", s, (long)warnpid);
  	else
  		printf("\n%s\n", s);
  }
  void
  warningat(char *s, int l) { /* ditto with line number. */
  	/* use alloca to call warning so one does not have to remember to set two breakpoints... */
  	char *sl = alloca(strlen(s) + 16);
  	sprintf(sl, "%s %d", s, l);
  	warning(sl);
  }
  #pragma auto_inline on
  
  void
  invalidCompactClassError(char *s) { /* Print a (compact) class index error message and exit. */
  #if SPURVM
  	printf("\nClass %s does not have the required class index\n", s);
  #else
  	printf("\nClass %s does not have the required compact class index\n", s);
  #endif
  	exit(-1);
  }
  
  /*
   * Define sigsetjmp and siglongjmp to be the most minimal setjmp/longjmp available on the platform.
   */
  #if WIN32
  # define sigsetjmp(jb,ssmf) setjmp(jb)
  # define siglongjmp(jb,v) longjmp(jb,v)
  #else
  # define sigsetjmp(jb,ssmf) _setjmp(jb)
  # define siglongjmp(jb,v) _longjmp(jb,v)
  #endif
+ 
+ #define odd(v) ((int)(v)&1)
+ #define even(v) (!!odd(v))
  '!

Item was changed:
  ----- Method: StackInterpreter class>>requiredMethodNames: (in category 'translation') -----
  requiredMethodNames: options
  	"return the list of method names that should be retained for export or other support reasons"
  	| requiredList |
  	requiredList := self exportAPISelectors: options.
+ 	requiredList addAll: (self objectMemoryClass requiredMethodNames: options).
- 	requiredList addAll: (NewObjectMemory requiredMethodNames: options).
  	"A number of methods required by VM support code, jitter, specific platforms etc"
  	requiredList addAll: #(
  		assertValidExecutionPointe:r:s:
  		characterForAscii:
  		findClassOfMethod:forReceiver: findSelectorOfMethod:
  			forceInterruptCheck forceInterruptCheckFromHeartbeat fullDisplayUpdate
  		getCurrentBytecode getFullScreenFlag getInterruptKeycode getInterruptPending
  			getSavedWindowSize getThisSessionID
  		interpret
  		loadInitialContext
  		primitiveFail primitiveFailFor: primitiveFlushExternalPrimitives printAllStacks printCallStack printContext:
  			printExternalHeadFrame printFramesInPage: printFrame: printHeadFrame printMemory printOop:
  				printStackPages printStackPageList printStackPagesInUse printStackPageListInUse
  		readableFormat: readImageFromFile:HeapSize:StartingAt:
  		setFullScreenFlag: setInterruptKeycode: setInterruptPending: setInterruptCheckChain:
  			setSavedWindowSize: success:
  		validInstructionPointer:inMethod:framePointer:).
  
  	"Nice to actually have all the primitives available"
  	requiredList addAll: (self primitiveTable select: [:each| each isSymbol]).
  
  	"InterpreterProxy is the internal analogue of sqVirtualMachine.c, so make sure to keep all those"
  	InterpreterProxy organization categories do:
  		[:cat |
  		((cat ~= 'initialize') and: [cat ~= 'private']) ifTrue:
  			[requiredList addAll: (InterpreterProxy organization listAtCategoryNamed: cat)]].
  
  	^requiredList!

Item was added:
+ ----- Method: StackInterpreter>>literal:ofMethod:put: (in category 'compiled methods') -----
+ literal: offset ofMethod: methodPointer put: oop
+ 	<option: #SpurMemoryManager>
+ 	<inline: true>
+ 	objectMemory storePointer: offset + LiteralStart ofObject: methodPointer withValue: oop
+ !

Item was changed:
  ----- Method: StackInterpreter>>pushLiteralVariable: (in category 'stack bytecodes') -----
  pushLiteralVariable: literalIndex
  	objectMemory hasSpurMemoryManagerAPI
  		ifTrue:
  			[| litVar |
  			 "push/store/popLiteralVariable all fetch a literal, and either read or write the literal's value field.
  			  The fetch of the literal needs an explicit check (otherwise we would have to scan all literals in
  			  all methods in the stack zone, and the entire method on return, and global variables are relatively
  			  rare; in my work image 8.7% of literals are globals)."
  			 litVar := self literal: literalIndex.
  			 (objectMemory isForwarded: litVar) ifTrue:
+ 				[litVar := self unfollow: litVar atIndex: literalIndex].
- 				[litVar := objectMemory followForwarded: litVar].
  			 self internalPush:
  				(objectMemory fetchPointer: ValueIndex ofObject: litVar)]
  		ifFalse:
  			[self internalPush:
  				(objectMemory fetchPointer: ValueIndex ofObject: (self literal: literalIndex))]!

Item was added:
+ ----- Method: StackInterpreter>>unfollow:atIndex: (in category 'compiled methods') -----
+ unfollow: litVar atIndex: literalIndex
+ 	<option: #SpurMemoryManager>
+ 	<inline: #never> "So rare it mustn't bulk up the common path"
+ 	| followed |
+ 	followed := objectMemory followForwarded: litVar.
+ 	self literal: literalIndex ofMethod: method put: followed.
+ 	^followed!