[VM][BUG][FIX] pushRemappableOop: stack overflow

David T. Lewis lewis at mail.msen.com
Sat Mar 16 22:32:27 UTC 2002


Bug: A primitive which calls #pushRemappableOop: can overflow the
remap buffer, which is declared with size 26. This crashes the VM
with a stack dump.

The pushRemappableOopFix-dtl change set modifies #pushRemappableOop:
and #popRemappableOop to fail gracefully and to permit a primitive to
check for failures.

It also increases RemapBufferSize in ObjectMemory from 25 to 250 in
ObjectMemory class>>initialize. This change is not necessary, but will
decrease the likelihood of failures at a small cost in memory.

The CrashMePlugin-dtl change set provides an entertaining demonstration
of the stack overflow problem, and can be used to verify the fix.

Caveats:
- A plugin which overflows the remap buffer and fails to check the
error flag will produce erroneous results and may still crash the VM.
It might be better to simply force an error stack dump when the remap
buffer is overflowed.
- ObjectMemory>>pushRemappableOop: calls a method in one of its subclasses
(Interpreter>>primitiveFail). This does not hurt anything, but it is not
a very nice thing to do.  (PhiHo: does this break your MOP partitioning
of ObjectMemory and Interpreter?)

Dave

-------------- next part --------------
'From Squeak3.1alpha of 27 September 2001 [latest update: #4347] on 16 March 2002 at 5:23:46 pm'!
"Change Set:		pushRemappableOopFix-dtl
Date:			16 March 2002
Author:			David T. Lewis

Bug: A primitive which calls #pushRemappableOop: can overflow the
remap buffer, which is declared with size 26. This crashes the VM
with a stack dump.

This change set modifies #pushRemappableOop: and #popRemappableOop
to fail gracefully and to permit a primitive to check for failures.

Changes:

Modified the ObjectMemory>>pushRemappableOop: and
ObjectMemory>>popRemappableOop methods to fail gracefully when there
is an overflow or underflow of the remap buffer. On buffer overflow,
the primitive error flag is set, so that diligent primitive writers may
add a check such as:
	self failed ifTrue: [^ interpreterProxy primitiveFail]

Also increased RemapBufferSize in ObjectMemory from 25 to 250 in
ObjectMemory class>>initialize. This change is not necessary, but will
decrease the likelihood of failu!
 res at a small cost in memory.

Caveats:
- A plugin which overflows the remap buffer and fails to check the
error flag will produce erroneous results and may still crash the VM.
It might be better to simply force an error stack dump when the remap
buffer is overflowed.
- ObjectMemory>>pushRemappableOop: calls a method in one of its subclasses
(Interpreter>>primitiveFail). This does not hurt anything, but it is not
a very nice thing to do.
"!


!ObjectMemory methodsFor: 'interpreter access' stamp: 'dtl 3/16/2002 16:41'!
popRemappableOop
	"Pop and return the possibly remapped object from the remap buffer. Answer nilObject if this request would underflow the buffer."

	| oop |
	(remapBufferCount == 0)
		ifTrue: [oop _ self nilObject]
		ifFalse: [oop _ remapBuffer at: remapBufferCount].
	(remapBufferCount > 0) ifTrue: [remapBufferCount _ remapBufferCount - 1].
	^ oop! !

!ObjectMemory methodsFor: 'interpreter access' stamp: 'dtl 3/16/2002 16:41'!
pushRemappableOop: oop
	"Record!
  the given object in a the remap buffer. Objects in this buffer are re
mapped when a compaction occurs. This facility is used by the interpreter to ensure that objects in temporary variables are properly remapped. Fail if the request would overflow the buffer."

	(remapBufferCount < RemapBufferSize)
		ifTrue: [remapBuffer at: (remapBufferCount _ remapBufferCount + 1) put: oop]
		ifFalse: [self primitiveFail]
! !


!ObjectMemory class methodsFor: 'initialization' stamp: 'dtl 3/16/2002 16:44'!
initialize
	"ObjectMemory initialize"

	"Translation flags (booleans that control code generation via conditional translation):"
	DoAssertionChecks _ false.  "generate assertion checks"
	DoBalanceChecks _ false. "generate stack balance checks"

	self initializeSpecialObjectIndices.
	self initializeObjectHeaderConstants.

	SmallContextSize _ 92.  "16 indexable fields"
	LargeContextSize _ 252.  "56 indexable fileds.  Max with single header word."
	LargeContextBit _ 16r40000.  "This bit set in method headers if large context is needed."
	CtxtTempFrameStart _ 6!
 .  "Copy of TempFrameStart in Interp"
	NilContext _ 1.  "the oop for the integer 0; used to mark the end of context lists"

	RemapBufferSize _ 250.
	RootTableSize _ 2500.  "number of root table entries (4 bytes/entry)"

	"tracer actions"
	StartField _ 1.
	StartObj _ 2.
	Upward _ 3.
	Done _ 4.! !

ObjectMemory initialize!
-------------- next part --------------
'From Squeak3.1alpha of 27 September 2001 [latest update: #4347] on 16 March 2002 at 5:18:06 pm'!
"Change Set:		CrashMePlugin-dtl
Date:			16 March 2002
Author:			David T. Lewis

Demonstrate overflow of the special objects stack. After applying
the pushRemappableOopFix changes, the VM will no longer crash
when you evaluate 'CrashMePlugin kaBoom'.
"!

InterpreterPlugin subclass: #CrashMePlugin
	instanceVariableNames: ''
	classVariableNames: ''
	poolDictionaries: ''
	category: 'VMConstruction-Plugins'!

!CrashMePlugin commentStamp: 'dtl 3/16/2002 11:20' prior: 0!
Provides a primitive with exercises the special objects stack. Evaluate CrashMePlugin class>>kaBoom to force a stack overflow. You may wish to save your image first.

After fixing the VM to provide error checking in ObjectMemory>>pushRemappableOop:, this example should fail gracefully rather than crashing the VM.!


!CrashMePlugin methodsFor: 'primitives' stamp: 'dtl 3/16/2002 11:51'!
primitiveLoadStackToDepth

	| dept!
 h idx |
	self export: true.
	depth _ interpreterProxy stackIntegerValue: 0.
	self cCode: 'fprintf(stderr,"depth is %d\n", depth); fflush(stderr)'.
	idx _ 0.
	[idx < depth] whileTrue:
		[self cCode: 'fprintf(stderr,"pushing an object, idx is %d\n", idx); fflush(stderr)'.
		interpreterProxy pushRemappableOop: interpreterProxy trueObject.
		idx _ idx + 1].
	idx _ 0.
	[idx < depth] whileTrue:
		[self cCode: 'fprintf(stderr,"popping an object, idx is %d\n", idx); fflush(stderr)'.
		interpreterProxy popRemappableOop.
		idx _ idx + 1].
	interpreterProxy pop: 2; pushInteger: depth
! !


!CrashMePlugin class methodsFor: 'crashing the VM' stamp: 'dtl 3/16/2002 11:23'!
kaBoom

	"CrashMePlugin kaBoom"

	^ self primLoadStackToDepth: 100
! !

!CrashMePlugin class methodsFor: 'primitive access' stamp: 'dtl 3/16/2002 16:40'!
primLoadStackToDepth: anInteger
	"Push anInteger objects onto the special objects stack, and pop them off.
	If anInteger is larger than the stack size, bad things may !
 happen. Answer
	anInteger for success, or a string message if the prim
itive fails."

	"self primLoadStackToDepth: 25"	"No problem"
	"self primLoadStackToDepth: 26"	"Stack overflow, may crash the VM"
	"self primLoadStackToDepth: 1000"	"Stack overflow, may crash the VM"

	<primitive: 'primitiveLoadStackToDepth' module: 'CrashMePlugin'>

	^ 'The primitive failed gracefully. This is good.'
! !

CrashMePlugin class removeSelector: #declareCVarsIn:!


More information about the Squeak-dev mailing list