Hi, recently, the source file size limit of 16M was raised to 32M, with the side-effect of only allowing 2 source files from then on. This somehow conflicts with my plans for a Squeak source code management system (Collage) which has separate source files for every application. To keep that path open while still supporting larger source files, I've written these changes which concentrate all translation between sourcePointers and source file indexes/positions in one single place.
I've chosen to replace the global variable SourceFiles, which is currently an Array of two FileStreams (normally), by an instance of a subclass of SourceFileArray, which can hold such files and additionally provides the appropriate mapping methods. Using this approach, it should be fairly easy to change source file access to a multi-file system as used by Collage, or a database or network server interface such as that used for the SqueakCVS project.
Please, SqC: Consider these changes for inclusion in the standard release. I think that apart from the better extensibility of this scheme, it has the big advantage of keeping all this ugly stuff in a central place, making its maintenance easier.
Note: These changesets must be loaded in the correct order, and you should load them after 2004BigSources-di because they have conflicting method definitions.
Cheers, Hans-Martin
'From Squeak2.8alpha of 19 January 2000 [latest update: #2005] on 26 April 2000 at 9:44:26 pm'! "Change Set: NewSources-2-hmm Date: 26 April 2000 Author: Hans-Martin Mosner hmm@heeg.de
Second part of the modification of the SourceFiles mechanism. In this change set, all methods that think they know about the structure of sourcePointers are replaced by versions that delegate to SourceFiles."!
!CompiledMethod methodsFor: 'source code management' stamp: 'hmm 4/26/2000 20:44'! fileIndex ^SourceFiles fileIndexFromSourcePointer: self sourcePointer! !
!CompiledMethod methodsFor: 'source code management' stamp: 'hmm 4/26/2000 20:45'! filePosition ^SourceFiles filePositionFromSourcePointer: self sourcePointer! !
!CompiledMethod methodsFor: 'source code management' stamp: 'hmm 4/26/2000 21:00'! setSourcePointer: srcPointer srcPointer = 0 ifTrue: [ self at: self size put: 0. ^self]. (srcPointer between: 16r1000000 and: 16r4FFFFFF) ifFalse: [self error: 'Source pointer out of range']. self at: self size put: (srcPointer bitShift: -24) + 251. 1 to: 3 do: [:i | self at: self size-i put: ((srcPointer bitShift: (i-3)*8) bitAnd: 16rFF)]! !
!CompiledMethod methodsFor: 'source code management' stamp: 'hmm 4/26/2000 21:02'! setSourcePosition: position inFile: fileIndex self setSourcePointer: (SourceFiles sourcePointerFromFileIndex: fileIndex andPosition: position)! !
!CompiledMethod methodsFor: 'source code management' stamp: 'hmm 4/26/2000 20:44'! sourcePointer "Answer the integer which can be used to find the source file and position for this method. The returned value is either 0 (if no source is stored) or a number between 16r1000000 and 16r4FFFFFF. The actual interpretation of this number is up to the SourceFileArray stored in the global variable SourceFiles."
| pos | self last < 252 ifTrue: [^ 0 "no source"]. pos _ self last - 251. self size - 1 to: self size - 3 by: -1 do: [:i | pos _ pos * 256 + (self at: i)]. ^pos! !
!RemoteString methodsFor: 'accessing' stamp: 'hmm 4/26/2000 21:28'! setSourcePointer: aSourcePointer sourceFileNumber _ SourceFiles fileIndexFromSourcePointer: aSourcePointer. filePositionHi _ SourceFiles filePositionFromSourcePointer: aSourcePointer! !
!RemoteString methodsFor: 'accessing' stamp: 'hmm 4/26/2000 20:47'! sourcePointer sourceFileNumber ifNil: [^ 0]. ^SourceFiles sourcePointerFromFileIndex: sourceFileNumber andPosition: filePositionHi! !
'From Squeak2.8alpha of 19 January 2000 [latest update: #2005] on 26 April 2000 at 9:44:17 pm'! "Change Set: NewSources-1-hmm Date: 26 April 2000 Author: Hans-Martin Mosner hmm@heeg.de
First part of a modification that concentrates all mapping from source pointers to actual source files in one place. This is needed for more sophisticated source code management schemes, such as that needed for Collage."!
SequenceableCollection subclass: #SourceFileArray instanceVariableNames: '' classVariableNames: '' poolDictionaries: '' category: 'System-Source Files'!
!SourceFileArray commentStamp: 'hmm 4/25/2000 21:56' prior: 0! This class is an abstract superclass for source code access mechanisms. It defines the messages that need to be understood by those subclasses that store and retrieve source chunks on files, over the network or in databases. The first concrete subclass, StandardSourceFileArray, supports access to the traditional sources and changes files. Other subclasses might implement multiple source files for different applications, or access to a network source server.! ]style[(254 23 184)f1,f1LStandardSourceFileArray Comment;,f1! SourceFileArray subclass: #StandardSourceFileArray instanceVariableNames: 'files ' classVariableNames: '' poolDictionaries: '' category: 'System-Source Files'!
!StandardSourceFileArray commentStamp: 'hmm 4/25/2000 21:58' prior: 0! This class implements the source file management behavior of traditional Squeak, with a sources file and a changes file. File positions are mapped such that those files can be up to 32MBytes in size.
Structure: files Array -- storing the actual source files !
!SourceFileArray methodsFor: 'accessing' stamp: 'hmm 4/26/2000 21:42'! at: index self subclassResponsibility! !
!SourceFileArray methodsFor: 'accessing' stamp: 'hmm 4/26/2000 21:43'! at: index put: aFileStream self subclassResponsibility! !
!SourceFileArray methodsFor: 'accessing' stamp: 'hmm 4/26/2000 21:09'! collect: aBlock ^(1 to: self size) collect: [:i | aBlock value: (self at: i)]! !
!SourceFileArray methodsFor: 'accessing' stamp: 'hmm 4/26/2000 21:43'! size self subclassResponsibility! !
!SourceFileArray methodsFor: 'sourcePointer conversion' stamp: 'hmm 4/25/2000 22:00'! fileIndexFromSourcePointer: anInteger "Return the index of a source file corresponding to the given source pointer." self subclassResponsibility! !
!SourceFileArray methodsFor: 'sourcePointer conversion' stamp: 'hmm 4/25/2000 22:00'! filePositionFromSourcePointer: anInteger "Return the position within a source file for the given source pointer." self subclassResponsibility! !
!SourceFileArray methodsFor: 'sourcePointer conversion' stamp: 'hmm 4/25/2000 22:01'! sourcePointerFromFileIndex: index andPosition: position "Return a sourcePointer encoding the given file index and position" self subclassResponsibility! !
!StandardSourceFileArray methodsFor: 'initialize-release' stamp: 'hmm 4/25/2000 21:20'! initialize files _ Array new: 2. files at: 1 put: (SourceFiles at: 1). files at: 2 put: (SourceFiles at: 2)! !
!StandardSourceFileArray methodsFor: 'accessing' stamp: 'hmm 4/25/2000 21:20'! at: index ^files at: index! !
!StandardSourceFileArray methodsFor: 'accessing' stamp: 'hmm 4/25/2000 21:20'! at: index put: aFile files at: index put: aFile! !
!StandardSourceFileArray methodsFor: 'accessing' stamp: 'hmm 4/25/2000 21:20'! size ^files size! !
!StandardSourceFileArray methodsFor: 'sourcePointer conversion' stamp: 'hmm 4/25/2000 21:44'! fileIndexFromSourcePointer: anInteger "Return the index of the source file which contains the source chunk addressed by anInteger" "This implements the recent 32M source file algorithm"
| hi | hi _ anInteger // 16r1000000. ^hi < 3 ifTrue: [hi] ifFalse: [hi - 2]! !
!StandardSourceFileArray methodsFor: 'sourcePointer conversion' stamp: 'hmm 4/25/2000 21:44'! filePositionFromSourcePointer: anInteger "Return the position of the source chunk addressed by anInteger" "This implements the recent 32M source file algorithm"
| hi lo | hi _ anInteger // 16r1000000. lo _ anInteger \ 16r1000000. ^hi < 3 ifTrue: [lo] ifFalse: [lo + 16r1000000]! !
!StandardSourceFileArray methodsFor: 'sourcePointer conversion' stamp: 'hmm 4/25/2000 21:48'! sourcePointerFromFileIndex: index andPosition: position | hi lo | "Return a source pointer according to the new 32M algorithm" ((index between: 1 and: 2) and: [position between: 0 and: 16r1FFFFFF]) ifFalse: [self error: 'invalid source code pointer']. hi _ index. lo _ position. lo >= 16r1000000 ifTrue: [ hi _ hi+2. lo _ lo - 16r1000000]. ^hi * 16r1000000 + lo! !
!StandardSourceFileArray class methodsFor: 'initialize-release' stamp: 'hmm 4/25/2000 21:19'! install "Replace SourceFiles by an instance of me with the standard sources and changes files. This only works if SourceFiles is either an Array or an instance of this class" "StandardSourceFileArray install" SourceFiles _ self new initialize! !
!StandardSourceFileArray reorganize! ('initialize-release' initialize) ('accessing' at: at:put: changesFile size sourcesFile) ('sourcePointer conversion' fileIndexFromSourcePointer: filePositionFromSourcePointer: sourcePointerFromFileIndex:andPosition:) !
"Postscript: After defining the necessary classes, install the standard SourceFileArray which defines a sourcePointer mapping compatible to the old way."
StandardSourceFileArray install!
Hans-Martin -
Please, SqC: Consider these changes for inclusion in the standard release. I think that apart from the better extensibility of this scheme, it has the big advantage of keeping all this ugly stuff in a central place, making its maintenance easier.
With all due respect, these were already pretty well contained. The entire change to doubled file sizes only changed 6 methods.
I am running with your changes now. Assuming all goes well, I'll send them out with next updates.
Note: These changesets must be loaded in the correct order, and you should load them after 2004BigSources-di because they have conflicting method definitions.
Yes, ahem, AND you should move the def of CompiledMethod>>sourcePointer in front of that for CompiledMethod>>fileIndex in the second file, if you don't want to crash your Squeak ;-).
- Dan
squeak-dev@lists.squeakfoundation.org