Hi Dave,

Just a short notice from my side, I have been using this preference in two of my most frequently used images so far and have neither experienced any unexpected hick-ups* nor any new slowdowns where source files access appeared as a bottle neck in the time profiler again. Because these slowdowns on my machine have always been sporadic only, this is not yet a formal proof, but overall this change seems to work very well & was long overdue. Thanks a lot! :-)

*Well, there was one recent incident with wrong pointers in one of my changes files, but since this happened in my throw-away image which I use to do everything useful and useless (opening multiple instances in parallel, disabling source files access altogether, and much more weird experiments), it is very likely that there was a different cause for this incident.

Best,
Christoph

---
Sent from Squeak Inbox Talk

On 2022-02-16T18:42:34+00:00, commits at source.squeak.org wrote:

> David T. Lewis uploaded a new version of Compression to project The Trunk:
> http://source.squeak.org/trunk/Compression-dtl.61.mcz
> 
> ==================== Summary ====================
> 
> Name: Compression-dtl.61
> Author: dtl
> Time: 15 February 2022, 10:38:51.880942 pm
> UUID: 64e55d6a-3ee9-46c3-b62c-91815ba8ad34
> Ancestors: Compression-cmm.60
> 
> Allow a compressed copy of the sources file to be held in object memory without file system dependency. Adds class CompressedSources, extending CompressedSourceStream to hold its segmentFile (normally an .stc file on disk) in a ReadWriteStream rather than on the file system.
> 
> Use the 'Cache sources file' preference in category 'Files' to enable use of the internal sources file. When the preference is later disabled, a new sources file will be written to the default directory if no existing sources file can otherwise be located.
> 
> =============== Diff against Compression-cmm.60 ===============
> 
> Item was added:
> + CompressedSourceStream subclass: #CompressedSources
> +     instanceVariableNames: ''
> +     classVariableNames: 'CachedSources'
> +     poolDictionaries: ''
> +     category: 'Compression-Streams'!
> + 
> + !CompressedSources commentStamp: 'dtl 1/16/2022 17:39' prior: 0!
> + A CompressedSources is a CompressedSourceStream that holds its sources in the image rather than in an external file.
> + !
> 
> Item was added:
> + ----- Method: CompressedSources class>>cachedSources (in category 'accessing') -----
> + cachedSources
> +     ^CachedSources!
> 
> Item was added:
> + ----- Method: CompressedSources class>>fromSourceFileArray (in category 'instance creation') -----
> + fromSourceFileArray
> +     "Answer a new instance created from the existing SourceFiles"
> + 
> +     "CompressedSources fromSourceFileArray"
> + 
> +     | f sourcesName |
> +     f := SourceFiles first readOnlyCopy.
> +     sourcesName := f localName.
> +     (sourcesName endsWith: 'sources')
> +         ifTrue: [^self fromSourcesFile: f].
> +     ^f asCompressedSources.
> + !
> 
> Item was added:
> + ----- Method: CompressedSources class>>fromSourcesFile: (in category 'private') -----
> + fromSourcesFile: f
> + 
> +     | cf |
> +     f binary. "binary to preserve utf8 encoding"
> +     cf := (CompressedSources on: (ReadWriteStream with: ByteArray new))
> +                 segmentSize: 65536 maxSize: f size.
> + 
> +     "Copy the sources"
> + 'Compressing Sources File...'
> +     displayProgressAt: Sensor cursorPoint
> +     from: 0 to: f size
> +     during:
> +         [:bar | f position: 0.
> +         [f atEnd] whileFalse:
> +             [cf nextPutAll: (f next: 65536).
> +             bar value: f position]].
> +     ^cf.
> + !
> 
> Item was added:
> + ----- Method: CompressedSources class>>internalizeSources (in category 'accessing') -----
> + internalizeSources
> + 
> +     <preference: 'Cache sources file'
> +     category: 'Files'
> +     description: 'If true, a compressed sources file will be kept in the image to avoid file system access'
> +     type: #Boolean>
> +     ^CachedSources notNil
> + !
> 
> Item was added:
> + ----- Method: CompressedSources class>>internalizeSources: (in category 'accessing') -----
> + internalizeSources: internalize
> +     internalize
> +         ifNotNil: [| msg |
> +             msg := 'No external ' translated , Smalltalk sourceFileVersionString , ' sources file found' translated.
> +             internalize
> +                 ifTrue: [CachedSources := self fromSourceFileArray position: 0]
> +                 ifFalse: [Smalltalk locateSourcesEntry
> +                         ifNil: [((self confirm: msg , ', save a local copy?' translated)
> +                                     and: [self saveSourcesToLocalFile])
> +                                 ifFalse: [^ msg , ' (preference not changed)' translated]].
> +                     CachedSources := nil].
> +             Smalltalk closeSourceFiles; openSourceFiles]!
> 
> Item was added:
> + ----- Method: CompressedSources class>>saveSourcesToLocalFile (in category 'accessing') -----
> + saveSourcesToLocalFile
> +     "Save the compressed sources to a .sources file in the default directory. Answer true for success"
> + 
> +     | sourcesName fs |
> +     sourcesName := Smalltalk sourceFileVersionString , '.sources'.
> +     (FileDirectory default fileExists: sourcesName)
> +         ifTrue: [self error: sourcesName , ' already exists in the default directory'.
> +             ^ false].
> +     [[| sourceData |
> +     fs := FileStream newFileNamed: sourcesName.
> +     fs binary.
> +     sourceData := ProgressInitiationException
> +         display: 'converting compressed sources'
> +         during: [:bar | CachedSources uncompressedBytes: bar].
> +     fs nextPutAll: sourceData]
> +         ensure: [fs close]]
> +         on: Error
> +         do: [:e | 
> +             self notify: 'failed writing ' , sourcesName.
> +             ^ false].
> +     ^ true!
> 
> Item was added:
> + ----- Method: CompressedSources>>asCompressedSources (in category 'converting') -----
> + asCompressedSources
> +     ^self!
> 
> Item was added:
> + ----- Method: CompressedSources>>readOnlyCopy (in category 'file open/close') -----
> + readOnlyCopy
> +     "Not required for source file management, just answer a copy"
> +     ^self copy!
> 
> Item was added:
> + ----- Method: CompressedSources>>reopen (in category 'file open/close') -----
> + reopen
> +     self position: 0.
> + !
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20220328/1363cb28/attachment-0001.html>