David T. Lewis uploaded a new version of System to project The Trunk: http://source.squeak.org/trunk/System-dtl.1374.mcz
==================== Summary ====================
Name: System-dtl.1374 Author: dtl Time: 25 November 2022, 2:00:24.392665 pm UUID: f4b3b5ed-db5c-445a-be6b-3f29ea6552e3 Ancestors: System-dtl.1373
Support use of ChangeLogStream for changes log maintained within the image.
=============== Diff against System-dtl.1373 ===============
Item was changed: ----- Method: SmalltalkImage>>openSourcesAndChanges:forImage: (in category 'image, changes names') ----- openSourcesAndChanges: changesName forImage: imageName "Open the changes and sources files and install them in SourceFiles. Inform the user of problems regarding write permissions or CR/CRLF mixups." "Note: SourcesName and imageName are full paths; changesName is a local name." | sources changes msg wmsg | msg := 'Squeak cannot locate {1}.
Please check that the file is named properly and is in the same directory as this image.'. wmsg := 'Squeak cannot write to {1}.
Please check that you have write permission for this file.
You won''t be able to save this image correctly until you fix this.'.
sources := Smalltalk openSources. sources ifNotNil: [sources setConverterForCode]. + changes := SourceFileArray cachedChanges + ifNil: [Smalltalk openChanges: changesName forImage: imageName]. - changes := Smalltalk openChanges: changesName forImage: imageName. changes ifNotNil: [changes setConverterForCode]. - ((sources == nil or: [sources atEnd]) and: [Preferences valueOfFlag: #warnIfNoSourcesFile]) ifTrue: [Smalltalk platformName = 'Mac OS' ifTrue: [msg := msg , ' Make sure the sources file is not an Alias.']. self inform: (msg format: { 'the sources file named ' , self sourcesName })].
(changes == nil and: [Preferences valueOfFlag: #warnIfNoChangesFile]) ifTrue: [self inform: (msg format: { 'the changes file named ' , changesName })].
((Preferences valueOfFlag: #warnIfNoChangesFile) and: [changes notNil]) ifTrue: [changes isReadOnly ifTrue: [self inform: (wmsg format: { 'the changes file named ' , changesName })].
((changes next: 200) includesSubstring: String crlf) ifTrue: [self inform: ('The changes file named <b>{1}</b> has been injured by an unpacking utility. Line endings were changed from Cr to CrLf.<br><br>Please set the preferences in your decompressing program to <b>do not convert text files</b> and unpack the system again.' translated format: { changesName }) asTextFromHtml]].
SourceFiles := Array with: sources with: changes!
Item was changed: ----- Method: SmalltalkImage>>saveAs: (in category 'sources, changes log') ----- saveAs: newName "Save the image under that new name." newName ifNil:[^ self]. + + (SourceFileArray internalizeChanges or: (SourceFiles at: 2) isNil) ifFalse: - (SourceFiles at: 2) ifNotNil: [self closeSourceFiles; "so copying the changes file will always work" saveChangesInFileNamed: (self fullNameForChangesNamed: newName)]. self saveImageInFileNamed: (self fullNameForImageNamed: newName)!
Item was changed: ----- Method: SmalltalkImage>>saveAsNewVersion (in category 'sources, changes log') ----- saveAsNewVersion "Save the image/changes using the next available version number." "Smalltalk saveAsNewVersion" | newName changesName aName anIndex | aName := FileDirectory baseNameFor: (FileDirectory default localNameFor: self imageName). anIndex := aName lastIndexOf: FileDirectory dot asCharacter ifAbsent: [nil]. (anIndex notNil and: [(aName copyFrom: anIndex + 1 to: aName size) isAllDigits]) ifTrue: [aName := aName copyFrom: 1 to: anIndex - 1].
newName := FileDirectory default nextNameFor: aName extension: FileDirectory imageSuffix. changesName := self fullNameForChangesNamed: newName.
+ (SourceFileArray internalizeChanges or: (SourceFiles at: 2) isNil) ifFalse: [ + "Check to see if there is a .changes file that would cause a problem if we saved a new .image file with the new version number" + (FileDirectory default fileOrDirectoryExists: changesName) + ifTrue: + [^ self inform: - "Check to see if there is a .changes file that would cause a problem if we saved a new .image file with the new version number" - (FileDirectory default fileOrDirectoryExists: changesName) - ifTrue: - [^ self inform: 'There is already .changes file of the desired name, ', newName, ' curiously already present, even though there is no corresponding .image file. Please remedy manually and then repeat your request.'].
+ self closeSourceFiles; "so copying the changes file will always work" + saveChangesInFileNamed: (self fullNameForChangesNamed: newName) + ]. - (SourceFiles at: 2) ifNotNil: - [self closeSourceFiles; "so copying the changes file will always work" - saveChangesInFileNamed: (self fullNameForChangesNamed: newName)]. self saveImageInFileNamed: (self fullNameForImageNamed: newName)
!
Hi David,
this seems backwards to me. It seems to me that one would want to cache the sources but keep the changes gfile for crash recovery. Sicne the changes file should mbe empty at the start of a release this would mean that all sources were cached. What am I getting wrong? And how does your scheme do crash recovery, if at all?
On Fri, Nov 25, 2022 at 11:00 AM commits@source.squeak.org wrote:
David T. Lewis uploaded a new version of System to project The Trunk: http://source.squeak.org/trunk/System-dtl.1374.mcz
==================== Summary ====================
Name: System-dtl.1374 Author: dtl Time: 25 November 2022, 2:00:24.392665 pm UUID: f4b3b5ed-db5c-445a-be6b-3f29ea6552e3 Ancestors: System-dtl.1373
Support use of ChangeLogStream for changes log maintained within the image.
=============== Diff against System-dtl.1373 ===============
Item was changed: ----- Method: SmalltalkImage>>openSourcesAndChanges:forImage: (in category 'image, changes names') ----- openSourcesAndChanges: changesName forImage: imageName "Open the changes and sources files and install them in SourceFiles. Inform the user of problems regarding write permissions or CR/CRLF mixups." "Note: SourcesName and imageName are full paths; changesName is a local name." | sources changes msg wmsg | msg := 'Squeak cannot locate {1}.
Please check that the file is named properly and is in the same directory as this image.'. wmsg := 'Squeak cannot write to {1}.
Please check that you have write permission for this file.
You won''t be able to save this image correctly until you fix this.'.
sources := Smalltalk openSources. sources ifNotNil: [sources setConverterForCode].
changes := SourceFileArray cachedChanges
ifNil: [Smalltalk openChanges: changesName forImage:
imageName].
changes := Smalltalk openChanges: changesName forImage: imageName. changes ifNotNil: [changes setConverterForCode].
((sources == nil or: [sources atEnd]) and: [Preferences valueOfFlag:
#warnIfNoSourcesFile]) ifTrue: [Smalltalk platformName = 'Mac OS' ifTrue: [msg := msg , ' Make sure the sources file is not an Alias.']. self inform: (msg format: { 'the sources file named ' , self sourcesName })].
(changes == nil and: [Preferences valueOfFlag:
#warnIfNoChangesFile]) ifTrue: [self inform: (msg format: { 'the changes file named ' , changesName })].
((Preferences valueOfFlag: #warnIfNoChangesFile) and: [changes
notNil]) ifTrue: [changes isReadOnly ifTrue: [self inform: (wmsg format: { 'the changes file named ' , changesName })].
((changes next: 200) includesSubstring: String crlf) ifTrue: [self inform: ('The changes file
named <b>{1}</b> has been injured by an unpacking utility. Line endings were changed from Cr to CrLf.<br><br>Please set the preferences in your decompressing program to <b>do not convert text files</b> and unpack the system again.' translated format: { changesName }) asTextFromHtml]].
SourceFiles := Array with: sources with: changes!
Item was changed: ----- Method: SmalltalkImage>>saveAs: (in category 'sources, changes log') ----- saveAs: newName "Save the image under that new name." newName ifNil:[^ self].
(SourceFileArray internalizeChanges or: (SourceFiles at: 2) isNil)
ifFalse:
(SourceFiles at: 2) ifNotNil: [self closeSourceFiles; "so copying the changes file will
always work" saveChangesInFileNamed: (self fullNameForChangesNamed: newName)]. self saveImageInFileNamed: (self fullNameForImageNamed: newName)!
Item was changed: ----- Method: SmalltalkImage>>saveAsNewVersion (in category 'sources, changes log') ----- saveAsNewVersion "Save the image/changes using the next available version number." "Smalltalk saveAsNewVersion"
| newName changesName aName anIndex | aName := FileDirectory baseNameFor: (FileDirectory default
localNameFor: self imageName). anIndex := aName lastIndexOf: FileDirectory dot asCharacter ifAbsent: [nil]. (anIndex notNil and: [(aName copyFrom: anIndex + 1 to: aName size) isAllDigits]) ifTrue: [aName := aName copyFrom: 1 to: anIndex - 1].
newName := FileDirectory default nextNameFor: aName extension:
FileDirectory imageSuffix. changesName := self fullNameForChangesNamed: newName.
(SourceFileArray internalizeChanges or: (SourceFiles at: 2) isNil)
ifFalse: [
"Check to see if there is a .changes file that would cause
a problem if we saved a new .image file with the new version number"
(FileDirectory default fileOrDirectoryExists: changesName)
ifTrue:
[^ self inform:
"Check to see if there is a .changes file that would cause a
problem if we saved a new .image file with the new version number"
(FileDirectory default fileOrDirectoryExists: changesName)
ifTrue:
'There is already .changes file of the desired name, ', newName, ' curiously already present, even though there is no corresponding .image file. Please remedy manually and then repeat your request.'].[^ self inform:
self closeSourceFiles; "so copying the changes file will
always work"
saveChangesInFileNamed: (self
fullNameForChangesNamed: newName)
].
(SourceFiles at: 2) ifNotNil:
[self closeSourceFiles; "so copying the changes file will
always work"
saveChangesInFileNamed: (self
fullNameForChangesNamed: newName)]. self saveImageInFileNamed: (self fullNameForImageNamed: newName)
!
On 2022-11-25, at 11:28 AM, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi David,
this seems backwards to me. It seems to me that one would want to cache the sources but keep the changes gfile for crash recovery. Sicne the changes file should mbe empty at the start of a release this would mean that all sources were cached. What am I getting wrong?
Generally we don't condense sources for most releases, so the changes does tend to be around. The 5.3 release changes was ~18Mb for example.
And how does your scheme do crash recovery, if at all?
I don't think this version of Dave's idea does, which is certainly something I'd like to see. Having the changes 'file' cached in-image is fine but having at least an option to make it write-through to an outside destination would be good. There's a lot of different ways we could do this.
Last time I tried a similar approach I made the source pointer stuff be an object that could be a Smallint to point into a file or a String or a link of some sort to something I don't even remember. That meant you could mix'n'match where source came from, which seemed plausibly interesting. You might even make it go via a web service that serves up the source and stores new source & doits in a shareable manner, merging in with some monticello capabilities.
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Oxymorons: New classic
On Fri, Nov 25, 2022 at 11:49:00AM -0800, tim Rowledge wrote:
Last time I tried a similar approach I made the source pointer stuff be an object that could be a Smallint to point into a file or a String or a link of some sort to something I don't even remember. That meant you could mix'n'match where source came from, which seemed plausibly interesting. You might even make it go via a web service that serves up the source and stores new source & doits in a shareable manner, merging in with some monticello capabilities.
I think there are a lot of really interesting things that might be done with this idea. I really hope someone picks up on it and tries a few experiments.
There is certainly no reason that SourceFiles must have exactly two elements, and no reason that source pointers cannot point to things other than file positions.
Dave
Hi Eliot,
On Fri, Nov 25, 2022 at 11:28:58AM -0800, Eliot Miranda wrote:
Hi David,
this seems backwards to me. It seems to me that one would want to cache the sources but keep the changes gfile for crash recovery.
You can already do that, see the "Cache sources file" preference.
Sicne the changes file should mbe empty at the start of a release this would mean that all sources were cached. What am I getting wrong? And how does your scheme do crash recovery, if at all?
There is no crash recovery.
Some earlier discussion (well I suppose it was more of a monologue) is at http://lists.squeakfoundation.org/pipermail/squeak-dev/2022-October/222570.h....
The original 1996 implementation is referenced here http://lists.squeakfoundation.org/pipermail/squeak-dev/1998-March/007670.htm...
As Dan said at that time: "It has one wonderful property - you can run for long periods without spinning up the disk of a portable computer. -- and the analogous awful property -- if anything crashes, there's nothing to replay :-(
Nowadays we are not so worried about wearing out the bearings on our hard drives, but we do have cases where it is useful to be able to move an image around without carrying extra disk file references along with it.
This is definitely /not/ intended for use in distributing Squeak release images. And putting the changes file back out on your file system is just a matter of "SourceFileArray internalizeChanges: false" (a preference).
Here are some use cases that are of interest to me:
1) Forking the running VM on Unix with #forkSqueak. The changes file management in this case works by accident, but it really would be better if two or more images were not attempting to share the same changes file on disk.
2) SqueakJS. It would be nice to run an image on SqueakJS without worrying about keeping the associated files with the image.
3) SnapshotServer resume image on another server, one of my hobby projects at http://www.squeaksource.com/SnapshotServer
I should note that although the sources file is compressed when the source file is inboarded ("Cache sources file" preference), so such optimization is attempted for "Cache changes file". That might be a topic for some other day.
Dave
On Fri, Nov 25, 2022 at 03:51:45PM -0500, David T. Lewis wrote:
I should note that although the sources file is compressed when the source file is inboarded ("Cache sources file" preference), so such optimization is attempted for "Cache changes file". That might be a topic for some other day.
I meant to say "no such optimization is attempted", sorry for the typo.
Dave
squeak-dev@lists.squeakfoundation.org