Dave Lewis uploaded a new version of VMMaker to project VM Maker: http://www.squeaksource.com/VMMaker/VMMaker-dtl.189.mcz
==================== Summary ====================
Name: VMMaker-dtl.189 Author: dtl Time: 6 October 2010, 12:17:37 pm UUID: 29192b6e-491d-4bc6-a209-5ba1c8625da0 Ancestors: VMMaker-dtl.188
VMMaker 4.3.3
Allow a standard interpreter VM to load and run an image that was saved from a Cog or StackInterpreter VM. On image load, the storage format of Float objects is returned to normalized word ordering (different from platform word order for little endian platforms) and unrecognized header flags are cleared. The image will be saved in standard interpreter format (image format 6504 for a 32-bit image), which may subsequently be loaded by a Cog or StackInterpreter VM.
Works for 32-bit Cog images (image format 6505) on both 32-bit and 64-bit host (compile -m32 or -m64). Support for 64-bit image formats is not yet implemented, see comment in #normalizeFloatOrderingInImage.
This could stand a bit of testing before folks rely on it too heavily, but it should permit Cog to be used for developing images that may later need to be loaded into a standard VM. I have loaded an image saved from Cog (format 6505) into a standard VM, saved it in the older 6504 format, then read that image back into a Cog VM. There are no obvious problems, and the Float values are preserved as expected.
Eliot, the changes are attached.
Dave
On Wed, Oct 06, 2010 at 06:17:56AM +0000, squeak-dev-noreply@lists.squeakfoundation.org wrote:
Dave Lewis uploaded a new version of VMMaker to project VM Maker: http://www.squeaksource.com/VMMaker/VMMaker-dtl.189.mcz
==================== Summary ====================
Name: VMMaker-dtl.189 Author: dtl Time: 6 October 2010, 12:17:37 pm UUID: 29192b6e-491d-4bc6-a209-5ba1c8625da0 Ancestors: VMMaker-dtl.188
VMMaker 4.3.3
Allow a standard interpreter VM to load and run an image that was saved from a Cog or StackInterpreter VM. On image load, the storage format of Float objects is returned to normalized word ordering (different from platform word order for little endian platforms) and unrecognized header flags are cleared. The image will be saved in standard interpreter format (image format 6504 for a 32-bit image), which may subsequently be loaded by a Cog or StackInterpreter VM.
Works for 32-bit Cog images (image format 6505) on both 32-bit and 64-bit host (compile -m32 or -m64). Support for 64-bit image formats is not yet implemented, see comment in #normalizeFloatOrderingInImage.
This is really nice, because it will cut the number of development images by half! Thanks, David.
On 6 October 2010 07:32, David T. Lewis lewis@mail.msen.com wrote:
This could stand a bit of testing before folks rely on it too heavily, but it should permit Cog to be used for developing images that may later need to be loaded into a standard VM. I have loaded an image saved from Cog (format 6505) into a standard VM, saved it in the older 6504 format, then read that image back into a Cog VM. There are no obvious problems, and the Float values are preserved as expected.
Eliot, the changes are attached.
Dave
On Wed, Oct 06, 2010 at 06:17:56AM +0000, squeak-dev-noreply@lists.squeakfoundation.org wrote:
Dave Lewis uploaded a new version of VMMaker to project VM Maker: http://www.squeaksource.com/VMMaker/VMMaker-dtl.189.mcz
==================== Summary ====================
Name: VMMaker-dtl.189 Author: dtl Time: 6 October 2010, 12:17:37 pm UUID: 29192b6e-491d-4bc6-a209-5ba1c8625da0 Ancestors: VMMaker-dtl.188
VMMaker 4.3.3
Allow a standard interpreter VM to load and run an image that was saved from a Cog or StackInterpreter VM. On image load, the storage format of Float objects is returned to normalized word ordering (different from platform word order for little endian platforms) and unrecognized header flags are cleared. The image will be saved in standard interpreter format (image format 6504 for a 32-bit image), which may subsequently be loaded by a Cog or StackInterpreter VM.
Works for 32-bit Cog images (image format 6505) on both 32-bit and 64-bit host (compile -m32 or -m64). Support for 64-bit image formats is not yet implemented, see comment in #normalizeFloatOrderingInImage.
Yay!
I'm wondering wether this can help make the transition to Cog be less painful. Does anyone have ideas in that direction? E.g., there still is need for the interpreter, to run pre-closure images. Many production systems use those (e.g the currently deployed Etoys releases), we don't want a Squeak VM update break them.
- Bert -
On 06.10.2010, at 05:02, Igor Stasenko wrote:
This is really nice, because it will cut the number of development images by half! Thanks, David.
On 6 October 2010 07:32, David T. Lewis lewis@mail.msen.com wrote:
This could stand a bit of testing before folks rely on it too heavily, but it should permit Cog to be used for developing images that may later need to be loaded into a standard VM. I have loaded an image saved from Cog (format 6505) into a standard VM, saved it in the older 6504 format, then read that image back into a Cog VM. There are no obvious problems, and the Float values are preserved as expected.
Eliot, the changes are attached.
Dave
On Wed, Oct 06, 2010 at 06:17:56AM +0000, squeak-dev-noreply@lists.squeakfoundation.org wrote:
Dave Lewis uploaded a new version of VMMaker to project VM Maker: http://www.squeaksource.com/VMMaker/VMMaker-dtl.189.mcz
==================== Summary ====================
Name: VMMaker-dtl.189 Author: dtl Time: 6 October 2010, 12:17:37 pm UUID: 29192b6e-491d-4bc6-a209-5ba1c8625da0 Ancestors: VMMaker-dtl.188
VMMaker 4.3.3
Allow a standard interpreter VM to load and run an image that was saved from a Cog or StackInterpreter VM. On image load, the storage format of Float objects is returned to normalized word ordering (different from platform word order for little endian platforms) and unrecognized header flags are cleared. The image will be saved in standard interpreter format (image format 6504 for a 32-bit image), which may subsequently be loaded by a Cog or StackInterpreter VM.
Works for 32-bit Cog images (image format 6505) on both 32-bit and 64-bit host (compile -m32 or -m64). Support for 64-bit image formats is not yet implemented, see comment in #normalizeFloatOrderingInImage.
-- Best regards, Igor Stasenko AKA sig.
Yes, that is the intent. It should help with the transition to Cog in several ways:
1) It should now be safe to use Cog on your favorite image, even if you think you might want to go back to the old image format.
2) If you save an image from a Cog VM in the newer image format (6505) and distribute it to folks who may not be able to run Cog for one reason or another, it will now be possible for them to load that image back into a traditional VM.
3) If you want to use Cog to build an an image for wide distribution (the upcoming Squeak 4.2 release for example) and you want to distribute the image in the older image format (probably a prudent choice for the next round of releases), you can now load and save the image one time in a traditional VM, and the image will be put back into 6504 format.
Caveat: I just finished doing this last night and it is only lightly tested. I don't anticipate any problems, but you never know ;)
The traditional VM should still run pre-closure images (format 6502) and will continue to save them in pre-closure 6502 format. Eliot set this up so that the conversion from 6502 pre-closure format to 6504 format happens automagically, but only when you first actually compile something with closure support. But I guess this should be tested to make sure I did not break it ;)
Dave
On Wed, Oct 06, 2010 at 06:53:48AM -0700, Bert Freudenberg wrote:
Yay!
I'm wondering wether this can help make the transition to Cog be less painful. Does anyone have ideas in that direction? E.g., there still is need for the interpreter, to run pre-closure images. Many production systems use those (e.g the currently deployed Etoys releases), we don't want a Squeak VM update break them.
- Bert -
On 06.10.2010, at 05:02, Igor Stasenko wrote:
This is really nice, because it will cut the number of development images by half! Thanks, David.
On 6 October 2010 07:32, David T. Lewis lewis@mail.msen.com wrote:
This could stand a bit of testing before folks rely on it too heavily, but it should permit Cog to be used for developing images that may later need to be loaded into a standard VM. I have loaded an image saved from Cog (format 6505) into a standard VM, saved it in the older 6504 format, then read that image back into a Cog VM. There are no obvious problems, and the Float values are preserved as expected.
Eliot, the changes are attached.
Dave
On Wed, Oct 06, 2010 at 06:17:56AM +0000, squeak-dev-noreply@lists.squeakfoundation.org wrote:
Dave Lewis uploaded a new version of VMMaker to project VM Maker: http://www.squeaksource.com/VMMaker/VMMaker-dtl.189.mcz
==================== Summary ====================
Name: VMMaker-dtl.189 Author: dtl Time: 6 October 2010, 12:17:37 pm UUID: 29192b6e-491d-4bc6-a209-5ba1c8625da0 Ancestors: VMMaker-dtl.188
VMMaker 4.3.3
Allow a standard interpreter VM to load and run an image that was saved from a Cog or StackInterpreter VM. On image load, the storage format of Float objects is returned to normalized word ordering (different from platform word order for little endian platforms) and unrecognized header flags are cleared. The image will be saved in standard interpreter format (image format 6504 for a 32-bit image), which may subsequently be loaded by a Cog or StackInterpreter VM.
Works for 32-bit Cog images (image format 6505) on both 32-bit and 64-bit host (compile -m32 or -m64). Support for 64-bit image formats is not yet implemented, see comment in #normalizeFloatOrderingInImage.
-- Best regards, Igor Stasenko AKA sig.
On 06.10.2010, at 08:10, David T. Lewis wrote:
Yes, that is the intent. It should help with the transition to Cog in several ways:
- It should now be safe to use Cog on your favorite image, even if you
think you might want to go back to the old image format.
- If you save an image from a Cog VM in the newer image format (6505)
and distribute it to folks who may not be able to run Cog for one reason or another, it will now be possible for them to load that image back into a traditional VM.
- If you want to use Cog to build an an image for wide distribution (the
upcoming Squeak 4.2 release for example) and you want to distribute the image in the older image format (probably a prudent choice for the next round of releases), you can now load and save the image one time in a traditional VM, and the image will be put back into 6504 format.
Caveat: I just finished doing this last night and it is only lightly tested. I don't anticipate any problems, but you never know ;)
The traditional VM should still run pre-closure images (format 6502) and will continue to save them in pre-closure 6502 format. Eliot set this up so that the conversion from 6502 pre-closure format to 6504 format happens automagically, but only when you first actually compile something with closure support. But I guess this should be tested to make sure I did not break it ;)
Oh. That is great news - so you are saying that the Cog VM is able to run old images, so it is safe to upgrade the VM anyway? E.g. I should be able to run Etoys as-is on Cog, even being able to save the image, until it is converted to closures?
- Bert -
Dave
On Wed, Oct 06, 2010 at 06:53:48AM -0700, Bert Freudenberg wrote:
Yay!
I'm wondering wether this can help make the transition to Cog be less painful. Does anyone have ideas in that direction? E.g., there still is need for the interpreter, to run pre-closure images. Many production systems use those (e.g the currently deployed Etoys releases), we don't want a Squeak VM update break them.
- Bert -
On 06.10.2010, at 05:02, Igor Stasenko wrote:
This is really nice, because it will cut the number of development images by half! Thanks, David.
On 6 October 2010 07:32, David T. Lewis lewis@mail.msen.com wrote:
This could stand a bit of testing before folks rely on it too heavily, but it should permit Cog to be used for developing images that may later need to be loaded into a standard VM. I have loaded an image saved from Cog (format 6505) into a standard VM, saved it in the older 6504 format, then read that image back into a Cog VM. There are no obvious problems, and the Float values are preserved as expected.
Eliot, the changes are attached.
Dave
On Wed, Oct 06, 2010 at 06:17:56AM +0000, squeak-dev-noreply@lists.squeakfoundation.org wrote:
Dave Lewis uploaded a new version of VMMaker to project VM Maker: http://www.squeaksource.com/VMMaker/VMMaker-dtl.189.mcz
==================== Summary ====================
Name: VMMaker-dtl.189 Author: dtl Time: 6 October 2010, 12:17:37 pm UUID: 29192b6e-491d-4bc6-a209-5ba1c8625da0 Ancestors: VMMaker-dtl.188
VMMaker 4.3.3
Allow a standard interpreter VM to load and run an image that was saved from a Cog or StackInterpreter VM. On image load, the storage format of Float objects is returned to normalized word ordering (different from platform word order for little endian platforms) and unrecognized header flags are cleared. The image will be saved in standard interpreter format (image format 6504 for a 32-bit image), which may subsequently be loaded by a Cog or StackInterpreter VM.
Works for 32-bit Cog images (image format 6505) on both 32-bit and 64-bit host (compile -m32 or -m64). Support for 64-bit image formats is not yet implemented, see comment in #normalizeFloatOrderingInImage.
-- Best regards, Igor Stasenko AKA sig.
On Wed, Oct 06, 2010 at 09:10:46AM -0700, Bert Freudenberg wrote:
On 06.10.2010, at 08:10, David T. Lewis wrote:
Yes, that is the intent. It should help with the transition to Cog in several ways:
- It should now be safe to use Cog on your favorite image, even if you
think you might want to go back to the old image format.
- If you save an image from a Cog VM in the newer image format (6505)
and distribute it to folks who may not be able to run Cog for one reason or another, it will now be possible for them to load that image back into a traditional VM.
- If you want to use Cog to build an an image for wide distribution (the
upcoming Squeak 4.2 release for example) and you want to distribute the image in the older image format (probably a prudent choice for the next round of releases), you can now load and save the image one time in a traditional VM, and the image will be put back into 6504 format.
Caveat: I just finished doing this last night and it is only lightly tested. I don't anticipate any problems, but you never know ;)
The traditional VM should still run pre-closure images (format 6502) and will continue to save them in pre-closure 6502 format. Eliot set this up so that the conversion from 6502 pre-closure format to 6504 format happens automagically, but only when you first actually compile something with closure support. But I guess this should be tested to make sure I did not break it ;)
Oh. That is great news - so you are saying that the Cog VM is able to run old images, so it is safe to upgrade the VM anyway? E.g. I should be able to run Etoys as-is on Cog, even being able to save the image, until it is converted to closures?
That is a good question. I'm not sure about this (and I'm away from Squeak now and cannot check). I'm assuming that Cog will always save the image as 6505 (closures + float reordering), and that loading it back into a standard VM would then result in a 6504 image format. It might be possible to put that image all the way back into 6502 format, because it presumably would contain no actual closure bytecodes at this point. I had not actually thought about it until you asked just now.
So the short answer is no, it is probably not safe to save an Etoys image from Cog if you will ever need to run it on an old pre-closure VM. But we should try it and find out for sure.
Dave
On 06.10.2010, at 09:33, David T. Lewis wrote:
On Wed, Oct 06, 2010 at 09:10:46AM -0700, Bert Freudenberg wrote:
On 06.10.2010, at 08:10, David T. Lewis wrote:
Yes, that is the intent. It should help with the transition to Cog in several ways:
- It should now be safe to use Cog on your favorite image, even if you
think you might want to go back to the old image format.
- If you save an image from a Cog VM in the newer image format (6505)
and distribute it to folks who may not be able to run Cog for one reason or another, it will now be possible for them to load that image back into a traditional VM.
- If you want to use Cog to build an an image for wide distribution (the
upcoming Squeak 4.2 release for example) and you want to distribute the image in the older image format (probably a prudent choice for the next round of releases), you can now load and save the image one time in a traditional VM, and the image will be put back into 6504 format.
Caveat: I just finished doing this last night and it is only lightly tested. I don't anticipate any problems, but you never know ;)
The traditional VM should still run pre-closure images (format 6502) and will continue to save them in pre-closure 6502 format. Eliot set this up so that the conversion from 6502 pre-closure format to 6504 format happens automagically, but only when you first actually compile something with closure support. But I guess this should be tested to make sure I did not break it ;)
Oh. That is great news - so you are saying that the Cog VM is able to run old images, so it is safe to upgrade the VM anyway? E.g. I should be able to run Etoys as-is on Cog, even being able to save the image, until it is converted to closures?
That is a good question. I'm not sure about this (and I'm away from Squeak now and cannot check). I'm assuming that Cog will always save the image as 6505 (closures + float reordering), and that loading it back into a standard VM would then result in a 6504 image format. It might be possible to put that image all the way back into 6502 format, because it presumably would contain no actual closure bytecodes at this point. I had not actually thought about it until you asked just now.
So the short answer is no, it is probably not safe to save an Etoys image from Cog if you will ever need to run it on an old pre-closure VM. But we should try it and find out for sure.
Dave
If Cog could run an unmodified pre-closure image that would solve most compatibility issues indeed. Being able to save in the old 6502 format would be an added bonus, but might not even be necessary.
If Cog cannot do this then we need to figure out a way to cleanly install interpreter and Cog VMs side-by-side.
- Bert -
On Wed, Oct 6, 2010 at 9:46 AM, Bert Freudenberg bert@freudenbergs.dewrote:
On 06.10.2010, at 09:33, David T. Lewis wrote:
On Wed, Oct 06, 2010 at 09:10:46AM -0700, Bert Freudenberg wrote:
On 06.10.2010, at 08:10, David T. Lewis wrote:
Yes, that is the intent. It should help with the transition to Cog in several ways:
- It should now be safe to use Cog on your favorite image, even if you
think you might want to go back to the old image format.
- If you save an image from a Cog VM in the newer image format (6505)
and distribute it to folks who may not be able to run Cog for one
reason
or another, it will now be possible for them to load that image back into a traditional VM.
- If you want to use Cog to build an an image for wide distribution
(the
upcoming Squeak 4.2 release for example) and you want to distribute the image in the older image format (probably a prudent choice for the next round of releases), you can now load and save the image one time in a traditional VM, and the image will be put back into 6504 format.
Caveat: I just finished doing this last night and it is only lightly tested. I don't anticipate any problems, but you never know ;)
The traditional VM should still run pre-closure images (format 6502) and will continue to save them in pre-closure 6502 format. Eliot set this up so that the conversion from 6502 pre-closure format to 6504 format happens automagically, but only when you first actually compile something with closure support. But I guess this should be tested to make sure I did not break it ;)
Oh. That is great news - so you are saying that the Cog VM is able to run old images, so it is safe to upgrade the VM anyway? E.g. I should be able to run Etoys as-is on Cog, even being able to save the image, until it is converted to closures?
That is a good question. I'm not sure about this (and I'm away from Squeak now and cannot check). I'm assuming that Cog will always save the image as 6505 (closures + float reordering), and that loading it back into a standard VM would then result in a 6504 image format. It might be possible to put that image all the way back into 6502 format, because it presumably would contain no actual closure bytecodes at this point. I had not actually thought about it until you asked just now.
So the short answer is no, it is probably not safe to save an Etoys image from Cog if you will ever need to run it on an old pre-closure VM. But we should try it and find out for sure.
Dave
If Cog could run an unmodified pre-closure image that would solve most compatibility issues indeed. Being able to save in the old 6502 format would be an added bonus, but might not even be necessary.
If Cog cannot do this then we need to figure out a way to cleanly install interpreter and Cog VMs side-by-side.
Neither of the Cog VMs can run pre-closure images. They depend on the closure change to allow context-to-stack mapping. I go into this in detail on my blog but essentially executing BLockContexts forces non-stack discipline (non-LIFO) for activations and requires synchronising the VM's hidden stack state with context stack state, whereas the closure scheme produces only stack discipline (LIFO) and eliminates any need to synchronise hidden stack state with context stack state.
cheers Eliot
- Bert -
On Wednesday 06 Oct 2010 10:16:35 pm Bert Freudenberg wrote:
If Cog cannot do this then we need to figure out a way to cleanly install interpreter and Cog VMs side-by-side.
This should not be a problem for Unix ports. The squeak.sh can be patched to detect VM image format and then launch the right VM for it. The magic file is attached. e.g.
$ file -m squeak.magic *.image Squeak3.9-7067.image: Squeak Image Classic Squeak4.2-10548-alpha.image: Squeak Image Standard
Subbu
On Wed, Oct 6, 2010 at 10:58 AM, K. K. Subramaniam kksubbu.ml@gmail.comwrote:
On Wednesday 06 Oct 2010 10:16:35 pm Bert Freudenberg wrote:
If Cog cannot do this then we need to figure out a way to cleanly install interpreter and Cog VMs side-by-side.
This should not be a problem for Unix ports. The squeak.sh can be patched to detect VM image format and then launch the right VM for it. The magic file is attached. e.g.
$ file -m squeak.magic *.image Squeak3.9-7067.image: Squeak Image Classic Squeak4.2-10548-alpha.image: Squeak Image Standard
Subbu
BTW, the following might be useful too, which dumps much of the image header (at least on linux & Mac OS X): #!/bin/sh if [ ( $# -eq 0 ) -o ( "$1" = "--help" ) -o ( "$1" = "-?" ) ]; then echo "usage: $0 imagefile [imagefile...]" exit 0 fi if [ $# -eq 1 ]; then hexdump -n 44 -e \ '"version : %u\n" "header sz: %u\n" "data size: %u\n" "base addr: 0x%08x\n" "soa oop : 0x%08x\n" "last hash: 0x%08x\n" 1/2 "window y: %d" 1/2 " x: %d" 1/4 " flags (full screen,float order...): 0x%x\n" "extra mem: %u\n" 1/2 "n stk pg : %u\n" 1/2 "code kb : %u\n" "eden byt : %u\n"' "$1" exit 0 fi for f in "$@"; do echo hexdump -n 44 -e \ ""$f: version : %u\n" "$f: header sz: %u\n" "$f: data size: %u\n" "$f: base addr: 0x%08x\n" "$f: soa oop : 0x%08x\n" "$f: last hash: 0x%08x\n" 1/2 "$f: window y: %d" 1/2 " x: %d" 1/4 " flags (full screen,float order...): 0x%x\n" "$f: extra mem: %u\n" 1/2 "$f: n stk pg : %u\n" 1/2 "$f: code kb : %u\n" "$f: eden byt : %u\n"" "$f" done
Eliot,
Very neat. Thanks.
It also needs to check for magic at offset 512 and recover from big endian encoding. See comment for readImageFromFileHe.... method in VMMaker:
--- This method detects when the image was stored on a machine with the opposite byte ordering from this machine and swaps the bytes automatically. Furthermore, it allows the header information to start 512 bytes into the file, since some file transfer programs for the Macintosh apparently prepend a Mac- specific header of this size. Note that this same 512 bytes of prefix area could also be used to store an exec command on Unix systems, allowing one to launch Smalltalk by invoking the image name as a command. ---
Subbu
vm-dev@lists.squeakfoundation.org
-
Bert Freudenberg -
David T. Lewis -
Eliot Miranda -
Igor Stasenko -
K. K. Subramaniam -
squeak-dev-noreply＠lists.squeakfoundation.org