I tend to switch back and forth between a variety of VMs and images (Spur, Cog, interpreter VM etc). Lately I have been doing my updates from a format 68002 64-bit image. Just curious, is anybody else out there using a 64-bit image on a regular basis and keeping it updated from the trunk development stream?
Dave
On 16 July 2014 04:27, David T. Lewis lewis@mail.msen.com wrote:
I tend to switch back and forth between a variety of VMs and images (Spur, Cog, interpreter VM etc). Lately I have been doing my updates from a format 68002 64-bit image. Just curious, is anybody else out there using a 64-bit image on a regular basis and keeping it updated from the trunk development stream?
I'm not, and I really should, in CI. What would I need to do to set up such a thing?
frank
Dave
On Wed, Jul 16, 2014 at 09:14:20AM +0100, Frank Shearar wrote:
On 16 July 2014 04:27, David T. Lewis lewis@mail.msen.com wrote:
I tend to switch back and forth between a variety of VMs and images (Spur, Cog, interpreter VM etc). Lately I have been doing my updates from a format 68002 64-bit image. Just curious, is anybody else out there using a 64-bit image on a regular basis and keeping it updated from the trunk development stream?
I'm not, and I really should, in CI. What would I need to do to set up such a thing?
I was just mostly curious if anyone else is just using a 64 bit image on a day to day basis.
As far as CI jobs on build.squeak.org, we would need to install the squeakvm64 VM on box3. I put the deb in the expected place but it is not installed.
/root/localdebs/squeakvm64_20131020-1_i386.deb
Note that this deb is *only* for use on box3. If you want to run the 64-bit image yourself, please use the official distribution on squeakvm.org/unix.
The trunk image is regularly being traced to 64 bits in the CI job at http://build.squeak.org/job/Squeak%2064-bit%20image/ so the output of that job should be usable.
Dave
On 16 July 2014 12:48, David T. Lewis lewis@mail.msen.com wrote:
On Wed, Jul 16, 2014 at 09:14:20AM +0100, Frank Shearar wrote:
On 16 July 2014 04:27, David T. Lewis lewis@mail.msen.com wrote:
I tend to switch back and forth between a variety of VMs and images (Spur, Cog, interpreter VM etc). Lately I have been doing my updates from a format 68002 64-bit image. Just curious, is anybody else out there using a 64-bit image on a regular basis and keeping it updated from the trunk development stream?
I'm not, and I really should, in CI. What would I need to do to set up such a thing?
I was just mostly curious if anyone else is just using a 64 bit image on a day to day basis.
As far as CI jobs on build.squeak.org, we would need to install the squeakvm64 VM on box3. I put the deb in the expected place but it is not installed.
/root/localdebs/squeakvm64_20131020-1_i386.deb
Note that this deb is *only* for use on box3. If you want to run the 64-bit image yourself, please use the official distribution on squeakvm.org/unix.
I'd probably build the 64-bit image from source: it makes things easier for the other build agents, which (a) aren't on box3 and (b) aren't always Linux. He says, knowing that all non-Linux build agents are currently off-line...
The CI build scripts behind SqueakTrunk, ExternalPackages and so on already build 32-bit interpreter VMs from source, so extending the process to 64 should be easy.
The trunk image is regularly being traced to 64 bits in the CI job at http://build.squeak.org/job/Squeak%2064-bit%20image/ so the output of that job should be usable.
Ah, handy. So I could build a 64 bit Interpreter from source, and just pull in the output of the above job as the image to use?
frank
Dave
On Wed, Jul 16, 2014 at 01:02:55PM +0100, Frank Shearar wrote:
On 16 July 2014 12:48, David T. Lewis lewis@mail.msen.com wrote:
On Wed, Jul 16, 2014 at 09:14:20AM +0100, Frank Shearar wrote:
On 16 July 2014 04:27, David T. Lewis lewis@mail.msen.com wrote:
I tend to switch back and forth between a variety of VMs and images (Spur, Cog, interpreter VM etc). Lately I have been doing my updates from a format 68002 64-bit image. Just curious, is anybody else out there using a 64-bit image on a regular basis and keeping it updated from the trunk development stream?
I'm not, and I really should, in CI. What would I need to do to set up such a thing?
I was just mostly curious if anyone else is just using a 64 bit image on a day to day basis.
As far as CI jobs on build.squeak.org, we would need to install the squeakvm64 VM on box3. I put the deb in the expected place but it is not installed.
/root/localdebs/squeakvm64_20131020-1_i386.deb
Note that this deb is *only* for use on box3. If you want to run the 64-bit image yourself, please use the official distribution on squeakvm.org/unix.
I'd probably build the 64-bit image from source: it makes things easier for the other build agents, which (a) aren't on box3 and (b) aren't always Linux. He says, knowing that all non-Linux build agents are currently off-line...
The CI build scripts behind SqueakTrunk, ExternalPackages and so on already build 32-bit interpreter VMs from source, so extending the process to 64 should be easy.
There's nothing really to build, just use a SystemTracer to convert whichever image you want to start with into 68002 64-bit format. You can use the existing CI job for reference, but that's really all there is to it. This is similar to what you're doing when you convert an image to the new Spur format (image format 6521 for 32-bit Spur, and image format 68019 for 64-bit Spur).
The trunk image is regularly being traced to 64 bits in the CI job at http://build.squeak.org/job/Squeak%2064-bit%20image/ so the output of that job should be usable.
Ah, handy. So I could build a 64 bit Interpreter from source, and just pull in the output of the above job as the image to use?
Yes, just pass the --image64 parameter to the unix/cmake/configure script, and you will have VM for 64-bit images. If you install it along with the normal squeakvm VM, the /usr/local/bin/squeak script figures out the right VM executable to use based on image format.
Dave
On 16-07-2014 00:27, David T. Lewis wrote:
It works but:
a) Seems to be slow (although I'm used to the cogvm which is faster than SqueakVM) b) Somethings just itch my ears like: b.1) Compiler recompileAll fails (and shouldn't) b.2) Smalltalk condenseChanges that works for 32bit images work, fails in 64bit image.
b.2 is not that important but b.1 shows that there is code that cannot be re-compiled and it's not a good sign.
(using trunk updated 64bit image and 4.10.2-2614_64bit SqueakVM)
I tend to switch back and forth between a variety of VMs and images (Spur, Cog, interpreter VM etc). Lately I have been doing my updates from a format 68002 64-bit image. Just curious, is anybody else out there using a 64-bit image on a regular basis and keeping it updated from the trunk development stream?
Dave
On Fri, Jul 18, 2014 at 04:01:45PM -0300, Casimiro de Almeida Barreto wrote:
On 16-07-2014 00:27, David T. Lewis wrote:
(using trunk updated 64bit image and 4.10.2-2614_64bit SqueakVM)
I tend to switch back and forth between a variety of VMs and images (Spur, Cog, interpreter VM etc). Lately I have been doing my updates from a format 68002 64-bit image. Just curious, is anybody else out there using a 64-bit image on a regular basis and keeping it updated from the trunk development stream?
It works but:
a) Seems to be slow (although I'm used to the cogvm which is faster than SqueakVM)
It is definitely slower than Cog. I recall from a few years ago (using whatever PC I had at the time) that the 64-bit image ran slower than the 32-bit image. However, on the PC that I am using now, the 64-bit image is somewhat faster than the 32-bit image. I cannot explain the difference. I expect the 64-bit image to be slower due to various inefficiencies in the use of sqInt as a default data type throughout the VM (which is a 64 bit integer when the VM is compiled for the 64-bit image). Maybe some CPUs handle this difference more efficiently than others, I really don't know.
b) Somethings just itch my ears like: b.1) Compiler recompileAll fails (and shouldn't) b.2) Smalltalk condenseChanges that works for 32bit images work, fails in 64bit image.
Running the full unit test suite will also lead to some unpleasantness.
b.2 is not that important but b.1 shows that there is code that cannot be re-compiled and it's not a good sign.
That is not good, and it might be worthwhile to find the cause.
Dave
On Fri, Jul 18, 2014 at 04:01:45PM -0300, Casimiro de Almeida Barreto wrote:
On 16-07-2014 00:27, David T. Lewis wrote:
I tend to switch back and forth between a variety of VMs and images (Spur, Cog, interpreter VM etc). Lately I have been doing my updates from a format 68002 64-bit image. Just curious, is anybody else out there using a 64-bit image on a regular basis and keeping it updated from the trunk development stream?
It works but:
a) Seems to be slow (although I'm used to the cogvm which is faster than SqueakVM) b) Somethings just itch my ears like: b.1) Compiler recompileAll fails (and shouldn't) b.2) Smalltalk condenseChanges that works for 32bit images work, fails in 64bit image.
b.2 is not that important but b.1 shows that there is code that cannot be re-compiled and it's not a good sign.
(using trunk updated 64bit image and 4.10.2-2614_64bit SqueakVM)
I fixed a small but nasty bug in one of the primitives that appears to have been the cause of quite a few problems:
http://lists.squeakfoundation.org/pipermail/vm-dev/2014-September/016443.htm...
With freshly compiled VM, Compiler recompileAll and Smalltalk condenseChanges are working again in a 64-bit image.
Dave
2014-09-07 16:21 GMT+02:00 David T. Lewis lewis@mail.msen.com:
On Fri, Jul 18, 2014 at 04:01:45PM -0300, Casimiro de Almeida Barreto wrote:
On 16-07-2014 00:27, David T. Lewis wrote:
I tend to switch back and forth between a variety of VMs and images
(Spur, Cog,
interpreter VM etc). Lately I have been doing my updates from a format
68002
64-bit image. Just curious, is anybody else out there using a 64-bit
image on
a regular basis and keeping it updated from the trunk development
stream?
It works but:
a) Seems to be slow (although I'm used to the cogvm which is faster than SqueakVM) b) Somethings just itch my ears like: b.1) Compiler recompileAll fails (and shouldn't) b.2) Smalltalk condenseChanges that works for 32bit images work, fails in 64bit image.
b.2 is not that important but b.1 shows that there is code that cannot be re-compiled and it's not a good sign.
(using trunk updated 64bit image and 4.10.2-2614_64bit SqueakVM)
I fixed a small but nasty bug in one of the primitives that appears to have been the cause of quite a few problems:
http://lists.squeakfoundation.org/pipermail/vm-dev/2014-September/016443.htm...
With freshly compiled VM, Compiler recompileAll and Smalltalk condenseChanges are working again in a 64-bit image.
Dave
Ah, the beauty of C signed arithmetic model... We should try to avoid it as frequently as we can.
On 08.09.2014, at 08:35, Nicolas Cellier nicolas.cellier.aka.nice@gmail.com wrote:
Ah, the beauty of C signed arithmetic model... We should try to avoid it as frequently as we can.
Indeed. Not just C though. In JavaScript I discovered that (42 << 32) == 42. Seriously.
- Bert -
On 9/8/14, Bert Freudenberg bert@freudenbergs.de wrote:
On 08.09.2014, at 08:35, Nicolas Cellier nicolas.cellier.aka.nice@gmail.com wrote:
Ah, the beauty of C signed arithmetic model... We should try to avoid it as frequently as we can.
Indeed. Not just C though. In JavaScript I discovered that (42 << 32) == 42.
Firefox version 27 Menu Tools / Web Developer / WebConsole
paste (42 << 32) == 42. into command line CONFIRMED.
Thanks, Bert, for this info.
Seriously.
- Bert -
Thanks David for the fix. Very nice that the Smalltalk primitive can just fail when the shift amount is too large.
Bert Freudenberg wrote:
Indeed. Not just C though. In JavaScript I discovered that (42 << 32) == 42.
Thanks Bert.
Also confirmed on Google Chrome 36.0 (V8 3.26) and Node v0.10.
Right shift is the same (42 >> 32) == 42. And (42 >> 35) == 5
Did you have to work around it when designing SqueakJS ? My guess is not: I guess that the SqueakJS VM is a purely 32 bit machine, and that this behaviour won't affect Smalltalk programs.
Have fun! David
p.s.
I wonder if this is a restriction imposed by the language standard, or is merely a side effect of a common implementation choice. The MDN reference page says "Bitwise operators treat their operands as a sequence of 32 bits". Which, with hindsight, leaves shifts of more than 31 bits undefined. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/...
However, my naive reading of this paragraph of the ECMAScript 5 standard suggests that wrapping around a bit shift is just plain wrong, and that it should yield 0 or fail (until a 64 bit Javascript standard is written) http://www.ecma-international.org/ecma-262/5.1/#sec-11.7
On 08.09.2014, at 16:45, David Corking lists@dcorking.com wrote:
Bert Freudenberg wrote:
Indeed. Not just C though. In JavaScript I discovered that (42 << 32) == 42.
Thanks Bert.
Also confirmed on Google Chrome 36.0 (V8 3.26) and Node v0.10.
Right shift is the same (42 >> 32) == 42. And (42 >> 35) == 5
It's in the ECMAScript standard. At least it's the same everywhere.
Did you have to work around it when designing SqueakJS ?
Yes: https://github.com/bertfreudenberg/SqueakJS/commit/6d694d0733
My guess is not: I guess that the SqueakJS VM is a purely 32 bit machine, and that this behaviour won't affect Smalltalk programs.
The VM needs to provide the bitShift primitive for SmallIntegers. That is, it needs to return a correct result if that fits into a SmallInt, and fail otherwise (it could also generate a LargeInt, but doesn't have to).
In Squeak, the single "bitShift:" methods shifts both left and right depending on the argument's sign, whereas in most other languages, you cannot shift by a negative amount. So there is one thing to watch out when implementing a VM.
I'm not really sure what happens in C, but in JS you get this nicety when shifting by a negative number:
16 << -8 == 268435456
So the VM needs to handle negative shifts separately anyway, and in JS it needs to special-case shifts larger than 31, too.
Have fun! David
p.s.
I wonder if this is a restriction imposed by the language standard, or is merely a side effect of a common implementation choice. The MDN reference page says "Bitwise operators treat their operands as a sequence of 32 bits". Which, with hindsight, leaves shifts of more than 31 bits undefined. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/...
They're well defined, they just operate on 32 bit signed values:
1<<31 == -2147483648
That is, the result of shifts and bitwise operators is always done in 32 bits. But there's no direct way to detect overflow (even though other JS integer operations are accurate to 53 bits [because there are no actual integers]).
However, my naive reading of this paragraph of the ECMAScript 5 standard suggests that wrapping around a bit shift is just plain wrong, and that it should yield 0 or fail (until a 64 bit Javascript standard is written) http://www.ecma-international.org/ecma-262/5.1/#sec-11.7
Marvel at 11.7.1 step 7:
• Let shiftCount be the result of masking out all but the least significant 5 bits of rnum, that is, compute rnum & 0x1F.
- Bert -
On 08.09.2014, at 18:28, Bert Freudenberg bert@freudenbergs.de wrote:
On 08.09.2014, at 16:45, David Corking lists@dcorking.com wrote:
Did you have to work around it when designing SqueakJS ?
Yes: https://github.com/bertfreudenberg/SqueakJS/commit/6d694d0733
Yikes. I just spotted another bug in that function. Can you see it, too?
(SmallIntegerTests did not uncover that bug, they all pass. Most tests assume a correct VM)
(IntegerTest and IntegerDigitLogicTest contain too many brute-force tests, which time out on a slow VM. SqueakJS passes 44 of 60)
(SqueakJS could use testers. But it's not much fun, opening a TestRunner in 4.5 takes like 30 seconds)
- Bert -
On 08.09.2014, at 18:28, Bert Freudenberg bert@freudenbergs.de wrote:
On 08.09.2014, at 16:45, David Corking lists@dcorking.com wrote:
Did you have to work around it when designing SqueakJS ?
Yes: https://github.com/bertfreudenberg/SqueakJS/commit/6d694d0733
Yikes. I just spotted another bug in that function. Can you see it, too?
I think it should be using an unsigned right shift like this:
if ((shifted>>>shiftCount) === smallInt) return shifted;
Dave
(SmallIntegerTests did not uncover that bug, they all pass. Most tests assume a correct VM)
(IntegerTest and IntegerDigitLogicTest contain too many brute-force tests, which time out on a slow VM. SqueakJS passes 44 of 60)
(SqueakJS could use testers. But it's not much fun, opening a TestRunner in 4.5 takes like 30 seconds)
- Bert -
On 08.09.2014, at 19:44, David T. Lewis lewis@mail.msen.com wrote:
On 08.09.2014, at 18:28, Bert Freudenberg bert@freudenbergs.de wrote:
https://github.com/bertfreudenberg/SqueakJS/commit/6d694d0733
Yikes. I just spotted another bug in that function. Can you see it, too?
I think it should be using an unsigned right shift like this:
if ((shifted>>>shiftCount) === smallInt) return shifted;
Dave
No, "bitShift:" is supposed to be signed. This is my test if the result still fits in 32 bits, by reversing the shift. I *think* that part is correct ;)
E.g. -55 << 5 ==> -1760 -1760 >> 5 ==> -55
meaning we can use the result but
-555555555 << 5 ==> -597908576 -597908576 >> 5 ==> -18684643
so we cannot use the result because bits got dropped.
- Bert -
On Mon, Sep 08, 2014 at 08:35:55AM +0200, Nicolas Cellier wrote:
2014-09-07 16:21 GMT+02:00 David T. Lewis lewis@mail.msen.com:
On Fri, Jul 18, 2014 at 04:01:45PM -0300, Casimiro de Almeida Barreto wrote:
On 16-07-2014 00:27, David T. Lewis wrote:
I tend to switch back and forth between a variety of VMs and images
(Spur, Cog,
interpreter VM etc). Lately I have been doing my updates from a
format
68002
64-bit image. Just curious, is anybody else out there using a 64-bit
image on
a regular basis and keeping it updated from the trunk development
stream?
It works but:
a) Seems to be slow (although I'm used to the cogvm which is faster
than
SqueakVM) b) Somethings just itch my ears like: b.1) Compiler recompileAll fails (and shouldn't) b.2) Smalltalk condenseChanges that works for 32bit images work, fails in 64bit image.
b.2 is not that important but b.1 shows that there is code that cannot be re-compiled and it's not a good sign.
(using trunk updated 64bit image and 4.10.2-2614_64bit SqueakVM)
I fixed a small but nasty bug in one of the primitives that appears to
have
been the cause of quite a few problems:
http://lists.squeakfoundation.org/pipermail/vm-dev/2014-September/016443.htm...
With freshly compiled VM, Compiler recompileAll and Smalltalk condenseChanges are working again in a 64-bit image.
Dave
Ah, the beauty of C signed arithmetic model... We should try to avoid it as frequently as we can.
Actually, in this case it was not related to signed arithmetic. It was a word size issue involving a variable that took the default sqInt declaration (which may be 64 bits) when it should have been explicitly declared as a 32 bit variable in order to match the logic elsewhere in the primitive. The shift operation itself was working fine, but the primitive was trimming the result to 32 bits without failing the primitive.
Arguably the primitive should be updated to use a 64 bit variable so that larger shift amounts will succeed. But I don't know if this might have performance implications on various platforms, so I did not make that change.
Dave
squeak-dev@lists.squeakfoundation.org
-
Bert Freudenberg -
Casimiro de Almeida Barreto -
David Corking -
David T. Lewis -
Frank Shearar -
H. Hirzel -
Nicolas Cellier