Vm-dev

vm-dev@lists.squeakfoundation.org

10 participants
17027 discussions

VM timers in a bit of a mess
by Tim Rowledge 14 Jun '05

14 Jun '05

For some reason I just had to take a look a some of the timer code and spotted some niggles. In sq.h the three timer functions (ioMSecs, ioLowResMSecs and iMicroMSecs) are declared and then macros made to allow for a plain clock.h based version. However, sometime back ioMicroMSecs() got added to the VM proxy structure in sqVirtualMachine. I think this means you can't actually compile unless there is a 'real' implementation of ioMicroMSecs() somewhere - could be wrong but that appears to be the case. At the very least the comment in sq.h is wrong about ioMicroMSecs only being used in gc code and needs correcting. I imagine getting rid of the related macro would be sensible too. So far as I can see we all have concrete implementations of ioMicroMSecs() sitting around, though they vary a bit RISC OS returns the sqInt of the millisecond count Mac returns int of some mangling of microsec values unix returns sqInt of mangled gettimeofday() result win32 returns int of a tick count & 0x3FFFFFFF (similar with ioMSecs) In the (I think recently added) GC statistics, when we return the various times an assumption is made about the time values being SmallInt values, which isn't neccessarily true for all but win32. Note that ioMSecs() is always seen in interp.c along with '&MillisecondClockMask' to make sure it is SmallInt safe. We could either require ioMicroMSecs to be moduloed down or add the guard in the Interpreter code. Likewise we could have ioMSecs do a modulo and remove the code in Interpreter. Thoughts on cleaning this up? tim -- Tim Rowledge, tim(a)rowledge.org, http://www.rowledge.org/tim Computing Dictionary: Recursive: (see Recursive)

1 0

Re: Unix [Mac OS] VM config options?
by Brent Fulgham 09 Jun '05

09 Jun '05

--- goran(a)krampe.se wrote: > ... I was wondering - is there a documented list > of all configure options when building? In the same vein, I was curious if anyone had been building the current VM under Mac OS 10.4.1? I got some compiler errors when I tried last week, but was not sure if this was due to ongoing work on the VM. -Brent

4 4

fetchWordofObject
by John M McIntosh 07 Jun '05

07 Jun '05

/* This message is deprecated. Use fetchLong32, fetchLong64 or fetchPointer */ sqInt fetchWordofObject(sqInt fieldIndex, sqInt oop) { error("deprecated -- do not use -- and then delete me"); } Yes, but Croquet invokes the FFI plugin which invokes fetchWordofObject, and dies... Try Croquet and a mac carbon 3.8.8b2 VM and see what happens. I'll further note the suggestion to use fetchLong32... is not possible because I don't see fetchLong32 as an entry in the sqVirtualMachine.h So I'd suggest we change fetchWordofObject back to the original logic, to avoid toasting all the plugins that use it? -- ======================================================================== === John M. McIntosh <johnmci(a)smalltalkconsulting.com> 1-800-477-2659 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== ===

3 13

Unix VM config options?
by goran＠krampe.se 07 Jun '05

07 Jun '05

Hi guys an Ian especially! :) I am maintaining a "squeak" and a "squeak-beta" (VM) module for Lunar Linux (a source distro) and I was wondering - is there a documented list of all configure options when building? I am aware of the 64/32 option of course (and my Lunar module now properly asks about that when building) but I was wondering about the rest? Of course I could dig into configure etc and try to figure this out but... ;) regards, Göran

1 0

64bit VM latest
by Tim Rowledge 24 May '05

24 May '05

After much hacking I have a new alpha of VMMaker for 64 bit clean code. I found a quite long list of changes neccessary, some of which might be controversial. Before I release this widely I'd like to see if we can all agree with the changes. Image fileins: I've cut down all the 'normal' image changes to a bare minimum for release into the 3.8 update stream. See attached VMM38-64bit-imageUpdates/1/cs. I have _not_ done anything with any of the image code that is not directly related to making VMMaker work. SystemTracer etc were left alone. I also made a 'remove all leftover VMM code' file to get the few remaining vmm related methods out of the main image. See RemoveLeftoverVMMbits-38b4/1/cs. VMMaker changes: Added GCInstrumentJMMImprovementsAR64bit.3.cs - needs several int->sqInt changes - adds several prims using named-in-no-module approach - removes a couple of instvars and classvars - alters prim table to correct a couple of missed obsoletes Clean out new gc & instrumentation code to changeset. SystemDictionary stuff to new fileout for update stream (VMM38-gc-instrument-image). LowSpaceAndInterruptHandler-2-dtl needs to go into update stream. VMMLowSpace- dtl-tpr-2 is corrected VM changes, included in VMM. Lowspace problem in initializeMemoryFirstFree: which steals memory just when we are short of it. I recommend setting the lowspacethreshold to 3-4Mb instead of 200kb. We could do some tweaking in the VM code to help but when you're down to 200kb things are going to get ugly no matter what. Virtual memory might stay execution a while but even that will run out sometime. add fix for FloatArrat>primitiveDivFloatArray quick-0 test. fix primVMParameters to merge GC changes and wordSize revert #isIntegerValue: to use EOR trick - works for 32 & 64 bit machine, I think. Explicit range check will not work for actual 64bits. fix storeInteger:ofObject:withValue: to skip root oop check and many storePointerUnchecked where SmallInt is used, save checking for young. (several %speedup) make pop:thenPushIntegerOf:, check sends of pop: and pushInteger: add an instantiate but don't fill, used for small classes like points and the one that will annoy Ian, macros for longAtput etc in sqMemoryAccess.h This proved neccessary after John & I discovered a noticable drop in benchmark results when using the inlined accessor functions. Examing the compiled code for both ARM and PPC showed that the inlining was being done (quite neatly I thought for ARM) but not quite neatly enough in some crucial cases. It was causing a 20% or so overall hit and 40% for tinyBenchmarks, a bit less for OSX. Still to do: there are lots of places where potential signed/unsigned/oop comparisons are done and might cause problems. 'oop < endOfMemory' is pretty common in the interp.c file. There is a stricly temporary .mcz file on my website that should be readable as http://www.rowledge.org/tim/squeak/SqFiles/release/VMMaker-tpr.27.mcz and I should have the changed sqMemoryAccess.h file committed sometime today. tim -- Tim Rowledge, tim(a)rowledge.org, http://www.rowledge.org/tim Useful Latin Phrases:- Romani quidem artem amatoriam invenerunt. = You know, the Romans invented the art of love.

2 2

sqFilenameFromString changes cause os-9 compile to fail.
by John M McIntosh 24 May '05

24 May '05

This macro was #define sqFilenameFromString(dst, src, num) \ if (1) { \ int i; \ for (i = 0; i < num; i++) { \ dst[i] = *((char *) (src + i)); \ } \ dst[num] = 0;\ } that got changed to this #define sqFilenameFromString(dst, src, num) \ do { \ int i; \ for (i= 0; i < num; ++i) \ (dst)[i]= (src)[i]; \ dst[num]= '\0'; \ } while(0) however the old codewarrior compiler errors with (dst)[i] = (src)[i] pointer/array required say in sqInt sqFileDeleteNameSize(sqInt sqFileNameIndex, sqInt sqFileNameSize) { char cFileName[1000]; This is not a problem on OS-X because we override the sqFilenameFromString in order to convert the HFS+ file name to posix UTF-8 file names. I'm wonder here if src which is sqFileNameIndex which define as sqInt which is int, then don't we really need proper casting here? -- ======================================================================== === John M. McIntosh <johnmci(a)smalltalkconsulting.com> 1-800-477-2659 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== ===

1 0

Re: lowspace signalling and handling issues
by Lex Spoon 14 May '05

14 May '05

Tim Rowledge <tim(a)rowledge.org> wrote: > Has anyone ever done any testing to see just how many passes of compacting can > be survived? If we checked the actual free space (perhaps the lowSpaceSignal > state is better) and refused to give any to the fwdBlock table would we simply > be thrashing a bit more, or doomed? Or should I just give up because nobody > can be bothered to actually think about how to do things right for a change? Oh don't stop now! If it does not go into the standard VM then let's make a Does Things Right VM. There are plenty of people who would live dangerously by using a safer dev environment. Err, you know what I mean. On this particular issue, I have long wondered why the low-space threshold is still unusably low. Fancy tricks aside, it seems like a no brainer to increase this to a few megabytes. To look to the future a little, we probably should have a second-level debugging approach which puts you in a new morphic project. It is quite possible that the reason you ran out of memory, is still going to be in effect if you stay in the same project. Two errors in a row--of any kind--should dump you into a new morphic project. It's not as nice as debugging in the original project, but sometimes it is the only way to survive. This is a general mechanism that applies to memory problems as well as plenty of others. Consider, e.g., a morph with all its variables set to nil. While we're on the subject, I'd love to see someone take a shot at the old idea of catching infinite recursions. If the stack gets deep, then pop up a debugger and ask if everything is okay. As I recall, you had some ideas for implementing this in a simple way but it never happened. Recursion is one of the ways that we run out of space, but checking for out-of-space is not the nicest way to handle it. Overall, I strongly feel that Squeak should be a *safe* language. It's a great property and it is in our roots. Right now, there are a lot of easy ways to accidentally wedge your image.... and many of them have simple solutions should anyone care to spearhead this. Cheers, -Lex

1 0

lowspace signalling and handling issues
by Tim Rowledge 04 May '05

04 May '05

After having problem trying to debug some TK4 code that blew up with lowspace problems but never let me catch and debug, I spent some time adding the lowspace-process stuff we recently discussed. I had to make a few alterations to match it up with the latest 64bit clean code but no problems with that part. After building a VM I started testing with some of the methods in SystemDictionary 'memory space' - in particular #useUpMemory. It is perhaps fortunate that I did since the other #useUp* methods pretty much work once the lowspace process is caught by the vm and passed up to the image. After a _lot_ of head scratching by John & I we found that with a gazillion tiny objects (Links are the smallest possible objects that can exist on their own, plain Objects would have to be contained in a collection and so would cost the same 3 words per object) cause a catastrophic GC explosion. What happens is that memory fills up until we get to signal lowspace and then we are in danger. Depending upon the exact size of object memory in use the 200kb used as the lowSpaceThreshold can be gobbled up in one swallow by the initializeMemoryFirstFree: method making sure there is a byte per object that survived the markPhase. In using useUpMemory we can get to having 4 bytes of free space when the next allocate is attempted.... Ka-Boom. Expanding the lowSpaceThreshold (along with the VM changes to report the process and avoid the accidental problem of interrupting eventTickler) to a couple of mb makes it ok on my machine and the threshold can be a lot lower with the other tests that create bigger (hence fewer) objects (hence smaller fwdTable needs). In the worst case, we could have a very large OM filled with very small objects all surviving markPhase; in such a case we would need an additional 1/12 of OM available for the fwdTable. So for a 30Mb objectmemory we ought to set the lowSPaceThreshold to 30/13 => 2.31Mb + actual space needed to run the notifier/debugger etc for reasonable safety. Or hide the 2.31 Mb away so the image never even knows it is there. If you are using virtual memory and a limit of 512Mb then you should perhaps secrete 40Mb some where safe. This assumes that we really need to have one byte per object of course. The original rationale was to keep the number of compact loops down to eight (see Dan's comment in initializeMemoryFirstFree:) for Alan's large demo image. The nicest solution would be to come up with a way to do our GC & compacting without needing any extra space. Commence headscratching now... John suggested making sure the fwd gets less than the byte-per-object if things are tight, and accpting the extra compaction loops. Good news- with the vm change and 2Mb lowSpaceThreshold I can probably go back and find my TK4 problem(s). Bad news- consider Tweak. With lots of processes whizzing away, merely stopping the one that did the allocation and triggered the lowspace is not going to be much good. Stopping everything except the utterly essential stuff to debug the lowspace will be needed. Probably. More bad news- somehow, going from VMM37b5 to b6 cost 40% of performance on my machine :-( Bugger. tim -- Tim Rowledge, tim(a)sumeru.stanford.edu, http://sumeru.stanford.edu/tim Every bug you find is the last one.

3 7

FloatArrayPlugin & 64bitting
by Tim Rowledge 04 May '05

04 May '05

FloatArrayPlugin doesn't feel well when we generate it from the 64bit VMMaker alpha. It has a clause:- } rcvrPtr = ((float *) (interpreterProxy->firstIndexableField(rcvr))); /* Check if any of the argument's values is zero */ argPtr = ((float *) (interpreterProxy->firstIndexableField(arg))); for (i = 0; i <= (length - 1); i += 1) { if ((longAt(argPtr + i)) == 0) { return interpreterProxy->primitiveFail(); } which upsets the typing of longAt since it expects an sqInt. I can trivially alter it to cast the rcvrPtr to an sqInt for the use of the longAt and it compiles/runs ok. I have no idea if a 64bit machine will have floats of 64 bits (ie the same as sqInt) or if they stay 32bit (ie we have to use plain int, which may upset longAt on a 64bit machine) or what. And since I have no 64bit setup, I can't say I'm massively bothered right now. But somebody probably is and if anyone cares to suggest the nicest fix I'll happily include it. tim -- Tim Rowledge, tim(a)sumeru.stanford.edu, http://sumeru.stanford.edu/tim No single raindrop believes it is to blame for the flood

3 9

Re: FloatArrayPlugin & 64bitting
by Dave Mason 30 Apr '05

30 Apr '05

>>>>> On Fri, 29 Apr 2005 16:32:09 -0400, Dave Mason <dmason(a)sarg.ryerson.ca> said: I have the source (without VMMaker because of a permission problem), and I looked at FloatArrayPlugin. The code there looks fine if the longAt is replaced with floatAt or intAt. It stores a block of floats starting at the first field in the object. However there definitely is *potential* for a problem if you had a collection of fields some of which were floats. Here's where there could be a problem on 64 bit, where sizeof(sqInt)==8 and sizeof(float)==4. Consider a block of memory: abcdefgh ijklmnop qrstuvwx 1stIndexableField returns the address of 'a'; assume the following code: sqInt argOop = oopForPointer(interpreterProxy->firstIndexableField(arg)); float* argFlo = (float *)(interpreterProxy->firstIndexableField(arg)); sqInt* argOopp = (sqInt *)(interpreterProxy->firstIndexableField(arg)); floatAt(argOop) gets abcd floatAtPointer(argFlo) gets abcd floatAtPointer(argOopp) gets abcd Fine so far, but: floatAt(argOop+1) gets bcde (or error) floatAtPointer(argFlo+1) gets efgh floatAtPointer(argOopp+1) gets ijkl floatAt(argOop+2) gets cdef (or error) floatAtPointer(argFlo+2) gets ijkl floatAtPointer(argOopp+2) gets qrst As far as I can see, abcd, ijkl, and qrst are the right answers (if they are fields containing floats), with efgh being garbage and ijkl not being the correct answer in the 3rd example. So *anywhere* you are skipping through a bunch of fields, you need to do the pointer arithmetic with sqInt*. As I said before, this is a pretty subtle error, but a quick grep through the code for 1stIndexableField shows a lot of _potential_ problems, although all in plugins, and many of them have to do with arrays, so they probably aren't. All of this may be obvious to Ian and the rest of you, but I'm trying to learn a bit about the code, and this was a useful exercise for me. ../Dave

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev