Squeak-dev May 2001

squeak-dev@lists.squeakfoundation.org

143 participants
597 discussions

RE: Multiple disjoint object memory in Smalltalk
by Raab, Andreas 29 May '01

29 May '01

John, > Now I can't speak for the comment... > And I wonder if it's fixed? > See self longAt: oop put: (header bitAnd: AllButRootBit). clears that > Root Bit everywhere. But I'd think ObjectMemory>>clearRootsTable > should do that which is called from fullGC which is called from > ObjectMemory>>primitiveSnapshot. > > So anyone know if this is still a bug? Or is it dead. I think it's still there. The problem was very hard to make happen to begin with; it was related to having a young object referenced from the context that performs the primitive snapshot (for instance changing the startup message could trigger it). For some weird reason the context invoking the primitive had the root bit set when coming up again and this lead to tons of crazy things. - Andreas

1 0

re: running directly from RAM on wince?
by Craig Latta 29 May '01

29 May '01

> How far have you gotten? It's not a five-minute thing, but it's going well. I introduced an additional preprocessor variable ("MEMORYMAPPING"), which, when true, changes the definition of sqImageFile and much of the code that uses it. I have twelve places left to fix at the moment... > Do you want to send over your changes? Yes, after I see if it works. thanks, -C -- Craig Latta composer and computer scientist craig.latta(a)netjam.org www.netjam.org crl(a)watson.ibm.com Smalltalkers do: [:it | All with: Class, (And love: it)]

1 0

RE: running directly from RAM on wince?
by Raab, Andreas 29 May '01

29 May '01

Noel, > LOL That's OK; we're on the same page now. Yup. Back to Craig's original question ;-) So as I see it you should be able to basically map everything in, then run it. The only issue remaining is that one must never quit without saving the image. In effect, the image would always be an up-to-date memory snapshot and therefore the only thing required would be to make sure that when you exit it you can start it again. It might even work without a snapshot (to an extent, the question is what happens to certain resources) but if you snapshot you should be perfectly fine. > I do note that to grown/shrink the allocation, we'd have to close it, > allocate a new mapping, and re-map. This might not even be a problem as long as CE doesn't commit mapped memory before you even start using to it (if that's not true it might be possible to play around with VirtualAlloc and see if that helps). As long as it doesn't commit the memory before it is used you can simply map a large enough region to reserve address space and keep some internal pointer that tells you where the current end of the committed region is - effectively that's what I'm doing too, only using VirtualAlloc for reserving the address space and then handing out continuous portions of it. There *is* of course a question what happens when your external storage has not enough capacity to back the entire requested region - do you have any idea what happens in this case?! Cheers, - Andreas

2 1

RE: pending file primitives (EH?)
by Raab, Andreas 29 May '01

29 May '01

Noel, > Hence the reason to find a better way to integrate memory > mapped files. And this is really why I had asked earlier > about disjoint heaps, because I see a lot of open issues here. > Apparently, my inquiry was mis-understood as a simple inquiry > into garbage collection and memory management. [snip] No it wasn't. The key issue when dealing with multiple object memories *is* garbage collection. It is because the garbage collector needs to know what objects are still alive, where it can move objects around and when and where to stop. And do all of this in real-time. I've been talking with Dan about some of this stuff recently (the idea was to see if we could provide an OM on a per project basis) and we've had some pretty cute ideas (though nothing real workable for the moment). So I suggest we'll both watch the other thread and see if somebody has an outstandingly interesting idea ;-) Cheers, - Andreas

2 1

[ENH] Hierarchical selection of Projects for 'Jump'
by JArchibald＠aol.com 29 May '01

29 May '01

All-- A bug in hierarchy ordering is corrected with this fileIn. This fileIn is complete -- previous is replaced by this. >From previous submission: => 4/3/01 12:52:07 AM EDT, JArchibald(a)aol.com => << The attached change set modifies the behavior of the menu used to navigate to other projects. The current menu presents (bottom part of menu) a list of Project names in alphabetical order. It seems to be more evocative if this section of the menu presents a hierarchical list instead. This change set provides for this functionality. ... Adjustments are made to PopUpMenu to present the same appearance in MVC projects. >> Enjoy, Jerry. ____________________________ Jerry L. Archibald systemObjectivesIncorporated ____________________________

1 0

RE: Beyond gzip
by Richard A. O'Keefe 29 May '01

29 May '01

Alan Kay <Alan.Kay(a)disney.com> wrote: A scheme like this was used in BBN Lisp in the late sixties ....d worked well given the propensity of most lists to run in the cdr direction .... BBN Lisp became Interlisp became Interlisp-D. Let's see if I can remember how Interlisp-D worked (does Medley still work this way, I wonder)? Addresses were 24 bits, addressable units were 16-bit words. Pages were 256 words (512 bytes). Tagging was BiBoP (Big Bag of Pages; use the page number to index an array of bytes and there's the 8-bit tag). List cells had the form +--------+------------------------+ |CDR F| CAR | +--------+------------------------+ 8 bits 24 bits Note that this is 2 words (32 bits), so that a list-cell-within-page address must be even. I think there were four cases: (CDR,F) = (0,x) CDR is NIL (CDR,F) = (0,~x) whole cell has been redirected to CAR (CDR,F) = (x,0) CDR is cell 2x in this page (CDR,F) = (x,1) cell 2x in this page holds pointer to CDR. I'm not terribly sure about the first two any more. On a machine with 4MB of physical memory, 32MB of virtual memory, and (if you were terribly terribly lucky) 80MB of disc, this made it possible to do an amazing amount of useful work on what was basically a 16-bit machine. CAR and CDR were done in microcode, and were sufficiently complex that they were done once. Xerox Quintus Prolog, on the same hardware, used +--------+------------------------+ |car tag | car word ptr | |cdr tag | cdr word ptr | +--------+------------------------+ for list cells (except for tag location, pretty much obvious). Car and cdr were so simple that they were replicated about a dozen times in various microinstructions, and XQP ended up running list processing code about 5 times as fast as Interlisp-D (my measurements). In short, CDR-coding and similar tricks are great for compression, but they don't come free. Interlisp-D: multiple threads of control, user defined data types, a fast straightforward window system, fully networked, 16-bit characters and virtual keyboards, it was 1984 and I didn't know how lucky I was.

1 0

RE: running directly from RAM on wince?
by Raab, Andreas 29 May '01

29 May '01

Noel, > > [You] could be pretty much okay; all that's needed is to > > "bridge the gap" between old and young space during GC > > What gap? There is no gap. The whole point is to give > Squeak exactly what it expects: a single, contiguous, allocated > region for the object space. Sorry, I misunderstood you here. I thought you wanted to map the image file but then run out of heap space. I wasn't aware that you were trying to run the entire thing out of the memory mapped portion. > You might want to check the 3rd link I put in the prior > message. You'll find a layout of the WinCE memory > architecture; pictures and everything. Yup. Very useful stuff (so let's recite it :-) http://msdn.microsoft.com/library/periodic/period98/memory.htm BTW, one thing that got me scratching my head was the following: "Memory-mapped files that are backed up by named files currently are supported only for read-only access. Unnamed memory-mapped files-those typically backed up by the page file-are supported. This is a little strange since Windows CE does not support a page file." A little??? Oh well. I interpret the first part basically as you cannot write to pages of files mapped from the disk. Which in turn would imply that you can't use it for mapping the Squeak image in. Or is this outdated information?! Cheers, - Andreas

1 0

RE: running directly from RAM on wince?
by Raab, Andreas 29 May '01

29 May '01

Hi Noel, > So now consider the alternative, intended as an OPTION for suitable > limited-memory devices: memory mapped files. A 6 MB image in > the internal store takes up whatever compressed space in the > object store is necessary to hold it. A 6 MB image stored > externally takes up zero memory from my device. > What about the heap? At run-time, the address space necessary to > map that file is separate from the 32 MB allotment given to > the program, and the RAM used to hold those pages is managed > by the OS depending upon available memory. Just for clarification, do I understand you correctly, that want to use memory mapped files for basically providing a swap space and an address space extension?! In this case, your key problem will be where exactly the file gets mapped into memory - there is a fundamental assumption built into the VM that old space comes before young space. As long as this happens in the "right" order (e.g., the mapped pages come before the heap) you could be pretty much okay; all that's needed is to "bridge the gap" between old and young space during GC (which is not exactly easy but certainly doable; you could for instance consider making the mapped files what VisualWorks calls "perm space" and simply never do a GC on it). BTW, one thing that sounds strange is that CE only gives you a 32MB address space. Is this still true?! I would've guessed that if there's handling of memory mapped files and all that stuff it should be beyound the 32MB barrier by now... Cheers, - Andreas

2 1

Re: Multiple disjoint object memory in Smalltalk
by Lex Spoon 29 May '01

29 May '01

woods(a)weird.com (Greg A. Woods) wrote: > [ On Sunday, May 27, 2001 at 12:08:54 (-0500), Lex Spoon wrote: ] > > Subject: Re: Multiple disjoint object memory in Smalltalk > > > > Let it not pass, that being able to use multiple heaps also allows for a > > very easy-to-implement grow/shrink mechanism. The current mechanism is > > pretty hard to implement on Unix; actually, I don't really have a clue > > how to do it at all without using Linux-specific extensions. > > Since you don't say what Linux-only features you have in mind, won't > mmap() do (maybe not with ideal efficiency, but...)? > I made a stab at it based on mmap and munmap, but I ran into problems. Am I missing something? An initial problem is that on many Unices the default is to mmap at the *end* of the address space, so that there is no room to grow segments. My solution to this was to mmap as much as I could to begin with; grow becomes trivial, and shrink involves an munmap followed by an mmap of the end of the space -- thus releasing some pages of memory used while hanging on to the relevant region of the address space. A new problem arises, though: occasionally the second mmap() fails and I end up with a hole in the middle of the heap. The rules that Linux (never mind any other Unix) has for how much phony memory I can allocate are opaque to me, so this error to happen randomly. Maybe when this happens it would work to deallocate pages at the end of the mmap-ed region, and to try the mmap() again. I dunno, it also might *not* work, and who knows what will happen on the next Unix? This approach really seems to push the limits. One Linux-specific feature of interest is mremap(). Or maybe not -- it still seems to have problems. Alternatively, an excellent Linux solution would be to replace malloc() with something that allocates at the end of the address space. Then, the perfectly suited brk() is made available for Squeak's heap. Unfortunately, the "replace malloc" part is going to be a little different on different Unices. To contrast all of this discussion, if Squeak had an ObjectMemory that could manage multiple chunks of memory, then you could use plain-old, extremely-portable malloc() and free(). Lex

1 0

Unix release (was: Re: Best way to deal with OS events?)
by Bert Freudenberg 29 May '01

29 May '01

On Mon, 28 May 2001, Kevin Fisher wrote: > Speaking of the official UNIX source, there have been a lot of patches > and fixes floating around the list the past couple of months (like the > serial fixes from last week, for example). Does anyone know when a > new 'official' UNIX tarball with all the patches and goodies might > appear? *sigh* AWGTHTGTTA? The last official unix release is 2.7, dated January 2000. Go bug Ian Piumarta about blessing a 3.0 VM (or even 2.8, considering the Stable Squeak discussion). There isn't a single place where all the patches are collected and integrated. The best one currently is Lex Spoon's http://www.cc.gatech.edu/~lex/squeak/ Squeak.sourceforge.net was set up by members of this list to make collaboration on VM issues easier, on the occasion of the delayed 2.8 unix VM. However, the main VM porters (of all platforms) were hesitant to take the official sources to the public CVS. Andreas for Win32 at least tried, but found the CVS tools for his platform insufficient. We haven't touched the CVS for almost a year now. Maybe we should just import Ian's official 3.0pre2 sources as Vendor Branch and work from that? http://cvsbook.red-bean.com/cvsbook.html#Tracking_Third-Party_Sources__Vend… This would allow us to be relatively independent of when a new release comes out, and it would clearly mark the distinction between official sources and community patches. Comments are welcome, volunteers even more so. -- Bert

2 1

← Newer
1
...
5
6
7
8
9
10
11
...
60
Older →

Jump to page: