<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"><html><head><meta content="text/html;charset=UTF-8" http-equiv="Content-Type"></head><body ><div style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;"><div>Hi Marcel,<br></div><div><br></div><div>Thanks for the reply.<br></div><div><br></div><div>I have no clue on what happens regarding memory/disk. That the entire Smalltalk image runs in memory is a completely new concept to me.<br></div><div><br></div><div>My perception was that the image was "storage and initial state" and , like a db, the stuff resides on disk to be referenced and pulled into memory when needed.<br></div><div><br></div><div>Honestly, I have no clue.<br></div><div><br></div><div>(R , the most important statistics app on the planet is image based too. I wonder if that is pulled into memory too)</div><div><br></div><div>What I am going to do is load the 75Gb into the image as objects that wrap XMLDocument (the file has many, many XMLDocuments in it) and see what happens.<br></div><div>If that fails, I will go the PostgresDB route and store the data there and build objects on the fly from database calls.<br></div><div><br></div><div>I was really hoping to deal with just objects as the geek in me thinks that would be totally cool.<br></div><div><br></div><div>Anyhoo, it will be a fun experiment.<br></div><div><br></div><div>I will be sidelined for several days. I have a job interview--pure smalltalk--with Dolphin, VisualWorks(?) and a port to pharo coming up and I need to install and study Dolphin and do the mooc on pharo in prep.<br></div><div><br></div><div><br></div><div>cordially,<br></div><div><br></div><div>t</div><div><br></div><div class="zmail_extra_hr" style="border-top: 1px solid rgb(204, 204, 204); height: 0px; margin-top: 10px; margin-bottom: 10px; line-height: 0px;"><br></div><div class="zmail_extra" data-zbluepencil-ignore="true"><div><br></div><div id="Zm-_Id_-Sgn1">---- On Mon, 18 Oct 2021 11:14:44 -0400 <b>Marcel Taeumel <marcel.taeumel@hpi.de></b> wrote ----<br></div><div><br></div><blockquote style="margin: 0px;"><div><div id="x_-2061953719__MailbirdStyleContent" style="font-size : 10pt; font-family : Arial; color : #000000; text-align : left;" dir="ltr"><div>Hi Timothy --<br></div><div><br></div><div>I am not aware of basic support for automatic paging/swapping of the object space in the OSVM. Wouldn't the GC become more challenging to implement, too? Yet, I recall that there has been some PhD work on that topic ... probably using a pre-Spur VM ... Well, you can use (Native)ImageSegment (primitives 98 and 99) to manually manage part of the object space, right?<br></div><div><br></div><div>Best,<br></div><div>Marcel<br></div><div><br></div><div><br></div><div class="x_-2061953719mb_sig"><br></div><blockquote class="x_-2061953719history_container" style="border-left-style : solid; border-width : 1px; margin-top : 20px; margin-left : 0px; padding-left : 10px; min-width : 500px;"><p style="margin-top: 10px;"><span class="colour" style="color: rgb(170, 170, 170); margin-top: 10px;">Am 18.10.2021 13:45:16 schrieb gettimothy via Squeak-dev <<a href="mailto:squeak-dev@lists.squeakfoundation.org" target="_blank">squeak-dev@lists.squeakfoundation.org</a>>:</span><br></p><div style="font-family : Arial, Helvetica, sans-serif;"><div style="font-family : Verdana, Arial, Helvetica, sans-serif; font-size : 10pt;"><div>Sorry for the formatting, my mail reader doesn't format replies "correctly"<br></div><div><br></div><div><br></div><div>Thank you for the reply.<br></div><div><br></div><div>When you write...<br></div><div><br></div><blockquote style="border : 1px solid rgb(204, 204, 204); padding : 7px; background-color : rgb(245, 245, 245); min-width : 500px;"><div><div>The bad news is that you need more than 73GB RAM as well. <br></div><div>I presume even 256GB will not suffice in practice because objects probably <br></div><div>use more memory than the xml text. <br></div></div></blockquote><div>Are you stating that every time I run Squeak, the entirety of the image is loaded into RAM?<br></div><div>I am/was operating under the presupposition that the Squeak Image is rather like that of Linux Swap Space or a Postgres DB, which both exist on hard drives.<br></div><div>And that, Cog/Spur would pull the portions it needs from the image on Disk and put them in memory as needed.<br></div><div><br></div><div><br></div><div>re:<br></div><div><br></div><blockquote style="border : 1px solid rgb(204, 204, 204); padding : 7px; background-color : rgb(245, 245, 245); min-width : 500px;"><div><div>#timeProfile is your friend. <br></div><div><br></div></div></blockquote><div><br></div><div>thanks! added and I will run it on both squeak and pharo to look at differences.<br></div><div><br></div><div>re:<br></div><div><br></div><div><blockquote style="border : 1px solid rgb(204, 204, 204); padding : 7px; background-color : rgb(245, 245, 245); min-width : 500px;"><blockquote style="border : 1px solid rgb(204, 204, 204); padding : 7px; background-color : rgb(245, 245, 245); min-width : 500px;"><div><div>It's because you use the wrong priority. Use #userBackgroundPriority <br></div><div>instead of #lowIOPriority. <br></div><div><br></div></div></blockquote></blockquote><div>I have another (I am sure, wrong) preconception that a higher priority results in higher executions speed.<br></div><div>I suspect that it just manages interrupts, etc and that if their are none, then the execution speed is unaffected.<br></div><div>Is the latter true?<br></div><div>thx for your patience.<br></div><div>t<br></div></div><div><br></div><div><br></div><div>On Sun, 17 Oct 2021, gettimothy via Squeak-dev wrote: <br></div><blockquote style="border : 1px solid rgb(204, 204, 204); padding : 7px; background-color : rgb(245, 245, 245); min-width : 500px;"><div><div>> Eliot, <br></div><div>> <br></div><div>> Thank you for the reply <br></div><div>> <br></div><div>> <br></div><div>> That sounds like fun. To be clear, I can set these VM parameters on the running image? Or do I need to build a VM with those parameters pre-set? <br></div><div>> <br></div><div>> <br></div><div>> <br></div><div>> . Space is no problem : <br></div><div>> <br></div><div>> bash-4.3$ df -h /bulkstorage/ <br></div><div>> Filesystem Size Used Avail Use% Mounted on <br></div><div>> /dev/sdd1 3.6T 86G 3.4T 3% /bulkstorage <br></div><div>> <br></div><div>> that is where the 73Gb xml file is. <br></div><div><br></div><div>The bad news is that you need more than 73GB RAM as well. <br></div><div>I presume even 256GB will not suffice in practice because objects probably <br></div><div>use more memory than the xml text. <br></div><div><br></div><div>> <br></div><div>> <br></div><div>> Squeak 5.3 is very slow compared to the pharo8. <br></div><div>> <br></div><div>> I have a pristine image of both running that SAX parse on the same file. <br></div><div>> <br></div><div>> pharo : <br></div><div>> <br></div><div>> ping: two million elements. Time: 0:00:04:12.13878234600296 <br></div><div>> <br></div><div>> <br></div><div>> <br></div><div>> squeak5.3 (I upped the priority too): <br></div><div>> <br></div><div>> fs roughly 100000 elements in 5 minutes <br></div><div><br></div><div>#timeProfile is your friend. <br></div><div><br></div><div>> <br></div><div>> <br></div><div>> for some reason, pharo lets me run the thing and still interact with the system, while on squeak, the process has priority over my inputs. <br></div><div><br></div><div>It's because you use the wrong priority. Use #userBackgroundPriority <br></div><div>instead of #lowIOPriority. <br></div><div><br></div><div><br></div><div>Levente<br></div></div></blockquote><div><br></div></div><div><br></div></div></blockquote></div><div><br></div></div></blockquote></div><div><br></div></div><br></body></html>