<div dir="ltr">Hi everybody,<div> I'm forwarding an email I sent to Eliot as he suggested. It is about changes I'm doing to the VM to get type information dynamically.</div><div><br></div><div> Thanks Eliot!</div><div> Hernan.</div><div><br><div class="gmail_quote"><div dir="ltr">---------- Forwarded message ---------<br>From: <strong class="gmail_sendername" dir="auto">Hernan Wilkinson</strong> <span dir="ltr"><<a href="mailto:hernan.wilkinson@10pines.com" target="_blank">hernan.wilkinson@10pines.com</a>></span><br>Date: Thu, Oct 4, 2018 at 7:09 AM<br>Subject: Re: Help for a crazy idea I have<br>To: Eliot Miranda <<a href="mailto:eliot.miranda@gmail.com" target="_blank">eliot.miranda@gmail.com</a>><br></div><br><br><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Hi Eliot!<div> thank you for your answer. </div><div> I look at Slots as you suggested. They could work but, at least as far I could see and play with them, they would not work for all the cases I have in mind, for example I could not use slots to keep the type of temporary variables, parameters or methods return, etc. If I'm mistaken please let me know.</div><div><br></div><div> So, I continued with my idea and changed ClassDescription adding a new inst. var. called "instanceVariablesTypes", that points to an array of arrays. Each position of the first one points to the types of the inst. var. whose index correspond the position. Very simple and it works fine. Performance impact is very low. At the end of the email is the code I wrote at the VM in case you want to see it. </div><div> Now I'm working on the temporary variables types using AdditionalMethodState.</div><div><br></div><div> But the main reason I'm writing to you again is because I found a rare behavior and an error when running the simulator with the Cuis image. I'm using the StackInterpreter. The problems are are:</div><div>1) The simulator running a Cuis image does more scavengings than running the simulator with a Squeak image, even thought the Cuis image size is 9 mb and the Squeak one is 50 mb. (I know the image size has nothing to do with the young space size). So, is there a way to configure the young space size or a way to see why it triggers so many scavengings?</div><div>2) When I added the inst. var "instanceVariableTypes" to ClassDescription, some pointer ends up invalid. Doing the same with Squeak works fine. When adding that inst var in an Cuis image using the simulator, an assertion stops the simulator execution. There is another detail, this does not happen with 32 bit Cuis images/vm, only with 64 bit Cuis images. Do you know what could be happening? (I'm attaching the simulator's stack report just is case you have time to look at it, but it is very simple to reproduce).</div><div>3) What is the difference between "<primitive: 105 error: ec>" and just "<primitive: 105>" ?</div><div>I had to replace in Cuis all the first one with the second one because the simulator did not work with methods using the first one (example: Array>>replaceFrom: start to: stop with: replacement startingAt: repStart)</div><div><br></div><div> Anyway, it is very interesting what I'm achieving with just only keeping the types for inst. vars. I could significantly improve auto complete in the class browser for example, do some type checking, etc.</div><div> I'll keep you posted with the changes I'm making.</div><div> If you could help me with the 3 problems I mentioned above it would be great :-)</div><div><br></div><div> Cheers!</div><div> Hernan</div><div>-----------------</div><div>Here is the code I changed in the VM:</div><div><br></div><div><div>storeAndPopReceiverVariableBytecode</div><div><span style="white-space:pre-wrap"> </span>| rcvr top instVarIndex |</div><div><span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>rcvr := self receiver.</div><div><span style="white-space:pre-wrap"> </span>top := self internalStackTop.</div><div><span style="white-space:pre-wrap"> </span>instVarIndex := currentBytecode bitAnd: 7.</div><div><span style="white-space:pre-wrap"> </span>self internalPop: 1.</div><div><span style="white-space:pre-wrap"> </span>objectMemory</div><div><span style="white-space:pre-wrap"> </span>storePointerImmutabilityCheck: instVarIndex</div><div><span style="white-space:pre-wrap"> </span>ofObject: rcvr</div><div><span style="white-space:pre-wrap"> </span>withValue: top.</div><div><span style="white-space:pre-wrap"> </span>self fetchNextBytecode.</div><div><span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>self keepInstanceVariableTypeInformationFor: top in: rcvr at: instVarIndex. "<--- added this"</div><div><br></div><div><div>extendedStoreBytecodePop: popBoolean</div><div><span style="white-space:pre-wrap"> </span>| descriptor variableType variableIndex value rcvr |</div><div><span style="white-space:pre-wrap"> </span><inline: true></div><div><span style="white-space:pre-wrap"> </span>descriptor := self fetchByte.</div><div><span style="white-space:pre-wrap"> </span>variableType := descriptor >> 6 bitAnd: 3.</div><div><span style="white-space:pre-wrap"> </span>variableIndex := descriptor bitAnd: 63.</div><div><span style="white-space:pre-wrap"> </span>value := self internalStackTop.</div><div><span style="white-space:pre-wrap"> </span>popBoolean ifTrue: [ self internalPop: 1 ].</div><div><span style="white-space:pre-wrap"> </span>variableType = 0 ifTrue:</div><div><span style="white-space:pre-wrap"> </span>[rcvr := self receiver.</div><div><span style="white-space:pre-wrap"> </span>objectMemory storePointerImmutabilityCheck: variableIndex ofObject: rcvr withValue: value.</div><div><span style="white-space:pre-wrap"> </span>self keepInstanceVariableTypeInformationFor: value in: rcvr at: variableIndex. "<--- added this"</div><div><span style="white-space:pre-wrap"> </span>^ self fetchNextBytecode].</div><div><span style="white-space:pre-wrap"> </span>variableType = 1 ifTrue:</div><div><span style="white-space:pre-wrap"> </span>[ self fetchNextBytecode.</div><div><span style="white-space:pre-wrap"> </span>^self temporary: variableIndex in: localFP put: value].</div><div><span style="white-space:pre-wrap"> </span>variableType = 3 ifTrue:</div><div><span style="white-space:pre-wrap"> </span>[self storeLiteralVariable: variableIndex withValue: value.</div><div><span style="white-space:pre-wrap"> </span>^ self fetchNextBytecode].</div><div><span style="white-space:pre-wrap"> </span>self error: 'illegal store'</div></div><div><br></div><div><div>keepInstanceVariableTypeInformationFor: anAssignedObject in: rcvr at: instVarIndex</div><div><br></div><div><span style="white-space:pre-wrap"> </span>| assignedObjectClass assignedObjectClassTag instVarTypes rcvrClass rcvrClassTag types |</div><div><span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>"(objectMemory isForwarded: rcvr) ifTrue: [ self halt ]."</div><div><span style="white-space:pre-wrap"> </span>rcvrClassTag := objectMemory fetchClassTagOf: rcvr.</div><div><span style="white-space:pre-wrap"> </span>self deny: (objectMemory isForwardedClassTag: rcvrClassTag).</div><div><span style="white-space:pre-wrap"> </span>rcvrClass := objectMemory classForClassTag: rcvrClassTag.</div><div><span style="white-space:pre-wrap"> </span>self deny: rcvrClass isNil.</div><div><span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>"(objectMemory isForwarded: top) ifTrue: [ self halt ]."</div><div><span style="white-space:pre-wrap"> </span>assignedObjectClassTag := objectMemory fetchClassTagOf: anAssignedObject.</div><div><span style="white-space:pre-wrap"> </span>self deny: (objectMemory isForwardedClassTag: assignedObjectClassTag).</div><div><span style="white-space:pre-wrap"> </span>assignedObjectClass := objectMemory classForClassTag: assignedObjectClassTag.</div><div><span style="white-space:pre-wrap"> </span>self deny: assignedObjectClass isNil.</div><div><span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>instVarTypes := objectMemory followObjField: 5 ofObject: rcvrClass. "TODO: use constant instead of 5"</div><div><span style="white-space:pre-wrap"> </span>instVarTypes = objectMemory nilObject ifTrue: [ ^self ].</div><div><span style="white-space:pre-wrap"> </span>types := objectMemory followObjField: instVarIndex ofObject: instVarTypes.</div><div><span style="white-space:pre-wrap"> </span>0 to: 9 do: [ :index | | typeAtIndex | "TODO: use array size and not 9. Check for types == nil"</div><div><span style="white-space:pre-wrap"> </span>typeAtIndex := objectMemory followObjField: index ofObject: types.</div><div><span style="white-space:pre-wrap"> </span>typeAtIndex == assignedObjectClass ifTrue: [ ^self ].</div><div><span style="white-space:pre-wrap"> </span>typeAtIndex == objectMemory nilObject ifTrue: [ ^objectMemory storePointer: index ofObject: types withValue: assignedObjectClass ]].</div><div><span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span>"self halt: 'out of space for types"</div><div><span style="white-space:pre-wrap"> </span></div><div><span style="white-space:pre-wrap"> </span></div><div><br></div><div><span style="white-space:pre-wrap"> </span></div><div><br></div><div><br></div><div><span style="white-space:pre-wrap"> </span></div></div><div><span style="white-space:pre-wrap"> </span></div></div><div><br></div></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, Sep 19, 2018 at 1:48 PM Eliot Miranda <<a href="mailto:eliot.miranda@gmail.com" target="_blank">eliot.miranda@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Hernan,<div><br><div class="gmail_quote"><div dir="ltr">On Sun, Sep 16, 2018 at 1:43 PM Hernan Wilkinson <<a href="mailto:hernan.wilkinson@10pines.com" target="_blank">hernan.wilkinson@10pines.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr">Hi Eliot,<div> I have had this idea for a long time, some guys at the university implemented in pharo using metalinks (Marcus's compiler capability) but it was a toy implementation... a few days ago I cloned the cog repo to show people at work how easy is to run the Smalltalk vm as a simulation and see how everything is implemented, change it, etc. so I started to implement this idea but of course, I need help :-)</div><div> The idea is simple: Every time an object is assigned to a variable, I want to keep its class to have "real type inference". </div><div> The idea is based on the fact that Smalltalk is always running, so "class" information will be generated with real-running code. Of course, we will have to derive type info from that class info, but that is a problem to be solved in the image space.</div><div> So, I did some experiments and modified the bytecodes that store inst. vars. (#storeAndPopReceiverVariableBytecode and #extendedStoreBytecodePop:) to keep that info, but they doit in a dictionary that belongs to the simulator space. </div><div> What I want to do is to store that info in a way that should be accesible from the image. </div><div> I added an inst. var. to ClassDescription called 'instanceVariablesTypes' that it is initialized with an array of the number of inst. vars. of the class and at each index there is a Set (it could be a Dictionary or an Array...) where I want to store the assigned object class to the inst. var</div><div> The problem I have is that I do not know how to manage those objects from the image space. I need to be able to add objects to the Set that keeps the classes of an inst. var, make the array of instanceVariablesTypes change accordingly to changes in the class, etc.</div><div> So basically my question are:</div><div>1) is there a way to manage a Set or a Dictionary from the vm but that was created at "the image". The method lookup has the code to lookup in a dict., but I have not seen code to create, add, remove elements</div><div>2) if there is not, I could do everything with arrays, but to do so I need to know how an array can be instantiated from the vm</div><div>3) Any other suggestion?</div></div></div></div></blockquote><div><br></div><div>Yes, I would look at Slots. With Slots you should b e sable to manage everything at the image and work on a subset of all the classes. Kernel classes should be avoided since one will get into an infinite recursion (and indeed using the Vm would be a solution here). The problem with using the VM is that one has to use very low-level data structures and tackle the JIT to get performance, It is not easy, whereas Slots should get you the information you need very very cheaply.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><br></div><div>I know this idea would slow down the execution of the vm, but my idea is based on the fact that while programming (development time) the speed is not so critical, so we could have an option to not do this on runtime time (or different vms...)</div><div>I also know that keeping the type info for inst. vars. is not enough, the same has to be done with methods (parameters, temporaries and return type), class variables, and keeping types for collections should be based per instance and not per class (like generics do in other languages: Array<T>), etc. Also, I do not know how this idea could conflict with jiting (I played with the stack vm only), etc. </div><div>There are a lot of things to do, but I think having this information available from the image would help a lot on tools like refactoring, syntax highlighting, type checking, etc etc.</div></div></div></div></blockquote><div><br></div><div>Again, using Slots instead of the VM would make it much easier to access the information at the image level.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><br></div><div> Well, the email is too long already :-) </div><div> I hope you can send me some ideas to implement this.</div></div></div></div></blockquote><div><br></div><div>Take a look at Slots. If that doesn't work then indeed I can help you do it in the VM. But I warn you; it'll be hairy (but fun!! :-) ).</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><br></div><div> Thanks!</div><div> Hernan</div><div><br>-- <br><div dir="ltr" class="m_-3435010932069740053m_8581783856547631054m_1003532533774346377m_-3340512138370000042gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><span style="font-family:tahoma,sans-serif;font-size:xx-small;border-collapse:collapse"><strong><span style="font-size:8pt"><span><span style="font-size:small"><font size="2"><span style="font-weight:normal"><span style="font-weight:bold">Hernán Wilkinson</span><br>Agile Software Development, Teaching & Coaching</span></font></span></span></span></strong></span></div><div><span style="font-family:tahoma,sans-serif;font-size:xx-small;border-collapse:collapse"><strong><span style="font-size:8pt"><span><span style="font-size:small"><font size="2"><span style="font-weight:normal">Phone: +54-011</span></font></span></span></span></strong></span><font face="tahoma, sans-serif" size="2">-4893-2057</font></div><div><strong style="font-family:tahoma,sans-serif;font-size:xx-small"><span style="font-size:8pt"><span style="font-size:small"><font size="2"><span style="font-weight:normal">Twitter: @HernanWilkinson</span></font></span></span></strong></div><div><span style="font-family:tahoma,sans-serif;font-size:xx-small;border-collapse:collapse"><strong><span style="font-size:8pt"><span><span style="font-size:small"><font size="2"><span style="font-weight:normal">site: <a href="http://www.10pines.com/" style="color:rgb(17,65,112)" target="_blank">http://www.10Pines.com</a></span></font></span></span></span></strong></span></div><div><font face="tahoma, sans-serif"><span style="border-collapse:collapse">Address: Alem 896</span></font>, Floor 6, Buenos Aires, Argentina</div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_-3435010932069740053m_8581783856547631054m_1003532533774346377gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><span style="font-size:small;border-collapse:separate"><div>_,,,^..^,,,_<br></div><div>best, Eliot</div></span></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_-3435010932069740053m_8581783856547631054gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><span style="font-family:tahoma,sans-serif;font-size:xx-small;border-collapse:collapse"><strong><span style="font-size:8pt"><span><span style="font-size:small"><font size="2"><span style="font-weight:normal"><span style="font-weight:bold">Hernán Wilkinson</span><br>Agile Software Development, Teaching & Coaching</span></font></span></span></span></strong></span></div><div><span style="font-family:tahoma,sans-serif;font-size:xx-small;border-collapse:collapse"><strong><span style="font-size:8pt"><span><span style="font-size:small"><font size="2"><span style="font-weight:normal">Phone: +54-011</span></font></span></span></span></strong></span><font face="tahoma, sans-serif" size="2">-4893-2057</font></div><div><strong style="font-family:tahoma,sans-serif;font-size:xx-small"><span style="font-size:8pt"><span style="font-size:small"><font size="2"><span style="font-weight:normal">Twitter: @HernanWilkinson</span></font></span></span></strong></div><div><span style="font-family:tahoma,sans-serif;font-size:xx-small;border-collapse:collapse"><strong><span style="font-size:8pt"><span><span style="font-size:small"><font size="2"><span style="font-weight:normal">site: <a href="http://www.10pines.com/" style="color:rgb(17,65,112)" target="_blank">http://www.10Pines.com</a></span></font></span></span></span></strong></span></div><div><font face="tahoma, sans-serif"><span style="border-collapse:collapse">Address: Alem 896</span></font>, Floor 6, Buenos Aires, Argentina</div></div></div></div></div></div></div></div></div></div></div></div>
</div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_-3435010932069740053gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><span style="font-family:tahoma,sans-serif;font-size:xx-small;border-collapse:collapse"><strong><span style="font-size:8pt"><span><span style="font-size:small"><font size="2"><span style="font-weight:normal"><span style="font-weight:bold">Hernán Wilkinson</span><br>Agile Software Development, Teaching & Coaching</span></font></span></span></span></strong></span></div><div><span style="font-family:tahoma,sans-serif;font-size:xx-small;border-collapse:collapse"><strong><span style="font-size:8pt"><span><span style="font-size:small"><font size="2"><span style="font-weight:normal">Phone: +54-011</span></font></span></span></span></strong></span><font face="tahoma, sans-serif" size="2">-4893-2057</font></div><div><strong style="font-family:tahoma,sans-serif;font-size:xx-small"><span style="font-size:8pt"><span style="font-size:small"><font size="2"><span style="font-weight:normal">Twitter: @HernanWilkinson</span></font></span></span></strong></div><div><span style="font-family:tahoma,sans-serif;font-size:xx-small;border-collapse:collapse"><strong><span style="font-size:8pt"><span><span style="font-size:small"><font size="2"><span style="font-weight:normal">site: <a href="http://www.10pines.com/" style="color:rgb(17,65,112)" target="_blank">http://www.10Pines.com</a></span></font></span></span></span></strong></span></div><div><font face="tahoma, sans-serif"><span style="border-collapse:collapse">Address: Alem 896</span></font>, Floor 6, Buenos Aires, Argentina</div></div></div></div></div></div></div></div></div></div></div></div></div></div>