Vm-dev October 2015

vm-dev@lists.squeakfoundation.org

29 participants
86 discussions

ffi-test-main.c fails on FreeBSD
by Gary Dunn 12 Aug '16

12 Aug '16

In conjunction with getting Open Cobalt running on FreeBSD I have been looking into the problems with SqueakFFIPrims on FreeBSD and have reached a point that is beyond my understanding. I am not sure if the fix should come from the VM side or the FreeBSD port maintainer. I needed to get source code back into the ports tree so I did a make in /usr/ports/lang/squeak. I already had the 3.9 tarball, so that just repopulated the source and did the initial build. In the "files" folder I found patch files typical for a FreeBSD port. These patch source files from a generic tarball to work with FreeBSD. The one of interest was patch-platforms__unix__plugins__SqueakFFIPrims__ffi-config which contains ----x-----x----- --- platforms/unix/plugins/SqueakFFIPrims/ffi-config.org Wed Apr 26 20:27:53 2006 +++ platforms/unix/plugins/SqueakFFIPrims/ffi-config Wed Apr 26 20:29:00 2006 @@ -39,6 +39,7 @@ case ${abi} in linux) abi=sysv;; + freebsd*) abi=sysv;; darwin*) abi=darwin;; *) abi=libffi; lib="-lffi";; esac -----x-----x----- I then had the folder work/Squeak-3.9-7/platforms/unix/plugins/SqueakFFIPrims In it is a file 00README which explains how to test FFI. It says to run ffi-test-config but I have no such file. It says to type "make" to build a test suite but the build failed. It says to try with make CPU=any ABI=libffi LIB=-lffi which also failed. I traced the problem to the gcc search paths for includes and libraries -- turns out FreeBSD only searches /usr/include and /usr/lib when the required files are in /usr/local/include and /usr/local/lib. After some fiddling I finally had a "main" to test. It failed: ffi-test-main.c failed at line 361. Here is a bit from that file, ending with line 361: ffiInitialize(); for (ul= 0; ul < 15; ++ul) ffiPushSingleFloat(fa[ul]); GO(FFITypeSingleFloat, many); f= ffiReturnFloatValue(); ffiCleanup(); CHECK(f - ff < FLT_EPSILON); I worked out the CHECK macro but could not find what FLT_EPSILON is. At that point I decided to stop and come to you for help. Where do I go from here? -- Gary Dunn, Honolulu osp(a)aloha.com http://openslate.net/ http://e9erust.blogspot.com/ Sent from Slate001

5 8

Tasks of the heartbeat code
by Holger Freyther 27 Jul '16

27 Jul '16

Dear Eliot, I tried to understand the basic load of the VM and I have two questions for the heartbeat code. Is it right that there are two concerns for the code? 1.) Call forceInterruptCheckFromHeartbeat periodically 2.) Update the clock as seen by the image 1st is useful to interrupt Processes like […] repeat with a pre-emptive scheduler? On a different note do backwards jumps generate interrupt checks too? 2nd is this an optimization to not have to generate the clock when the primitive is being used? thanks holger

4 12

VM Recipe: How to build an interpreter VM on Linux
by David T. Lewis 07 Feb '16

07 Feb '16

How to build an interpreter VM from source on a typical Linux or Unix system: Install development tools for your Linux system (32-bit libraries are not required). Install CMake, either from your Linux distribution or from www.cmake.org Install Subversion, either from your Linux distributiion or from https://subversion.apache.org/ Make a new, empty, directory for your work. In that directory, download the full platforms sources from the Subversion repository. Download everything, whether you need it or not. The Subversion directory information will be used in the build process, so do not cut corners to save space. $ svn co http://squeakvm.org/svn/squeak/trunk/platforms Also in the new directory, download the generated sources. This is an up to date copy of source files generated from the Smalltalk source code in VMMaker. You can generate them yourself by loading VMMaker, but using the files from the Subversion repository is a faster way to get started. $ svn co http://squeakvm.org/svn/squeak/trunk/src You will now have ./platforms and ./src directories with the latest source code for the VM. Make another subdirectory called ./build that you will use to build the VM. $ mkdir build Change to the build directory. $ cd build Copy the example Makefile into your build directory. $ cp ../platforms/unix/cmake/Makefile.example Makefile Build the VM $ make Install the VM $ sudo make install You now have a VM that runs that classic Squeak image formats, including the 64-bit version (see http://build.squeak.org/job/Squeak%2064-bit%20image/). This is a traditional context interpreter VM. It will not run the latest high performance Spur image formats. Please refer to the Cog and Spur VM development at http://www.mirandabanda.org/cogblog/ and http://www.mirandabanda.org/files/Cog/VM/ for the latest Cog and Spur VM development, and for VMs that support the new Spur image formats. Dave

5 7

Re: [squeak-dev] vm crash from BalloonEngine (maybe B2DPlugin or BitBltPlugin)
by Nicolai Hess 06 Jan '16

06 Jan '16

2015-04-08 20:02 GMT+02:00 karl ramberg <karlramberg(a)gmail.com>: > Confirmed. > > Karl > > On Wed, Apr 8, 2015 at 5:33 PM, Nicolai Hess <nicolaihess(a)web.de> wrote: > >> In order to reproduce it, draw a rectangle with radial gradient fill, >> with direction >> and normal, both of length 1. >> > This is not the reason why it crashes. It happens for other values as well: |form canvas fill direction normal| direction := 4@0. normal := 0@2. form := Form extent:300@300 depth:32. canvas := form getCanvas. fill := GradientFillStyle new colorRamp:{0 -> Color white . 0.5 -> Color green . 1 -> Color blue};origin:400@400 ;direction:direction;normal:normal;radial:true;yourself. canvas fillRectangle: (0@0 extent:800@800) fillStyle: fill. Display getCanvas drawImage: form at: 300@300 This uses a different value for direction and normal and the fill is defined for a larger area (800x800) -> CRASH Maybe, this is a rare bug, because, normally you don't create a small gradient t o fill a large area. But a vm crash is not nice, I would like to get this fixed. I am not sure what is the exact cause, but it may come from a signed integer overflwo, It looks like Balloon uses fixed point arithmetik (binary scaling). This fixed point representation (16.16 bit fixed point represented in a 32 integer) tend to overflow, especially for operations like computing the squared length of a vector x*x+y*y -> overflow -> result is a negative number. Possible solutions: downscale prior to the multiplication -> lost of precision use a larger datatype for intermediate results. It looks like B2D uses the first option (look at senders of accurateLengthOfwith and squaredLenghtOfwith Maybe there is something wrong with the other code that multiplies the values, or the 16.16 bit fixed point is just not large enough for some fills? Any ideas? Nicolai > >> This one works (no vm crash): >> |form canvas fill direction normal| >> direction := 1@0. >> normal := 0@0. >> form := Form extent:300@300 depth:32. >> canvas := form getCanvas. >> fill := GradientFillStyle new colorRamp:{0 -> Color white . 0.5 -> Color >> green . 1 -> Color blue};origin:150@150 >> ;direction:direction;normal:normal;radial:true;yourself. >> canvas fillRectangle: (50@50 extent:200@200) fillStyle: fill. >> Display getCanvas drawImage: form at: 300@300. >> >> >> direction := 1@0. >> normal := 0@0. >> -> OK >> >> direction := 0@0. >> normal := 0@1. >> -> OK >> >> direction := 1@0. >> normal := 0@1. >> -> VM-Crash (Pharo and Squeak) >> >> Pharo: >> VM Version: Cog VM 4.0.0 (release) from Feb 16 2015 >> Compiler: gcc 4.6.2 >> Interpreter Build: NBCoInterpreter >> NativeBoost-CogPlugin-EstebanLorenzano.21 uuid: >> 4d9b9bdf-2dfa-4c0b-99eb-5b110dadc697 Feb 16 2015 >> Cogit Build: NBCogit NativeBoost-CogPlugin-EstebanLorenzano.21 uuid: >> 4d9b9bdf-2dfa-4c0b-99eb-5b110dadc697 Feb 16 2015 >> Source Version: https://github.com/pharo-project/pharo-vm.git Commit: >> 55fbd0b0279998a4a33852f5afb3d213986cc3ae Date: 2015-02-16 17:54:00 +0100 >> By: Esteban Lorenzano <estebanlm(a)gmail.com> Jenkins build #14895 >> >> Current byte code: -1 >> Primitive index: 117 >> Smalltalk stack dump: >> 0x24c830 M BalloonEngine>copyLoopFaster 0x6b09b80: a(n) BalloonEngine >> 0x24c848 M BalloonEngine>copyBits 0x6b09b80: a(n) BalloonEngine >> 0x24c860 M BalloonEngine>postFlushIfNeeded 0x6b09b80: a(n) BalloonEngine >> 0x24c884 I >> BalloonEngine>drawRectangle:fill:borderWidth:borderColor:transform: >> 0x6b09b80: a(n) BalloonEngine >> 0x24c8b8 I BalloonCanvas>drawRectangle:color:borderWidth:borderColor: >> 0x6b099c0: a(n) BalloonCanvas >> 0x24c8e8 I BalloonCanvas>fillRectangle:basicFillStyle: 0x6b099c0: a(n) >> BalloonCanvas >> >> >> Squeak: >> VM Version: Cog VM 4.0.0 (release) from Aug 22 2013 >> Compiler: gcc 3.4.4 (cygming special, gdc 0.12, using dmd 0.125) >> Interpreter Build: CoInterpreter VMMaker.oscog-eem.331 uuid: >> 37d2e4b0-2f37-4e2d-8313-c63637785e59 Aug 22 2013 >> Cogit Build: StackToRegisterMappingCogit VMMaker.oscog-eem.333 uuid: >> 84da9cb8-7f30-4cb7-b4fb-239a11f63b54 Aug 22 2013 >> Source Version: VM: r2776 http://www.squeakvm.org/svn/squeak/branches/Cog >> Plugins: r2545 >> http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins >> >> Current byte code: -1 >> Primitive index: 117 >> >> >> nicolai >> >> >> >> >

4 11

Do we have the new primitive?? [WAS] Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?
by Mariano Martinez Peck 16 Dec '15

16 Dec '15

> All I can say is that I am impressed by the numbers it is really much >> faster. >> I still don't understand why I send this email with a subject say >> IdentitySet because what I really need is a fast/large IdentityDictionary >> :( Anyway, there's a place where we can use this LargeIdentitySet in Fuel >> I think). >> >> So Levente, you say this is not possible to adapt this for dictionary? >> can >> we contact Eliot to provide such a primitive? >> > > As promised, I uploaded my LargeIdentityDictionary implementation to > http://leves.web.elte.hu/**squeak/**LargeIdentityDictionary.st<http://leves.web.elte.hu/squeak/LargeIdentityDictionary.st>. > The numbers will be a bit worse compared to LargeIdentitySet, because of > the lack of the primitive, but it's still 2-3x faster than other solutions > (IdentityDictionary, PluggableIdentityDictionary, subclassing, etc). I'm > about to propose this primitive with other improvements on the vm-dev list. > > Hi Eliot/Levente. What is the status of this? Do we have already the new primitive? If true, how can we adapt LargeIdentitySet to use such new primitive? Thanks! > > Levente > > >> thanks >> >> On Fri, Dec 16, 2011 at 3:28 PM, Levente Uzonyi <leves(a)elte.hu> wrote: >> >> On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: >>> >>> On 16.12.2011 03:26, Levente Uzonyi wrote: >>> >>>> >>>> >>>>> How about my numbers? :) >>>>> >>>>> "Preallocate objects, so we won't count gc time." >>>>> n := 1000000. >>>>> objects := Array new: n streamContents: [ :stream | >>>>> n timesRepeat: [ stream nextPut: Object new ] ]. >>>>> >>>>> set := IdentitySet new: n. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "4949" >>>>> >>>>> set := LargeIdentitySet new. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "331" >>>>> >>>>> set := (PluggableSet new: n) >>>>> hashBlock: [ :object | object identityHash * 4096 + object class >>>>> identityHash * 64 ]; "Change this to #basicIdentityHash in Pharo" >>>>> equalBlock: [ :a :b | a == b ]; >>>>> yourself. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "5511" >>>>> >>>>> >>>>> I also have a LargeIdentityDictionary, which is relatively fast, but >>>>> not >>>>> as fast as LargeIdentitySet, because (for some unknown reason) we don't >>>>> have a primitive that could support it. If we had a primitive like >>>>> primitive 132 which would return the index of the element if found or >>>>> 0 if >>>>> not, then we could have a really fast LargeIdentityDictionary. >>>>> >>>>> >>>>> Levente >>>>> >>>>> Hehe yes, if writing a version fully exploiting the limited range, >>>> that's >>>> probably the approach I would go for as well. >>>> (IAssuming it's the version at http://leves.web.elte.hu/** >>>> squeak/LargeIdentitySet.st<htt**p://leves.web.elte.hu/squeak/** >>>> LargeIdentitySet.st<http://leves.web.elte.hu/squeak/LargeIdentitySet.st> >>>> > >>>> ) >>>> >>>> Mariano commented in the version at http://www.squeaksource.com/** >>>> FuelExperiments <http://www.squeaksource.com/**FuelExperiments<http://www.squeaksource.com/FuelExperiments>> >>>> that it's >>>> >>>> slow for them, which I guess is due to not adopting #identityHash calls >>>> to >>>> #basicIdentityHash calls for Pharo: >>>> ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 >>>> So it basically uses 1 bucket instead of 4096... Whoops. :) >>>> >>>> Uploaded a new version to the MC repository which is adapted for Pharo, >>>> on the same machine my numbers were taken from, it does the same test >>>> as I >>>> used above in 871 ms. (181 with preallocation). >>>> >>>> >>> Cool. One more thing: in Squeak the method using primitive 132 directly >>> was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >>> this was also added to Pharo, then the #pointsTo: sends should be changed >>> to #instVarsInclude:, otherwise Array can be reported as included even if >>> it wasn't added. >>> I'll upload my LargeIdentityDictionary implementation to the same place >>> this evening, since it's still 2-3 factor faster than other solutionts >>> and >>> there seem to be demand for it. >>> >>> >>> Levente >>> >>> >>> Cheers, >>>> Henry >>>> >>>> >>>> >>>> >>> >> >> -- >> Mariano >> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >> >> > -- Mariano http://marianopeck.wordpress.com

4 12

Re: [Pharo-dev] Random forest in Pharo
by Robert Withers 04 Dec '15

04 Dec '15

(+vm-dev) Esteban, thanks very much for your views. You write: On 10/16/2015 07:53 AM, Esteban Lorenzano wrote: > > So… my plan is: > > - NB to FFI port (should be ready soon) > - spur32 bits (should be ready soon, we are just waiting the NB to FFI) > - spur64 bits (Eliot is working on the JIT, I need to manage the migration of Pharo) > - FFI 64bits (0% done here… just some stubs waiting for effort). But we want to move Pharo to 64bits… we are losing a train here... > - FFI MT and of course you are buried in existing tasks, especially the transition to 64-bits. It is my humble opinion, without really knowing, that 64-bit would have to be redone after the MTVM completes. Kinda why I would desire MTVM before 64-bits, then I assume less work would be needed overall. But that's my uneducated guess. Perhaps Eliot could shine some light for us on the progress of MTVM and what's needed. Because of that assumption I've made and without the responsibilities you have, Esteban, but recognizing modernizing NB to FFI, my desired list is: - NB to FFI port (should be ready soon) - spur32 bits (should be ready soon, we are just waiting the NB to FFI) - FFI MT - spur64 bits (Eliot is working on the JIT, I need to manage the migration of Pharo) - FFI 64bits (0% done here… just some stubs waiting for effort). But we want to move Pharo to 64bits… we are losing a train here... > If *anyone* wants to collaborate in any part of this list (is a MASSIVE effort), I would be very happy and willing to provide as much assistance/guiding/peer programming as I can. Yes, please, and I appreciate you willingness to mentor. Best, Robert On 10/16/2015 07:53 AM, Esteban Lorenzano wrote: > >> On 16 Oct 2015, at 13:22, Robert Withers <robert.w.withers(a)gmail.com> wrote: >> >> Bonjour Stef, >> >> I believe the change everyone would be waiting for is a Multi-threaded VM, that supports threaded callouts and callbacks. So I suppose the questions become: > > multi thread ffi vm exists as a prototype, and according to its author (Eliot) needs no less than 4 months of someone skilled working on it to finish it. > >> - will existing NB syntax require change with the introduction of a MTVM? > > nothing will be equal… why? because you will need to provide callbacks. There is no magic here, if you want something to happen while doing something else, you need to provide a way to be called back when that thing finishes. > > now we have: > > nbCall:module: > > to call FFI (will change to a more generic ffiCall:module:, btw… but we’ll keep old notation some versions to provide backward compatibility) > > So I can imagine an API like this: > > ffiAsyncCall:callback:module: > > So yes… you need change. But that’s inevitable… Of course you can do a fwk forking in image and starting a continuation, etc. But well, I will let you guys to think on that (also you still will need the fork) :) > >> - to what degree will NB syntax need to be extended to support threaded callouts. > > as described, I suppose. No exact idea until we implement. > >> - what NB syntax changes are needed for callbacks. > > NB used some magic there. Shadow classes and etc. We do not need/want that anymore. > > now you have a new FFICallback api, where you can do: > > FFICallback > signature: #(int (int handle, int *pitch, int x, int y, int w, int h)) > block: [ :handle :pitch :x :y :w :h | > pitch signedLongAt: 1 put: (self get_stride: handle). > self get_data: handle ] > > (I extracted that example from my ongoing port of Athens) > >> - timing: when is MTVM expected and can we operate on the leading edge? > > we are lacking resources: Is just me (in the Pharo part, of course… Eliot and some others work also in the exclusively VM part, but they have other agendas). > > So… my plan is: > > - NB to FFI port (should be ready soon) > - spur32 bits (should be ready soon, we are just waiting the NB to FFI) > - spur64 bits (Eliot is working on the JIT, I need to manage the migration of Pharo) > - FFI 64bits (0% done here… just some stubs waiting for effort). But we want to move Pharo to 64bits… we are losing a train here... > - FFI MT > > So… not very hight priority. According as how I see things now, I *think* I can have that some point of 2017… yes, as far as that. Also because I have a lot other stuff to do, not just the lowlevel parts. > > Of course this is not good and we have talked with some people to speed up this. We talked with Igor about taking the MT stuff (yes, we also believe is VERY important), but he fade in the blur… I didn’t hear from him since 3 months (I’m still waiting for some feedback on OSWindow work he did, he)… yes I’m a bit frustrated. > > If *anyone* wants to collaborate in any part of this list (is a MASSIVE effort), I would be very happy and willing to provide as much assistance/guiding/peer programming as I can. > > hope this helps > > Esteban > >> >> Personally, in SqueakElib, I have my network and session layers working for encrypted connections, so I just have the presentation layer to go. Once done, I would be very interested in working in the VM with whomever is also interested with an eye towards threading (MTVM). I am no expert here so it will be a challenge I am looking forward to. >> >> Regards, >> Robert >> >> On 10/16/2015 06:59 AM, stepharo wrote: >>> >>> In pharodays presentations we explictly mentioned our strategy: >>> - we keep NB syntax >>> - we change the assembly generation to be able to get Spur. >>> >>> Stef >>> >>> I can tell you that if I could get a lua kind of jit now I would sign >>> immediately and also for having pharo as a dll. >>> Now it takes time to do soemthing. >>> >>> Le 15/10/15 17:35, Robert Withers a écrit : >>>> >>>> I am reticent to invest in learning FFI that is changing without an idea >>>> of the direction of the change. So >>> >>> >> > >

10 20

keyUp event modifier bitmap of is shifted by 3 on OSX
by Kruck, Bastian 27 Nov '15

27 Nov '15

Hello Folks, when looking for how Cog passes keystrokes to the image, we found that the modifier bitmap of the keyUp events is left shifted by 3 bits on OS X. This seems to be a bug in Mac OS specific event handling. sqMacUIEvents.c usually unshifts the mouse buttons away before passing it to the image, but it doesn’t do that for keyup events. Something that I oversee here? This could be a fix: Index: platforms/Mac OS/vm/sqMacUIEvents.c =================================================================== --- platforms/Mac OS/vm/sqMacUIEvents.c (revision 3312) +++ platforms/Mac OS/vm/sqMacUIEvents.c (working copy) @@ -1717,7 +1717,7 @@ //fprintf(stdout,"\nrawkey up %i",ioMSecs()); key = findInKeyMap(keyCode); if (key != -1) { - enterKeystroke ( EventTypeKeyboard,keyCode, EventKeyUp, 0, ModifierStateCarbon(event,0)); + enterKeystroke ( EventTypeKeyboard,keyCode, EventKeyUp, 0, ModifierStateCarbon(event,0) >> 3); } removeFromKeyMap(keyCode); result = eventNotHandledErr; Unfortunately, I couldn’t verify the fix as I am running Yosemite which seems to not be supported yet for building from source. I appended the changeset which we used to find that issue. Steps to reproduce: 1) install our changeset into a Squeak4.5 image on OS X 2) run `TempSettings printEvents: true.` 3) open a Transcript 4) Press cmd+a 5) The fifth item of that Array should be 8 on the third time (but is 8<<3 instead) #(2 15512120 0 1 8 0 0 1) #(2 15512120 97 0 8 97 0 1) #(2 15512198 0 2 64 0 0 1) Tobias commented that this bug could be duplicated to some other places (which we didn’t verify yet). I am not sure, how many projects already rely on this behavior and how to go on with that. BR, Bastian Kruck

4 5

hello
by Sebastián Krynski 04 Nov '15

04 Nov '15

Hello my name is Sebastián Krynski from Buenos Aires, Argentina. Just writing to let you know that I'm starting to work on adding an atomic swap operator in Pharo , guaranteed not to be interrupted by the VM. The operator will be used this way: a :=: b meaning 'swap variable a with variable b'. In order to do this I will be modifying the VM and the Compiler . I'm working under the direction of INRIA and Gabriela Arévalo.

2 2

64bits spur DataStream assumes SmallInteger fit in 32bits
by Nicolas Cellier 01 Nov '15

01 Nov '15

Hi, as the title tells, DataStream>>writeInteger: anInteger "PRIVATE -- Write the contents of a SmallInteger." byteStream nextInt32Put: anInteger "signed!!!!!" Which strategy do we adopt? I suggest using a fake LargeInteger... objectToStore := (self objectIfBlocked: anObject) objectForDataStream: self. objectToStore == anObject ifFalse: [typeID := self typeIDFor: objectToStore]. Of course, we must also care of LargeInteger materialization in 64bits spur. I'm going to fix it unless someone has a better idea. Nicolas

3 2

Problem while updating SecureHashAlgorithm
by Nicolas Cellier 31 Oct '15

31 Oct '15

Hi, anyone else got a problem while updating SecureHashAlgorithm? The symptom is a SyntaxError window telling we cannot store into TA. TA is a new class variable, so we normally should be able to write into it. But the compiler fails because the classPool contains a ClassBinding instead of an Association for #TA... If I recompile SecureHashAlgorithm without the new class vars, then compile with the class vars again via the Browser, then I correctly get a classPool populated with Association only and I can proceed from the SyntaxError. Any idea where these ClassBinding came from? Nicolas

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev October 2015