Vm-dev March 2014

vm-dev@lists.squeakfoundation.org

23 participants
104 discussions

ffi-test-main.c fails on FreeBSD
by Gary Dunn 12 Aug '16

12 Aug '16

In conjunction with getting Open Cobalt running on FreeBSD I have been looking into the problems with SqueakFFIPrims on FreeBSD and have reached a point that is beyond my understanding. I am not sure if the fix should come from the VM side or the FreeBSD port maintainer. I needed to get source code back into the ports tree so I did a make in /usr/ports/lang/squeak. I already had the 3.9 tarball, so that just repopulated the source and did the initial build. In the "files" folder I found patch files typical for a FreeBSD port. These patch source files from a generic tarball to work with FreeBSD. The one of interest was patch-platforms__unix__plugins__SqueakFFIPrims__ffi-config which contains ----x-----x----- --- platforms/unix/plugins/SqueakFFIPrims/ffi-config.org Wed Apr 26 20:27:53 2006 +++ platforms/unix/plugins/SqueakFFIPrims/ffi-config Wed Apr 26 20:29:00 2006 @@ -39,6 +39,7 @@ case ${abi} in linux) abi=sysv;; + freebsd*) abi=sysv;; darwin*) abi=darwin;; *) abi=libffi; lib="-lffi";; esac -----x-----x----- I then had the folder work/Squeak-3.9-7/platforms/unix/plugins/SqueakFFIPrims In it is a file 00README which explains how to test FFI. It says to run ffi-test-config but I have no such file. It says to type "make" to build a test suite but the build failed. It says to try with make CPU=any ABI=libffi LIB=-lffi which also failed. I traced the problem to the gcc search paths for includes and libraries -- turns out FreeBSD only searches /usr/include and /usr/lib when the required files are in /usr/local/include and /usr/local/lib. After some fiddling I finally had a "main" to test. It failed: ffi-test-main.c failed at line 361. Here is a bit from that file, ending with line 361: ffiInitialize(); for (ul= 0; ul < 15; ++ul) ffiPushSingleFloat(fa[ul]); GO(FFITypeSingleFloat, many); f= ffiReturnFloatValue(); ffiCleanup(); CHECK(f - ff < FLT_EPSILON); I worked out the CHECK macro but could not find what FLT_EPSILON is. At that point I decided to stop and come to you for help. Where do I go from here? -- Gary Dunn, Honolulu osp(a)aloha.com http://openslate.net/ http://e9erust.blogspot.com/ Sent from Slate001

5 8

Do we have the new primitive?? [WAS] Re: [Pharo-project] IdentitySet but using #hash rather than #identityHash ?
by Mariano Martinez Peck 16 Dec '15

16 Dec '15

> All I can say is that I am impressed by the numbers it is really much >> faster. >> I still don't understand why I send this email with a subject say >> IdentitySet because what I really need is a fast/large IdentityDictionary >> :( Anyway, there's a place where we can use this LargeIdentitySet in Fuel >> I think). >> >> So Levente, you say this is not possible to adapt this for dictionary? >> can >> we contact Eliot to provide such a primitive? >> > > As promised, I uploaded my LargeIdentityDictionary implementation to > http://leves.web.elte.hu/**squeak/**LargeIdentityDictionary.st<http://leves.web.elte.hu/squeak/LargeIdentityDictionary.st>. > The numbers will be a bit worse compared to LargeIdentitySet, because of > the lack of the primitive, but it's still 2-3x faster than other solutions > (IdentityDictionary, PluggableIdentityDictionary, subclassing, etc). I'm > about to propose this primitive with other improvements on the vm-dev list. > > Hi Eliot/Levente. What is the status of this? Do we have already the new primitive? If true, how can we adapt LargeIdentitySet to use such new primitive? Thanks! > > Levente > > >> thanks >> >> On Fri, Dec 16, 2011 at 3:28 PM, Levente Uzonyi <leves(a)elte.hu> wrote: >> >> On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: >>> >>> On 16.12.2011 03:26, Levente Uzonyi wrote: >>> >>>> >>>> >>>>> How about my numbers? :) >>>>> >>>>> "Preallocate objects, so we won't count gc time." >>>>> n := 1000000. >>>>> objects := Array new: n streamContents: [ :stream | >>>>> n timesRepeat: [ stream nextPut: Object new ] ]. >>>>> >>>>> set := IdentitySet new: n. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "4949" >>>>> >>>>> set := LargeIdentitySet new. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "331" >>>>> >>>>> set := (PluggableSet new: n) >>>>> hashBlock: [ :object | object identityHash * 4096 + object class >>>>> identityHash * 64 ]; "Change this to #basicIdentityHash in Pharo" >>>>> equalBlock: [ :a :b | a == b ]; >>>>> yourself. >>>>> Smalltalk garbageCollect. >>>>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. "5511" >>>>> >>>>> >>>>> I also have a LargeIdentityDictionary, which is relatively fast, but >>>>> not >>>>> as fast as LargeIdentitySet, because (for some unknown reason) we don't >>>>> have a primitive that could support it. If we had a primitive like >>>>> primitive 132 which would return the index of the element if found or >>>>> 0 if >>>>> not, then we could have a really fast LargeIdentityDictionary. >>>>> >>>>> >>>>> Levente >>>>> >>>>> Hehe yes, if writing a version fully exploiting the limited range, >>>> that's >>>> probably the approach I would go for as well. >>>> (IAssuming it's the version at http://leves.web.elte.hu/** >>>> squeak/LargeIdentitySet.st<htt**p://leves.web.elte.hu/squeak/** >>>> LargeIdentitySet.st<http://leves.web.elte.hu/squeak/LargeIdentitySet.st> >>>> > >>>> ) >>>> >>>> Mariano commented in the version at http://www.squeaksource.com/** >>>> FuelExperiments <http://www.squeaksource.com/**FuelExperiments<http://www.squeaksource.com/FuelExperiments>> >>>> that it's >>>> >>>> slow for them, which I guess is due to not adopting #identityHash calls >>>> to >>>> #basicIdentityHash calls for Pharo: >>>> ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 >>>> So it basically uses 1 bucket instead of 4096... Whoops. :) >>>> >>>> Uploaded a new version to the MC repository which is adapted for Pharo, >>>> on the same machine my numbers were taken from, it does the same test >>>> as I >>>> used above in 871 ms. (181 with preallocation). >>>> >>>> >>> Cool. One more thing: in Squeak the method using primitive 132 directly >>> was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >>> this was also added to Pharo, then the #pointsTo: sends should be changed >>> to #instVarsInclude:, otherwise Array can be reported as included even if >>> it wasn't added. >>> I'll upload my LargeIdentityDictionary implementation to the same place >>> this evening, since it's still 2-3 factor faster than other solutionts >>> and >>> there seem to be demand for it. >>> >>> >>> Levente >>> >>> >>> Cheers, >>>> Henry >>>> >>>> >>>> >>>> >>> >> >> -- >> Mariano >> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >> >> > -- Mariano http://marianopeck.wordpress.com

4 12

Suspect single float types
by Nicolas Cellier 05 May '14

05 May '14

I read in CCodeGenerator>>computeKernelReturnTypes fetchFloatAt:into: #float storeFloatAt:from: #float fetchFloatAtPointer:into: #float storeFloatAtPointer:from: #float fetchSingleFloatAt:into: #float storeSingleFloatAt:from: #float fetchSingleFloatAtPointer:into: #float storeSingleFloatAtPointer:from: #float Ain't these #double?

2 2

PharoVM on Windows & Code::Blocks for debugging
by phil＠highoctane.be 01 Apr '14

01 Apr '14

I'd like to know if someone managed to get Code::Blocks to work with the MinGW configuration. I've the build running from the command line. I can also configure C::B to use makefiles. Now, there seems to be some glitches and was curious to know if someone had a working setup. (Yes, one can use gdb in the command line mode, but it is ugly. And TUI doens't work on Windows). There are many interesting pieces of code in the Windows platform code. And I'd like (at one point) to have a real headless Pharo.exe There are also the pieces to get the Pharo.exe run as a service natively. I've been studying this with SourceNavigator and done basic gdb work. But for larger work, I'd like to have a decent IDE. NOTE: my email appears to bounce on the vm-dev list, please someone help me sort that out. Phil

2 1

Re: [Pharo-dev] threading in Pharo
by Eliot Miranda 31 Mar '14

31 Mar '14

Hi Ben, On Wed, Mar 26, 2014 at 7:04 AM, Ben Coman <btc(a)openinworld.com> wrote: > Eliot Miranda wrote: > > > > > On Tue, Mar 25, 2014 at 10:21 AM, Eliot Miranda <eliot.miranda(a)gmail.com>wrote: > >> Hi Igor, >> >> >> you have a point but I disagree. The scheduler is defined in the >> implementation section of the blue book. It could be more explicit, but >> the blue book scheduler is a known and simple system. In my threading work >> I've made sure to preserve its semantics (cooperative within priorities, >> preemptive across priorities, thread switch at activation of non-primitive >> sends and backward branches). The only serious bug I know of (that >> preempting sends a process to the back of its run-queue) was addressed in >> Cog and the VW VM. We can and should write up the semantics, but they're >> not undefined, they're simple and they do work. >> > > Oops. I lie. There is a serious bug with Semaphore>>critical: and the > like. There's a suspension point after signal and before block evaluation > that can result in deadlock if a higher priority process signals the > semaphore. Since the higher priority process is running the lower priority > process never makes progress to run the ensure block or cvlear the flag or > whatever it needs to do, and hence the system deadlocks. This requires > thought ;-) So I do know of one outstanding bug. > > My grasp of concurrency controls hasn't been tested in 20 years, so > naively I would say: If a lower priority process "L" holds** a semaphore > when the higher priority process "H" signals the semaphore, temporarily > raise the priority of the "L", or temporarily lower the priority of "H". > Is there something simple that I missing? > There are a couple of obvious solutions. One is to avoid the suspension point somehow, another is adding priority inversion. Adding priority inversion to the scheduler is a significant change which would increase complexity. You yourself have glossed over detail by saying "temporarily". I've glossed over a lot with "somehow". How would these "temporarily"s and "somehow"s be implemented? Your design sketches are appreciated. cc'ing vm-dev and squeak dev lists. this is an important discussion. > cheers -ben > > > > >> Eliot (phone) >> >> On Mar 25, 2014, at 10:11 AM, Igor Stasenko <siguctua(a)gmail.com> wrote: >> >> >> >> >> On 25 March 2014 17:31, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: >> >>> Hi Igor, >>> >>> >>> On Tue, Mar 25, 2014 at 5:05 AM, Igor Stasenko <siguctua(a)gmail.com>wrote: >>> >>>> >>>> >>>> >>>> On 24 March 2014 22:54, phil(a)highoctane.be <phil(a)highoctane.be> wrote: >>>> >>>>> On Mon, Mar 24, 2014 at 8:23 PM, Alexandre Bergel < >>>>> alexandre.bergel(a)me.com> wrote: >>>>> >>>>>> >> I am working on a memory model for expandable collection in Pharo. >>>>>> Currently, OrderedCollection, Dictionary and other expandable collections >>>>>> use a internal array to store their data. My new collection library recycle >>>>>> these array instead of letting the garbage collector dispose them. I simply >>>>>> insert the arrays in an ordered collection when an array is not necessary >>>>>> anymore. And I remove one when I need one. >>>>>> > >>>>>> > Hm, is that really going to be worth the trouble? >>>>>> >>>>>> This technique reduces the consumption of about 15% of memory. >>>>>> >>>>>> >> At the end, #add: and #remove: are performed on these polls of >>>>>> arrays. I haven't been able to spot any problem regarding concurrency and I >>>>>> made no effort in preventing them. I have a simple global collection and >>>>>> each call site of "OrderedCollection new" can pick an element of my global >>>>>> collection. >>>>>> >> >>>>>> >> I have the impression that I simply need to guard the access to >>>>>> the global poll, which is basically guarding #add: #remove: and #includes: >>>>>> > >>>>>> > One of the AtomicCollections might be the right things for you? >>>>>> >>>>>> I will have a look at it. >>>>>> >>>>>> >> What is funny, is that I did not care at all about multi-threading >>>>>> and concurrency, and I have not spotted any problem so far. >>>>>> > >>>>>> > There isn't any 'multi-threading' like in Java, you got a much more >>>>>> control version: cooperative on the same priority, preemptive between >>>>>> priorities. >>>>>> > So, I am not surprised. And well, these operations are likely not >>>>>> to be problematic when they are racy, except when the underling data >>>>>> structure could get into an inconsistent state itself. The overall >>>>>> operations (adding/removing/searing) are racy on the application level >>>>>> anyway. >>>>>> > >>>>>> > However, much more interesting would be to know what kind of >>>>>> benefit do you see for such reuse? >>>>>> > And especially, with Spur around the corner, will it still pay off >>>>>> then? Or is it an application-specific optimization? >>>>>> >>>>>> I am exploring a new design of the collection library of Pharo. Not >>>>>> all the (academic) ideas will be worth porting into the mainstream of >>>>>> Pharo. But some of them yes. >>>>>> >>>>>> Thanks for all your help guys! You're great! >>>>>> >>>>>> Cheers, >>>>>> Alexandre >>>>>> >>>>>> -- >>>>>> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: >>>>>> Alexandre Bergel http://www.bergel.eu >>>>>> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. >>>>>> >>>>>> >>>>>> >>>>>> >>>>> An interesting method I stumbled upon which may help in >>>>> understanding how these things do work. >>>>> >>>>> BlockClosure>>valueUnpreemptively >>>>> "Evaluate the receiver (block), without the possibility of >>>>> preemption by higher priority processes. Use this facility VERY sparingly!" >>>>> "Think about using Block>>valueUninterruptably first, and think >>>>> about using Semaphore>>critical: before that, and think about redesigning >>>>> your application even before that! >>>>> After you've done all that thinking, go right ahead and use it..." >>>>> | activeProcess oldPriority result semaphore | >>>>> activeProcess := Processor activeProcess. >>>>> oldPriority := activeProcess priority. >>>>> activeProcess priority: Processor highestPriority. >>>>> result := self ensure: [activeProcess priority: oldPriority]. >>>>> >>>>> >>>> I would not recommend you to use this method for anything. >>>> This method heavily relies on how process scheduler works, and in >>>> case of any changes, it may break everything. >>>> For the sake of programming, one shall never assume there is a way to >>>> "stop the world while i busy doing something". >>>> >>> >>> Really? Surely any system as interactive as Smalltalk can benefit >>> from a stop-the-rest-of-the-world scheduling facility, and surely packaging >>> it as BlockClosure>>valueUnpreemptively would be a convenient way of doing >>> so. Surely the right attitude for an implementor of a threading system for >>> Smalltalk would be "Sure, I can implement that, even in a truly concurrent, >>> multi-processor environment". It may take some doing but it's an important >>> facility to have. It shouldn't be abused, but when you need it, you need >>> it. >>> >>> >> There should be hard guarantees from VM to do it. Right now there's none. >> That's my point. >> Like special primitive(s) for disabling interrupts/scheduling and >> enabling it back again. >> Let us be realistic: the above implementation is based on insider's >> knowledge how scheduling works, lacking any notion of contract between VM >> and image. >> Right now it just based on implementation detail rather than on >> guaranteed and well defined semantic. >> >> It is no doubt, sometimes you may need such hammer to stop the world. >> And it is no doubt (to me) that one should avoid using it unless it is >> impossible to do otherwise. >> >> >> >>> -- >>> best, >>> Eliot >>> >> >> >> >> -- >> Best regards, >> Igor Stasenko. >> >> > > > -- > best, > Eliot > > > -- best, Eliot

2 1

when the vm crashes...
by Bob Arning 30 Mar '14

30 Mar '14

Is there enough information in the attached file to point to a possible cause? Cheers, Bob

4 13

Changes to ObjectMemory needed to get StackInterpreter to compile from generated source
by gettimothy 30 Mar '14

30 Mar '14

Howdy. I have a mcz VMMaker.oscog-tty.661.mcz in my local package-cache if you would like me to push it live. Here is the low-down. Compiling the StackInterpreter from the code currently in SVN in the stacksrc directory works fine--it is probably an artifact from the last time Eliot generated the StackInterpreter source code before moving on to other things. However, using VMMaker,cleaning out that src tree and generating new source results in code that will not compile due to mesage sends to nowhere in NewObjectMemroy>>allInstancesOf: The compile errors look like this: ...oscogvm/stacksrc/vm/gcc3x-interp.c:10108: undefined reference to `minLargeHeaderSize' ...oscogvm/stacksrc/vm/gcc3x-interp.c:10109: undefined reference to `sizeHeaderputBodySize' ...oscogvm/stacksrc/vm/gcc3x-interp.c:10180: undefined reference to `sizeHeaderputBodySize' the unimplemented methods are sizeHeader:putBodySize and minLargeHeaderSize I have not studied the VM architectures yet*, so I just flipped a coin, and took a wild guess at fixing things. For sizeHeader:putBodySize, I did a cut-n-paste from baseHeader:put: sizeHeader: oop putBodySize:aWord ^ self longAt: oop put: aWord. for minLargeHeaderSize, I returned BaseHeaderSize (Hey! BaseHeaderSize just HAS to be a valid header size, soooooo, let's make that the minimum one!). minLargeHeaderSize ^BaseHeaderSize Compilation worked and the image runs. 16 failures and 2 errors in test run. If the above is not correct, I will revisit this after my plugin study and vm architecture study. . * (I will be studying them soon, I feel pretty good about compiling from scratch now and will be moving to Plugins tomorrow--after that, 64 bits or bust baby!--which implies studying the vm architecture in depth )

1 0

StackInterpreter builds....
by gettimothy 29 Mar '14

29 Mar '14

I was able to compile and run the StackInterpreter Image ----- /home/tty/usr/src/smalltalk/runStackVM32/Squeak4.6-13700.image Squeak4.6 latest update: #13700 Current Change Set: Unnamed1 Image format 6505 (32 bit) Virtual Machine --------------- /home/wm/usr/local/stackVM32/lib/squeak/4.0-2886/squeak Croquet Closure Stack VM [StackInterpreter VMMaker.oscog-eem.658] Unix built on Mar 29 2014 14:05:29 Compiler: 4.8.2 platform sources revision VM: r2886 http://www.squeakvm.org/svn/squeak/branches/Cog Date: 2014-03-28 19:11:48 -0400 Plugins: r2545 http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins StackInterpreter VMMaker.oscog-eem.658 uuid: e40c5fd5-9990-433f-bd76-2f8d0324e24f Mar 29 2014 but there is much work I need to do and things to learn yet. I had to remove most of the plugins from plugins.int/ext to get the thing to compile and will be adding them back one-by-one to try to get my mind about this. I am thinking that some plugins look in /lib64 instead of /lib for their libraries--that kind of stuff. Once I get fluent at this, we can discuss the port to native 64 or perhaps you would prefer I do some plugin bugs on Cog. Hopefully I can start doing something useful next week. Thanks again for your help tty

1 0

This is the StackInterpreter?
by gettimothy 29 Mar '14

29 Mar '14

What is worse than things not working is when things start working and I don't know why. After a hiatus in PHP land, I am back to Squeak Land and lo and behold, the Stack Interpreter 32 bit compiled and runs the 4.6 image. That is if what you see below is in fact the Stack Interpreter. Please let me know if it is not Image ----- /home/tty/usr/src/smalltalk/runStackVM32/Squeak4.6-13700.image Squeak4.6 latest update: #13700 Current Change Set: Unnamed1 Image format 6505 (32 bit) Virtual Machine --------------- /home/wm/usr/local/stackVM32/lib/squeak/4.0-2881/squeak Croquet Closure Cog VM [CoInterpreter VMMaker.oscog-eem.641] Unix built on Mar 28 2014 12:22:20 Compiler: 4.8.2 platform sources revision VM: r2881 http://www.squeakvm.org/svn/squeak/branches/Cog Date: 2014-03-13 20:45:37 -0400 Plugins: r2545 http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins CoInterpreter VMMaker.oscog-eem.641 uuid: c0305b66-db2d-45aa-8d25-267732aa1784 Mar 28 2014 StackToRegisterMappingCogit VMMaker.oscog-eem.641 uuid: c0305b66-db2d-45aa-8d25-267732aa1784 Mar 28 2014 If so, then we have a second data point of building a Stack Interpreter on a 64 bit machine with 32 bit compat libs. file squeak squeak: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), not stripped An interesting point of note is of note is that I was getting ready to debug a vm that would not run. It was built with this configure command. ../../../platforms/unix/config/configure CC="gcc -m32" CXX="g++ -m32" --without-npsqueak CFLAGS="-g -O2 -msse2 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DNDEBUG -DITIMER_HEARTBEAT=1 -DNO_VM_PROFILE=1 -DCOGMTVM=0 -DDEBUGVM=0" LIBS=-lpthread --x-libraries=/usr/lib --prefix=/home/tty/usr/local/stackVM32 and the result was a vm that built, but would hang waiting on what I believed was a signal. Setting out to debug this non-running vm, I changed to -O2 flag to -Og which rhymes with 'd' and that spells "debugging". ../../../platforms/unix/config/configure CC="gcc -m32" CXX="g++ -m32" --without-npsqueak CFLAGS="-g -Og -msse2 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DNDEBUG -DITIMER_HEARTBEAT=1 -DNO_VM_PROFILE=1 -DCOGMTVM=0 -DDEBUGVM=0" LIBS=-lpthread --x-libraries=/usr/lib --prefix=/home/tty/usr/local/stackVM32 And, lo and behold, no more trouble in River City. It ran.

2 5

Re: [Pharo-dev] Capturing stdout
by phil＠highoctane.be 29 Mar '14

29 Mar '14

On Windows, there is work to do on the VM platform sources to get the command line output to work properly. There are a set of MsgBox calls in there where one could use a nice little printf/puts. I've been looking at it but there is more than meets the eye there. Phil On Fri, Mar 28, 2014 at 10:23 PM, Max Leske <maxleske(a)gmail.com> wrote: > Maybe... Since Pharo now has a command line handler, you could redirect > stdout to the vm process? Will probably not work while the image is running > I guess. But maybe it will (see for example > http://serverfault.com/questions/178457/can-i-send-some-text-to-the-stdin-o… > ) > > > On 28.03.2014, at 21:56, Sean P. DeNigris <sean(a)clipperadams.com> wrote: > > > Emilio Oca-3 wrote > >> I managed to do that in Windows using... ProcessWrapper... > > > > As it's for a core test, I need to find a way to do it without loading > > additional libraries. I could pipe the output to a tmp file if there's > > nothing... > > > > > > > > ----- > > Cheers, > > Sean > > -- > > View this message in context: > http://forum.world.st/Capturing-stdout-tp4751320p4751521.html > > Sent from the Pharo Smalltalk Developers mailing list archive at > Nabble.com. > > > > >

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Vm-dev March 2014