Squeak-dev April 2009

squeak-dev@lists.squeakfoundation.org

104 participants
168 discussions

[ANN] A new scheduler + VM changes alpha-release
by Igor Stasenko 01 May '09

01 May '09

Hello guys, i hope you are interested in having a rock-solid & safe concurrency in smalltalk, despite it green-threaded. A new implementation of scheduler will allow us to improve stuff and be no longer dependant from VM side scheduling policy. You can find the changes on mantis: http://bugs.squeak.org/view.php?id=7345 Please, give me a feedback, if you can successfully build VM with new changes on all platforms. I used a pretty old VMMaker package version for my VM: VMMaker-dtl.91 Will try to see if it can be built with latest one , available on SqS. Andreas (& others) - do you have any tests/benchmarks, how to measure a new scheduler overhead comparing to old one? Personally, i didn't noticed much speed degradation :) And last thing, i need an advice how to implement a #callbackEnter: properly. Its currently leaved unchanged, using old VM scheduling policy. So, its unsafe to use plugins with callbacks if you running under new scheduler. A callbackEnter remembers the active process, suspends it and then activates it back when callback is done. The problem is, that squeak 3.10 doesn't have any image-side code which i could change to play with callbacks :( A fix for callbacks should be pretty easy: - at entry, deactivate active process (which should be NOT the interrupt process) & switch to interrupt process, so it could schedule next available process. - when leaving switch from interrupt process to previously remembered active process. So, a certain preconditions should be met: - a code which calls a primitives which using callbacks should never run in interruptProcess - a code , which returns from a callback should run in interruptProcess -- Best regards, Igor Stasenko AKA sig.

4 14

[RFI] A way to browse code saved with a *.pr project.
by Jerome Peace 01 May '09

01 May '09

Hi all, Problem: You develop an application in a project under one set of circumstances. Save the project out to disk. Them want to load it back in another entirely different set of circumstances. Some Browser or morph or something saved with the project doesn't want to load back in.* The story I would like to see implemented is: In addition to the option of loading the project 1) can I opt instead to load just the code associated with the project into a code browser. And/Or 2) save the code associated with the project as a fileout to the disk w/o loading the project at all. (Then filelist would allow the many things it knows to do with filed out code.) This story is strictly for .pr files. I know to generally save out a projects change set first before saving a project. I am looking for a way to ease things, particularly when I forget the extra steps. Yours in curiosity and service, --Jerome Peace *[ For me typically, I might be going from a fresh squeak 3.10.2 + development project over to etoys. Typically one gets tripped up by the etoys morphExtentionPlus. Or equally difficult from one of damian's squeak-dev images back to a fresh 3.10.2. In which case its the presence of an omnibrowser that creates loading problems.

2 1

Multi-core VMs (was: Suspending process fix)
by Michael van der Gulik 01 May '09

01 May '09

On 4/29/09, Igor Stasenko <siguctua(a)gmail.com> wrote: > As for moving to multi-cores.. yes, as Gulik suggests, its like adding > a new dimension: > - local scheduler for each core > - single global scheduler for freezing everything > > This, of course, if we could afford running same object memory over > multiple cores. Handling interpreter/object memory state(s) with > multiple cores is not trivial thing. Implementing it isn't hard. It's fixing all the bugs we'll find that's hard. There'll be bugs in the image and in the VM, and it'll be a good 30 years before we've found them all. *Most* parts of the VM will continue working fine. The parts that will break... er... some of the parts that will break are: * garbage collection. * allocating memory for new objects. * primitives and devices. * pointer swapping *might* need to be atomic (become:, becomeForward:). * Semaphore signalling. * (more things???) Most other things should work fine if we fire up a second interp() on another pthread which shares the same object memory. Writing to object slots (aka instance variables) should continue to work fine provided that the write itself occurs in a single atomic machine code instruction and that the object that the new reference points to is already allocated and initialized. Creating new objects could be improved by having each scheduler have it's own eden space. But then the garbage collection becomes more complex. The other alternative is to have a global image lock and whoever whines about it can implement a better solution. I think that's it. The only important thing shared between different OS threads (pthreads?) would be the object memory, and all you really do with the object memory is write or read from object slots, create new objects and run garbage collection. Also, it shouldn't always be necessary to freeze the entire image. If every scheduler has its own eden space and its own list of processes that nothing else is allowed to modify, then there's no need to freeze the whole VM for scheduler work. The only time the whole VM will need to freeze is for garbage collection, but even being very intelligent with a new GC design can avoid that. Gulik. -- http://gulik.pbwiki.com/

6 7

Java to Smalltalk conversion (on comp.lang.smalltalk)
by Davide Grandi 30 Apr '09

30 Apr '09

Hello to everyone, I've just posted a message on comp.lang.smalltalk (and a few redirects on some st mailing list) about the definition of a process that could allows the translation of a java app into smalltalk. I'm looking for pointers, suggestions, caveat or plain impressions on that topic. I think that an eventually feasible solution will touch many interesting points of the computer science : Source Configuration Management systems, refactoring, [java] parsing and analysis, automated testing, dependency analysis, open source class libraries, ... Feel free to post your comments here or on c.l.s. Best regards and happy squeaking ! Davide -- Ing. Davide Grandi davide.grandi(a)mclink.it

1 0

Re: New scheduling policies [Was: Re: Re: Suspending process fix]
by Andreas Raab 30 Apr '09

30 Apr '09

Hi Jecel - That is actually great news. I had absolutely no idea that the Self scheduler was implemented in Self (I had naturally assumed it was implemented in C). The fact that it uses the same idea is a great validation of our thoughts here ;-) Do you by any chance know how Self performed in heavy process switch benchmarks? Has anyone ever assessed the overhead of the scheduler? Cheers, - Andreas Jecel Assumpcao Jr wrote: > I am sorry that this week I won't be able to really participate in this > thread, but I couldn't help mentioning that you guys are reinventing the > Self scheduler and so might want to take a look at that. > > It includes a single primitive called TWAINS (transfer and wait for an > interrupt or signal) for switching from one process to another. When > Self starts up, it runs a single thread. At some point the scheduler is > started - it is a process just like any other. The only thing special > about the scheduler is that it calls the TWAINS primitives and none of > the others ever do so. There is no protection against this, however. > > -- Jecel > P.S.: the scheduler difference was what made a friend of mine drop > Squeak and use Self for his PhD project several years ago > > >

2 1

Sunit bug ?
by Mariano Martinez Peck 30 Apr '09

30 Apr '09

Hi folks! Today, at work, a friend ask me about a method in TestResource because he thought it was a bug. We look at it, and for us, it could be a bug. So, I ask to you. See TestResource class side isAvailable ------------------------ isAvailable ^self current notNil and: [self current isAvailable] ------------------------ the first part (self current notNil) ALWAYS return true because of the implementation of self current: ------------------------ current current isNil ifTrue: [current := self new]. ^current ------------------------- The second part too. Because the implementation of self available (object side method): ------------------------- isAvailable "override to provide information on the readiness of the resource" ^true ------------------------- Ok, you can override and change this method in subclasses, however, the first part is ALWAYS true. Is this correct ? or isAvailable should be: isAvailable ^ self current isAvailable Thanks! Mariano

2 1

Re: Jabber install problem?
by John Chludzinski 30 Apr '09

30 Apr '09

> I think that's a really old version (Michael, if I'm right can you > update the SqueakMap entry?). Try loading the latest from > http://www.squeaksource.com/Jabber <http://www.squeaksource.com/Jabber> I went to this site and downloaded: Jabber-jf.8.mcz <http://www.squeaksource.com/Jabber/Jabber-jf.8.mcz>. I noticed KomServices-jf.6.mcz <http://www.squeaksource.com/Jabber/KomServices-jf.6.mcz> listed. I'm currently using the latest Comanche (via SqueakMap) + Seaside (also via SqueakMap). Do I need to update Comanche (with KomServices-jf.6.mcz <http://www.squeaksource.com/Jabber/KomServices-jf.6.mcz> )? ---John

4 6

New scheduling policies [Was: Re: [squeak-dev] Re: Suspending process fix]
by Igor Stasenko 30 Apr '09

30 Apr '09

I got a new VM build, which is ready to be tested with new scheduler(after i implement it). I rewrote the external signaling stuff & interrupt checking. Now its not signals any semaphores. Instead, i added a primitive which explicitly fetching all pending signals to array and flushing pending signals VM internal buffer. Then in interrupt checker i simply switch active process to special 'interrupt process' (or scheduler process - Andreas), if there any pending signals to handle. What does it means for language side? It means a very cool thing: you are no longer obliged to use semaphores to respond to signals! You can register any object in external objects table. And new scheduler will simply do: externalObjects := Smalltalk externalObjects. signalIndexes do: [:index | (externalObjects at: i) handleExternalSignal. ] so, as long as your registered object responds to #handleExternalSignal, you are free to choose what to do in response to signal. Semaphores, of course will signal themselves. After replacing scheduler with new model, the VM will no longer need to know anything about semaphores. This is because any scheduling related stuff will become 100% language-side specific. So, that with new model, multiple primitives become obsolete: primitiveYield primitiveWait primitiveSuspend primitiveSignal primitiveResume instead of them there are two new primitives: primitiveTransferToProcess "sets an ActiveProcess to new process, sets an InterruptedProcess to the process which was active set a ProcessAction to anAction object " primitiveFetchPendingSignals "primitive, fill an array (first argument)with special objects indexes, needed to be signaled. Returns a number of signals being filled. Or negative number indicating that array is not big enough to fetch all signals at once. Primitive fails if first argument is not array. " 2009/4/29 Igor Stasenko <siguctua(a)gmail.com>: > 2009/4/29 Andreas Raab <andreas.raab(a)gmx.de>: >> Igor Stasenko wrote: >>> >>> I came to an idea , you might be interested in. >>> As many of us know, some CPUs having a special mode - interrupt mode. >>> What if we introduce the interrupt mode for scheduler? >> >> [... snip ...] >>> >>> Now i trying to imagine, how a basic stuff might look like(please >>> correct me if its utterly wrong way ;), if we will be able to use >>> interrupt mode. >> >> This is actually along similar lines of thought that I had when I was >> thinking of how to get rid of the builtin VM scheduling behavior. The main >> thought that I had was that the VM may have a "special" process - the >> scheduler process (duh!) which it runs when it doesn't know what else to do. >> The VM would then not directly schedule processes after semaphore signals >> but rather put them onto a "ready" queue that can be read by the scheduler >> process and switch to the scheduler process. The scheduler process decides >> what to run next and resumes the process via a primitive. Whenever an >> external signal comes in, the VM automatically activates the scheduler >> process and the scheduler process then decides whether to resume the >> previously running process or to switch to a different process. >> >> In a way this folds the timer process into the scheduler (which makes good >> sense from my perspective because much of the work in the timer is stuff >> that could be more effectively take place in the scheduler). The >> implementation should be relatively straightforward - just add a scheduler >> process and a ready list to the special objects, and wherever the VM would >> normally process switch you just switch to the scheduler. Voila, there is >> your user-manipulable scheduler ;-) And obviously, anything that is run out >> of the scheduler process is by definition non-interruptable because there is >> simply nothing to switch to! >> > > Very nice indeed. That's even better that my first proposal. > ProcessorScheduler>>schedulingProcessLoop > [ > self handlePendingSignalsAndActions. > activeProcess ifNil: [ self idle ] ifNotNil: [ self > primitiveTransferControlTo: activeProcess]. > ] repeat. > > and when any process, somehow stops running > (suspend/wait/terminate/interrupted etc), VM will again switch to > scheduler process loop. > > What is important in having it, that there is guarantee to be not > preempted by anything. Simply by having this, many > concurrency/scheduling related problems can be solved by language-side > implementation, without fear of having gotchas from VM side. > > Also, VM doesn't needs to know details about priorities, suspending, > etc etc.. - which means that we can simplify VM considerably and > implement same parts on the language side, where everything is late > bound :) > > As for moving to multi-cores.. yes, as Gulik suggests, its like adding > a new dimension: > - local scheduler for each core > - single global scheduler for freezing everything > > This, of course, if we could afford running same object memory over > multiple cores. Handling interpreter/object memory state(s) with > multiple cores is not trivial thing. > > If we going to keep more isolated model (islands, hydra ) then we need > no/minimal changes to scheduler - each scheduler serves own island and > receives asynchronous signals from other collegues through shared > queue. > >> Cheers, >> - Andreas >> > > > -- > Best regards, > Igor Stasenko AKA sig. > -- Best regards, Igor Stasenko AKA sig.

5 6

Jabber install problem?
by John Chludzinski 29 Apr '09

29 Apr '09

I tried to install (file in) the Jabber package for Squeak (via SqueakMap) and got a download error when it attempted a UTF8 conversion on some bullets: "Dispatch this presence message according to type: · available Default. Signals that you are online. · unavailable You are no longer available. Allows for offline options to be set. · subscribe An attempt to subscribe to the users presence · subscribed The subscribe attempt was successful. · unsubscribe A user is unsubscribing from your presence. The server handles the actual unsubscription, as is sent just for notification. · unsubscribed A presence unsubscribe completed succesfully. · probe A query to a users presence. The server processes these requests. Note that clients do not need to handle this type - it is server only." I removed these bullets and save the file, then exited the image without saving. Got back into Squeak and then filed in (installed) the modified file. It worked fine BUT is there something additional that I stopped when I was installing from SqueakMap and the error occurred? BTW, I'm using the latest image for Windows - 3.10.2-7179. ---John

3 2

Need help with identifying the problem
by Igor Stasenko 29 Apr '09

29 Apr '09

In VM w/o closures, i implemented a new primitive, #primitiveInterrupt, which looks like following: primitiveInterrupt "primitive" | context interruptProcess sched oldProc | self export: true. context := self stackTop. sched := self schedulerPointer. "check that we using a new scheduler , which having additional instance variables: - interruptProcess - interruptedProcess" (self lastPointerOf: sched) >= (InterruptedProcessIndex * BytesPerWord + BaseHeaderSize) ifFalse: [ ^ self primitiveFail ]. oldProc := self fetchPointer: ActiveProcessIndex ofObject: sched. interruptProcess := self fetchPointer: InterruptProcessIndex ofObject: sched. oldProc == interruptProcess ifTrue: [ "we are already in interrupt, proceed as with sending #value to block" ^ self primitiveValue ]. self storePointer: InterruptedProcessIndex ofObject: sched withValue: oldProc. self storePointer: SuspendedContextIndex ofObject: interruptProcess withValue: context. self putToSleep: oldProc. self transferTo: interruptProcess. self pop: 1 thenPush: self nilObject A primitive is hooked in BlockContext>>primInterrupt : primInterrupt "Primitive. Evaluate the block in special interrupt process" <primitive: 'primitiveInterrupt' module: ''> self primitiveFail And code, which is using it looks like following: Processor>>interruptWith: aBlock "Interrupt the currently running process to evaluate given block argument inside interrupt process" | result | [ result := aBlock value. activeProcess suspend. ] asContext primInterrupt. ^ result the VM crashing at attempt to return from aBlock value (it never gets to the point for evaluating 'activeProcess suspend'). if i write code like following: Processor>>interruptWith: aBlock "Interrupt the currently running process to evaluate given block argument inside interrupt process" | result | [ [ result := aBlock value. ] value activeProcess suspend. ] asContext primInterrupt. ^ result Then VM does not crashing, but silenty quits to OS. The only way how i managed to make it work, is to do it as: Processor>>interruptWith: aBlock "Interrupt the currently running process to evaluate given block argument inside interrupt process" [ self runBlock: aBlock ] asContext primInterrupt. and runBlock: aBlock is: runBlock: aBlock aBlock value. activeProcess suspend. it works fine, except that i unable to use the returned result. - i could put it to instance variable.. but its a bit awkward :( I suspect this somehow connected with the return to context who having sender = nil. (a bottom context of process always having sender = nil). But i wonder, why my code not working, while code, which used for #fork, it does? Any suggestions? newProcess "Answer a Process running the code in the receiver. The process is not scheduled." <primitive: 19> "Simulation guard" ^ Process forContext: [self value. Processor terminateActive] asContext priority: Processor activePriority here, the initial context of newly created process will point to: [self value. Processor terminateActive] and in my case, an initial context of interrupt process is set in #primitiveInterrupt and points to: [ result := aBlock value. activeProcess suspend. ] why i works for above case, but not for mine, no idea... -- Best regards, Igor Stasenko AKA sig.

2 2

Jump to page: