[squeak-dev] Re: too many copiedValues [was Interesting survey about smalltalk]

Nicolas Cellier nicolas.cellier.aka.nice at gmail.com
Mon Jun 21 22:26:08 UTC 2010


2010/6/22 Eliot Miranda <eliot.miranda at gmail.com>:
> Hi Nicolas,
>
> On Mon, Jun 21, 2010 at 2:59 PM, Nicolas Cellier
> <nicolas.cellier.aka.nice at gmail.com> wrote:
>>
>> 2010/6/21 Nicolas Cellier <nicolas.cellier.aka.nice at gmail.com>:
>> > Compiler hack is available for Squeak 3.9 at
>> > http://bugs.squeak.org/view.php?id=2918
>> > Load the 3 .cs in indicated order.
>> >
>> > I will publish an update for latest trunk at the same place when I'll
>> > have a bit of time...
>> >
>>
>> Finally, I published a Compiler version for latest trunk in this
>> SqueakSource repository
>> Plus a hack or two to make external interface loadable in closure VM
>> (there is an additional limitation in closure that is max number of
>> block copied values<16)
>
> I'm interested to see the code that exceeds this limitation.  Hopefully
> it'll be temporary, but it'll take a bytecode redesign to overcome, so I
> can't fix this anytime soon.  Sorry it has bitten you!

Oh, I guess one wouldn't code so many temps.
It's some kind of auto-generated code for FFI call of LAPACK functions.
LAPACK is written in FORTRAN.
FORTRAN has all parameters passed by reference.
Consequence is that many arguments passed to a LAPACK function (like
dimensions of matrices) require an allocation.
I generated the allocation for passing some scalar inputs inside a
block pretected with an #ensure: ensuring deallocation...
But many LAPACK arguments means many temps and two cases of copiedValues > 15.

Methods are:

LapackLibrary>>hegvdWithitype: itype jobz: jobz uplo: uplo n: n a: a
lda: lda b: b ldb: ldb w: w work: work lwork: lwork rwork: rwork
lrwork: lrwork iwork: iwork liwork: liwork
LapackLibrary>>tgexcWithwantq: wantq wantz: wantz n: n a: a lda: lda
b: b ldb: ldb q: q ldq: ldq z: z ldz: ldz ifst: ifst ilst: ilst work:
work lwork: lwork

in http://www.squeaksource.com/Smallapack/Smallapack-External-nice.16.mcz

pre-requisite is FFI and hack in
http://www.squeaksource.com/Smallapack/Compiler-nice.147.mcz
in order to compile methods with more than 15 arguments...

Nicolas

>>
>> Sometimes, it gets a bit tricky to initialize Smallapack.
>> I did something like:
>>
>> LapackMatrix initialize.
>> LapackMatrix allSubclasses do: [:e | [e initializeClassInstVars] ifError:
>> []].
>>
>> Nicolas
>>
>> > Then the latest Smallapack code is at
>> > http://www.squeaksource.com/Smallapack.html
>> >
>> > If you are on linux, you'd better use a relatively new VM correcting
>> > http://bugs.squeak.org/view.php?id=3929
>> >
>> > Nicolas
>> >
>> >
>> > 2010/6/21 arul selvan <arul.selvan at gmail.com>:
>> >> nicolas,
>> >>
>> >> i will be interested to test if a port for squeak is available
>> >>
>> >> arul
>> >>
>> >>
>> >> On Mon, Jun 21, 2010 at 12:41 PM, Nicolas Cellier
>> >> <nicolas.cellier.aka.nice at gmail.com> wrote:
>> >>> I started Smallapack for this purpose.
>> >>> Smallapack interfaces BLAS+LAPACK thru FFI.
>> >>> Very much like numpy...
>> >>>
>> >>> Smallapack works quite well in VW and Dolphin...
>> >>> (In fact it did work very well since 1990 with st80 user
>> >>> primitives...)
>> >>> ...Unfirtunately not so in Squeak (VM crach possibly).
>> >>>
>> >>> I'd like to put more time in it, but so far there has not been so much
>> >>> interest.
>> >>>
>> >>> Nicolas
>> >>>
>> >>> 2010/6/21 Jimmie Houchin <jdev at cyberhaus.us>:
>> >>>> On 6/20/2010 10:41 AM, Lawson English wrote:
>> >>>>>
>> >>>>> On 6/20/10 6:08 AM, Nicolas Cellier wrote:
>> >>>>>>
>> >>>>>> 2010/6/20 Michael Haupt<mhaupt at gmail.com>:
>> >>>>>>>
>> >>>>>>> Hi Nicolas,
>> >>>>>>>
>> >>>>>>> On Sun, Jun 20, 2010 at 11:17 AM, Nicolas Cellier
>> >>>>>>> <nicolas.cellier.aka.nice at gmail.com>  wrote:
>> >>>>>>>>
>> >>>>>>>> About 8) :  True, every single operation results in memory
>> >>>>>>>> allocation
>> >>>>>>>> / garbage collection, a burden for number crunching.
>> >>>>>>>
>> >>>>>>> really?
>> >>>>>>>
>> >>>>>>> There is this nice book by Didier Besset called "Object-Oriented
>> >>>>>>> Implementation of Numerical Methods. An Introduction with Java and
>> >>>>>>> Smalltalk.: An Introduction with Java and Smalltalk". It can't be
>> >>>>>>> *that* bad. :-)
>> >>>>>>
>> >>>>>> Agree, "not worse than Matlab" was the meaning of my message.
>> >>>>>>>>
>> >>>>>>>> My own answer was: use C/FORTRAN for optimized number crunching
>> >>>>>>>> functions. Use Smalltalk for any higher level/GUI function (via
>> >>>>>>>> DLLCC/FFI). We may have more than 1 hammer in your toolset!
>> >>>>>>>
>> >>>>>>> With GPU connectivity things emerging, number crunching might even
>> >>>>>>> be
>> >>>>>>> an interesting area for Smalltalk.
>> >>>>>>>
>> >>>>>>> Best,
>> >>>>>>> Michael
>> >>>>>>
>> >>>>>> Yes, this falls in vectorizing the operations.
>> >>>>>> But I would go for a GPU-BLAS implementation available to any
>> >>>>>> language
>> >>>>>> (Smalltalk and C as well).
>> >>>>>>
>> >>>>>> Nicolas
>> >>>>>
>> >>>>> How many parallel squeak processes would be required to = the speed
>> >>>>> of one
>> >>>>> native library for arbitrary precision math, or for other math
>> >>>>> intensive
>> >>>>> purposes?
>> >>>>>
>> >>>>> Lawson
>> >>>>
>> >>>> Hello,
>> >>>>
>> >>>> I would love to be using Squeak for my financial application.
>> >>>> Numerical
>> >>>> performance isn't currently what is stopping me. My problem is that I
>> >>>> require interfacing with a Windows COM dll and in a future version
>> >>>> with a
>> >>>> Java library. Hopefully at some point I will be able to port to
>> >>>> Squeak. I
>> >>>> would much prefer it to using Python, which is what I am currently
>> >>>> using.
>> >>>>
>> >>>> I didn't even know Squeak was in the running until I discovered the
>> >>>> Matrix
>> >>>> class. And for what I need to do it performs reasonably adequately.
>> >>>> However
>> >>>> Squeak does not to my knowledge have a comprehensive collection of
>> >>>> mathematics methods to be able to be applied to a variety of data.
>> >>>> Currently
>> >>>> I am using Python and Numpy which has a nicely optimized
>> >>>> Mathematics/Scientific set of functions using optimized C/Fortran
>> >>>> libraries.
>> >>>> I would love to see Squeak compete in this area. In fact the Numpy
>> >>>> people
>> >>>> are currently refactoring the library to turn it into a C library
>> >>>> usable by
>> >>>> other languages.
>> >>>>
>> >>>> Here is some samples from my experimentation.
>> >>>>
>> >>>> Some of what I am doing is doing rolling calculations over my
>> >>>> dataset.
>> >>>>
>> >>>> dataset is one weeks worth of OHLC data of a currency pair.
>> >>>>
>> >>>> In Squeak I have.
>> >>>>
>> >>>> ttr := [
>> >>>>  1 to: ((m rowCount) -500) do: [:i || row rowSum rowMax rowMin
>> >>>> rowMedian
>> >>>> rowAverage |
>> >>>>  row := (m atRows: i to: (499+i) columns: 5 to: 5).
>> >>>>  rowSum := row sum.
>> >>>>  rowMax := row max.
>> >>>>  rowMin := row min.
>> >>>>  rowMedian := row median.
>> >>>>  rowAverage := row average.
>> >>>>  omd add: {rowSum . rowMax . rowMin . rowMedian . rowAverage}]]
>> >>>> timeToRun.
>> >>>>
>> >>>> Squeak:  17 seconds,  with Cog 4.2 seconds  (nice work guys
>> >>>> (Eliot/Teleplace)
>> >>>>
>> >>>> In Python/Numpy I have.
>> >>>>
>> >>>> import numpy as np
>> >>>> def speedtest(array,omd):
>> >>>>    t1 = time.time()
>> >>>>    for i in range(0, (len(a)-500)):
>> >>>>        rowmax = np.max(a['bidclose'][i:i+500])
>> >>>>        rowmin = np.min(a['bidclose'][i:i+500])
>> >>>>        rowsum = np.sum(a['bidclose'][i:i+500])
>> >>>>        rowmedian = np.median(a['bidclose'][i:i+500])
>> >>>>        rowmean = np.mean(a['bidclose'][i:i+500])
>> >>>>        omd.append((rowsum, rowmax, rowmin, rowmedian, rowmean))
>> >>>>    return time.time()-t1
>> >>>>
>> >>>> Python:  .7 seconds
>> >>>>
>> >>>> Python/Numpy performs well, is reasonably nice to work with. But I
>> >>>> would
>> >>>> give up the performance to be able to use Squeak. The live
>> >>>> environment and
>> >>>> debugging would be invaluable for experimentation.
>> >>>>
>> >>>> Hopefully this will give you some idea.
>> >>>>
>> >>>> Jimmie
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>
>> >>
>> >
>>
>
>
>
>
>



More information about the Squeak-dev mailing list