[squeakdev] Re: too many copiedValues [was Interesting survey about
smalltalk]
Nicolas Cellier
nicolas.cellier.aka.nice at gmail.com
Mon Jun 21 22:26:08 UTC 2010
2010/6/22 Eliot Miranda <eliot.miranda at gmail.com>:
> Hi Nicolas,
>
> On Mon, Jun 21, 2010 at 2:59 PM, Nicolas Cellier
> <nicolas.cellier.aka.nice at gmail.com> wrote:
>>
>> 2010/6/21 Nicolas Cellier <nicolas.cellier.aka.nice at gmail.com>:
>> > Compiler hack is available for Squeak 3.9 at
>> > http://bugs.squeak.org/view.php?id=2918
>> > Load the 3 .cs in indicated order.
>> >
>> > I will publish an update for latest trunk at the same place when I'll
>> > have a bit of time...
>> >
>>
>> Finally, I published a Compiler version for latest trunk in this
>> SqueakSource repository
>> Plus a hack or two to make external interface loadable in closure VM
>> (there is an additional limitation in closure that is max number of
>> block copied values<16)
>
> I'm interested to see the code that exceeds this limitation. Hopefully
> it'll be temporary, but it'll take a bytecode redesign to overcome, so I
> can't fix this anytime soon. Sorry it has bitten you!
Oh, I guess one wouldn't code so many temps.
It's some kind of autogenerated code for FFI call of LAPACK functions.
LAPACK is written in FORTRAN.
FORTRAN has all parameters passed by reference.
Consequence is that many arguments passed to a LAPACK function (like
dimensions of matrices) require an allocation.
I generated the allocation for passing some scalar inputs inside a
block pretected with an #ensure: ensuring deallocation...
But many LAPACK arguments means many temps and two cases of copiedValues > 15.
Methods are:
LapackLibrary>>hegvdWithitype: itype jobz: jobz uplo: uplo n: n a: a
lda: lda b: b ldb: ldb w: w work: work lwork: lwork rwork: rwork
lrwork: lrwork iwork: iwork liwork: liwork
LapackLibrary>>tgexcWithwantq: wantq wantz: wantz n: n a: a lda: lda
b: b ldb: ldb q: q ldq: ldq z: z ldz: ldz ifst: ifst ilst: ilst work:
work lwork: lwork
in http://www.squeaksource.com/Smallapack/SmallapackExternalnice.16.mcz
prerequisite is FFI and hack in
http://www.squeaksource.com/Smallapack/Compilernice.147.mcz
in order to compile methods with more than 15 arguments...
Nicolas
>>
>> Sometimes, it gets a bit tricky to initialize Smallapack.
>> I did something like:
>>
>> LapackMatrix initialize.
>> LapackMatrix allSubclasses do: [:e  [e initializeClassInstVars] ifError:
>> []].
>>
>> Nicolas
>>
>> > Then the latest Smallapack code is at
>> > http://www.squeaksource.com/Smallapack.html
>> >
>> > If you are on linux, you'd better use a relatively new VM correcting
>> > http://bugs.squeak.org/view.php?id=3929
>> >
>> > Nicolas
>> >
>> >
>> > 2010/6/21 arul selvan <arul.selvan at gmail.com>:
>> >> nicolas,
>> >>
>> >> i will be interested to test if a port for squeak is available
>> >>
>> >> arul
>> >>
>> >>
>> >> On Mon, Jun 21, 2010 at 12:41 PM, Nicolas Cellier
>> >> <nicolas.cellier.aka.nice at gmail.com> wrote:
>> >>> I started Smallapack for this purpose.
>> >>> Smallapack interfaces BLAS+LAPACK thru FFI.
>> >>> Very much like numpy...
>> >>>
>> >>> Smallapack works quite well in VW and Dolphin...
>> >>> (In fact it did work very well since 1990 with st80 user
>> >>> primitives...)
>> >>> ...Unfirtunately not so in Squeak (VM crach possibly).
>> >>>
>> >>> I'd like to put more time in it, but so far there has not been so much
>> >>> interest.
>> >>>
>> >>> Nicolas
>> >>>
>> >>> 2010/6/21 Jimmie Houchin <jdev at cyberhaus.us>:
>> >>>> On 6/20/2010 10:41 AM, Lawson English wrote:
>> >>>>>
>> >>>>> On 6/20/10 6:08 AM, Nicolas Cellier wrote:
>> >>>>>>
>> >>>>>> 2010/6/20 Michael Haupt<mhaupt at gmail.com>:
>> >>>>>>>
>> >>>>>>> Hi Nicolas,
>> >>>>>>>
>> >>>>>>> On Sun, Jun 20, 2010 at 11:17 AM, Nicolas Cellier
>> >>>>>>> <nicolas.cellier.aka.nice at gmail.com> wrote:
>> >>>>>>>>
>> >>>>>>>> About 8) : True, every single operation results in memory
>> >>>>>>>> allocation
>> >>>>>>>> / garbage collection, a burden for number crunching.
>> >>>>>>>
>> >>>>>>> really?
>> >>>>>>>
>> >>>>>>> There is this nice book by Didier Besset called "ObjectOriented
>> >>>>>>> Implementation of Numerical Methods. An Introduction with Java and
>> >>>>>>> Smalltalk.: An Introduction with Java and Smalltalk". It can't be
>> >>>>>>> *that* bad. :)
>> >>>>>>
>> >>>>>> Agree, "not worse than Matlab" was the meaning of my message.
>> >>>>>>>>
>> >>>>>>>> My own answer was: use C/FORTRAN for optimized number crunching
>> >>>>>>>> functions. Use Smalltalk for any higher level/GUI function (via
>> >>>>>>>> DLLCC/FFI). We may have more than 1 hammer in your toolset!
>> >>>>>>>
>> >>>>>>> With GPU connectivity things emerging, number crunching might even
>> >>>>>>> be
>> >>>>>>> an interesting area for Smalltalk.
>> >>>>>>>
>> >>>>>>> Best,
>> >>>>>>> Michael
>> >>>>>>
>> >>>>>> Yes, this falls in vectorizing the operations.
>> >>>>>> But I would go for a GPUBLAS implementation available to any
>> >>>>>> language
>> >>>>>> (Smalltalk and C as well).
>> >>>>>>
>> >>>>>> Nicolas
>> >>>>>
>> >>>>> How many parallel squeak processes would be required to = the speed
>> >>>>> of one
>> >>>>> native library for arbitrary precision math, or for other math
>> >>>>> intensive
>> >>>>> purposes?
>> >>>>>
>> >>>>> Lawson
>> >>>>
>> >>>> Hello,
>> >>>>
>> >>>> I would love to be using Squeak for my financial application.
>> >>>> Numerical
>> >>>> performance isn't currently what is stopping me. My problem is that I
>> >>>> require interfacing with a Windows COM dll and in a future version
>> >>>> with a
>> >>>> Java library. Hopefully at some point I will be able to port to
>> >>>> Squeak. I
>> >>>> would much prefer it to using Python, which is what I am currently
>> >>>> using.
>> >>>>
>> >>>> I didn't even know Squeak was in the running until I discovered the
>> >>>> Matrix
>> >>>> class. And for what I need to do it performs reasonably adequately.
>> >>>> However
>> >>>> Squeak does not to my knowledge have a comprehensive collection of
>> >>>> mathematics methods to be able to be applied to a variety of data.
>> >>>> Currently
>> >>>> I am using Python and Numpy which has a nicely optimized
>> >>>> Mathematics/Scientific set of functions using optimized C/Fortran
>> >>>> libraries.
>> >>>> I would love to see Squeak compete in this area. In fact the Numpy
>> >>>> people
>> >>>> are currently refactoring the library to turn it into a C library
>> >>>> usable by
>> >>>> other languages.
>> >>>>
>> >>>> Here is some samples from my experimentation.
>> >>>>
>> >>>> Some of what I am doing is doing rolling calculations over my
>> >>>> dataset.
>> >>>>
>> >>>> dataset is one weeks worth of OHLC data of a currency pair.
>> >>>>
>> >>>> In Squeak I have.
>> >>>>
>> >>>> ttr := [
>> >>>> 1 to: ((m rowCount) 500) do: [:i  row rowSum rowMax rowMin
>> >>>> rowMedian
>> >>>> rowAverage 
>> >>>> row := (m atRows: i to: (499+i) columns: 5 to: 5).
>> >>>> rowSum := row sum.
>> >>>> rowMax := row max.
>> >>>> rowMin := row min.
>> >>>> rowMedian := row median.
>> >>>> rowAverage := row average.
>> >>>> omd add: {rowSum . rowMax . rowMin . rowMedian . rowAverage}]]
>> >>>> timeToRun.
>> >>>>
>> >>>> Squeak: 17 seconds, with Cog 4.2 seconds (nice work guys
>> >>>> (Eliot/Teleplace)
>> >>>>
>> >>>> In Python/Numpy I have.
>> >>>>
>> >>>> import numpy as np
>> >>>> def speedtest(array,omd):
>> >>>> t1 = time.time()
>> >>>> for i in range(0, (len(a)500)):
>> >>>> rowmax = np.max(a['bidclose'][i:i+500])
>> >>>> rowmin = np.min(a['bidclose'][i:i+500])
>> >>>> rowsum = np.sum(a['bidclose'][i:i+500])
>> >>>> rowmedian = np.median(a['bidclose'][i:i+500])
>> >>>> rowmean = np.mean(a['bidclose'][i:i+500])
>> >>>> omd.append((rowsum, rowmax, rowmin, rowmedian, rowmean))
>> >>>> return time.time()t1
>> >>>>
>> >>>> Python: .7 seconds
>> >>>>
>> >>>> Python/Numpy performs well, is reasonably nice to work with. But I
>> >>>> would
>> >>>> give up the performance to be able to use Squeak. The live
>> >>>> environment and
>> >>>> debugging would be invaluable for experimentation.
>> >>>>
>> >>>> Hopefully this will give you some idea.
>> >>>>
>> >>>> Jimmie
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>
>> >>
>> >
>>
>
>
>
>
>
More information about the Squeakdev
mailing list
