[squeak-dev] Smallapack

Yoshiki Ohshima Yoshiki.Ohshima at acm.org
Mon Sep 12 17:53:47 UTC 2016


Sorry for coming back to this again...

I just tried to run Smallapack on 5.1, but it fails.  Specifically, I
modified SqueakPluginsBuiltInOrLocalOnly to be false, restart the
session and run:

Installer ss
       project: 'MetacelloRepository';
       install: 'ConfigurationOfSmallapack'.
((Smalltalk at: #ConfigurationOfSmallapack) project version: #development) load.

and call initialization (which may not be necessary):

CBlasLibrary install. CLapackLibrary install. LapackMatrix
resetBlasInterfaces; resetLapackInterfaces.

and try to run the tests.  But the tests fails.  This is from the same
directory where 5.0 based VM and image can run; so those dylibs are
copied into the same directory.  I also suspected the capitalization
of the dylib file names but it also was not the case.

Can anybody else confirm that this works, or does not work, for you?

On Fri, Aug 12, 2016 at 3:46 PM, Yoshiki Ohshima
<Yoshiki.Ohshima at acm.org> wrote:
> Thanks!
>
> I had not been able to focus on getting this to work so it's been
> almost two weeks now but as finally the basic case finally going, I am
> now happy.  Thanks for the help!
>
>
> On Fri, Aug 12, 2016 at 3:07 PM, Nicolas Cellier
> <nicolas.cellier.aka.nice at gmail.com> wrote:
>> Thanks Yoshiki, new configuration published.
>>
>> 2016-08-12 22:59 GMT+02:00 Yoshiki Ohshima <Yoshiki.Ohshima at acm.org>:
>>>
>>> Ok, now I'm getting some success!
>>>
>>> I start over with the VM: Croquet Closure Cog VM [CoInterpreterPrimitives
>>> VMMaker.oscog-eem.1044] Squeak Cog Spur 5.0.3248
>>>
>>> and followed Nicolas' instruction in this thread.  The image is 32-bit 5.0
>>> image.  In this setting all tests passed.  Yay.  To run the above mentioned
>>> example:
>>>
>>> -------------------
>>> a := LapackDGEMatrix rows: #(#(3 2 4) #(2 -5 -1) #(1 -2 2)).
>>> b := LapackDGEMatrix rows: #(#(1) (2) (3)).
>>> c := LapackLeastSquareProblem matrix: a rhsMatrix: b.
>>>
>>> c solution.
>>> -------------------
>>>
>>> I needed to fix the processSVD to make it work.  The changes are to
>>> change:
>>>
>>> rank := WordArray new: 1.
>>>
>>> into:
>>>
>>> rank := ExternalLongArray new: 1.
>>>
>>> (otherwise #arrayPointer is not understood by 'rank') and also the return
>>> value from gelssWithm: has to be stored in 'info' (otherwise it will be
>>> always nil).
>>>
>>>
>>>
>>> On Thu, Aug 11, 2016 at 6:05 PM, Yoshiki Ohshima <Yoshiki.Ohshima at acm.org>
>>> wrote:
>>>>
>>>>  I have not reached the conclusion but would like to give some status
>>>> report.
>>>>
>>>> My test case is simple:
>>>>
>>>> -------------------
>>>> a := LapackDGEMatrix rows: #(#(3 2 4) #(2 -5 -1) #(1 -2 2)).
>>>> b := LapackDGEMatrix rows: #(#(1) (2) (3)).
>>>> c := LapackLeastSquareProblem matrix: a rhsMatrix: b.
>>>>
>>>> c solution.
>>>> -------------------
>>>>
>>>> If I evaluate this, the external call for 'dgesvd_' in
>>>>
>>>> CLapackDLibrary>>xgesvdWithjobu:jobvt:m:n:a:lda:s:u:ldu:vt:ldvt:work:lwork:info:
>>>> fails.
>>>>
>>>> But if I fix it by rewriting the cdecl line to:
>>>>
>>>> <cdecl: long 'dgesvd_'( char * char * long * long * double * long *
>>>> double * double * long * double * long * double * long * long * )
>>>> module: 'libLAPACK'>
>>>>
>>>> from
>>>>
>>>> <cdecl: long 'dgesvd_'( char * char * long * long * double * long *
>>>> double * double * long * double * long * double * long * long * )>
>>>>
>>>> it passes.
>>>>
>>>> However,  the value that gets stored in the 's' instance variable of
>>>> LapackSVDecomposition is a LapackDGEMatrix, and when the resulting
>>>> value is used in the #defaultTolerance, I get DNU for '*".
>>>>
>>>>
>>>>
>>>> On Tue, Aug 9, 2016 at 3:26 PM, Yoshiki Ohshima <Yoshiki.Ohshima at acm.org>
>>>> wrote:
>>>> > Correction: (sorry)
>>>> >
>>>> > In a 64 bit image, TestCBlas runs fine but the ones listed above fails
>>>> > in a 32-bit image.
>>>> >
>>>> > On Tue, Aug 9, 2016 at 3:12 PM, Yoshiki Ohshima
>>>> > <Yoshiki.Ohshima at acm.org> wrote:
>>>> >> Thanks!
>>>> >>
>>>> >> I am trying this from a vanilla 5.0 image and I see that
>>>> >> ConfigurationOfSmallapack-nice.18 is loaded.  Now, however, the
>>>> >> following tests from Smallapack-SUnitTests category fail:
>>>> >>
>>>> >> #('TestCBlas>>#testCsscal' 'TestCBlas>>#testSaxpy'
>>>> >> 'TestCBlas>>#testSgemv' 'TestCBlas>>#testSgemvTrans'
>>>> >> 'TestCBlas>>#testSger' 'TestCBlas>>#testSscal' 'TestCBlas>>#testStrsm'
>>>> >> 'TestLapackMatrix>>#testMatrixProduct' 'TestLapackMatrix>>#testSum'
>>>> >> 'TestRandMatrix>>#testOperationTiming')
>>>> >>
>>>> >> As I wrote, TestCBlas used to be all green.  I'll check what has
>>>> >> changed since -nice.16...
>>>> >>
>>>> >> On Sat, Aug 6, 2016 at 4:56 PM, Nicolas Cellier
>>>> >> <nicolas.cellier.aka.nice at gmail.com> wrote:
>>>> >>> This should be fixed in ConfigurationOfSmallapack-nice.18
>>>> >>> BTW, I think you can provide the absolute path to the .dylib instead
>>>> >>> of
>>>> >>> copying in local repository...
>>>> >>>
>>>> >>>
>>>> >>> 2016-08-06 23:29 GMT+02:00 Nicolas Cellier
>>>> >>> <nicolas.cellier.aka.nice at gmail.com>:
>>>> >>>>
>>>> >>>> Hi Yoshiki,
>>>> >>>> thanks for reporting, I'll try better...
>>>> >>>>
>>>> >>>>
>>>> >>>> 2016-08-01 0:48 GMT+02:00 Yoshiki Ohshima <Yoshiki.Ohshima at acm.org>:
>>>> >>>>>
>>>> >>>>> I see you have some changes but it appears that evaluating the
>>>> >>>>> Installer do it goes into an infinite loop of #moduleName and
>>>> >>>>> #masOsxModuleName.
>>>> >>>>>
>>>> >>>>> (Thanks again!)
>>>> >>>>>
>>>> >>>>> On Sat, Jul 30, 2016 at 8:23 AM, Yoshiki Ohshima
>>>> >>>>> <Yoshiki.Ohshima at acm.org> wrote:
>>>> >>>>> > But some of TestLapackMatrix tests fail.  A few external
>>>> >>>>> > functions
>>>> >>>>> > cannot be found, it looks like.
>>>> >>>>> >
>>>> >>>>> > On Sat, Jul 30, 2016 at 7:55 AM, Yoshiki Ohshima
>>>> >>>>> > <Yoshiki.Ohshima at acm.org> wrote:
>>>> >>>>> >> Great!
>>>> >>>>> >>
>>>> >>>>> >> Before (I got into a meeting and then entered the "Friday mode",
>>>> >>>>> >> I was
>>>> >>>>> >> going down the path of trying to call the Framework functions
>>>> >>>>> >> but
>>>> >>>>> >> copying files anyway was a simpler solution for now.
>>>> >>>>> >>
>>>> >>>>> >> Yes, I got all tests green.  Thank  you!
>>>> >>>>> >>
>>>> >>>>> >> On Fri, Jul 29, 2016 at 3:24 PM, Nicolas Cellier
>>>> >>>>> >> <nicolas.cellier.aka.nice at gmail.com> wrote:
>>>> >>>>> >>> OK, what I did on my Mac:
>>>> >>>>> >>>
>>>> >>>>> >>> 1) look under the Squeak app and edit the Contents/Info.plist
>>>> >>>>> >>> 2) change value of SqueakPluginsBuiltInOrLocalOnly to "No"
>>>> >>>>> >>>    otherwise library loading is restricted to the Squeak app
>>>> >>>>> >>> bundle
>>>> >>>>> >>> 3) copy the veclib framework library files (dylib) in same
>>>> >>>>> >>> directory
>>>> >>>>> >>> as
>>>> >>>>> >>> squeak image
>>>> >>>>> >>> 4) launch Squeak
>>>> >>>>> >>> 5) install Smallapack
>>>> >>>>> >>>     follow instruction from
>>>> >>>>> >>>
>>>> >>>>> >>>
>>>> >>>>> >>> https://github.com/nicolas-cellier-aka-nice/smallapack/wiki/SmallapackSqueak
>>>> >>>>> >>> 6) change CBlasLibrary class>>moduleName 'libcblas.dylib' ->
>>>> >>>>> >>> 'libBlas.dylib'
>>>> >>>>> >>>     nowadays, cblas and blas are in the same dylib...
>>>> >>>>> >>> 7) change CLapackLibrary class>>moduleName 'libclapack.dylib'
>>>> >>>>> >>> ->
>>>> >>>>> >>> 'libLapack.dylib'
>>>> >>>>> >>>     idem
>>>> >>>>> >>> 8) re-initialize the cache (I know, I know, there are too
>>>> >>>>> >>> many...)
>>>> >>>>> >>>     CBlasLibrary install. CLapackLibrary install. LapackMatrix
>>>> >>>>> >>> resetBlasInterfaces; resetLapackInterfaces.
>>>> >>>>> >>> 9) run the TestCBlas suite
>>>> >>>>> >>>
>>>> >>>>> >>> It should be green
>>>> >>>>> >>> I will commit the changes later, and will probably implement
>>>> >>>>> >>> moduleNames as
>>>> >>>>> >>> a Preference (pragma oriented).
>>>> >>>>> >>> No need to override code anymore :)
>>>> >>>>> >>>
>>>> >>>>> >>> I think step 3) is necessary because of ioLoadModuleRaw() in
>>>> >>>>> >>> platforms/iOS/vm/OSX/sqMacUnixExternalPrims.m
>>>> >>>>> >>>
>>>> >>>>> >>>
>>>> >>>>> >>> https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/iOS/vm/OSX/sqMacUnixExternalPrims.m
>>>> >>>>> >>> It would only look those frameworks:
>>>> >>>>> >>>
>>>> >>>>> >>>                 static char *frameworks[]=
>>>> >>>>> >>>                 {
>>>> >>>>> >>>                         "",
>>>> >>>>> >>>                         "/CoreServices.framework/Frameworks",
>>>> >>>>> >>>
>>>> >>>>> >>> "/ApplicationServices.framework/Frameworks",
>>>> >>>>> >>>                         "/Carbon.framework/Frameworks",
>>>> >>>>> >>>                         0
>>>> >>>>> >>>                 };
>>>> >>>>> >>>
>>>> >>>>> >>> But I did step 3) before I tried 1) + 2), adn did not retry, so
>>>> >>>>> >>> maybe
>>>> >>>>> >>> I'm
>>>> >>>>> >>> wrong...
>>>> >>>>> >>> Scanning all the frameworks is not a solution. And what if we
>>>> >>>>> >>> want a
>>>> >>>>> >>> specific version?
>>>> >>>>> >>> It would be far better to be able to specify the path to the
>>>> >>>>> >>> library
>>>> >>>>> >>> from
>>>> >>>>> >>> within the image like VW...
>>>> >>>>> >>>
>>>> >>>>> >>>
>>>> >>>>> >>> 2016-07-29 19:41 GMT+02:00 Nicolas Cellier
>>>> >>>>> >>> <nicolas.cellier.aka.nice at gmail.com>:
>>>> >>>>> >>>>
>>>> >>>>> >>>>
>>>> >>>>> >>>>
>>>> >>>>> >>>> 2016-07-29 19:28 GMT+02:00 Nicolas Cellier
>>>> >>>>> >>>> <nicolas.cellier.aka.nice at gmail.com>:
>>>> >>>>> >>>>>
>>>> >>>>> >>>>>
>>>> >>>>> >>>>>
>>>> >>>>> >>>>> 2016-07-29 19:02 GMT+02:00 Yoshiki Ohshima
>>>> >>>>> >>>>> <Yoshiki.Ohshima at acm.org>:
>>>> >>>>> >>>>>>
>>>> >>>>> >>>>>> First question:
>>>> >>>>> >>>>>>
>>>> >>>>> >>>>>> The Mac OS comes with Accelerate.Framework and that contains
>>>> >>>>> >>>>>> BLAS.
>>>> >>>>> >>>>>> But probably I still should compile those libraries, right?
>>>> >>>>> >>>>>>
>>>> >>>>> >>>>>
>>>> >>>>> >>>>> No, it's better to link to accelerated library.
>>>> >>>>> >>>>> I don't have a Mac handy here to verify how to link to it
>>>> >>>>> >>>>> though.
>>>> >>>>> >>>>> I'll be able to check latter in the evening
>>>> >>>>> >>>>>
>>>> >>>>> >>>>
>>>> >>>>> >>>>
>>>> >>>>> >>>> I've downloaded the code, and I see it now: the library names
>>>> >>>>> >>>> are
>>>> >>>>> >>>> hardcoded (see implementors of moduleName).
>>>> >>>>> >>>> For Mac it is libblas.dylib and libcblas.dylib
>>>> >>>>> >>>> Also note that there is a preference for switching to cblas
>>>> >>>>> >>>> (blas
>>>> >>>>> >>>> functions with C interface).
>>>> >>>>> >>>> This should be faster because we pass some parameters by value
>>>> >>>>> >>>> rather than
>>>> >>>>> >>>> allocating them and pass reference...
>>>> >>>>> >>>>
>>>> >>>>> >>>> Module names could also be replaced by Preferences eventually,
>>>> >>>>> >>>> but
>>>> >>>>> >>>> by now,
>>>> >>>>> >>>> you'll have to override...
>>>> >>>>> >>>>
>>>> >>>>> >>>>>>
>>>> >>>>> >>>>>>
>>>> >>>>> >>>>>> On Thu, Jul 28, 2016 at 4:11 PM, Yoshiki Ohshima
>>>> >>>>> >>>>>> <Yoshiki.Ohshima at acm.org> wrote:
>>>> >>>>> >>>>>> > Thanks!
>>>> >>>>> >>>>>> >
>>>> >>>>> >>>>>> > On Thu, Jul 28, 2016 at 4:04 PM, Nicolas Cellier
>>>> >>>>> >>>>>> > <nicolas.cellier.aka.nice at gmail.com> wrote:
>>>> >>>>> >>>>>> >> Hi Yoshiki,
>>>> >>>>> >>>>>> >>
>>>> >>>>> >>>>>> >> Thanks for inquiring about Smallapack.
>>>> >>>>> >>>>>> >>
>>>> >>>>> >>>>>> >> This problem has been solved in 2011 as the post tells.
>>>> >>>>> >>>>>> >> Moreover, it was about alignment of squeak objects that
>>>> >>>>> >>>>>> >> was on
>>>> >>>>> >>>>>> >> multiple of 4
>>>> >>>>> >>>>>> >> on SqueakV3 memory.
>>>> >>>>> >>>>>> >> Spur is 8 byte aligned, so the problem would have also
>>>> >>>>> >>>>>> >> vanished
>>>> >>>>> >>>>>> >> without any
>>>> >>>>> >>>>>> >> patch for those being patient :)
>>>> >>>>> >>>>>> >>
>>>> >>>>> >>>>>> >> For the 15 arguments limit, Smallapack comes with it's
>>>> >>>>> >>>>>> >> own
>>>> >>>>> >>>>>> >> compiler,
>>>> >>>>> >>>>>> >> so it's
>>>> >>>>> >>>>>> >> a non issue.
>>>> >>>>> >>>>>> >> Maybe I should make the documentation more clear on
>>>> >>>>> >>>>>> >>
>>>> >>>>> >>>>>> >>
>>>> >>>>> >>>>>> >>
>>>> >>>>> >>>>>> >> https://github.com/nicolas-cellier-aka-nice/smallapack/wiki/SmallapackSqueak
>>>> >>>>> >>>>>> >> ?
>>>> >>>>> >>>>>> >>
>>>> >>>>> >>>>>> >> Unfortunately, there's no Sparse Matrix representation in
>>>> >>>>> >>>>>> >> Lapack.
>>>> >>>>> >>>>>> >> If you know of a good package for that, it could be
>>>> >>>>> >>>>>> >> integrated.
>>>> >>>>> >>>>>> >>
>>>> >>>>> >>>>>> >> If you have other questions, don't hesitate to ask.
>>>> >>>>> >>>>>> >>
>>>> >>>>> >>>>>> >> cheers
>>>> >>>>> >>>>>> >>
>>>> >>>>> >>>>>> >> Nicolas
>>>> >>>>> >>>>>> >>
>>>> >>>>> >>>>>> >> 2016-07-29 0:22 GMT+02:00 Yoshiki Ohshima
>>>> >>>>> >>>>>> >> <Yoshiki.Ohshima at acm.org>:
>>>> >>>>> >>>>>> >>>
>>>> >>>>> >>>>>> >>> I am trying to do a bit of linear algebra stuff that
>>>> >>>>> >>>>>> >>> involves
>>>> >>>>> >>>>>> >>> to
>>>> >>>>> >>>>>> >>> solve
>>>> >>>>> >>>>>> >>> a sparse 2D matrix (for a variation of doing least
>>>> >>>>> >>>>>> >>> square
>>>> >>>>> >>>>>> >>> fit).
>>>> >>>>> >>>>>> >>>
>>>> >>>>> >>>>>> >>> There was a message from Nicolas:
>>>> >>>>> >>>>>> >>>
>>>> >>>>> >>>>>> >>>
>>>> >>>>> >>>>>> >>>
>>>> >>>>> >>>>>> >>>
>>>> >>>>> >>>>>> >>> http://lists.squeakfoundation.org/pipermail/squeak-dev/2011-August/161113.html
>>>> >>>>> >>>>>> >>>
>>>> >>>>> >>>>>> >>> Is this where it stands today, too?  It looks like that
>>>> >>>>> >>>>>> >>> arg
>>>> >>>>> >>>>>> >>> count
>>>> >>>>> >>>>>> >>> problem is still there in 5.0, but is it in a way
>>>> >>>>> >>>>>> >>> non-issue as
>>>> >>>>> >>>>>> >>> it is
>>>> >>>>> >>>>>> >>> still FFI based?
>>>> >>>>> >>>>>> >>>
>>>> >>>>> >>>>>> >>> Thanks!
>>>> >>>>> >>>>>> >>>
>>>> >>>>> >>>>>> >>> --
>>>> >>>>> >>>>>> >>> -- Yoshiki
>>>> >>>>> >>>>>> >>>
>>>> >>>>> >>>>>> >>
>>>> >>>>> >>>>>> >>
>>>> >>>>> >>>>>> >>
>>>> >>>>> >>>>>> >>
>>>> >>>>> >>>>>> >
>>>> >>>>> >>>>>> >
>>>> >>>>> >>>>>> >
>>>> >>>>> >>>>>> > --
>>>> >>>>> >>>>>> > -- Yoshiki
>>>> >>>>> >>>>>>
>>>> >>>>> >>>>>>
>>>> >>>>> >>>>>>
>>>> >>>>> >>>>>> --
>>>> >>>>> >>>>>> -- Yoshiki
>>>> >>>>> >>>>>>
>>>> >>>>> >>>>>
>>>> >>>>> >>>>
>>>> >>>>> >>>
>>>> >>>>> >>>
>>>> >>>>> >>>
>>>> >>>>> >>>
>>>> >>>>> >>
>>>> >>>>> >>
>>>> >>>>> >>
>>>> >>>>> >> --
>>>> >>>>> >> -- Yoshiki
>>>> >>>>> >
>>>> >>>>> >
>>>> >>>>> >
>>>> >>>>> > --
>>>> >>>>> > -- Yoshiki
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> --
>>>> >>>>> -- Yoshiki
>>>> >>>>>
>>>> >>>>
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> -- Yoshiki
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > -- Yoshiki
>>>>
>>>>
>>>>
>>>> --
>>>> -- Yoshiki
>>>
>>>
>>>
>>>
>>> --
>>> -- Yoshiki
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>
>
>
> --
> -- Yoshiki



-- 
-- Yoshiki


More information about the Squeak-dev mailing list