[squeak-dev] Looking for real-world Magma application
nmanzanos at gmail.com
Tue Nov 2 22:02:02 UTC 2010
> Norberto, i suspect that you doing something wrong. Scanning 76000
> records by iterating though entire collection
> when looking for duplicates is indeed slow. And sure thing, time to
> add new item will grow exponentially once collection grows.
> And its not really matters what kind of DB you will use. It will be
> slow everywhere, if you need to scan entire dataset before adding new
> item to it.
Hey!!!! I'm not scanning the entire data base. What about indexes?
I tried to add all objects before and perform the comparision after, and
it takes even more time.
I made this before using traditional technology, SQL and all that, and I
never have to wait several days for the result.
> So, i suspect that there is something wrong with the way how you using
> and because of that, performance degrades from O(log(n)) down to O(n),
> for each operation.
> Sure, its hard to say anything without looking at actual code.
Ok. Let's see the code
"this method creates the repository"
| thePath theSession |
thePath := FileDirectory default pathName assurePath,'\magma'.
((FileDirectory on: thePath) fileExists:'objects')
ifFalse:[ (Smalltalk at: #MagmaSession)
(Smalltalk at: #MagmaRepositoryController) delete:
(Smalltalk at: #MagmaRepositoryController)
root: (IdentityDictionary new
at: #works put: self workCatalog;
at: #expressions put: self expressionCatalog ;
at: #manifestations put: self
theSession := (Smalltalk at: #MagmaSession) openLocal:
theSession connectAs: 'nor'.].
"these method creates the collections (workCatalog, expressionCatalog,
collection:= MagmaCollection new.
"these two methods add and find the objects"
| found |
found:= self find: anObject.
ifTrue: [self collection add: anObject ]
ifFalse: [ self merger merge: anObject on: found ].
| found |
found:= (self collection where:
reader read: #authorizedName
at: anAccessPoint authorizedName]).
"authorizedNamed is a string"
As I allready said, merging is not the problem, so I omit the methods for
Transaction is made using an interator which let you change the transaction
step. So, that's not a problem. I'm making commits every 300 addings.
"This is the main method"
BibHuma instance useMagmaCollections.
ferberizator iterator: (MagmaIterator newForIsis list: (1 to: 2000)).
"records to migrate to Magma"
ferberizator iterator transactionStep: 300.
session := MagmaSession openLocal: FileDirectory default pathName
session connectAs: 'nm'.
ferberizator iterator session: session.
session readStrategy: (MaReadStrategy minimumDepth: 0).
> > Norberto
> Best regards,
> Igor Stasenko AKA sig.
Instituto de Investigaciones en Humanidades y Ciencias Sociales (IdIHCS)
FaHCE/UNLP - CONICET
Calle 48 e/ 6 y 7 s/Nº - 8º piso - oficina 803
Tel: +54-221-4230125 interno 262
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Squeak-dev