I am pleased to announce the latest Magma upgrade! The main thrust of this release are bug fixes but, primarily, one particular bug which required a significant amount of work; complete transparency of "pseudo-persistent" MagmaCollectionReaders. By pseudo-persistent I mean Readers that are the result-sets of a query on a new, automatically created MagmaCollection borned from using one of the "luxury" query features (either distinctness or sorting by other than the most-optimal attribute) on another MagmaCollection. Although these new Reader/Collection pairs, are not (initially, at least) truly persistent because they are not referenced by the persistent object model, they do, like persistent collections, require support by back-end (hash-index) files and managed by a server object.
Until now, attempting to further query these "result set" Readers or attach them to the persistent graph did not work. Now it does. Further, thanks to Hilaire's great suggestion, further querying of Readers is fully symmetrical with MagmaCollections, i.e., you can use #where: and friends on a Reader.
I don't think Brent will like this but.. along with this is the notion that the back-end files for "result sets" are no longer deleted simply because the creator session disconnects. They can't be. Just because a session disconnects doesn't mean its done with the MagmaCollectionReader in its image. For example, the disconnect could just be a side-effect of saving the image. So how are back-end files ever removed? Via MagmaCompressor.
So, applications that allow users to run many hundreds of "luxury" queries, will accumulate quite a few result-set support files. This would pose a problem for the Magma server, up until now, because the server would open all known MagmaCollection and index files upon starting, and any queries would create (and open) files that would remain open until the server was shut down. No longer.
Now, all files are now opened lazily! The Magma server should start more quickly now. Further, to mitigate the sheer number of open file handles for query result sets, Magma now employs a "file pool" to limit the number of simultaneously-open files. The user may set the maximum number of open files via #filePoolSize: (the default is 60), and oldest-accessed files are closed. Not only does this allow a good approach to dealing with many query result sets, it is more ISP-friendly (some ISP's restrict the number of simultaneously-open files you may have).
For those familiar with Magma's meta-model (MagmaRepositoryDefinition), it no longer includes 'allLargeCollections'. This turned out to be significant addition-by-subtraction. There is a reduction in the model, code (including some special-case code) and number of byte-code instructions the computer must execute. A win win win all-around.
Caveats:
Ah, of course. Besides the aforementioned potential file explosion, it is now important to let Magma have the entire directory. __Do not keep any of your own important files in any Magma repository directory__ because MagmaRepositoryController>>#delete: now simply deletes all files in the directory.
Upgrading:
Guess what?! All existing databases are completely compatible with the new code. Why? Because, in spite of the change in Magma's meta model, the class mapping permits "dynamic evolution" of the class model, and removal of an instance variable is actually the simplest case.
Other fixes:
- Fixed bug with MagmaCollection>>#asArray:. Thanks to Göran's team! - Fixed at:ifOutOfBounds: sometimes giving error instead of valuing the outOfBoundsBlock reported by Göran. - Added Form and BitMap to default ReadStrategy, 99999 depth, thanks Hilaire for reporting. - ensure MaTransactionalFileStreams are always binary, this fixes the "Outage occurred" message in 3.9.
I spent a lot of time on this and believe it to be a quality release. It is, therefore, the new Magma baseline which I've posted to both the "Magma" project as well as the "MagmaTester" project.
Enjoy!