Searching magma string indices

[Chris Muller]

Hi Brent!

I think you've cleverly solved your own question..  By indexing the reversed value, its no different than a keyword index where you can already query on the presence of multiple keywords, as in:

    myArticles where: [ : each | (each keywords equals: 'Squeak') & (each keywords equals: 'Magma') ]

So you would just have to convert the wildcards to multiple conjunctions, for example, 'foo%bar' would just have to be translated to:

    self people where:
        [ : each |
        (each familyName startsWith: 'foo')
        & (each familyName endsWith: 'bar') ]

and the index would have something like:

    startsWith: aString
        self equals: aString

    endsWith: aString
        self equals: aString reversed

You don't even need a separate "reversed" index, just add two entries (the forward and reversed) for each object added to the collection.

Now, the expression '%foo%bar%' goes beyond the first requirement of just searching on a prefix and/or a suffix..  For this a different type of index would be needed that adds every possible value for "Fredfoon Tobart", the following values:

    fredfoon tobart
    redfoon tobart
    edfoon tobart
    dfoon tobart
    foon tobart
    oon tobart
    on tobart
    n tobart
     tobart
    tobart
    obart
    bart
    art
    rt
    t

Your '%foo%bar%' expression would then need to translate to

    (familyName from: 'foo' to: 'foo' maAlphabeticalNext)
    & (familyName from: 'bar' to: 'bar' maAlphabeticalNext)

to find the object(s) that had "Fredfoon Tobart".

...

The standard indexes included with Magma don't begin to scratch the surface of their real potential.  I find it fascinating to to talk about this stuff, thanks!

It's great to "see" you again buddy, welcome back,
  Chris



----- Original Message ----
From: Brent Pinkney <brent at zamail.co.za>
To: magma at lists.squeakfoundation.org
Sent: Thursday, March 1, 2007 12:15:07 AM
Subject: Searching magma string indices

Hi,

I have been thinking about how one could search Magma String indices for a given prefix or suffix. i.e. find all words starting with 'foo' or ending with 'bar'.


One can add SQL-ish "like" functionality for a prefix seach quite easily with the method:

    MaClause >> #like: aString
        "aString is a SQL-ish term and may include a trailing %"

        self from: aString upTo: (aString copyWithout: $%) maAlphabeticalNext
    !

For example: find all the people whose name starts with 'Jo':

    self people where: [ :p | p familyName like: 'Jo%' ].


This executes as a very efficient indixed search and will be debuting (in some form) in the next release of Lava.


However, finding all the strings ending with 'bar' is more troublesome as there is no way to map this to a #from:to:# expression on the Magma index.


UNLESS of course, we maintain another, hidden, String index on the collection with the hash of the string reversed.
That is

    self people where: [ :p | p familyName like '%Jo' ].

becomes:
    self people read: #familyName_reversed from: 'oJ' to: ...


It is very late here, so I have not been able to try it out, but I welcome comments.

What would be really nice is an index on expressions of the form '%foo%bar%'. Somehow I doubt this.



Cheers

Brent





_______________________________________________
Magma mailing list
Magma at lists.squeakfoundation.org
http://lists.squeakfoundation.org/mailman/listinfo/magma