[SqueakDBX] Character encoding and pooling in SqueakDBX and Glorp

[Mariano Martinez Peck]

On Thu, Aug 26, 2010 at 9:43 PM, Mariano Martinez Peck <
marianopeck at gmail.com> wrote:

>
>
> On Wed, Aug 25, 2010 at 7:20 AM, Panu Suominen <
> panu.j.m.suominen at gmail.com> wrote:
>
>> 2010/8/24 Mariano Martinez Peck <marianopeck at gmail.com>:
>>
>> > Yes. So that I can see it. Do you know if it breaks compatibility with
>> other
>> > databases or OS? because SqueakDBX classes like DBXConnection should be
>> > cross-backend.
>>
>> I don't know how cross-platform my fix is. Database part should be
>> fine. Postgresql can now detect the encoding.
>>
>
> Thanks a lot for the changeset. Now I will test with different OS and
> databases and I will let you know.
>
> now a couple of questions:
>
> - I guess a lot of people doesn't need UTF (or other) encoding. And I guess
> that the transformation brings some overhead in the query result mosty...(I
> should run the benchmarks and see if there is difference). But I was
> thinking if we could make this optional. I like the code and design you did.
> I think that a good idea would be to add some kind of NullEncoder which is
> set in:
>
>
Ok.... I run the benchmarks and I was rigth ;)

Before:

Benchmarking: DBXTinyBenchmarks (10 runs)
benchmarkInsert: 521.4 AVG
benchmarkInsertConverted: 795.9 AVG
benchmarkSelect10: 2.5 AVG
benchmarkSelect100: 17.5 AVG
benchmarkSelect1000: 167.4 AVG
benchmarkSelect10000: 1691.9 AVG


After:

Benchmarking: DBXTinyBenchmarks (10 runs)
benchmarkInsert: 643.2 AVG
benchmarkInsertConverted: 905.6 AVG
benchmarkSelect10: 3.8 AVG
benchmarkSelect100: 25.1 AVG
benchmarkSelect1000: 246.4 AVG
benchmarkSelect10000: 2577.6 AVG


In addition, several SqueakDBX tests were failing with your changes...In
order to have all green tests, I had to do:

fieldRawValue: anIndex on: aResultSet
    "Returns the value of the column at anIndex of the current row from
aResultSet"
    | raw |
    raw := (OpenDBX current apiQueryFieldValue: (aResultSet handle) index:
(anIndex - 1)).
    raw isNil ifFalse: [^aResultSet connection encoder
convertFromSystemString: raw].
    ^ nil


Panu, I really appreciate your help. I am happy to integrate this change
when we can make it option  :)




> DBXConnection >> initialize
>
> instead of TextConverter newForEncoding: #utf8.
>
> Then, for postgres, you will be using the one from  TextConverter
> newForEncoding:  (r rows first valueAt: 1).
>
> Even more, I would like to make it optional. Maybe we can create
> DBXCOnnection class >> platform: aPlatform settings: aConnectionSettings
> encode: anEncoder
> and that will do:
>
> platform: aPlatform settings: aConnectionSettings encoder: anEncoder
>     ^self new
>         platform: aPlatform;
>         settings: aConnectionSettings;
>         encoder: anEncoder;
>         yourself
>
> Finally, the  cleint code (your app code) creates the connection with that
> message :)
>
> or something like that....but the idea of making it optional not to have
> overhead if not required, and being able to be enabled from the app code.
>
> - What happens is the database supports and specific encoding that Pharo
> doesn't support?
>  TextConverter newForEncoding:  (r rows first valueAt: 1).
>
> will return nil, causing UndefinedObject dnu something.
>
> maybe we can just set again the NullEncoder in that case?  or raise an
> error ?  what do you think?
>
>
> Others database platforms just don't know anything about encoding
>> (they use default utf-8). If other smalltalk dialects have
>> TextConverter then it should be fine.
>> If they haven't then I can implement another abstraction layer to
>> provide compatibility between dialects.
>>
>> > If you want to can use another new package with your commit.
>> I am sorry, but I did not quite understand what you meant...
>>
>> Unfortunately I am a little busy at work right now so I did not had
>> time to learn how to use monticello properly to create version only
>> containing required changes.
>> For this reason I attached change set file to this mail.
>>
>
> Don't worry :)
>
> thanks!
>
>
>>
>> >> Also is there a connection pooling facility for SqueakDBX or Glorp?
>> > Yes, several.
>> Thank you.
>>
>> --
>> Panu
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeakdbx/attachments/20100826/7c086f42/attachment.htm