[SqueakDBX] Connection not Valid problem

Mariano Martinez Peck marianopeck at gmail.com
Fri Sep 25 01:16:08 UTC 2009


On Thu, Sep 24, 2009 at 10:12 PM, John Toohey <jt at parspro.com> wrote:

> I am running the OpenDBX-Core tests both from the TestRunner and the
> OBPackageBrowser. The tests run successfully until my DB runs out of
> connections. Latest Pharo, and latest GlorpDBX installed.
>
>
John: I have just noticed that a lot of SqueakDBX tests don't release the DB
connections, as you guessed. I have just commit. can you please test it and
let me know ? of course, you must first reboot or kill all the active
connections ;)



"Name: OpenDBX-Core-Mariano.230
Author: Mariano
Time: 24 September 2009, 10:15:03 pm
UUID: 19ce8340-3070-4495-a21d-cbe2b63aca0e
Ancestors: OpenDBX-Core-HMC.229

There were a lot of SqueakDBX tests that don't release the connection after
testing. So, this can result in a "Too many connections error". Now, all
tests release connections."







>
> On Thu, Sep 24, 2009 at 20:52, Mariano Martinez Peck <
> marianopeck at gmail.com> wrote:
>
>>
>>
>> On Thu, Sep 24, 2009 at 3:13 PM, John Toohey <jt at parspro.com> wrote:
>>
>>> Okay, I've installed SqueakGlorp into my existing image, and added the
>>> symlink to the VM for openDBX. I can get up to 44 tests to pass, but it
>>> seems that the tests are not releasing their DB connections, so after a
>>> while the rest fail with a "Too many connections error"
>>
>>
>> These tests you are talking about are SqueakDBX tests or Glorp tests when
>> using SqueakDBX as the driver ?
>>
>>
>>>
>>> Is this how it should be for those tests? I am just selecting the test
>>> package and selecting "run tests".
>>>
>>>
>> I need to know which tests are you running, if OpenDBX-Core-Tests or
>> GlorpTest. Anyway, tests should release connections. If this is not done, so
>> we have a bug in our tests.
>>
>>
>>
>>>
>>>
>>>
>>
>>>
>>> On Thu, Sep 24, 2009 at 12:24, Mariano Martinez Peck <
>>> marianopeck at gmail.com> wrote:
>>>
>>>>
>>>>
>>>> On Thu, Sep 24, 2009 at 3:06 PM, John Toohey <jt at parspro.com> wrote:
>>>>
>>>>> - I've managed to build it on a VMWare VM running Ubunto, and using the
>>>>> --disable-utils switch, I've built it on OSX also. However on both systems
>>>>> when I run the test suite I see this :-
>>>>
>>>>
>>>> Excellent news!!! I tell you: there is an utility called odbx-sql that
>>>> let you send queries trough OpenDBX. However, that utility requires external
>>>> libraries which seem to have problems, at least in Windows. So, when we
>>>> compile in Windows, we must disable that utility. But remember this is an
>>>> utility, not OpenDBX core or feature. It is not important. I didn't know it
>>>> has problems also with MacOS, so I will update the webpage. Thanks for the
>>>> testing!!!!
>>>>
>>>>
>>>>
>>>>>
>>>>> `--> ./odbxtest -b pgsql -h localhost -p 5432 -d juliet -u postgres -w
>>>>> postgres
>>>>>     ODBX_CAP_BASIC: 1
>>>>>     ODBX_CAP_LO: 0
>>>>>     ODBX_OPT_API_VERSION: 10200
>>>>>     ODBX_OPT_THREAD_SAFE: 1
>>>>>     ODBX_OPT_TLS: 1
>>>>>     ODBX_OPT_CONNECT_TIMEOUT: 1 (using)
>>>>>     ODBX_OPT_MULTI_STATEMENTS: 1 (using)
>>>>>     ODBX_OPT_PAGED_RESULTS: 0
>>>>>     ODBX_OPT_COMPRESS: 0
>>>>>     ODBX_CAP_BASIC: 1
>>>>>     ODBX_CAP_LO: 0
>>>>>     ODBX_OPT_API_VERSION: 10200
>>>>>     ODBX_OPT_THREAD_SAFE: 1
>>>>>     ODBX_OPT_TLS: 1
>>>>>     ODBX_OPT_CONNECT_TIMEOUT: 1 (using)
>>>>>     ODBX_OPT_MULTI_STATEMENTS: 1 (using)
>>>>>     ODBX_OPT_PAGED_RESULTS: 0
>>>>>     ODBX_OPT_COMPRESS: 0
>>>>> WARNING:  nonstandard use of \\ in a string literal
>>>>> LINE 1: SELECT * FROM odbxtest WHERE col = 'a''''\\''b'
>>>>>                                            ^
>>>>> HINT:  Use the escape string syntax for backslashes, e.g., E'\\'.
>>>>> Error in odbx_result(): ERROR:  relation "odbxtest" does not exist
>>>>>
>>>>> Error in odbx_result(): ERROR:  relation "odbxtest" does not exist
>>>>>
>>>>> Not sure if this means there is a problem with the build (I downloaded
>>>>> to latest one) or a problem with the test suite.
>>>>>
>>>>>
>>>> As you guessed this seems to be a problem with the test suit, but the
>>>> build is seems to be perfect. I will send this part to OpenDBX mailing list
>>>> and see if I receive some help. Thanks for the report.
>>>>
>>>>
>>>>> In order to continue and install GlorpDBX into my Pharo image, do I
>>>>> need to build a new image, as my current one already has Pharo-Glorp
>>>>> installed. I've never figured out how to uninstall packages in Smalltalk.
>>>>>
>>>>
>>>> Yeah. Me neither :(
>>>>
>>>> But, what you can do is install just SqueakDBX in that image and run the
>>>> SqueakDBX tests, using PostgreSQL to see if everything is ok. You should
>>>> have all green tests. If you get this, I would really appreciate if you can
>>>> make a binary for MacOS :)
>>>>
>>>> Then, you can take a new image and install GlorpDBX as explained in the
>>>> wiki.  Most of the tests should be green. I say "most" because there are
>>>> some that fails because of what I told you that we don't support large
>>>> database fields.
>>>>
>>>> Best,
>>>>
>>>> Mariano
>>>>
>>>>
>>>>>
>>>>>
>>>>> On Thu, Sep 24, 2009 at 11:15, Mariano Martinez Peck <
>>>>> marianopeck at gmail.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Sep 24, 2009 at 1:59 PM, John Toohey <jt at parspro.com> wrote:
>>>>>>
>>>>>>> I'm currently trying to build openDBX on OSX. I've finally found the
>>>>>>> postgres client libs and include files, and I've run ./configure
>>>>>>> -with-backends="pgsql", but when I run make I get this error :-
>>>>>>>
>>>>>>
>>>>>> Excellent you are giving SqueakDBX at least a try. The real thing is
>>>>>> this: The squeakDBX team is composed by 4 people. Only one has a Mac and he
>>>>>> is very busy with his work. I have ALL databases working on Linux and
>>>>>> Windows. And I have also prepared binaries for that because I can (dlls,
>>>>>> .deb and .rpm). But I cannot do it with Mac because I don't have a mac and I
>>>>>> don't know how binaries work on Mac. If you make it work, I would really
>>>>>> love you help me generating the binaries so that I can update them to
>>>>>> OpenDBX website.
>>>>>>
>>>>>> Which OpenDBX version are you trying to compile? You should try 1.4.3.
>>>>>>
>>>>>>
>>>>>> Can you try configuring this way please ?
>>>>>>
>>>>>> ./configure --disable-utils --with-backends="pgsql"
>>>>>>
>>>>>> If it works, I then explain you why that :)
>>>>>>
>>>>>>
>>>>>> Making all in po
>>>>>>> g++ -DHAVE_CONFIG_H -I. -I.. -I../lib
>>>>>>> -DLOCALEDIR=\"/usr/local/share/locale\"
>>>>>>> -DKEYWORDFILE=\"/usr/local/share/opendbx/keywords\"
>>>>>>> -I/opt/local/include/postgresql83  -g -O2 -MT odbx_sql-argmap.o -MD -MP -MF
>>>>>>> .deps/odbx_sql-argmap.Tpo -c -o odbx_sql-argmap.o `test -f 'argmap.cpp' ||
>>>>>>> echo './'`argmap.cpp
>>>>>>> mv -f .deps/odbx_sql-argmap.Tpo .deps/odbx_sql-argmap.Po
>>>>>>> g++ -DHAVE_CONFIG_H -I. -I.. -I../lib
>>>>>>> -DLOCALEDIR=\"/usr/local/share/locale\"
>>>>>>> -DKEYWORDFILE=\"/usr/local/share/opendbx/keywords\"
>>>>>>> -I/opt/local/include/postgresql83  -g -O2 -MT odbx_sql-commands.o -MD -MP
>>>>>>> -MF .deps/odbx_sql-commands.Tpo -c -o odbx_sql-commands.o `test -f
>>>>>>> 'commands.cpp' || echo './'`commands.cpp
>>>>>>> mv -f .deps/odbx_sql-commands.Tpo .deps/odbx_sql-commands.Po
>>>>>>> g++ -DHAVE_CONFIG_H -I. -I.. -I../lib
>>>>>>> -DLOCALEDIR=\"/usr/local/share/locale\"
>>>>>>> -DKEYWORDFILE=\"/usr/local/share/opendbx/keywords\"
>>>>>>> -I/opt/local/include/postgresql83  -g -O2 -MT odbx_sql-completion.o -MD -MP
>>>>>>> -MF .deps/odbx_sql-completion.Tpo -c -o odbx_sql-completion.o `test -f
>>>>>>> 'completion.cpp' || echo './'`completion.cpp
>>>>>>> mv -f .deps/odbx_sql-completion.Tpo .deps/odbx_sql-completion.Po
>>>>>>> g++ -DHAVE_CONFIG_H -I. -I.. -I../lib
>>>>>>> -DLOCALEDIR=\"/usr/local/share/locale\"
>>>>>>> -DKEYWORDFILE=\"/usr/local/share/opendbx/keywords\"
>>>>>>> -I/opt/local/include/postgresql83  -g -O2 -MT odbx_sql-odbx-sql.o -MD -MP
>>>>>>> -MF .deps/odbx_sql-odbx-sql.Tpo -c -o odbx_sql-odbx-sql.o `test -f
>>>>>>> 'odbx-sql.cpp' || echo './'`odbx-sql.cpp
>>>>>>> odbx-sql.cpp: In function 'int main(int, char**)':
>>>>>>> odbx-sql.cpp:228: error: invalid conversion from 'char* (*)(const
>>>>>>> char*, int)' to 'int (*)(const char*, int)'
>>>>>>> make[3]: *** [odbx_sql-odbx-sql.o] Error 1
>>>>>>> make[2]: *** [all-recursive] Error 1
>>>>>>> make[1]: *** [all-recursive] Error 1
>>>>>>> make: *** [all] Error 2
>>>>>>>
>>>>>>> Do you know of any pre-built binaries that will work on OSX?
>>>>>>>
>>>>>>
>>>>>>
>>>>>> I don't know, but I would love to have them!!!
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> On Wed, Sep 23, 2009 at 23:09, Mariano Martinez Peck <
>>>>>>> marianopeck at gmail.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Sep 23, 2009 at 11:27 PM, John Toohey <jt at parspro.com>wrote:
>>>>>>>>
>>>>>>>>> Hey,
>>>>>>>>> Thanks for looking into this for me. I'm just using Pharo as my app
>>>>>>>>> is using the beta of Seaside 3.0. I'm going to do as you suggested and put
>>>>>>>>> some debug statements in PGConnection, and see where that leads.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Ok, let us know what the problem was if you find it :)
>>>>>>>>
>>>>>>>>
>>>>>>>>> Is it a lot of changes from me to switch to DBX on Pharo?
>>>>>>>>>
>>>>>>>>
>>>>>>>> No. Your application shouldn't have any changes. You application
>>>>>>>> will use just Glorp. But in this case Glorp won't use the native PostgreSQL
>>>>>>>> driver, but SqueakDBX with PostgreSQL.
>>>>>>>>
>>>>>>>>
>>>>>>>>> I assume there are no changes in my Glorp code, but what else do I
>>>>>>>>> need to do.
>>>>>>>>>
>>>>>>>>
>>>>>>>> The only thing you must do is:
>>>>>>>>
>>>>>>>> - Install the PostgreSQL client library (if you don't have it
>>>>>>>> already installed).
>>>>>>>> - Compile or Install (if you are in Linux you can use the .rpm or
>>>>>>>> .deb) OpenDBX with PostgreSQL,
>>>>>>>>
>>>>>>>> You can read how to do it here: http://wiki.squeak.org/squeak/6129
>>>>>>>>
>>>>>>>> You must do it only once. Once this is done, everything is easy and
>>>>>>>> you can take advantage of all the SqueakDBX features (
>>>>>>>> http://wiki.squeak.org/squeak/6106). OpenDBX works perfect with
>>>>>>>> PostgreSQL in Linux, Mac and Windows. We have also have GlorpDBX working
>>>>>>>> with SqueakDBX and PostgreSQL and it is very easy to install:
>>>>>>>>
>>>>>>>> http://wiki.squeak.org/squeak/6130    (where it says GlorpDBX).
>>>>>>>>
>>>>>>>> The only limitation we have right now is that we don't support large
>>>>>>>> objects (blob, clob, etc) yet.
>>>>>>>>
>>>>>>>> If you try it and have any problem, please, let us know!!!
>>>>>>>>
>>>>>>>>
>>>>>>>>> I develop on OSX and deploy to Linux for production.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> No problem.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>>
>>>>>>>> Mariano
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Sep 23, 2009 at 21:03, Mariano Martinez Peck <
>>>>>>>>> marianopeck at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I was thinking if you have the problem in Squeak also or just
>>>>>>>>>> Pharo ?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Sep 23, 2009 at 8:12 PM, Mariano Martinez Peck <
>>>>>>>>>> marianopeck at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Sep 23, 2009 at 8:25 PM, John Toohey <jt at parspro.com>wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hi John! This list is SqueakDBX related, so perhaps I am not the
>>>>>>>>>>> best one to help you. I just make Glorp and PostgreSQL native driver to work
>>>>>>>>>>> on Pharo. Actually, the only thing I did is to have all green tests. No more
>>>>>>>>>>> than that. I don't have a Glorp + native postgresql driver application.
>>>>>>>>>>> Anyway, I will try to help you ;)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> My app frequently losses the connection the Postgres, and throws
>>>>>>>>>>>> a "Connection not Valid" exception. Unfortunately on the latest Pharo
>>>>>>>>>>>> images, there is a bug that causes this to write the stack trace to disk,
>>>>>>>>>>>> and then the image locks up.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> you meat there is a Pharo bug that when there is a walkback and
>>>>>>>>>>> the stacktrace is written to disk, it freezes the image ?  is there a bug
>>>>>>>>>>> report of that ? Could you reproduce that ?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>  I think the socket on the PGConnection class is being closed or
>>>>>>>>>>>> times out, and I need to know is there someway that I can debug this or if
>>>>>>>>>>>> its a know issue.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Yes, that's probably the case. I can see in code that the error
>>>>>>>>>>> 'Connection not valid' when the connection is not connected or if it is nil.
>>>>>>>>>>> If you see your stacktrace, you can see that the socket variable is nil.
>>>>>>>>>>> That's why the error. Now, the question is why the socket is nil ?
>>>>>>>>>>>
>>>>>>>>>>> I looked for the code to see where can be the socket set to nil,
>>>>>>>>>>> and I found some places:  PGConnection stTerminalError:  , stTerminated
>>>>>>>>>>> Perhaps you can put a "self halt" in those methods and then debug
>>>>>>>>>>> them.
>>>>>>>>>>>
>>>>>>>>>>> The PGConnection seems to do a lot of "self log: 'XXXX'". So,
>>>>>>>>>>> perhaps you can have a Transcript opened to see that.
>>>>>>>>>>>
>>>>>>>>>>> Finally, why don't you give a try to GlorpDBX + SqueakDBX ?   It
>>>>>>>>>>> works pretty well and has good performance.
>>>>>>>>>>>
>>>>>>>>>>> I hope this helps. Remember I am not a native postgresql driver
>>>>>>>>>>> guru. Perhaps you can ask also in squeak-dev or pharo mailing lists.
>>>>>>>>>>>
>>>>>>>>>>> Best,
>>>>>>>>>>>
>>>>>>>>>>> Mariano
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>  I am using the latest Pharo image, with the latest Pharo-Glorp
>>>>>>>>>>>> package, and the newest Postgres driver. I think this is the relevant part
>>>>>>>>>>>> of the stack trace :-
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> PGConnection(Object)>>error:
>>>>>>>>>>>> Receiver: a PGConnection
>>>>>>>>>>>> Arguments and temporary variables:
>>>>>>>>>>>>  aString: 'Connection not valid'
>>>>>>>>>>>> Receiver's instance variables:
>>>>>>>>>>>>  trace: 0
>>>>>>>>>>>> state: #Terminated
>>>>>>>>>>>>  events: an OrderedCollection()
>>>>>>>>>>>> socket: nil
>>>>>>>>>>>>  readBuffer: 'PblankT id ��name �$password �6client...etc...
>>>>>>>>>>>> readIndex: 384
>>>>>>>>>>>>  lastReadIndex: 384
>>>>>>>>>>>> writeBuffer: nil
>>>>>>>>>>>>  processId: 28132
>>>>>>>>>>>> secretKey: 1795051851
>>>>>>>>>>>>  sql: 'SELECT t1.id, t1.name, t1.password, t1.client, t1.role,
>>>>>>>>>>>> t1.mobile, t1.ema...etc...
>>>>>>>>>>>>  functionCallOid: nil
>>>>>>>>>>>> functionCallArgs: nil
>>>>>>>>>>>>  copyStream: nil
>>>>>>>>>>>> result: a PGResult
>>>>>>>>>>>>  connectionArgs: a PGConnectionArgs
>>>>>>>>>>>> notificationSubscribers: nil
>>>>>>>>>>>>  fieldConverters: an IdentityDictionary(16->[closure] in
>>>>>>>>>>>> PGConnection class>>boo...etc...
>>>>>>>>>>>>  sqlStringConverter: [closure] in PGConnection>>clientEncoding:
>>>>>>>>>>>> receivingTextConverter: an UTF8TextConverter
>>>>>>>>>>>>
>>>>>>>>>>>> PGConnection>>execute:
>>>>>>>>>>>> Receiver: a PGConnection
>>>>>>>>>>>> Arguments and temporary variables:
>>>>>>>>>>>>  sqlString: 'SELECT t1.id, t1.typename, t1.description,
>>>>>>>>>>>> t1.selectioncount, t1.pr...etc...
>>>>>>>>>>>>  Receiver's instance variables:
>>>>>>>>>>>> trace: 0
>>>>>>>>>>>>  state: #Terminated
>>>>>>>>>>>> events: an OrderedCollection()
>>>>>>>>>>>>  socket: nil
>>>>>>>>>>>> readBuffer: 'PblankT id ��name �$password �6client...etc...
>>>>>>>>>>>>  readIndex: 384
>>>>>>>>>>>> lastReadIndex: 384
>>>>>>>>>>>>  writeBuffer: nil
>>>>>>>>>>>> processId: 28132
>>>>>>>>>>>>  secretKey: 1795051851
>>>>>>>>>>>> sql: 'SELECT t1.id, t1.name, t1.password, t1.client, t1.role,
>>>>>>>>>>>> t1.mobile, t1.ema...etc...
>>>>>>>>>>>>  functionCallOid: nil
>>>>>>>>>>>> functionCallArgs: nil
>>>>>>>>>>>>  copyStream: nil
>>>>>>>>>>>> result: a PGResult
>>>>>>>>>>>>  connectionArgs: a PGConnectionArgs
>>>>>>>>>>>> notificationSubscribers: nil
>>>>>>>>>>>>  fieldConverters: an IdentityDictionary(16->[closure] in
>>>>>>>>>>>> PGConnection class>>boo...etc...
>>>>>>>>>>>>  sqlStringConverter: [closure] in PGConnection>>clientEncoding:
>>>>>>>>>>>> receivingTextConverter: an UTF8TextConverter
>>>>>>>>>>>>
>>>>>>>>>>>> SqueakDatabaseAccessor>>basicExecuteSQLString:
>>>>>>>>>>>> Receiver: a SqueakDatabaseAccessor
>>>>>>>>>>>> Arguments and temporary variables:
>>>>>>>>>>>>  aString: 'SELECT t1.id, t1.typename, t1.description,
>>>>>>>>>>>> t1.selectioncount, t1.prio...etc...
>>>>>>>>>>>>  result: nil
>>>>>>>>>>>> rowCollection: nil
>>>>>>>>>>>>  Receiver's instance variables:
>>>>>>>>>>>> connection: a PGConnection
>>>>>>>>>>>>  currentLogin: a Login(a PostgreSQLPlatform, 'postgres',
>>>>>>>>>>>> '847129', 'localhost_ju...etc...
>>>>>>>>>>>>  logging: true
>>>>>>>>>>>> reusePreparedStatements: nil
>>>>>>>>>>>>  deniedCommands: nil
>>>>>>>>>>>> mutex: a Semaphore()
>>>>>>>>>>>>  dependents: nil
>>>>>>>>>>>> isInTransaction: false
>>>>>>>>>>>>
>>>>>>>>>>>> Any help would be appreciated, as this is stopping me from going
>>>>>>>>>>>> to a Beta test at the moment.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> -JT
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> SqueakDBX mailing list
>>>>>>>>>>>> SqueakDBX at lists.squeakfoundation.org
>>>>>>>>>>>> http://lists.squeakfoundation.org/mailman/listinfo/squeakdbx
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> -JT
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> -JT
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> -JT
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> -JT
>>>
>>>
>>>
>>
>> _______________________________________________
>> SqueakDBX mailing list
>> SqueakDBX at lists.squeakfoundation.org
>> http://lists.squeakfoundation.org/mailman/listinfo/squeakdbx
>>
>>
>
>
> --
> -JT
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeakdbx/attachments/20090924/d6995370/attachment.htm


More information about the SqueakDBX mailing list