[SqueakDBX] Re: [opendbx] slow insert with sqlite
Mariano Martinez Peck
marianopeck at gmail.com
Tue Jul 5 21:08:12 UTC 2011
On Tue, Jul 5, 2011 at 10:50 PM, Alain Rastoul <alr.dev at free.fr> wrote:
> (sorry for sending this mail again, my pc was off for a long time and the
> message was dated from 2007, people who sort their messages would not see
> I've done a small program in Pharo 1.3 with glorp+opendbx that insert 1000
> rows in a customer table in a sqlite db.
> The 1000 insert takes 140 sec (very slow), but the Pharo profiler says that
> it spend 95%
> of the time waiting for input.
> (in InputEventPollingFetcher>> waitForInput)
> I was wondering if the queries are executed in another thread than the vm
> thread ?
Hi Alain. No. Squeak/Pharo's thread architecture is the so called green
thread, that is, only ONE OS thread is used. Internally, the language
reifies Process, Scheduler, #fork: , etc etc etc. But from the OS point of
view there is only one thread for the VM. So.....the regular FFI blocks the
VM. What does it mean? that while the C function called by FFI is being
executed, the WHOLE VM is block. Notihgn can happen at the same time.
Imagine the function that retrieves the results and needs to wait for
them.....TERRIBLE. So...if the backend does not support async quieries, then
you are screw and dbx may be slow in Pharo. Nothing to do.
However, some backends support async queries, and opendbx let us configure
this. This is explained in:
where it says "External call implementation"
You can see the list of backends that support async queries in here:
Notice that there is some room for improvements, but we didn't have time so
far. Hernik told us some good ideas. But since we didn't need more power so
far we couldn't find time to integrate his ideas. I am forwarding now the
emails to the mailing list. If you can take a look and provide code, it
would be awesome. Basically, it improves how and how much we wait in each
side: image and opendbx.
Finally, notice that Eliot is working in a multithreared FFI for Cog, but it
is not yet available as far as I know.
(I thought I've seen a document about opendbx architecture but could'nt find
> it on the site).
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> libopendbx-devel mailing list
> libopendbx-devel at lists.sourceforge.net
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the SqueakDBX