<div id="mb_3"><div><span class="q">On 8/2/06, <b class="gmail_sendername">Yanni Chiu</b> <<a href="mailto:yanni@rogers.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">yanni@rogers.com</a>> wrote:
</span></div><div><div><span class="q"><span class="gmail_quote"></span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Jeremy Shute wrote:<br>> GLORP is a dream to work with. It almost makes those spurious<br>> object-access patterns look free. :-) But, if you don't want to store a<br>> whole table in memory and you don't want to go twiddling down the whole
<br>> B-tree every time you do an object access, you want a cursor, and I<br>> haven't quite figured out how to get that to work...<br>><br>> On a side note, I achieved 10-12x the throughput with my prototype program
<br>> (written in a different language and dumping the serialized representation<br>> to disk), and I have moved on to yet another language to finish the job.<br>> *Sigh* One day I'll be able to use Squeak.<br>
<br>
Do you have to have GLORP in the critical path?<br>If you only have a few tables, maybe coding the SQL directly<br>is possible. Or, use GLORP for the bulk of your model, but<br>isolate the performance critical portion of the model in a
<br>separate subsystem, and use custom SQL for that portion.</blockquote></span></div><div><div><br>Yes,
it's very much in the critical path. I'm sorry but I'm still amazed
that it can't assemble and ship queries to Postgres as fast as I can
get data from a cable modem. That was a huge shocker -- I've got a
weak cable connection on one side and a disk on the other, and rifling
through strings and objects in RAM is the issue???
<br><br>I agree that there are solutions which involve direct-SQL
access, and making a mess of otherwise clean code (but a well isolated
mess, of course). I could also simply contribute to GLORP in order to
make it better. I would do this in a second if I thought it would get
me from point A to point B, GLORP is great software!
<br><br>As an off-topic side-note, in order to GET from point A to
point B, I addressed the problem by developing a "new to me" paradigm
for dealing with data of this type. So far, I think I did the right
thing. Like SQL (and unlike serialization or the Prevayler approach),
multiple programs can get access to the same objects from out-of-core
datastructures. But unlike SQL, the indices require ~1 disk seeks to
get at objects after a cache-miss (
i.e. hash based), columns can be in a much more structured format
(think of something similar to memcpy for a DOM tree, for instance),
etc.<br><br><br></div></div><div><span class="q"><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Having said that, unless you use cursors, the postgres<br>driver will pull the entire result set into memory.</blockquote></span></div><div><div><br>Sigh. I know. The options seem to be:<br><br>* Get the whole result set if it fits in memory.
<br>* Seek the same B-tree nodes over and over again if it doesn't (the root should be cached by the RDBMS, of course).<br></div><br>Cursors
would definitely be the answer to this, but I recognize that I am in
the minority in my need for them. Really, I don't think the cursor
would fix my 10-12x problem. For me, it's a matter of bypassing caches
and using prepared statements. But, I wanted to deal in objects, and
found a fine way to continue to do that without the overhead of OR
mapping.
</div><div><span class="q"><br><br><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Assuming you can get cursors working, I'd be surprised if
<br>you couldn't match the 10-12x increase you got using another
<br>language. Basically, the postgres driver just pulls bytes off<br>the socket and makes arrays of strings.<br></blockquote></span></div><br>I'm
betting that Squeak is capable of that 10-12x with proper massage. But
the 10-12x will simply match the prototype implementation, which in
turn has not been massaged. (In fact, both implementations are really
stupid in that they are SERIAL.) I have figured out how to use a proxy
object to get the GLORP sessions to be thread-safe, but the next
barrier will be lock contention as the serial implementation becomes a
simple producer/consumer queue.
<br><br>I would say that Squeak is currently state-of-the-art in terms
of programmer interface -- Seaside and GLORP are basically unrivalled
in their design and terseness. Avi didn't need any of the stuff I
mentioned to make a great piece of software. But, the subject was
"scalability", so I wanted to offer myself as a data point.
<br></div><div><span class="sg"><br>Jeremy</span></div></div>