<br><br><div class="gmail_quote">---------- Forwarded message ----------<br>From: <b class="gmail_sendername">Mariano Martinez Peck</b> <span dir="ltr"><<a href="mailto:marianopeck@gmail.com">marianopeck@gmail.com</a>></span><br>
Date: Wed, Jun 1, 2011 at 8:54 PM<br>Subject: Re: Increasing the performances of a Seaside application<br>To: Henrik Sperre Johansen <<a href="mailto:henrik.s.johansen@veloxit.no">henrik.s.johansen@veloxit.no</a>>, <a href="mailto:proyecto_relacional@googlegroups.com">proyecto_relacional@googlegroups.com</a><br>
<br><br><br><br><div class="gmail_quote"><div class="im">On Tue, May 31, 2011 at 11:30 PM, Henrik Sperre Johansen <span dir="ltr"><<a href="mailto:henrik.s.johansen@veloxit.no" target="_blank">henrik.s.johansen@veloxit.no</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#ffffff">
<span style="border-collapse:separate;color:rgb(0, 0, 0);font-family:'Times New Roman';font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span style="border-collapse:collapse;font-family:Verdana,Geneva,Helvetica,Arial,sans-serif;font-size:13px">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">
<div style="border-color:rgb(204, 204, 204)">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">Thanks. I was not clear. What we
actually do is:<br>
<br>
(code = OpenDBX resultTimeout) ifTrue: [ (Delay
forMilliseconds:<br>
(aQuerySettings timeout asMiliseconds)) wait ].<br>
<br>
Is that better? Even if it lets just run processes of the
same priority,<br>
this is good anyway because what we want is at least be
able to process<br>
other queries. Probably, those other processes are being
done from other<br>
Process.<br>
</blockquote>
<br>
</div>
It's a bit better. There's no starvation if the timeout is
greater than zero, but it's still a form of busy waiting, and
it limits the number of queries per second per connection to
at most 1000 (actually 1000 / timeout). To compare this with
our native implementation - PostgresV3 - I measured 6k+
queries per second per connection and it's still not optimized
for Cog (#perform: is slow on Cog).<br>
<font color="#888888"><br>
</font></blockquote>
<div style="border-color:rgb(204, 204, 204)"><br>
Thanks Levente. Unfortunatly I guess that's all we can do with
a blocking FFI :(<br>
<br>
Not really :)<br></div></span></span></div></blockquote></div><div><br><br>Thanks Henrik.<br><br>Before analyzing your suggestions, let me tell you something stupid we did in DBX that I have just realized. There are TWO different timeouts. <br>
<br>1) OpenDBX timeout: the one send by parameter to OpenDBX function:<br><a href="http://www.linuxnetworks.de/doc/index.php/OpenDBX/C_API/odbx_result" target="_blank">http://www.linuxnetworks.de/doc/index.php/OpenDBX/C_API/odbx_result</a><br>
that determinates the time the C function of OpenDBX waits for the result. <br><br>nextResultSet: aConnection querySettings: aQuerySettings onReturn: aBlock <br> "Returns the next resultSet from the last resultSet. When there is no more resultSets, <br>
the block is evaluated." <br> | handle err handleArray |<br> handleArray := WordArray with: 0.<br><br> err := OpenDBX current<br> apiQueryResult: aConnection handle<br> handle: handleArray<br>
<span style="background-color:rgb(255, 255, 51)"> timeout: aQuerySettings timeoutAsDBXTimeSpec</span><br> chunk: aQuerySettings pageSize.<br>......<br><br>2) SqueakDBX timeout: the time we wait in the IMAGE side once we got a timeout from OpenDBX. <br>
this is what I showed you:<div class="im"><br><br> (code = OpenDBX resultTimeout) ifTrue: [ (Delay forMilliseconds: (aQuerySettings timeout asMiliseconds)) wait ].<br><br><br></div>So....as you can see we are using both values for both things. This is not necessary and maybe stupid. <br>
<br>The default timeout now is 10 miliseconds<br><br>>> defaultTimeout<br> "10 miliseconds"<br> ^DBXQueryTimeout seconds: 0 microseconds: 10.<br><br>So...if I follow your a) you smartly recommend to use 100ms. And in this case you are talking about the OpenDBX timeout and only for the first time. This way most queries will be cought in the first try and even if they do not, we return fast. And then, for future calls of the same query (only if there is a timeout) we use a really short timeout. For example 1ms. The idea is to wait as much as possible in image side (Delay) rather than C.<br>
<br>At the same time, with b) you recommend you use an incremental SqueakDBX timeout (the Delay). So we can start with <span style="border-collapse:separate;color:rgb(0, 0, 0);font-family:'Times New Roman';font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span style="border-collapse:collapse;font-family:Verdana,Geneva,Helvetica,Arial,sans-serif;font-size:13px">1 ms and then grow 2 4 8 16 32 64 128 256 512 up to 1024. And if we get until 1024 we continue using that value? but isn't 1ms too small? because this value will be used if a timeout happened (the result took more than 100ms). So it is quite weird that it will be ready just 1ms after. No?<br>
<br></span></span>so...did I understand correctly ?<br> </div><div class="im"><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex"><div text="#000000" bgcolor="#ffffff">
<span style="border-collapse:separate;color:rgb(0, 0, 0);font-family:'Times New Roman';font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span style="border-collapse:collapse;font-family:Verdana,Geneva,Helvetica,Arial,sans-serif;font-size:13px"><div>
<br>
You could<br>
a) Use a default timeout for the first call which means it
actually completes more queries on the first try yet still
returns fast, say 100ms rather than 1ms.<br>
(For later calls just to check if it is possibly finished, you
probably want to block for as short a time as possible though)<br>
b) Use an exponentially growing value for the Delay rather
than a constant one, starting at 1ms and max some other value<br>
1 2 4 8 16 32 64 128 256 512 1024 for instance, polling once
per second shouldn't hurt other processes at all, yet give ok
responsiveness for queries > 1 seconds.<br>
<br>
This way, you (in the cases where potential is 9k queries
/sec) will have a hard cap at 10k queries (due to the 100ms
block time), and hurt those above that as little as possible
using Delays. What you don't have though, is a cap of around
1k, due to calls never completing in 1ms, and having to wait
(at least, I don't know the default value of aQuerySettings
timeout) 1ms for each due to minimum delay wait time
resolution.<br>
<br><br> </div></span></span></div></blockquote><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex"><div text="#000000" bgcolor="#ffffff">
<span style="border-collapse:separate;color:rgb(0, 0, 0);font-family:'Times New Roman';font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span style="border-collapse:collapse;font-family:Verdana,Geneva,Helvetica,Arial,sans-serif;font-size:13px"><div style="border-color:rgb(204, 204, 204)">
</div>
</span></span>Btw, unless the microseconds and seconds are
switched, this could be simpler (as well as misspelled :) ):<br>
DBXQueryTimeout >> asMiliseconds<br>
^ (self seconds * 1000) + (((self microseconds / 1000) asFloat)
integerPart asInteger )<br>
^ (self seconds * 1000) + (self microseconds // 1000)<br>
<br></div></blockquote></div><div><br>Thanks :)<br><br> </div><div class="im"><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex"><div text="#000000" bgcolor="#ffffff">
DBXTimeSpec also has an field called nseconds which contains
microseconds, rather confusing :)<br></div></blockquote></div><div><br>yes, I know. The problem was the OpenDBX/C uses that structure but from image side it was nicer to use microseconds hehehehe<br> </div><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">
<div text="#000000" bgcolor="#ffffff">
<br>
Cheers,<br>
Henry<br>
</div>
</blockquote></div><div><div></div><div class="h5"><br><br clear="all"><br>-- <br>Mariano<br><a href="http://marianopeck.wordpress.com" target="_blank">http://marianopeck.wordpress.com</a><br><br>
</div></div></div><br><br clear="all"><br>-- <br>Mariano<br><a href="http://marianopeck.wordpress.com" target="_blank">http://marianopeck.wordpress.com</a><br><br>