<br><br><div class="gmail_quote">---------- Forwarded message ----------<br>From: <b class="gmail_sendername">Henrik Sperre Johansen</b> <span dir="ltr"><<a href="mailto:henrik.s.johansen@veloxit.no">henrik.s.johansen@veloxit.no</a>></span><br>
Date: Fri, Jun 3, 2011 at 8:35 AM<br>Subject: Re: Increasing the performances of a Seaside application<br>To: Mariano Martinez Peck <<a href="mailto:marianopeck@gmail.com">marianopeck@gmail.com</a>><br><br><br><u></u>
<div bgcolor="#ffffff" text="#000000"><div class="im">
On 01.06.2011 20:54, Mariano Martinez Peck wrote:
<blockquote type="cite"><br>
<br>
<div class="gmail_quote">On Tue, May 31, 2011 at 11:30 PM, Henrik
Sperre Johansen <span dir="ltr"><<a href="mailto:henrik.s.johansen@veloxit.no" target="_blank">henrik.s.johansen@veloxit.no</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">
<div text="#000000" bgcolor="#ffffff"> <span style="border-collapse:separate;color:rgb(0, 0, 0);font-family:'Times New Roman';font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span style="border-collapse:collapse;font-family:Verdana,Geneva,Helvetica,Arial,sans-serif;font-size:13px">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">
<div style="border-color:rgb(204, 204, 204)">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">Thanks. I was not
clear. What we actually do is:<br>
<br>
(code = OpenDBX resultTimeout) ifTrue: [ (Delay
forMilliseconds:<br>
(aQuerySettings timeout asMiliseconds)) wait ].<br>
<br>
Is that better? Even if it lets just run
processes of the same priority,<br>
this is good anyway because what we want is at
least be able to process<br>
other queries. Probably, those other processes are
being done from other<br>
Process.<br>
</blockquote>
<br>
</div>
It's a bit better. There's no starvation if the
timeout is greater than zero, but it's still a form of
busy waiting, and it limits the number of queries per
second per connection to at most 1000 (actually 1000 /
timeout). To compare this with our native
implementation - PostgresV3 - I measured 6k+ queries
per second per connection and it's still not optimized
for Cog (#perform: is slow on Cog).<br>
<font color="#888888"><br>
</font></blockquote>
<div style="border-color:rgb(204, 204, 204)"><br>
Thanks Levente. Unfortunatly I guess that's all we can
do with a blocking FFI :(<br>
<br>
Not really :)<br>
</div>
</span></span></div>
</blockquote>
<div><br>
<br>
Thanks Henrik.<br>
<br>
Before analyzing your suggestions, let me tell you something
stupid we did in DBX that I have just realized. There are TWO
different timeouts. <br>
</div>
</div>
</blockquote></div>
Yes :)<div class="im"><br>
<blockquote type="cite">
<div class="gmail_quote">
<div>
<br>
1) OpenDBX timeout: the one send by parameter to OpenDBX
function:<br>
<a href="http://www.linuxnetworks.de/doc/index.php/OpenDBX/C_API/odbx_result" target="_blank">http://www.linuxnetworks.de/doc/index.php/OpenDBX/C_API/odbx_result</a><br>
that determinates the time the C function of OpenDBX waits for
the result. <br>
<br>
nextResultSet: aConnection querySettings: aQuerySettings
onReturn: aBlock <br>
"Returns the next resultSet from the last resultSet. When
there is no more resultSets, <br>
the block is evaluated." <br>
| handle err handleArray |<br>
handleArray := WordArray with: 0.<br>
<br>
err := OpenDBX current<br>
apiQueryResult: aConnection handle<br>
handle: handleArray<br>
<span style="background-color:rgb(255, 255, 51)">
timeout: aQuerySettings timeoutAsDBXTimeSpec</span><br>
chunk: aQuerySettings pageSize.<br>
......<br>
<br>
2) SqueakDBX timeout: the time we wait in the IMAGE side once
we got a timeout from OpenDBX. <br>
this is what I showed you:<br>
<br>
(code = OpenDBX resultTimeout) ifTrue: [ (Delay
forMilliseconds: (aQuerySettings timeout asMiliseconds)) wait
].<br>
<br>
<br>
So....as you can see we are using both values for both things.
This is not necessary and maybe stupid. <br>
</div>
</div>
</blockquote></div>
Yes :)<div class="im"><br>
<blockquote type="cite">
<div class="gmail_quote">
<div>
<br>
The default timeout now is 10 miliseconds<br>
<br>
>> defaultTimeout<br>
"10 miliseconds"<br>
^DBXQueryTimeout seconds: 0 microseconds: 10.<br>
<br>
So...if I follow your a) you smartly recommend to use 100ms.
And in this case you are talking about the OpenDBX timeout and
only for the first time. This way most queries will be cought
in the first try and even if they do not, we return fast. And
then, for future calls of the same query (only if there is a
timeout) we use a really short timeout. For example 1ms. The
idea is to wait as much as possible in image side (Delay)
rather than C.<br>
</div>
</div>
</blockquote></div>
This isn't milliseconds, this is microSeconds (when you use it for
the C call) :)<br>
I'm not sure of DBX implementation, if it returns immediately when
done or the entire period.<br>
Should be easily testable by setting a really long timeout, like 1
million (1second) and repeating a query you know only take a couple
of milliseconds to complete, say 10 times.<br>
If the test takes 10 seconds wall time, you know it wait entire
period, if not, then it's safe to set the timeout to the maximum
amount of time you feel it's acceptable to block the image. (at
first call, subsequent should still block for mimimum amount in C
and rather use the delay)<div class="im"><br>
<blockquote type="cite">
<div class="gmail_quote">
<div>
<br>
At the same time, with b) you recommend you use an incremental
SqueakDBX timeout (the Delay). So we can start with <span style="border-collapse:separate;color:rgb(0, 0, 0);font-family:'Times New Roman';font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span style="border-collapse:collapse;font-family:Verdana,Geneva,Helvetica,Arial,sans-serif;font-size:13px">1 ms and then grow 2 4 8 16 32 64 128 256 512 up to
1024. And if we get until 1024 we continue using that
value? but isn't 1ms too small? because this value will
be used if a timeout happened (the result took more than
100ms). So it is quite weird that it will be ready just
1ms after. No?<br>
<br>
</span></span>so...did I understand correctly ?<br>
</div>
</div>
</blockquote></div>
The timeout in C call is in microseconds, thus 100 means 1/10th of a
millisecond, not 100 milliseconds.<br>
Thus starting at 1ms delay makes more sense. Other than that, you
understood perfectly. <br><div class="im">
<br>
<blockquote type="cite">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">
<div text="#000000" bgcolor="#ffffff">
<span style="border-collapse:separate;color:rgb(0, 0, 0);font-family:'Times New Roman';font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span style="border-collapse:collapse;font-family:Verdana,Geneva,Helvetica,Arial,sans-serif;font-size:13px">
<div> <br>
You could<br>
a) Use a default timeout for the first call which
means it actually completes more queries on the first
try yet still returns fast, say 100ms rather than 1ms.<br>
(For later calls just to check if it is possibly
finished, you probably want to block for as short a
time as possible though)<br>
b) Use an exponentially growing value for the Delay
rather than a constant one, starting at 1ms and max
some other value<br>
1 2 4 8 16 32 64 128 256 512 1024 for instance,
polling once per second shouldn't hurt other processes
at all, yet give ok responsiveness for queries > 1
seconds.<br>
<br>
This way, you (in the cases where potential is 9k
queries /sec) will have a hard cap at 10k queries (due
to the 100ms block time), and hurt those above that as
little as possible using Delays. What you don't have
though, is a cap of around 1k, due to calls never
completing in 1ms, and having to wait (at least, I
don't know the default value of aQuerySettings
timeout) 1ms for each due to minimum delay wait time
resolution.<br>
<br>
<br>
</div>
</span></span></div>
</blockquote>
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">
<div text="#000000" bgcolor="#ffffff">
<span style="border-collapse:separate;color:rgb(0, 0, 0);font-family:'Times New Roman';font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span style="border-collapse:collapse;font-family:Verdana,Geneva,Helvetica,Arial,sans-serif;font-size:13px">
<div style="border-color:rgb(204, 204, 204)"> </div>
</span></span>Btw, unless the microseconds and seconds are
switched, this could be simpler (as well as misspelled :) ):<br>
DBXQueryTimeout >> asMiliseconds<br>
^ (self seconds * 1000) + (((self microseconds / 1000)
asFloat) integerPart asInteger )<br>
^ (self seconds * 1000) + (self microseconds // 1000)<br>
<br>
</div>
</blockquote>
<div><br>
Thanks :)<br>
</div>
</div>
</blockquote></div>
Of course, this round down. If you want it rounded UP to closes
millisecond, you can do:<br>
^ (self seconds * 1000) + (999 + self microseconds // 1000)<br>
Or if you want to round to nearest: <br>
^ (self seconds * 1000) + (500 + self microseconds // 1000)<div class="im"><br>
<blockquote type="cite">
<div class="gmail_quote">
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">
<div text="#000000" bgcolor="#ffffff"> DBXTimeSpec also has an
field called nseconds which contains microseconds, rather
confusing :)<br>
</div>
</blockquote>
<div><br>
yes, I know. The problem was the OpenDBX/C uses that structure
but from image side it was nicer to use microseconds hehehehe<br>
</div>
</div>
</blockquote>
<br></div>
The C struct contains microseconds, and that's what it's being used
as image-side as well :)<br>
Ie. it should be named microSeconds or something instead (mseconds
is too ambiguos )<br>
<br>
<br>
TLDR; You understood perfectly, using the same timeout for both
blocking in C, and waiting in image between C calls is not the best
idea.<br>
Using a longer initial C timeout ensures you get -better- than Delay
resolution response times in the cases where that is possible.<br>
<br>
Cheers,<br>
Henry<br>
<br>
PS. Is there a lock somewhere?<br>
What happens if you do two queries, how do you handle waiting for
both at the same time, and getting the correct result set to the
correct sender?<br>
<br>
</div>
</div><br><br clear="all"><br>-- <br>Mariano<br><a href="http://marianopeck.wordpress.com" target="_blank">http://marianopeck.wordpress.com</a><br><br>