On 5/11/08, Klaus D. Witzel <klaus.witzel at cobss.com> wrote:
>
> Out of curiosity, what would it be that you intend to let the users do
> while longish queries are tickling the db?
>
> I ask with this background in mind: over several years I headed a team of
> developers who produced 1.5 releases/year of a
> (fat+GUI)client<->(shared+fast)db server app with between 6 and 60 GB data
> etc, 50+ installation sites (institutions with their own db server) and 100+
> users, with 98% of the db data being updated over night (incrementally). We
> brought down the app's db response time from minutes to seconds but still
> never found any reason for using non-blocking queries, because nothing much
> was there that could have been done by the user in the meantime.
>
> Do you just want to see non-freezing queries running or have some other
> reason for that need?


Well..l was the one who asked the question in the first place, so I
guess I should try to explain myself!

First...I completely agree with nightly processing.
 We (two of us, not a team of developers!) do some of that now, but only in
one of our many data sources because we don't have an operational warehouse
yet and violate our license agreements every time we run batch processing on
our production systems!

However, we are using Smalltalk in a Healthcare environment where:

1.  We deal with many "special cases" based on a "per patient model."  We
have "inherited" data that was not set up with aggregate querying in mind.
 We are trying to improve the quality of said data, but this is a long term
project, and in the meantime we do a lot of "if-then's" that are much
simpler in "programming" mode than in SQL.  I'm sure the DBA's out there
might throw eggs, but how else can you get a timestamp out of an xml file
whose name and location is found in a field in a table?  What about systems
that don't even use tables?  Etc, etc, etc...  Healthcare is FUN if you like
something DIFFERENT all the time!!!

2.  We are just now implementing a data warehouse to enable cross-system
nightly processing.  For the weird systems mentioned above, we will
Smalltalk our way to "feed the machine," extracting data and creating input
files for the warehouse.  For this part of the nightly processing, is it
necessary to NOT lock up the VM?  Probably not.  Multiple images for each
system and so on.  But debugging is much easier when you can just "stop the
process."  Sometimes a "frozen" Squeak image is just not that easy to deal
with for novices such as ourselves.

3.  We support "research querying" for Six Sigma (process improvement)
activities, and are starting to use a Smalltalk image as an "interactive
querying tool" to deal with all the special cases mentioned above, generate
random lists, massage and format the data, etc...  In this "interactive
mode," it is nice to keep working in the image, planning your next "move,"
while your query chugs along.  In VisualWorks (still trying to swing the
license fee, and still wondering if it is
necessary), I can even start "trying out" some of the results while a
lengthy query is running.
 Some of our systems are so slow, and because we are "researching," we make
mistakes.  It's nice to be able to end a process when you forgot to save the
image before executing the query!

4.  One of our "big" applications is for data collection tools for
regulatory reporting.  Much of our data isn't even IN a computer yet.
 Employees abstract the data by hand from paper medical records.  We are
trying to get SOME of what they need if it is available by generating
nightly "work" lists as new patients are "coded" and end up in various
reporting populations, extracting what data we can (either from pre-built
queries or dealing with the special cases mentioned
above), and presenting Aida-based "forms" for them to fill in the
remaining data. At that point, I defer to the arguments presented
already with regards to the user "waiting"
for their form to load...

Then, of course, we will feed the combined data back into the warehouse...

So, in the end, a lot of our "customers" aren't users at all, but just us,
the programmers, interactively using Smalltalk as a tool or developing other
nightly
processing tools.  I just liked the threaded connections I tried out
in VisualWorks and
Smalltalk/X because it made it easier form ME to work with the image,
so I just wondered how hard it would be to implement.

Thanks for asking the question, though; I have been asking the same one of
myself for several months now as we have been struggling to "choose" a
Smalltalk product.  One thing is for sure, though, Smalltalk is DEFINITELY
the right tool for the right job amidst the natural objects and subclasses
of healthcare!

Hope some of that made sense,

Rob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20080511/e31a3fa6/attachment.htm