[Seaside] Enterprise Software V Ecommerce [in smalltalk/seaside/cincom]

[dirk newbold]

Hi all,

I’m a relative newbie (4 years) to programming and seaside, who
partnered up with some “old hands” (smalltalk/seaside is my first and
only language).

Initially we developed an Ecommerce (Istazaar) fashion platform for
designers to maintain their collections.  We stagnated last year due
to the time restraints on the “old hands” and I then began developing
a web based Enterprise Software (Vivport) platform for
document-management/knowledge-management/emails/tasks.

We now have two ventures at relatively equal stages of development.
I believe the Enterprise Software (Vivport) platform is the more
innovative however, now that Steve (one of the “old hands”) has some
time again he has been able to review its development (which
unfortunately I built unsupervised) and has major concerns in relation
to deploying this venture.

Instead of building the database; then building middleware, then UI;
I’ve basically just built a UI, Steve’s concerns with the outcomes of
this development process are outlined in his email to me below.

We were wondering if the smalltalk/seaside community has any views on
Steve’s concerns below that may influence our decision on which
venture to proceed with.
The platform has been developed to freely change between the user’s
personal account and their company(ies) account.  The personal
accounts are free so we would be attempting to server many users.

Any advice in relation to users/image, persistence, database, general
deployment etc for this type of venture (essentially a beefed up
gmail/cloud environment) would be greatly appreciated.

Thanks for your time.

Cheers,

Dirk


---------- Forwarded message ----------
From: Steven Berryman <Steven at berrymanelectrical.co.uk>
Date: Sat, Feb 18, 2012 at 6:07 AM
Subject: RE: Vivport V Istazaar
To: dirk newbold <dirkdirk at gmail.com>

Dirk,

As mentioned I’m concerned with the approach you have taken with
Vivport and in one way this technology can lead you into a false sense
of how easy things are (trust me you couldn’t have done it in .NET or
java without infrastructure). The difference with the ecommerce site
is that there is little shared data and I knew that the only real
issue was storing users but that would be a simple mapping issue to a
table. Even the registered objects were designed to make storing in
the database easier so objects didn’t directly hold onto a registered
object. What you have built is a very complicated network of objects
for the benefit of the UI which for a prototype is great but would
never work in the real world. The type of project you are building
with Vivport is infrastructure and in one way the last thing you
should have done is the UI. One system I worked on had 3-4 months of
designing the infrastructure before a line of code was written.



The problem you have is that you have assumed that all objects would
be in memory and are accessible via threads and this just isn’t
possible. If I look at my local Microsoft exchange server it’s
currently 50GB for 5 users and the machine only has 4GB of ram so not
only do you need to deal with large amounts of data but you have not
addressed any multi-threading issues. For example Smalltalk provides
structures such as SharedQueue to deal with any shared data in the
memory and I’m guessing you didn’t use these? When everything is in
memory the world is very easy but for a system like yours you would
need to start with the database and design the schema. Most searches
would be done directly in sql (assuming you are using a relational
database) so that way you reduce the amount of data going to the
client. So when you open up Microsoft Outlook it will run a query on
the database to get your inbox and the email headers so you don’t pull
large amounts of data across the network. As you start to click on
emails it will pull the data over as required.

So here is an example to show multi-threading issues (I have not used
your code but I’m sure you get the point)

Lets’ assume you have a folder with 10 items

Dirk - actions                                  Danny - actions
folder items do: [ :item
“goes through this loop twice”
                                                     delete the last
item in the folder
“code falls over because the last element
in the collection does not exist.” ]

I’m sure you could try something like the above but you would need to
put in a break point in the Dirk loop so you could then go in and to
the removal from a different session. Now imagine the same thing with
5,000 users logged into your server all running threads across your
code. Your code would have to be thread safe and this is not a simple
task especially trying to do it on code not designed that way from the
beginning. You could end up locking up all the other sessions if you
don’t carefully design and keep your critical sections to a minimum.

If you are using collections that are hashed they will fail in very
strange ways if you have other threads removing or adding items as you
are iterating through the collection. Basically in the ecommerce site
we could sort of ignore the shared data issue as we shared very little
data between sessions.

Also you have added features that would be very expensive when using
databases. For example the running man relies on the status of some
piece of data and of course when everything is in the same memory
space this is easy. In a database you would have to cross a
transaction boundary and even the fastest database servers we have
(costing 200k per server) you would be lucky for a single transaction
to take between 1-10 milliseconds (costly when you scale up users and
make too many database calls). This is why Microsoft exchange works
the way it does as you can imagine how slow it would be if 5000 people
logged into the server at the same time. So for most document
management systems they would have a large database at the back end
and all the searches would be done as stored procedures that could be
based on some proprietary algorithm and none of this would be done in
the language itself such as Smalltalk. Until you have a database
schema designed you can’t even think about the UI.

If you want to prove to yourself then try and create 100,000 users all
with about 10MB of data and test to see if it works. See if you can
log in a few sessions and send emails around.

With the ecommerce site I could imagine how to add a database at the
backend because it would be very simple. I would have no idea how you
get this into a database even if you used an object database you would
still need a defined object model. It’s also the reason why I liked
the ecommerce site as it would be simple from the database end as it
would only contain users, basic static data and designer’s data. Also
I can see how we could have multiple servers to handler more traffic
with the ecommerce site but would have no idea how we could do the
same with your design.

Hope this makes sense. Let’s catch up next week.

Steve