[Box-Admins] My daily rituals (was RE: Welcome Steve and Casey)

[Ken Causey]

Please don't take this as directed at you Steve, because it definitely
is not, but I'm going to take the opportunity of your mention of 'on the
weekend' to explain something that I think is obvious, and probably
really is, but maybe is not at the top of everyone's awareness.

For good or bad the box-admins team work is 24/7.  In practice it
doesn't often add up to that much time but first of all you never know
when disaster in its various forms will strike.  Furthermore I think it
is essential that every single day the server status is checked.

Here is a quick summary of what I do every single day, yes 7 days a week
every week (OK, every once in a while I miss a day):

login to the server via ssh as root

uptime
  Look at the days of uptime, is the number surprisingly small like 1
day or less?
  Look at the number of logged in users.  If it's not 1 (yourself) then
see who it is and have some confidence they aren't doing anything
nefarious.
  Look at the load average, is it in the normal range?

df
  Any significant changes in disk usage?
  Is it time to do some cleanup and make space?

free
  Any significant changes in RAM usage?

ps auwx
  Quickly glance through the process list looking for anything unusual. 
This task can lead to other inquiries when there are a lot of qmail
processes running or anything else seems unusual.  Only experience will
help judging what is unusual.

top (then hit the $c key to see the full command line which is essential
in identifying squeak processes)
  This is another way to look at the process list in particular
monitoring the memory and cpu levels and as with the rest of this
checking if things are more or less normal.

Further every day I visit source.squeak.org
  If the front page comes up I'm satisfied all is probably normal.

And bugs.squeak.org
  I check the manage page where I can quickly see any recent signups. 
We have someone who seems to test their anti-captcha technology and
signup with semi-random but consistent account names and user names but
they never actually do anything with the accounts.  I delete them
whenever I see them.  After that I check the bug list page with no
filtering and no project set so I can always see all changes in case
anyone is fooling around.
 
And that's usually it.  It probably doesn't take even 10 minutes most
days.  But doing it every day is essential in catching problems before
they become more serious and simply to have the knowledge to judge that
a change has occurred.

This is nothing new to anyone I suspect, basic Linux admin 101, but I
thought I should say it aloud to make sure.

Ken

> -------- Original Message --------
> Subject: Re: Welcome Steve and Casey
> From: Steven Elkins <sgelkins at gmail.com>
> Date: Sat, February 05, 2011 2:31 pm
> To: Ken Causey <ken at kencausey.com>
> Cc: box-admins at lists.squeakfoundation.org, casey.obrien.r at gmail.com
> 
> 
> I've been moderating squeak-dev for some time.  If there's something that I
> can do (1) on the weekend and (2) not too time-consuming, I might be able to
> help.
> 
> Cheers,
> Steve
> 
> On Sat, Feb 5, 2011 at 2:30 PM, Ken Causey <ken at kencausey.com> wrote:
> 
> > Welcome to the box-admins mailing list.  Feel free to introduce
> > yourselves (again perhaps) if you like; if you have any particular
> > interest or thoughts related to managing the community services please
> > contribute.
> >
> > Thanks,
> >
> > Ken
> >
> >
> 
> 
> -- 
> How wonderful it is that nobody need wait a single moment
> before starting to improve the world.       -- Anne Frank
> Paradise is exactly where you are
> right now...only much, much better.    -- Laurie Anderson