[Box-Admins] [Board] Backing up our servers
Levente Uzonyi
leves at caesar.elte.hu
Tue Apr 12 18:40:29 UTC 2022
Hi Marcel,
Data is backed up daily to a hard drive sitting on my desk.
Levente
On Tue, 12 Apr 2022, Marcel Taeumel wrote:
> Hi Levente --
>
> What's the current status on this? Where did you back up the data to? What is the timestamp of that backup? :-)
>
> Best,
> Marcel
>
> Am 28.03.2022 11:16:31 schrieb Levente Uzonyi <leves at caesar.elte.hu>:
>
> Hi Tony, hi all,
>
> @Tony:
> If you send me an ssh key, preferrably ed25519, I'll create a user for you
> on all the servers, so that you can log in with your key.
>
> @all:
> More than a week ago, I started to create backups of all servers using
> rdiff-backup[1]. It took a longer for the initial backup to complete
> than I expected due to the nature of the network setup we have: tcp is
> tunneled over tcp, which is known to be slow[2] but it surprised me how
> slow it turned out to be in practice (~1.66MB/s).
>
> The process copies the following directories: /etc, /home, /opt /root,
> /srv, /var/backups, /var/log, /var/mail, /var/spool, /var/www.
> If there are files that need to be preserved in any other directory on
> any server you are aware of, let me know.
>
>
> Levente
>
> [1] https://rdiff-backup.net/
> [2] http://sites.inka.de/~W1011/devel/tcp-tcp.html
>
> On Fri, 18 Mar 2022, Marcel Taeumel wrote:
>
> > +1 for giving Tony access to the servers
> > Best,
> > Marcel
> >
> > Am 17.03.2022 09:22:17 schrieb Tony Garnock-Jones :
> >
> > Hi David, Chris, all,
> >
> > Great, that README is a good initial survey of the machine. The backups
> > I run for the various cloud servers under my control are similar.
> >
> > Generally, I use rsnapshot [1] from cron to make incremental backups
> > across ssh (it uses rsync internally).
> >
> > This works great for unprivileged files, those readable by an account
> > you'd trust to log in to grab files to be backed up.
> >
> > For the privileged ones, as you note in your README, a cronjob running
> > as root on the source machines can archive the bits and pieces for later
> > pickup by the backup account.
> >
> > Levente, Tobias, I'd be happy to log into the boxes to do the initial
> > survey and set up some backups. I suppose the box-admins team should
> > decide if I'm to be trusted to do this! It does require root access at
> > least initially. If you all would prefer not to create a new privileged
> > unix account, etc, that's cool -- in that case I'd also be happy to
> > pair-sysadmin via tmux or screen or similar if that would be helpful.
> > And actually even in the case you would be happy letting me work with
> > root, I'd also welcome having a second set of screen/tmux eyes on what
> > I'm doing.
> >
> > Cheers,
> > Tony
> >
> > [1] https://rsnapshot.org/
> >
> >
> > On 3/17/22 01:47, David T. Lewis wrote:
> > > Hi Chris,
> > >
> > > Funny you should mention it, I was doing the same thing last
> > > weekend to make sure we could move the squeaksource.com service
> > > if needed. I had intended to share this in the board meeting
> > > today but was not able to attend. In any case, I'm attaching
> > > my "disaster recovery plan" notes (crudely formatted but better
> > > than nothing).
> > >
> > > For me it was a PITA dealing with session timeouts while
> > > download 30GB to my tired little old laptop PC, but at least
> > > I was able to do it after a few hours, and would be able to
> > > restore it to some new server as long as someone else could
> > > handle the sys admin things for ssh and port forwarding.
> > >
> > > I think between the two of us we could handle moving the two
> > > squeaksource services, although I don't know how to handle
> > > the rest of the services (mailing lists, etc). For that we
> > > would need some expertise from the box-admins, especially
> > > Levente and Tobias.
> > >
> > > Dave
> > >
> > >
> > > On Wed, Mar 16, 2022 at 03:58:09PM -0500, Chris Muller wrote:
> > >> Hi Tony, Dave, all,
> > >>
> > >> FYI, I had this *exact thought* yesterday when I was working on our
> > >> server. We "expect" cloud services to be redundant and backed up, but
> > >> for something this important it's necessary to have a backup system
> > >> under our control. Our code is an asset that can't be bought with money,
> > >> whereas the bandwidth is. I'm downloading a backup copy of the ./ss
> > >> directory at this moment which I need to test a server upgrade to 5.3
> > >> and the latest code. This will at least temporarily reduce our risk
> > >> of "total loss" until we can get a permanent backup solution.
> > >>
> > >> - Chris
> > >>
> > >> On Wed, Mar 16, 2022 at 12:52 PM David T. Lewis wrote:
> > >>>
> > >>> Let's make this an agenda item for our meeting (in a few minutes). I
> > >>> can share some work I did over the weekend to prepare a "poor person's"
> > >>> recovery plan for squeaksource.com. It's painful but it works.
> > >>>
> > >>> As for Rackspace backups, Levente and Tobias are most likely to know
> > >>> how and if this can be done. I don't know the answer, but I have a
> > >>> suspicion that the utility for "help me get my stuff off of Rackspace"
> > >>> is not something that Rackspace, Inc. would be in a big hurry to support.
> > >>>
> > >>> Dave
> > >>>
> > >>>
> > >>> On Wed, Mar 16, 2022 at 06:36:32PM +0100, Tony Garnock-Jones wrote:
> > >>>> Hello all,
> > >>>>
> > >>>> I'm writing on behalf of the Squeak Oversight Board - could someone get
> > >>>> in touch with me, please, to help me arrange backups of the important
> > >>>> bits of our servers with some urgency? We have become concerned with
> > >>>> disaster recovery and would like to quickly get something in place to at
> > >>>> least have *something* if the servers were to vanish in a puff of logic...
> > >>>>
> > >>>> Regards,
> > >>>> Tony
> >
> >
> >
>
>
>
More information about the Box-Admins
mailing list