Box-Admins November 2016

box-admins@lists.squeakfoundation.org

6 participants
15 discussions

Access to the new server(s)
by Tobias Pape 05 Apr '19

05 Apr '19

Dear all [ACTIONS AT END] with retroactive blessing of Levente, I have now prepared eight VMs on Rackspace. Here's the overview: ======================================================================================================================= Name Name (ext) intended use Unix Users Public Ports Private Ports Public IPv4 Private IPv4 ----------------------------------------------------------------------------------------------------------------------- ian ssh.squeak.org ssh-gateway ssh 1022 22 104.130.6.82 10.208.225.29 alan *.squeak.org webserver webteam 80, 443 22 104.239.229.92 10.176.200.8 adele lists.... mailinglists (tbd) 25, 587, 465 22, 8080 162.242.237.43 10.208.160.56 andreas -------- source.squeak chrismuller* -------- 22, 8080 irrelevant 10.208.161.222 dan -------- squeaksource davidlewis* -------- 22, 8080 irrelevant 10.176.197.150 ted -------- squeak wiki+map (tbd) -------- 22, 8080, 8081 irrelevant 10.176.130.111 david -------- jenkins (tbd) -------- 22, 8080 irrelevant 10.208.194.45 scott -------- misc (tbd) -------- 22, 8080, 8081 irrelevant 10.176.199.169 ======================================================================================================================= Currently, Levente and me have sudo on all these machines. Users with * also do. Note that _no_ server exposes SSH on port 22 on a public IP. This is intentional to narrow attack vectors for script kiddies. How to login? Ian is the ssh gateway so you have to connect to ian _first_ and use (1) local forwarding or (2) proxy jumping. I have installed the Public keys from most of you for the 'ssh' user on ian. Please verify by ssh -p1022 -lssh 104.130.6.82 you should see restrict shell, no commands # (you get out with crtl-d, ctrl-c, or killing ssh) How to reach the other servers? Example for 'andreas' variant (1): Do a local forward by ssh -AN -L22221:10.176.200.8:22 -p1022 -lssh 104.130.6.82 and then ssh -lYOURNAME -p22221 localhost (-N maybe optional, but then you see 'restrict shell, no commands #') Or in your .ssh/config you can put Host ian.squeak.org User ssh Hostname 104.130.6.82 Port 1022 LocalForward 222221 10.176.200.8:22 Host andreas.squeak.org User YOURNAME Hostname localhost Port 222221 And then say 'ssh -AN ian.squeak.org' and then 'ssh andreas.squeak.org' variant (2): (a) You have OpenSSH >= 7.3 Do a Jump with ssh -J ssh@104.130.6.82:1022 YOURNAME(a)10.176.200.8 Or in your .ssh/config you can put Host ian.squeak.org User ssh Hostname 104.130.6.82 Port 1022 Host andreas.squeak.org User YOURNAME Hostname 10.176.200.8 ProxyJump ian.squeak.org And then say 'ssh andreas.squeak.org' (b) You have OpenSSH >= 5.4 Do a Jump via ssh -o ProxyCommand="ssh -lssh -p1022 -W %h:%p 104.130.6.82" YOURNAME(a)10.176.200.8 Or in your .ssh/config you can put Host ian.squeak.org User ssh Hostname 104.130.6.82 Port 1022 Host andreas.squeak.org User YOURNAME Hostname 10.176.200.8 ProxyCommand ssh -W %h:%p ian.squeak.org And then say 'ssh andreas.squeak.org' (c) You have OpenSSH < 5.4 Use variant (1) We will shortly start RSYNC-ing over data from box3 and box4 as well as replicatiing DNS entries before switching over. [ACTION REQUIRED] - Who needs access to which servers? - Do we need Jenkins anymore? As always, questions appreciated. Best regards -Tobias

8 36

Re: [Box-Admins] State of the boxen
by Tobias Pape 22 Nov '16

22 Nov '16

Hi Brett On 22.11.2016, at 03:24, Brett Smith <brett(a)sfconservancy.org> wrote: > On 11/21/2016 06:21 PM, Tobias Pape wrote: > > > >> I'm happy to tell you that no active service is running on the boxen (save some infrastructure, but that's negligible). >> Reboot/Shutdown at will. >> > > > Tobias, > > Thank you very much for working with us through this whole process. I’m glad to see it squared away well before Gandi’s own deadline. > > I have destroyed the old VMs, so I certify your declaration that they’re deceased. :) I’m going to be traveling for the US Thanksgiving for the rest of the week, so I’ll be a little tricky to get a hold of. If any issues come up, please send e-mail and CC: squeak(a)sfconservancy.org. That will loop in other Conservancy staff who might be able to help. Thanks for working us through this and offering us this great opportunity :) Have a great weak and all the best for your holiday. Best regards -Tobias

1 0

Re: [Box-Admins] State of the boxen
by Tobias Pape 22 Nov '16

22 Nov '16

Hi Brett On 17.11.2016, at 18:20, Brett Smith <brett(a)sfconservancy.org> wrote: > On 11/07/2016 12:04 PM, Brett Smith wrote: > > > >> On 11/04/2016 07:18 PM, Tobias Pape wrote: >> >>> @Brett: I do not intend to upgrade any of those hosts anymore. We would need to power-cycle which is counterproductive now and unnecessary shortly, anyway. Ok with the SFC? >> Tobias, >> >> Yes, this is fine. They’re your boxes, so it’s your call. >> > > > Tobias, > > This is still fine with Conservancy, but unfortunately it’s not fine with Gandi. They’re going to do some maintenance starting at 11:00 UTC on the 22nd. As part of that, they will power cycle any boxes that haven’t been cycled since the patch became available. > > If you want to schedule a time to do that which would be less disruptive, let me know and I’ll be happy to coordinate with you. I’m generally available this weekend, so I could even work with you then if it would be helpful. I'm happy to tell you that no active service is running on the boxen (save some infrastructure, but that's negligible). Reboot/Shutdown at will. @Board,box-admins: I hereby declare box3, box4, and franken-box2 deceased. They are no more. They have expired and gone to meet their creator. They have ceased to exists. They're demised, they are no more, they kicked the bucket. These are ex-boxen. Best regards -Tobias

2 1

BOX3: services down
by Tobias Pape 11 Nov '16

11 Nov '16

Hi all, all services on box3 have now been stopped (besides ssh, obviously) Best regards -Tobias

1 0

ephemeral files
by Tobias Pape 11 Nov '16

11 Nov '16

Dear all on david:/srv I parked some files from box2/3/4 I did not yet dare to delete: /srv/ jenkins # box3:/var/lib/jenkins (most of it) old-apache/ # things here were referenced by the old webserver, see previous mails apache2config # box2:/etc/apache2/ bfav # box2:/home/bfav svn # box3:/home/svn updates # box3:/home/updates website # box3:/home/website Please decide what to do with them :) Best regards -Tobias

1 0

BOX4: services down
by Tobias Pape 11 Nov '16

11 Nov '16

Hi all, all services on box4 have now been stopped (besides ssh, obviously) Best regards -Tobias

1 0

State of the boxen
by Tobias Pape 11 Nov '16

11 Nov '16

Hi all [ACTION REQUIRED] so far the main services have been migrated: OK - move DNS OK - move main web server 27GB OK - move mailing lists (quite some of them) 10GB OK - move main application (source.squeak.org) 17GB OK - move squeaksource.com 18GB OK - move mantis 0.08GB OK - move squeak wiki + squeak map 5GB - move jenkins (maybe) 11GB OK - move rest (eg, codespeed)) This means: - box3 still only has jenkins - box4 now only piggy-backs box2 and houses a lonely dnscache. - box2 has an old apache with these services: board.squeak.org (is redirect: squeakboard.wordpress.com) boardblog.squeak.org (is redirect: squeakboard.wordpress.com) bfav.squeakfoundation.org download.squeak.org -> http://www.squeak.org/Download/ etoys.squeak.org exupery.squeak.org -> http://wiki.squeak.org/Exupery (should be: http://wiki.squeak.org/squeak/Exupery ?) jobs.squeak.org -> http://smalltalkjobs.dabbledb.com/ (DEAD) paste.squeak.org -> http://paste.lisp.org/new/squeak (PRACTICALLY DYSFUNCTIONAL) people.squeak.org -> http://people.squeakfoundation.org (DEAD) (www.)squeakfoundation.org -> http://www.squeak.org/Foundation -> http://www.squeak.org/board/ static.squeak.org DEAD svn.squeak.org:443 ?? UNKNOWN updates.squeakfoundation.org DEAD wwwtest.squeak.org ?? UNKNOWN [ACTION]: 1. What should we do with jenkins? It is a _major_ effort revive it as it managed to degenerate into limbo (no jobs will run successfully at all) I'd like to see it put to rest. 2. What about the apache services? @Brett: I do not intend to upgrade any of those hosts anymore. We would need to power-cycle which is counterproductive now and unnecessary shortly, anyway. Ok with the SFC? Best regards -Tobias

4 5

please take care for your personal belongings
by Tobias Pape 11 Nov '16

11 Nov '16

Hi all If you have any personal files in your home folder on box3 box4 box2-piggyback-on-box4 please look promptly whether you need them or not, because else they'll be gone in a week :) Best regards -Tobias

1 0

source.squeak.org scheduled maintenance
by Chris Muller 08 Nov '16

08 Nov '16

I would like to bring down source.squeak.org for a few minutes to update the image and take a back up of the indexed repository. I expect it to take no more than a few minutes.

1 0

root cause of squeaksource.com outage
by Chris Muller 07 Nov '16

07 Nov '16

I think I just stumbled onto the root cause of the image growth. I had downloaded the new production squeaksource.6.image (copy of the Oct-5 image) to my laptop to get a "data.obj" file exported. This export process runs in the background and takes a long time, so when I came back to check on it, I found two debuggers with the message "Port not available." What's happening is when the image is saved, it gets most of the way through SmalltalkImage>>snapshot:andQuit:embedded: so that the file is saved on disk, however, at the end it calls #processStartupList: whereupon SSRepository class>>#startUp: tells SSKom to start up. SSKom produces a debugger if you tell it to start when its already started. So the debuggers from the prior image save are slowly piling up within each subsequent image save, a slowly growing until it reached the limit. I guess the bug should still be there; I guess I'm puzzled why this didn't happen on box2.. I think we should work on moving squeaksource to the new image ASAP.

2 4

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Box-Admins November 2016