Box-Admins December 2013

box-admins@lists.squeakfoundation.org

8 participants
9 discussions

David!: SqueakSource.com problem
by Ken Causey 23 Jan '14

23 Jan '14

I don't always think of it but occasionally I visit www.squeaksource.com just to make sure that the front page at least looks right and that it is responding. Today I'm not getting the complete HTML source much less images or styling. Under such circumstances can one of us just kill the existing process and let daemontools restart it at least as a first attempt to fix the problem? I'll leave this to David this time. But guidance and comments on how others can help maintain this service, at least at a first approximation, would be appreciated. Apologies if I have overlooked something already stated. The team could use a better documentation strategy. Suggestions welcome, at least for discussion purposes. As usual the trick is finding something that a quorum of the ever changing group will actually use. Ken

2 8

How to connect to squeaksource.com image with a VNC client
by David T. Lewis 31 Dec '13

31 Dec '13

Updated today to provide secure VNC access to the squeaksource.com image. Details are in box3 ~ssdotcom/README. The squeaksource.com image allows VNC connections on display number 1 with a password of "Squeak01". However, for security reasons, this access can only be enabled by one of the box-admins with sudo privileges on box3.squeak.org. How to connect to the image with VNC: - Log on to box3.squeak.org and determine the PID of the squeakvm that is running the squeaksource.com image. Running /usr/bin/top is a quick way to do this. - Send a SIGUSR2 signal to the VM process. For example, if the pid is 12345, then the command would be "sudo kill -SIGUSR2 12345". - Within the next 60 seconds, use a VNC client on your local computer to connect to squeaksource.com:1 and log in with password "Squeak01". Dave

4 4

Re: [Seaside] SqueakSource/Seaside question - has anyone seen this problem before?
by Levente Uzonyi 23 Dec '13

23 Dec '13

On Sun, 22 Dec 2013, Bob Arning wrote: > David, > > Here is an alternative that tries to print the item rather than delete it. You can revert the previous fix, if you like. The problem with this solution is that it will print a string using UTF-8 encoding, but Seaside will return a XHTML page with ISO-8859-1 endocing. There are two ways to solve it: 1: use NCRs[1] to encode the names. This might be a problem, because one has to make sure that seaside won't encode the & characters. 2: change the default character encoding to UTF-8, which might lead to other problems. Levente [1]https://en.wikipedia.org/wiki/Numeric_character_reference > > Cheers, > Bob > > > 'From Squeak3.11alpha of 6 January 2010 [latest update: #8824] on 22 December 2013 at 10:32:55 am'! > > !WideString methodsFor: 'converting' stamp: 'raa 12/22/2013 10:31'! > printHtmlOn: aStream > > self squeakToUtf8 printHtmlOn: aStream! ! > > > On 12/22/13 12:03 AM, David T. Lewis wrote: > > Bob, > > This is absolutely great, thank you so much for the help. I just loaded > your patch into the running image, and the http://squeaksource.com web > interface is back to normal again. > > Dave > > > On Sat, Dec 21, 2013 at 11:25:59PM -0500, Bob Arning wrote: > > David, > > The short answer is that two users (St?ane Munioz St?ane Munioz) have > wide characters in their name and this does the damage. The method below > is a quick hack to filter these 2 out of the list and all seems good. > > Cheers, > Bob > > 'From Squeak3.11alpha of 6 January 2010 [latest update: #8824] on 21 > December 2013 at 11:21:03 pm'! > > !SSHome methodsFor: 'statistics'! > membersNew > | t1 | > t1 := self statistics membersNew. > ^ t1 > reject: [:t2 | t2 asString > anySatisfy: [:t3 | t3 asciiValue > 255]]! ! > > > > > On 12/21/13 9:25 PM, David T. Lewis wrote: > > On Sat, Dec 21, 2013 at 07:41:08PM -0500, Bob Arning wrote: > > If you click one of the links on the first page, the next page looks > good. If you then go back, the css from page 2 makes page1 look a little > better. So, the breakage is limited. Any way to share the image? > > Thanks Bob, > > A copy of the image is at http://box2.squeak.org/ss_image_for_bob.zip > > The zip is encripted, I will send the password to you in a separate email. > > Dave > > > > >

3 2

Re: [Seaside] SqueakSource/Seaside question - has anyone seen this problem before?
by David T. Lewis 22 Dec '13

22 Dec '13

Bob, This is absolutely great, thank you so much for the help. I just loaded your patch into the running image, and the http://squeaksource.com web interface is back to normal again. Dave On Sat, Dec 21, 2013 at 11:25:59PM -0500, Bob Arning wrote: > David, > > The short answer is that two users (St?ane Munioz St?ane Munioz) have > wide characters in their name and this does the damage. The method below > is a quick hack to filter these 2 out of the list and all seems good. > > Cheers, > Bob > > 'From Squeak3.11alpha of 6 January 2010 [latest update: #8824] on 21 > December 2013 at 11:21:03 pm'! > > !SSHome methodsFor: 'statistics'! > membersNew > | t1 | > t1 := self statistics membersNew. > ^ t1 > reject: [:t2 | t2 asString > anySatisfy: [:t3 | t3 asciiValue > 255]]! ! > > > > > On 12/21/13 9:25 PM, David T. Lewis wrote: > >On Sat, Dec 21, 2013 at 07:41:08PM -0500, Bob Arning wrote: > >>If you click one of the links on the first page, the next page looks > >>good. If you then go back, the css from page 2 makes page1 look a little > >>better. So, the breakage is limited. Any way to share the image? > >Thanks Bob, > > > >A copy of the image is at http://box2.squeak.org/ss_image_for_bob.zip > > > >The zip is encripted, I will send the password to you in a separate email. > > > >Dave > > > > >

1 0

Source.Squeak.org how-to-fix guidelines
by Ken Causey 21 Dec '13

21 Dec '13

I wish I could provide a TLDR, but really I ask you to try to persist in reading through this. I think, more than I originally expected when I started to write this, that it contains a fair amount of my philosophy regarding the maintenance of the Squeak servers and services. It certainly turned out much longer than I expected and could use an editor. Back to the original email: I mentioned in my recent email about an issue with squeaksource.com that it helps if those who know give guidance to everyone else about easy things that can be done to fix issues and how to make decisions about what to do. In that vein I will do my best to provide the same for source.squeak.org. Presumably much of this is also true for squeaksource.com but I'm not going to assume it. So the scenario is this: source.squeak.org is unresponsive or the page does not render in full; the first is much more common. First, because it has happened before, you need to ensure the problem is not at a higher/later level, that is that it is not a problem with Apache on the box2 server. The easiest way to do this is to check any other service, perhaps most other services, that run on box2 and see if they are working OK or not. Here is a list (probably not exhaustive): bugs.squeak.org (Apache/FastCGI PHP) www.squeak.org (Apache/AIDA Squeak) lists.squeakfoundation.org (Apache/C & Python CGI) ezmlm.squeak.org (Apache, defunct but still exists) The last two are notable in that they rely on very little outside of Apache to work. If Apache is working but those services are not working then the server is in bad shape indeed. So in brief if you go to ezmlm.squeak.org and get a page that says: Some old Mailing Lists Click Here for List Archives and Information and you can click the second line and get a list of defunct/dead mailing lists then Apache is probably not the source of the problem. If Apache is not working then the thing to do is: sudo /etc/init.d/apache2 restart If this gives a problem, or just doesn't work (be patient) then sudo /etc/init.d/apache2 stop Note the message printed and wait a moment, if it appears to have stopped fine then sudo /etc/init.d/apache2 start Again wait a moment, and then check ezmlm.squeak.org again. If this is is still not working then I guess it is time to reboot the server. However, I would ask that you really only do this as a last resort and that you not do so as a quick decision. First email box-admins, feel free to Cc me directly if you want. Wait some time, 30 minutes, an hour, two hours. You can make your own judgment call on that. I have and it's not always consistent. The point is that I or someone else may want to look at the situation first if only for information gathering purposes. Ultimately restarting the server once or twice without waiting for others to chime in is not going to get you in any trouble. A history of it is likely to start to annoy me, I assume it would annoy others as well. The reasoning is that while the system is in its broken state there is the possibility of gathering information about the problem that is not recoverable, at least not easily, once the system has been rebooted. OK but if rebooting is the answer then it is simply sudo reboot Be aware that the server does tend to take multiple minutes (It has been a while, I remember it seems like a long time, I don't remember how long it really tends to be) to be responsive again. My habit after I have been booted off the server is to ping squeak And you say 'Huh'? Yeah, this is beginning to digress, but I will persist nonetheless. For my own convenience some time ago I modified my /etc/hosts file. (Clearly we are getting off into the bushes and this is only directly applicable if you run Linux and friends locally, if you run MacOSX this may still apply to you pretty closely, but I don't know. Those of you on Windows: the fundamental facts are all true but the details have been changed. Google it.) $ cat /etc/hosts <snip> # utility 85.10.195.197 squeak 173.246.101.237 box3 173.246.104.42 box4 <snip> From the naming pattern you can guess that I started this before box3 and box4 existed. If you choose to do the same you can use any names you like, just don't mask any names used in your local network if there are any. Back to the point at hand: ken@neue:~$ ping squeak PING squeak (85.10.195.197) 56(84) bytes of data. 64 bytes from squeak (85.10.195.197): icmp_seq=1 ttl=47 time=128 ms 64 bytes from squeak (85.10.195.197): icmp_seq=2 ttl=47 time=128 ms 64 bytes from squeak (85.10.195.197): icmp_seq=3 ttl=47 time=127 ms This is what you want to see. While the server is restarting though, and the TCP/IP stack is down, ping is just going to be silent; but it will keep trying. So leave this and go back to whatever else you were doing. Check it occasionally and unless catastrophe has occurred you should in time see the above (of course your ping times will vary). At this point then the server has or is restarting. It has reached the point that the echo service is working. That however does not mean everything is yet working, that includes sshd and apache. Nonetheless you can go ahead and try to ssh to the server or check any of the web services. But if they don't immediately work, don't despair, minutes of time is required for everything to come back up at the best of times. Ultimately everything should start back up as normal. If it doesn't then it is time to call for help. OK, so end of the 'reboot the server' digression. The scenario is now this: source.squeak.org is not working but we have checked and we believe that Apache is fine. This tells us that the problem is almost certainly isolated to the source.squeak.org Squeak process. Probably the thing to do is kill it and let daemontools restart it. But don't do that without checking out the image that will be used first. $ ls -lh ~squeaksource/Squeak3.11-8824-SS.image -rw-r--r-- 1 squeaksource squeaksource 35M Dec 21 16:25 /home/squeaksource/Squeak3.11-8824-SS.image The main thing to look at here is the file size (5th column). The size of the file should be in this vicinity, it does grow but slowly. That is it grows slowly under normal conditions, sometimes, and this is a danger of the easy save-the-world persistence strategy we use, the image is saved when something has gone wrong and the heap has grown tremendously. I believe I have seen this file saved at between 150-200MB before. When restarting the image does not work then it is nearly always the case that this file is much larger than expected. I can't remember any case in which it has restarted properly when the file is larger than normal. If you forget to look and just kill the process (I will get to that shortly) it is not the end of the world. It may just mean that killing it and having it restarted does not fix the problem and in my opinion it is better to look at the image first and have some confidence that it is not corrupted. If it is corrupted you can find recent backups of the bulk of the filesystem under /var/cache/rsnapshot/. This directory will look like $ ls /var/cache/rsnapshot/ daily.0 daily.1 daily.2 daily.3 daily.4 daily.5 daily.6 daily.0 is the most recent backup (within the last 24 hours), daily.1 the next most recent, etc. What I might do then if I'm looking for a good backup image is this ~$ ls -lh /var/cache/rsnapshot/*/localhost/home/squeaksource/Squeak3.11-8824-SS.image -rw-r--r-- 1 squeaksource squeaksource 35M Dec 21 16:25 /var/cache/rsnapshot/daily.0/localhost/home/squeaksource/Squeak3.11-8824-SS.image -rw-r--r-- 1 squeaksource squeaksource 37M Dec 20 17:25 /var/cache/rsnapshot/daily.1/localhost/home/squeaksource/Squeak3.11-8824-SS.image -rw-r--r-- 1 squeaksource squeaksource 35M Dec 19 15:24 /var/cache/rsnapshot/daily.2/localhost/home/squeaksource/Squeak3.11-8824-SS.image -rw-r--r-- 1 squeaksource squeaksource 35M Dec 18 16:24 /var/cache/rsnapshot/daily.3/localhost/home/squeaksource/Squeak3.11-8824-SS.image -rw-r--r-- 1 squeaksource squeaksource 35M Dec 17 12:23 /var/cache/rsnapshot/daily.4/localhost/home/squeaksource/Squeak3.11-8824-SS.image -rw-r--r-- 1 squeaksource squeaksource 35M Dec 16 12:23 /var/cache/rsnapshot/daily.5/localhost/home/squeaksource/Squeak3.11-8824-SS.image -rw-r--r-- 1 squeaksource squeaksource 35M Dec 15 11:22 /var/cache/rsnapshot/daily.6/localhost/home/squeaksource/Squeak3.11-8824-SS.image It's not well formatted in this email but I hope you get the idea. If possible you want to use the most recent backup which will be found within the daily.0 directory. But it is very possible that the backup backed it up after it was corrupted, in which case you consider daily.1, and so on. In any case once you find a copy that looks like it is probably OK then make a backup copy of the corrupted image and changes just in case someone wants to take a look at it, then copy over both the image and changes from the backup you picked into the squeaksource home directory. Hopefully now killing any existing process and having daemontools start it back up will work. And to do this you first find have to identify the relevant process. One way is $ ps auwx | grep squeaks which should produce a list something like root 2150 0.0 0.0 1360 268 ? S Nov28 0:00 supervise squeaksource squeaks 2176 3.6 8.0 1051344 77724 ? S Nov28 1256:48 /usr/local/lib/squeak/3.11.3-2135/squeakvm -vm-display-none /home/squeaksource/Squeak3.11-8824-SS.image website 30990 25.2 10.8 1051420 105060 ? S 16:59 9:31 /usr/bin/squeakvm -vm-display=none /home/website/website/squeaksite.image kencaus 1409 0.0 0.0 1552 524 pts/0 S+ 17:37 0:00 grep squeaks The relevant one is the squeakvm process referencing the proper image of course, the second one in this list. At which point you would take the process ID (second column) and do $ kill 2176 for example. The ID will of course vary. Check again and if the process is stuck, the one with the given ID does not disappear from the list, then you may have to $ kill -9 2176 In any case if you repeatedly look at the relevant list of running processes you should soon see another ... squeakvm ... Squeak3.11-8824-SS.image process running with a new process ID and hopefully if you check http://source.squeak.org you will get what you expect. Let me note that the filtered list of processes above also includes, on the first line, the daemontools process that 'supervises' the source.squeak.org service. Note that if you don't see this in the list then there is probably a problem with daemontools itself and in any case when you kill the process I don't expect that a new one will be started. However I'm going to draw this to a close here and leave that for another time. Ken

2 2

SqueakSource/Seaside question - has anyone seen this problem before?
by David T. Lewis 21 Dec '13

21 Dec '13

Has anyone with experience on Seaside and/or SqueakSource ever seen symptoms like we are currently experiencing here: http://squeaksource.com Our squeaksource.com server has for some reason gotten itself confused and has a broken web interface. The repository itself is fine, so this is not an emergency. However, the web interface is broken and seems to be emitting chunks of incomplete html. I can restore things to working order by reloading from a backup image, but I would prefer to understand what has going wrong first. The problem is definitely in the image itself, as opposed to Apache or anything else about the server. When I run the image on my own PC at home, I get identical symptoms. I am no expert in Seaside, and working through it in a debugger is making my head spin so I'm going to take the easy way out and just ask - has anybody seen this kind of problem before? TIA, Dave

1 0

Disk space usage on box3
by Ken Causey 10 Dec '13

10 Dec '13

The disk space on box3 is beginning to get dangerously low on filesystem space. At the moment it is 94% full with 3.8GB of free space. Lately it seems to increase 1% every 2 or 3 days. Primary offenders seem to be /var/lib/jenkins/ 33GB /home/ssdotcom/ 18GB I hope one or both of you can find something to delete. Ken

6 26

SqueakSource and source.squeak.org banners switched!
by Chris Muller 03 Dec '13

03 Dec '13

Hi Dave, when I look at squeaksource.com, it has the green banner with title "Squeak Development" where source.squeak.org has the original _blue_ banner and says, "SqueakSource". How could this possibly happen?

2 4

Some squeaksource.com updates
by David T. Lewis 01 Dec '13

01 Dec '13

Mail notifications for squeaksource.com updates have now been enabled. This function was disabled when squeaksource.com was initially re-hosted on squeak.org, and should now once again work as expected. Earlier problems that had caused repeat mail notifications for existing updates have been resolved and are not expected to cause further problems. The welcome page for http://squeaksource.com has been updated for factual content, and hopefully to appear less alarming for casual visitors. The policy of disabling new project creation remains in effect. With the welcome page updates, I also removed the instructions for moving projects from SqS to the newer repositories, because it seems a bit too unwelcoming for a "welcome" page. However, this is important information, so we need to keep it available somewhere. Is this (or could it be) available on a FAQ somewhere? If so I will link to it from the SqS welcome page. Here is the part that I removed: <h2>Migration to SmalltalkHub</h2> <p>To migrate so SmalltalkHub use the following script</p> <div class="format" style="color: #CD0A0A; background-color: #FEF1EC; border: 2px solid #CD0A0A; padding: 5px; border-radius: 4px; -moz-border-radius: 4px; -webkit-border-radius: 4px;">| source goSource destination goDestination files destinationFiles | <br/> source := MCHttpRepository location: ''http://www.squeaksource.com/YOURPROJECT''. destination := MCSmalltalkhubRepository     owner: ''YOURNAME''     project: ''YOURPROJECT''     user: ''YOURNAME''     password: ''YOURPASSWORD''. <br/> goSource := Gofer new repository: source. goDestination := Gofer new repository: destination. <br/> files := source allVersionNames. <br/> (goSource allResolved select: [ :resolved | files anySatisfy: [ :each |     resolved name = each ] ]) do: [ :each | goSource package: each packageName ]. <br/> goSource fetch. "downloads all mcz on your computer" <br/> destinationFiles := destination allVersionNames. "checks what files are already at destination" files reject: [ :file | destinationFiles includes: file ] thenDo: [ :file |     goDestination version: file ]. "selects only the mcz that are not yet at destination" <br/> goDestination push. "sends everything to SmalltalkHub" <br/> self assert: destination allVersionNames sorted = files sorted. "checks we have exactly the same files at source and destination"</div> Thanks, Dave

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Box-Admins December 2013