On Sun, Nov 02, 2014 at 09:31:31AM -0600, Chris Muller wrote:
On Sat, Nov 1, 2014 at 8:07 PM, Levente Uzonyi leves@elte.hu wrote:
Hi All,
There's less than 3GB (out of 60) free disk space at the moment on box3, and it keeps decreasing.
What services are running on box3?
- squeaksource.com which currently uses about 17GB
- build.squeak.org - aka jenkins - which uses about 35 GB at the moment
- apache2 as a frontend for the websites, which uses negligible disk space
- a yet unused but configured nameserver, which uses negligible disk space
- a seemingly unused ftp server, which should probably be removed, but it
uses negligible disk space anyway
What are the possible solutions?
- increase the size of the disk. This can be done by the SFC, if the Board
requests it, and the SFC is willing to do it. In theory this can be done on the fly - without stopping the server[1]. How large should the increase be? - It depends on what the plans about jenkins are, but the more the merrier.
- move squeaksource.com to box4. box3 was planned to host nothing but
jenkins. There's plenty of space left on box4, so this is something we can do fairly easily.
- clean up jenkins. We can keep less build artifacts, delete unused projects
(if any), try to remove unused versions of softwares, try to share files using hard links, etc. But this is probably what requires the most effort.
Why do we need to devote more than HALF of box3 to Jenkins? 35GB of.... what? For goodness sake it runs the tests in a 20MB image and that's it! Why in the world does it need so much space? And what valuable output of Jenkins are we as a community, consuming from it that we should expend resources to "keep it growing" instead of putting it on a diet?
The ongoing growth is mainly from our Jenkins jobs. I don't see any obvious way to tell Jenkins to clean up after itself, and google does not offer any enlightenment. I suspect that most Jenkins sites may just be in the habit of cleaning up with cron jobs or perhaps even manually. We might need to do the same.
Just as an example, if we remove the older entries from ~jenkins/jobs/SqueakTrunk, we would free up 1GB from that job alone. But I think that all of the jobs require attention.
I think a few guidelines might help. My suggestions:
- Each Jenkin job must have an owner, identified in some recognizable way (such as in the job description on the build.squeak.org web page).
- The job owner is responsible for making sure the job gets cleaned up once in a while, either by doing the cleanup or by asking for help to get it done (not all of us are unix hackers, and that might be what is needed to clean some of these things up).
- Nothing should be removed without permission of the job owner.
- If the job owner cannot be identified or contacted, then the job may be archived and/or disabled as needed.
Dave