Hi All,
There's less than 3GB (out of 60) free disk space at the moment on box3, and it keeps decreasing.
What services are running on box3? - squeaksource.com which currently uses about 17GB - build.squeak.org - aka jenkins - which uses about 35 GB at the moment - apache2 as a frontend for the websites, which uses negligible disk space - a yet unused but configured nameserver, which uses negligible disk space - a seemingly unused ftp server, which should probably be removed, but it uses negligible disk space anyway
What are the possible solutions? - increase the size of the disk. This can be done by the SFC, if the Board requests it, and the SFC is willing to do it. In theory this can be done on the fly - without stopping the server[1]. How large should the increase be? - It depends on what the plans about jenkins are, but the more the merrier. - move squeaksource.com to box4. box3 was planned to host nothing but jenkins. There's plenty of space left on box4, so this is something we can do fairly easily. - clean up jenkins. We can keep less build artifacts, delete unused projects (if any), try to remove unused versions of softwares, try to share files using hard links, etc. But this is probably what requires the most effort.
We need a quick decision. In case of an emergency (less than 1GB free space), I'll try to move squeaksource.com to box4.
Levente
I guess we have two running instances of SS and if we merged them, then we’d only need one. As I understand it, we have source.squeak.org http://source.squeak.org/ which is the trunk SS. And then we have squeaksource.com http://squeaksource.com/, which was inherited from (I think it was) Berne. I always thought the box2 image would go to box4. Would it be possible to have one instance for all the files? Trunk projects and non-Trunk? I guess it would seem sort of odd to have two instances running in the same box. Perhaps it’s necessary. I don’t know. Can they be conflated?
Chris
On Nov 1, 2014, at 9:07 PM, Levente Uzonyi leves@elte.hu wrote:
Hi All,
There's less than 3GB (out of 60) free disk space at the moment on box3, and it keeps decreasing.
What services are running on box3?
- squeaksource.com which currently uses about 17GB
- build.squeak.org - aka jenkins - which uses about 35 GB at the moment
- apache2 as a frontend for the websites, which uses negligible disk space
- a yet unused but configured nameserver, which uses negligible disk space
- a seemingly unused ftp server, which should probably be removed, but it uses negligible disk space anyway
What are the possible solutions?
- increase the size of the disk. This can be done by the SFC, if the Board requests it, and the SFC is willing to do it. In theory this can be done on the fly - without stopping the server[1]. How large should the increase be?
- It depends on what the plans about jenkins are, but the more the merrier.
- move squeaksource.com to box4. box3 was planned to host nothing but jenkins. There's plenty of space left on box4, so this is something we can do fairly easily.
- clean up jenkins. We can keep less build artifacts, delete unused projects (if any), try to remove unused versions of softwares, try to share files using hard links, etc. But this is probably what requires the most effort.
We need a quick decision. In case of an emergency (less than 1GB free space), I'll try to move squeaksource.com to box4.
Levente
On Sat, 1 Nov 2014, Chris Cunnington wrote:
I guess we have two running instances of SS and if we merged them, then we’d only need one. As I understand it, we have source.squeak.org which is the trunk SS. And then we have squeaksource.com, which was inherited from (I think it was) Berne. I always thought the box2 image would go to box4. Would it be possible to have one instance for all the files? Trunk projects and non-Trunk? I guess it would seem sort of odd to have two instances running in the same box. Perhaps it’s necessary. I don’t know. Can they be conflated?
It's possible the merge the two instances, which would mean moving all projects from source.squeak.org to squeaksource.com, and not the other way around. It would require some manual work because there are duplicate accounts and projects. The merge would save a few tens of MBs of disk space and memory. But there would be some issues with the domains names, because both domain names are used in existing images and external links.
But this doesn't help with the current situation, and there's no problem running two instances on the same server. I guess this idea deserves a separate thread.
Levente
Chris
On Nov 1, 2014, at 9:07 PM, Levente Uzonyi <leves@elte.hu> wrote:
Hi All,
There's less than 3GB (out of 60) free disk space at the moment on box3, and it keeps decreasing.
What services are running on box3?
- squeaksource.com which currently uses about 17GB
- build.squeak.org - aka jenkins - which uses about 35 GB at the moment
- apache2 as a frontend for the websites, which uses negligible disk space
- a yet unused but configured nameserver, which uses negligible disk space
- a seemingly unused ftp server, which should probably be removed, but it uses negligible disk space anyway
What are the possible solutions?
- increase the size of the disk. This can be done by the SFC, if the Board requests it, and the SFC is willing to do it. In theory this can be done on the fly - without stopping the server[1].
How large should the increase be?
- It depends on what the plans about jenkins are, but the more the merrier.
- move squeaksource.com to box4. box3 was planned to host nothing but jenkins. There's plenty of space left on box4, so this is something we can do fairly easily.
- clean up jenkins. We can keep less build artifacts, delete unused projects (if any), try to remove unused versions of softwares, try to share files using hard links, etc. But this is probably what
requires the most effort.
We need a quick decision. In case of an emergency (less than 1GB free space), I'll try to move squeaksource.com to box4.
Levente
On Sat, Nov 1, 2014 at 8:07 PM, Levente Uzonyi leves@elte.hu wrote:
Hi All,
There's less than 3GB (out of 60) free disk space at the moment on box3, and it keeps decreasing.
What services are running on box3?
- squeaksource.com which currently uses about 17GB
- build.squeak.org - aka jenkins - which uses about 35 GB at the moment
- apache2 as a frontend for the websites, which uses negligible disk space
- a yet unused but configured nameserver, which uses negligible disk space
- a seemingly unused ftp server, which should probably be removed, but it
uses negligible disk space anyway
What are the possible solutions?
- increase the size of the disk. This can be done by the SFC, if the Board
requests it, and the SFC is willing to do it. In theory this can be done on the fly - without stopping the server[1]. How large should the increase be? - It depends on what the plans about jenkins are, but the more the merrier.
- move squeaksource.com to box4. box3 was planned to host nothing but
jenkins. There's plenty of space left on box4, so this is something we can do fairly easily.
- clean up jenkins. We can keep less build artifacts, delete unused projects
(if any), try to remove unused versions of softwares, try to share files using hard links, etc. But this is probably what requires the most effort.
Why do we need to devote more than HALF of box3 to Jenkins? 35GB of.... what? For goodness sake it runs the tests in a 20MB image and that's it! Why in the world does it need so much space? And what valuable output of Jenkins are we as a community, consuming from it that we should expend resources to "keep it growing" instead of putting it on a diet?
On Sun, Nov 02, 2014 at 09:31:31AM -0600, Chris Muller wrote:
On Sat, Nov 1, 2014 at 8:07 PM, Levente Uzonyi leves@elte.hu wrote:
Hi All,
There's less than 3GB (out of 60) free disk space at the moment on box3, and it keeps decreasing.
What services are running on box3?
- squeaksource.com which currently uses about 17GB
- build.squeak.org - aka jenkins - which uses about 35 GB at the moment
- apache2 as a frontend for the websites, which uses negligible disk space
- a yet unused but configured nameserver, which uses negligible disk space
- a seemingly unused ftp server, which should probably be removed, but it
uses negligible disk space anyway
What are the possible solutions?
- increase the size of the disk. This can be done by the SFC, if the Board
requests it, and the SFC is willing to do it. In theory this can be done on the fly - without stopping the server[1]. How large should the increase be? - It depends on what the plans about jenkins are, but the more the merrier.
- move squeaksource.com to box4. box3 was planned to host nothing but
jenkins. There's plenty of space left on box4, so this is something we can do fairly easily.
- clean up jenkins. We can keep less build artifacts, delete unused projects
(if any), try to remove unused versions of softwares, try to share files using hard links, etc. But this is probably what requires the most effort.
Why do we need to devote more than HALF of box3 to Jenkins? 35GB of.... what? For goodness sake it runs the tests in a 20MB image and that's it! Why in the world does it need so much space? And what valuable output of Jenkins are we as a community, consuming from it that we should expend resources to "keep it growing" instead of putting it on a diet?
The ongoing growth is mainly from our Jenkins jobs. I don't see any obvious way to tell Jenkins to clean up after itself, and google does not offer any enlightenment. I suspect that most Jenkins sites may just be in the habit of cleaning up with cron jobs or perhaps even manually. We might need to do the same.
Just as an example, if we remove the older entries from ~jenkins/jobs/SqueakTrunk, we would free up 1GB from that job alone. But I think that all of the jobs require attention.
I think a few guidelines might help. My suggestions:
- Each Jenkin job must have an owner, identified in some recognizable way (such as in the job description on the build.squeak.org web page).
- The job owner is responsible for making sure the job gets cleaned up once in a while, either by doing the cleanup or by asking for help to get it done (not all of us are unix hackers, and that might be what is needed to clean some of these things up).
- Nothing should be removed without permission of the job owner.
- If the job owner cannot be identified or contacted, then the job may be archived and/or disabled as needed.
Dave
On Sun, Nov 02, 2014 at 12:02:57PM -0500, David T. Lewis wrote:
On Sun, Nov 02, 2014 at 09:31:31AM -0600, Chris Muller wrote:
On Sat, Nov 1, 2014 at 8:07 PM, Levente Uzonyi leves@elte.hu wrote:
Hi All,
There's less than 3GB (out of 60) free disk space at the moment on box3, and it keeps decreasing.
<snip>
Just as an example, if we remove the older entries from ~jenkins/jobs/SqueakTrunk, we would free up 1GB from that job alone. But I think that all of the jobs require attention.
Oops, my arithmetic was off by a zero, it looks more like 9 or 10GB could be freed up here.
Frank, I made an off-line backup (tar backup to my home PC) of everything in the ~jenkins/jobs/SqueakTrunk/builds directory from build 177 through build 699. That includes all of the images built from 2013-02-21 through 2013-12-19.
Is there value in keeping those build artifacts on line? If not, may I have your permission to delete them, leaving the remaining builds from 2013-12-19 through the present on line? I can give you a copy of the backup, or move it to some other location if we want to keep it on line.
Thanks,
Dave
On Sun, 2 Nov 2014, David T. Lewis wrote:
On Sun, Nov 02, 2014 at 12:02:57PM -0500, David T. Lewis wrote:
On Sun, Nov 02, 2014 at 09:31:31AM -0600, Chris Muller wrote:
On Sat, Nov 1, 2014 at 8:07 PM, Levente Uzonyi leves@elte.hu wrote:
Hi All,
There's less than 3GB (out of 60) free disk space at the moment on box3, and it keeps decreasing.
<snip>
Just as an example, if we remove the older entries from ~jenkins/jobs/SqueakTrunk, we would free up 1GB from that job alone. But I think that all of the jobs require attention.
Oops, my arithmetic was off by a zero, it looks more like 9 or 10GB could be freed up here.
Frank, I made an off-line backup (tar backup to my home PC) of everything in the ~jenkins/jobs/SqueakTrunk/builds directory from build 177 through build 699. That includes all of the images built from 2013-02-21 through 2013-12-19.
Is there value in keeping those build artifacts on line? If not, may I have your permission to delete them, leaving the remaining builds from 2013-12-19 through the present on line? I can give you a copy of the backup, or move it to some other location if we want to keep it on line.
I think build information should be kept "forever", because it's nice to see progress over time. And I also see some value in keeping not too old build artifacts, because it allows me to quickly see what was the situation X builds ago. After a bit of googling, I found that there's a "Discard Old Builds" option[1][2], which can simply delete all build information based on some simple rules. But there's also a plugin[3] which does exactly what I would like to see. One can specify to keep the build information with it, while deleting the build artifacts of older builds. So I think we (I mean someone who has access to jenkins) should install and configure that plugin for the critical projects. These are SqueakTrunk and ReleaseSqueakTrunk, which use 17GB of disk space at the moment.
Levente
[1] https://stackoverflow.com/questions/7994379/free-space-in-jenkins-deleting-b... [2] http://blog.enterpriselab.ch/tdmarti/2011/05/13/delete-artifacts-in-jenkins/ [3] https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin
Thanks,
Dave
On 3 November 2014 01:23, Levente Uzonyi leves@elte.hu wrote:
On Sun, 2 Nov 2014, David T. Lewis wrote:
On Sun, Nov 02, 2014 at 12:02:57PM -0500, David T. Lewis wrote:
On Sun, Nov 02, 2014 at 09:31:31AM -0600, Chris Muller wrote:
On Sat, Nov 1, 2014 at 8:07 PM, Levente Uzonyi leves@elte.hu wrote:
Hi All,
There's less than 3GB (out of 60) free disk space at the moment on box3, and it keeps decreasing.
<snip>
Just as an example, if we remove the older entries from ~jenkins/jobs/SqueakTrunk, we would free up 1GB from that job alone. But I think that all of the jobs require attention.
Oops, my arithmetic was off by a zero, it looks more like 9 or 10GB could be freed up here.
Frank, I made an off-line backup (tar backup to my home PC) of everything in the ~jenkins/jobs/SqueakTrunk/builds directory from build 177 through build 699. That includes all of the images built from 2013-02-21 through 2013-12-19.
Is there value in keeping those build artifacts on line? If not, may I have your permission to delete them, leaving the remaining builds from 2013-12-19 through the present on line? I can give you a copy of the backup, or move it to some other location if we want to keep it on line.
I think build information should be kept "forever", because it's nice to see progress over time. And I also see some value in keeping not too old build artifacts, because it allows me to quickly see what was the situation X builds ago. After a bit of googling, I found that there's a "Discard Old Builds" option[1][2], which can simply delete all build information based on some simple rules. But there's also a plugin[3] which does exactly what I would like to see. One can specify to keep the build information with it, while deleting the build artifacts of older builds. So I think we (I mean someone who has access to jenkins) should install and configure that plugin for the critical projects. These are SqueakTrunk and ReleaseSqueakTrunk, which use 17GB of disk space at the moment.
I'll second Levente's comment about keeping build artifacts to reproduce issues. This is critical in the Trunk process, because we don't have a linearisable development history. "Update number" means only "the sum of the version numbers of all installed packages". Not only that, I don't think there's any way of saying "please update my Squeak image from 4.5-N to 4.5-M", so we can't recreate the starting conditions of a build.
I've installed the Discard Old Build plugin. I'll need to restart Jenkins for the plugin to register.
I'll wait a while before doing that, in case anyone needs to say anything.
frank
Levente
[1] https://stackoverflow.com/questions/7994379/free-space-in-jenkins-deleting-b... [2] http://blog.enterpriselab.ch/tdmarti/2011/05/13/delete-artifacts-in-jenkins/ [3] https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin
Thanks,
Dave
On Mon, Nov 03, 2014 at 12:54:20PM +0000, Frank Shearar wrote:
On 3 November 2014 01:23, Levente Uzonyi leves@elte.hu wrote:
On Sun, 2 Nov 2014, David T. Lewis wrote:
On Sun, Nov 02, 2014 at 12:02:57PM -0500, David T. Lewis wrote:
On Sun, Nov 02, 2014 at 09:31:31AM -0600, Chris Muller wrote:
On Sat, Nov 1, 2014 at 8:07 PM, Levente Uzonyi leves@elte.hu wrote:
Hi All,
There's less than 3GB (out of 60) free disk space at the moment on box3, and it keeps decreasing.
<snip>
Just as an example, if we remove the older entries from ~jenkins/jobs/SqueakTrunk, we would free up 1GB from that job alone. But I think that all of the jobs require attention.
Oops, my arithmetic was off by a zero, it looks more like 9 or 10GB could be freed up here.
Frank, I made an off-line backup (tar backup to my home PC) of everything in the ~jenkins/jobs/SqueakTrunk/builds directory from build 177 through build 699. That includes all of the images built from 2013-02-21 through 2013-12-19.
Is there value in keeping those build artifacts on line? If not, may I have your permission to delete them, leaving the remaining builds from 2013-12-19 through the present on line? I can give you a copy of the backup, or move it to some other location if we want to keep it on line.
I think build information should be kept "forever", because it's nice to see progress over time. And I also see some value in keeping not too old build artifacts, because it allows me to quickly see what was the situation X builds ago. After a bit of googling, I found that there's a "Discard Old Builds" option[1][2], which can simply delete all build information based on some simple rules. But there's also a plugin[3] which does exactly what I would like to see. One can specify to keep the build information with it, while deleting the build artifacts of older builds. So I think we (I mean someone who has access to jenkins) should install and configure that plugin for the critical projects. These are SqueakTrunk and ReleaseSqueakTrunk, which use 17GB of disk space at the moment.
I'll second Levente's comment about keeping build artifacts to reproduce issues. This is critical in the Trunk process, because we don't have a linearisable development history. "Update number" means only "the sum of the version numbers of all installed packages". Not only that, I don't think there's any way of saying "please update my Squeak image from 4.5-N to 4.5-M", so we can't recreate the starting conditions of a build.
I've installed the Discard Old Build plugin. I'll need to restart Jenkins for the plugin to register.
I'll wait a while before doing that, in case anyone needs to say anything.
Thanks Frank,
+1
I also like Levente's suggestion. Please restart Jenkins whenever you like :)
Dave
Levente
[1] https://stackoverflow.com/questions/7994379/free-space-in-jenkins-deleting-b... [2] http://blog.enterpriselab.ch/tdmarti/2011/05/13/delete-artifacts-in-jenkins/ [3] https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin
Thanks,
Dave
On 3 November 2014 13:02, David T. Lewis lewis@mail.msen.com wrote:
On Mon, Nov 03, 2014 at 12:54:20PM +0000, Frank Shearar wrote:
On 3 November 2014 01:23, Levente Uzonyi leves@elte.hu wrote:
On Sun, 2 Nov 2014, David T. Lewis wrote:
On Sun, Nov 02, 2014 at 12:02:57PM -0500, David T. Lewis wrote:
On Sun, Nov 02, 2014 at 09:31:31AM -0600, Chris Muller wrote:
On Sat, Nov 1, 2014 at 8:07 PM, Levente Uzonyi leves@elte.hu wrote: > > Hi All, > > There's less than 3GB (out of 60) free disk space at the moment on > box3, and > it keeps decreasing. >
<snip>
Just as an example, if we remove the older entries from ~jenkins/jobs/SqueakTrunk, we would free up 1GB from that job alone. But I think that all of the jobs require attention.
Oops, my arithmetic was off by a zero, it looks more like 9 or 10GB could be freed up here.
Frank, I made an off-line backup (tar backup to my home PC) of everything in the ~jenkins/jobs/SqueakTrunk/builds directory from build 177 through build 699. That includes all of the images built from 2013-02-21 through 2013-12-19.
Is there value in keeping those build artifacts on line? If not, may I have your permission to delete them, leaving the remaining builds from 2013-12-19 through the present on line? I can give you a copy of the backup, or move it to some other location if we want to keep it on line.
I think build information should be kept "forever", because it's nice to see progress over time. And I also see some value in keeping not too old build artifacts, because it allows me to quickly see what was the situation X builds ago. After a bit of googling, I found that there's a "Discard Old Builds" option[1][2], which can simply delete all build information based on some simple rules. But there's also a plugin[3] which does exactly what I would like to see. One can specify to keep the build information with it, while deleting the build artifacts of older builds. So I think we (I mean someone who has access to jenkins) should install and configure that plugin for the critical projects. These are SqueakTrunk and ReleaseSqueakTrunk, which use 17GB of disk space at the moment.
I'll second Levente's comment about keeping build artifacts to reproduce issues. This is critical in the Trunk process, because we don't have a linearisable development history. "Update number" means only "the sum of the version numbers of all installed packages". Not only that, I don't think there's any way of saying "please update my Squeak image from 4.5-N to 4.5-M", so we can't recreate the starting conditions of a build.
I've installed the Discard Old Build plugin. I'll need to restart Jenkins for the plugin to register.
I'll wait a while before doing that, in case anyone needs to say anything.
Thanks Frank,
+1
I also like Levente's suggestion. Please restart Jenkins whenever you like :)
Well, that worked. So maybe a first start would be to say we want at most 100 builds, and discard aborted builds. (SqueakTrunk currently has 792 builds, so this ought to cap its disk usage at ~1G).
And then the same again for ReleaseSqueakTrunk.
frank
Dave
Levente
[1] https://stackoverflow.com/questions/7994379/free-space-in-jenkins-deleting-b... [2] http://blog.enterpriselab.ch/tdmarti/2011/05/13/delete-artifacts-in-jenkins/ [3] https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin
Thanks,
Dave
On Mon, 3 Nov 2014, Frank Shearar wrote:
On 3 November 2014 13:02, David T. Lewis lewis@mail.msen.com wrote:
On Mon, Nov 03, 2014 at 12:54:20PM +0000, Frank Shearar wrote:
On 3 November 2014 01:23, Levente Uzonyi leves@elte.hu wrote:
On Sun, 2 Nov 2014, David T. Lewis wrote:
On Sun, Nov 02, 2014 at 12:02:57PM -0500, David T. Lewis wrote:
On Sun, Nov 02, 2014 at 09:31:31AM -0600, Chris Muller wrote: > > On Sat, Nov 1, 2014 at 8:07 PM, Levente Uzonyi leves@elte.hu wrote: >> >> Hi All, >> >> There's less than 3GB (out of 60) free disk space at the moment on >> box3, and >> it keeps decreasing. >>
<snip>
Just as an example, if we remove the older entries from ~jenkins/jobs/SqueakTrunk, we would free up 1GB from that job alone. But I think that all of the jobs require attention.
Oops, my arithmetic was off by a zero, it looks more like 9 or 10GB could be freed up here.
Frank, I made an off-line backup (tar backup to my home PC) of everything in the ~jenkins/jobs/SqueakTrunk/builds directory from build 177 through build 699. That includes all of the images built from 2013-02-21 through 2013-12-19.
Is there value in keeping those build artifacts on line? If not, may I have your permission to delete them, leaving the remaining builds from 2013-12-19 through the present on line? I can give you a copy of the backup, or move it to some other location if we want to keep it on line.
I think build information should be kept "forever", because it's nice to see progress over time. And I also see some value in keeping not too old build artifacts, because it allows me to quickly see what was the situation X builds ago. After a bit of googling, I found that there's a "Discard Old Builds" option[1][2], which can simply delete all build information based on some simple rules. But there's also a plugin[3] which does exactly what I would like to see. One can specify to keep the build information with it, while deleting the build artifacts of older builds. So I think we (I mean someone who has access to jenkins) should install and configure that plugin for the critical projects. These are SqueakTrunk and ReleaseSqueakTrunk, which use 17GB of disk space at the moment.
I'll second Levente's comment about keeping build artifacts to reproduce issues. This is critical in the Trunk process, because we don't have a linearisable development history. "Update number" means only "the sum of the version numbers of all installed packages". Not only that, I don't think there's any way of saying "please update my Squeak image from 4.5-N to 4.5-M", so we can't recreate the starting conditions of a build.
I've installed the Discard Old Build plugin. I'll need to restart Jenkins for the plugin to register.
I'll wait a while before doing that, in case anyone needs to say anything.
Thanks Frank,
+1
I also like Levente's suggestion. Please restart Jenkins whenever you like :)
Well, that worked. So maybe a first start would be to say we want at most 100 builds, and discard aborted builds. (SqueakTrunk currently has 792 builds, so this ought to cap its disk usage at ~1G).
If you mean the build artifacts for the last 100 successful builds, then yes, it sounds good to me.
Levente
And then the same again for ReleaseSqueakTrunk.
frank
Dave
Levente
[1] https://stackoverflow.com/questions/7994379/free-space-in-jenkins-deleting-b... [2] http://blog.enterpriselab.ch/tdmarti/2011/05/13/delete-artifacts-in-jenkins/ [3] https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin
Thanks,
Dave
On 03-11-2014, at 4:54 AM, Frank Shearar frank.shearar@gmail.com wrote:
Not only that, I don't think there's any way of saying "please update my Squeak image from 4.5-N to 4.5-M", so we can't recreate the starting conditions of a build.
Actually I think that we can do this. Take a look at UpdateStreamDownloader class>applyUpdatesFromDiskToUpdaeNumber:stopIfGap:
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Klingon Code Warrior:- 9) "A TRUE Klingon warrior does not comment his code!"
On 3 November 2014 19:12, tim Rowledge tim@rowledge.org wrote:
On 03-11-2014, at 4:54 AM, Frank Shearar frank.shearar@gmail.com wrote:
Not only that, I don't think there's any way of saying "please update my Squeak image from 4.5-N to 4.5-M", so we can't recreate the starting conditions of a build.
Actually I think that we can do this. Take a look at UpdateStreamDownloader class>applyUpdatesFromDiskToUpdaeNumber:stopIfGap:
No, because decomposing a sum into its summands is not unambiguous. If you and I have two images, and your Network package is version 222 and Kernel package is 501, while my Network package is version 221 and Kernel is version 502, all other packages being the same, _our update numbers are identical_.
There are a few synchronisation points, in the form of MCMs, in the history of the update stream, but in between those, no, you have no way of knowing - from only the update number - what the versions of packages are that ought to make up that "version" of the image.
frank
tim
tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Klingon Code Warrior:- 9) "A TRUE Klingon warrior does not comment his code!"
On 3 November 2014 18:35, Levente Uzonyi leves@elte.hu wrote:
On Mon, 3 Nov 2014, Frank Shearar wrote:
On 3 November 2014 13:02, David T. Lewis lewis@mail.msen.com wrote:
On Mon, Nov 03, 2014 at 12:54:20PM +0000, Frank Shearar wrote:
On 3 November 2014 01:23, Levente Uzonyi leves@elte.hu wrote:
On Sun, 2 Nov 2014, David T. Lewis wrote:
On Sun, Nov 02, 2014 at 12:02:57PM -0500, David T. Lewis wrote: > > > On Sun, Nov 02, 2014 at 09:31:31AM -0600, Chris Muller wrote: >> >> >> On Sat, Nov 1, 2014 at 8:07 PM, Levente Uzonyi leves@elte.hu >> wrote: >>> >>> >>> Hi All, >>> >>> There's less than 3GB (out of 60) free disk space at the moment on >>> box3, and >>> it keeps decreasing. >>>
<snip>
> Just as an example, if we remove the older entries from > ~jenkins/jobs/SqueakTrunk, > we would free up 1GB from that job alone. But I think that all of the > jobs > require attention.
Oops, my arithmetic was off by a zero, it looks more like 9 or 10GB could be freed up here.
Frank, I made an off-line backup (tar backup to my home PC) of everything in the ~jenkins/jobs/SqueakTrunk/builds directory from build 177 through build 699. That includes all of the images built from 2013-02-21 through 2013-12-19.
Is there value in keeping those build artifacts on line? If not, may I have your permission to delete them, leaving the remaining builds from 2013-12-19 through the present on line? I can give you a copy of the backup, or move it to some other location if we want to keep it on line.
I think build information should be kept "forever", because it's nice to see progress over time. And I also see some value in keeping not too old build artifacts, because it allows me to quickly see what was the situation X builds ago. After a bit of googling, I found that there's a "Discard Old Builds" option[1][2], which can simply delete all build information based on some simple rules. But there's also a plugin[3] which does exactly what I would like to see. One can specify to keep the build information with it, while deleting the build artifacts of older builds. So I think we (I mean someone who has access to jenkins) should install and configure that plugin for the critical projects. These are SqueakTrunk and ReleaseSqueakTrunk, which use 17GB of disk space at the moment.
I'll second Levente's comment about keeping build artifacts to reproduce issues. This is critical in the Trunk process, because we don't have a linearisable development history. "Update number" means only "the sum of the version numbers of all installed packages". Not only that, I don't think there's any way of saying "please update my Squeak image from 4.5-N to 4.5-M", so we can't recreate the starting conditions of a build.
I've installed the Discard Old Build plugin. I'll need to restart Jenkins for the plugin to register.
I'll wait a while before doing that, in case anyone needs to say anything.
Thanks Frank,
+1
I also like Levente's suggestion. Please restart Jenkins whenever you like :)
Well, that worked. So maybe a first start would be to say we want at most 100 builds, and discard aborted builds. (SqueakTrunk currently has 792 builds, so this ought to cap its disk usage at ~1G).
If you mean the build artifacts for the last 100 successful builds, then yes, it sounds good to me.
Results of setting just those two builds:
frankshearar@box3-squeak:~$ df -h Filesystem Size Used Avail Use% Mounted on /dev/xvda1 60G 38G 19G 68% / tmpfs 504M 0 504M 0% /lib/init/rw udev 10M 80K 10M 1% /dev tmpfs 504M 4.0K 504M 1% /dev/shm tmpfs 24K 16K 8.0K 67% /var/gandi
Rather better!
frank
Levente
And then the same again for ReleaseSqueakTrunk.
frank
Dave
Levente
[1]
https://stackoverflow.com/questions/7994379/free-space-in-jenkins-deleting-b... [2]
http://blog.enterpriselab.ch/tdmarti/2011/05/13/delete-artifacts-in-jenkins/ [3] https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin
Thanks,
Dave
box-admins@lists.squeakfoundation.org