I'm going to try to summarize as simply as I can the servers statuses, the original plan, modifications to the plan, and what has been done.
First our assets:
Hetzner - box2.squeak.org (85.10.195.197) - Our aging outgoing server that has been our singular server for many years now and hosted all online services.
Stats: ancient version of Debian, full hardware server, 1GB RAM, 150GB HD
Gandi.net - box3.squeak.org (173.246.101.237) - This server was originally allocated and setup at the request of Frank for use as a continuous integration server and hosts build.squeak.org, a Jenkins instance. As far as I am concerned this is still the primary purpose of this server. Currently this is also hosting our 'recovered' squeaksource.com. David can speak to details about the plan for this.
Stats: current and up to date Debian Squeeze (32bit), virtual server, 1GB RAM, 60GB HD
Gandi.net - box4.squeak.org (173.246.104.42) - This server is intended to be the primary replacement host for box2.squeak.org. I'll speak more to the status of this server below.
Stats: current and up to date Debian Wheezy (64bit), virtual server, 4GB RAM, 100GB HD
Both Gandi.net virtual servers are provided kindly by Gandi.net at no charge and through an agreement and with the assistance of the Software Freedom Consortium and Bradley Kuhn.
The plan:
In brief the goal is to replicate all existing services hosted by the old server using primarily box4 and secondarily box3. My intention had been to setup mailman and mailing list hosting on box3 to reduce the load on box4. However this was prior to setting up squeaksource.com on box3 and so this is now unlikely without changes.
The intention is to use up to date software and use leaner software when possible, for example: nginx for top-level HTTP, qmail for SMTP, djbdns for DNS, etc. Both classic Squeak VM and Cog VM would be available. Super user access is consistently through sudo which logs actions in case of problems. Every individual with access to the server will have their own personal account. How to handle access to 'shared' resources is still somewhat to be determined.
Status:
On both servers an administrative log is being kept in /root/admin-log.txt with a hopefully consistent format and under RCS. Those with access should feel free to review these for more detail.
DNS (djbdns) has been setup on both box3 and box4. box4 is the primary DNS and box3 the secondary and automatically updates from box4. Repeated emails to the holders of squeak.org and squeakfoundation.org to update the root servers to point to these have met with total silence. This has been frustrating.
Daemontools has been installed and whenever 'service' software needs to be maintained and does not come with ready support for init, Daemontools can and should be used.
Whenever software needs to be installed which is not readily installable via apt-get (or a newer version is desired) checkinstall should be used to build and install a debian package so that all installed software can be consistently managed.
Classic SqueakVM 4.10.2.2614 was built and a debian package produced and installed. Some testing was done but issues may remain and a newer version may well be available. All .sources files were placed in a central location so that none should be needed per project. This will need to be updated (links?) whenever a new VM version is installed.
NTP server was installed.
Nginx was installed on box4 but little configuration has been done.
I repeatedly tried to build and install Cog but ran into multiple problems. Even beyond build problems one issue is that Cog is built to name the executable squeak and as such collides with the Classic VM. Having both VMs seems necessary given the age of some of the Squeak hosted services. Having Cog is desirable for more recent services and updates to the old ones.
Note that work on both VMs was some time back and the issues I had would need to be revisited and evaluated.
Ken
Thanks a lot for this summary, it helps me a lot!
A couple of notes inline below.
Dave
On Thu, Oct 10, 2013 at 10:04:17AM -0500, Ken Causey wrote:
I'm going to try to summarize as simply as I can the servers statuses, the original plan, modifications to the plan, and what has been done.
First our assets:
Hetzner - box2.squeak.org (85.10.195.197) - Our aging outgoing server that has been our singular server for many years now and hosted all online services.
Stats: ancient version of Debian, full hardware server, 1GB RAM, 150GB HD
Gandi.net - box3.squeak.org (173.246.101.237) - This server was originally allocated and setup at the request of Frank for use as a continuous integration server and hosts build.squeak.org, a Jenkins instance. As far as I am concerned this is still the primary purpose of this server. Currently this is also hosting our 'recovered' squeaksource.com. David can speak to details about the plan for this.
It sounds like box4 would be a better host for squeaksource.com. If so, I don't mind moving it from box3 to box4. I will need an account on box4 in order to do this.
Stats: current and up to date Debian Squeeze (32bit), virtual server, 1GB RAM, 60GB HD
Gandi.net - box4.squeak.org (173.246.104.42) - This server is intended to be the primary replacement host for box2.squeak.org. I'll speak more to the status of this server below.
Stats: current and up to date Debian Wheezy (64bit), virtual server, 4GB RAM, 100GB HD
Both Gandi.net virtual servers are provided kindly by Gandi.net at no charge and through an agreement and with the assistance of the Software Freedom Consortium and Bradley Kuhn.
The plan:
In brief the goal is to replicate all existing services hosted by the old server using primarily box4 and secondarily box3. My intention had been to setup mailman and mailing list hosting on box3 to reduce the load on box4. However this was prior to setting up squeaksource.com on box3 and so this is now unlikely without changes.
The intention is to use up to date software and use leaner software when possible, for example: nginx for top-level HTTP, qmail for SMTP, djbdns for DNS, etc. Both classic Squeak VM and Cog VM would be available. Super user access is consistently through sudo which logs actions in case of problems. Every individual with access to the server will have their own personal account. How to handle access to 'shared' resources is still somewhat to be determined.
Status:
On both servers an administrative log is being kept in /root/admin-log.txt with a hopefully consistent format and under RCS. Those with access should feel free to review these for more detail.
DNS (djbdns) has been setup on both box3 and box4. box4 is the primary DNS and box3 the secondary and automatically updates from box4. Repeated emails to the holders of squeak.org and squeakfoundation.org to update the root servers to point to these have met with total silence. This has been frustrating.
Daemontools has been installed and whenever 'service' software needs to be maintained and does not come with ready support for init, Daemontools can and should be used.
Whenever software needs to be installed which is not readily installable via apt-get (or a newer version is desired) checkinstall should be used to build and install a debian package so that all installed software can be consistently managed.
Classic SqueakVM 4.10.2.2614 was built and a debian package produced and installed. Some testing was done but issues may remain and a newer version may well be available. All .sources files were placed in a central location so that none should be needed per project. This will need to be updated (links?) whenever a new VM version is installed.
NTP server was installed.
Nginx was installed on box4 but little configuration has been done.
I repeatedly tried to build and install Cog but ran into multiple problems. Even beyond build problems one issue is that Cog is built to name the executable squeak and as such collides with the Classic VM. Having both VMs seems necessary given the age of some of the Squeak hosted services. Having Cog is desirable for more recent services and updates to the old ones.
I can help with this if you like. On my own PC, I am in the habit of just renaming the Cog script to /usr/local/bin/cog and installing the classic VM in its normal /usr/local/bin/squeak location. If nobody objects, I can do something similar here. If we need to build a VM, it's not hard for me to do (after all, the "executable instructions" are running on Jenkins), although on principle I'd prefer to run official released VMs from squeakvm.org and from Eliot's site if possible.
Note that work on both VMs was some time back and the issues I had would need to be revisited and evaluated.
Ken
On 10/10/2013 01:36 PM, David T. Lewis wrote:
Thanks a lot for this summary, it helps me a lot!
A couple of notes inline below.
Dave
On Thu, Oct 10, 2013 at 10:04:17AM -0500, Ken Causey wrote:
First our assets:
Hetzner - box2.squeak.org (85.10.195.197) - Our aging outgoing server that has been our singular server for many years now and hosted all online services.
Stats: ancient version of Debian, full hardware server, 1GB RAM, 150GB HD
Gandi.net - box3.squeak.org (173.246.101.237) - This server was originally allocated and setup at the request of Frank for use as a continuous integration server and hosts build.squeak.org, a Jenkins instance. As far as I am concerned this is still the primary purpose of this server. Currently this is also hosting our 'recovered' squeaksource.com. David can speak to details about the plan for this.
It sounds like box4 would be a better host for squeaksource.com. If so, I don't mind moving it from box3 to box4. I will need an account on box4 in order to do this.
I disagree and my reasoning is that in all probability squeaksource.com is 'bigger' than mailman and its associated infrastructure. I am concerned about whether box3 and box4 constitute sufficient resources for all of the services we wish to support. If squeaksource.com will fit on box3 which is the smaller of the two systems then it doesn't make sense to me to move it to box4.
Of course this assumes that squeaksource.com can be made to work on box3 alongside build.squeak.org. In any case I think it best to continue as is and make decisions about which services to position on which server when we have more information about how they actually fit and not just speculation.
By the way I very strongly agree with the position that squeaksource.com should be purely read-only at this point. In fact I'm becoming a bit of a nut and I find 'burn the disk packs' reverberating through my head regularly these days.
Stats: current and up to date Debian Squeeze (32bit), virtual server, 1GB RAM, 60GB HD
Gandi.net - box4.squeak.org (173.246.104.42) - This server is intended to be the primary replacement host for box2.squeak.org. I'll speak more to the status of this server below.
Stats: current and up to date Debian Wheezy (64bit), virtual server, 4GB RAM, 100GB HD
I repeatedly tried to build and install Cog but ran into multiple problems. Even beyond build problems one issue is that Cog is built to name the executable squeak and as such collides with the Classic VM. Having both VMs seems necessary given the age of some of the Squeak hosted services. Having Cog is desirable for more recent services and updates to the old ones.
I can help with this if you like. On my own PC, I am in the habit of just renaming the Cog script to /usr/local/bin/cog and installing the classic VM in its normal /usr/local/bin/squeak location. If nobody objects, I can do something similar here. If we need to build a VM, it's not hard for me to do (after all, the "executable instructions" are running on Jenkins), although on principle I'd prefer to run official released VMs from squeakvm.org and from Eliot's site if possible.
Did you really think it would be as simple as that?
I guess I wasn't clear. In my opinion everything that is not installed purely within a user's home directory should be installed as a Debian package. When it comes to items like Cog which have not yet been 'officially' packaged, we should do it ourselves. checkinstall provides a mechanism which makes it relatively easy to produce a Debian package which is good enough without too much effort. But it has limited flexibility. Another solution is to properly build a Debian package for Cog. This is more work and given the frequency with which Cog was, and I presume still is, changing this seemed like it would definitely not be worth the effort until Cog stabilizes.
I will create an account for you and email the information to you separately. (Actually your email system has been blocking me from emailing you directly for some time, probably my fault with letting my SPF record rot. I have hopefully fixed it now and perhaps I will not receive a failure message from sending this email.) However I did want you to understand my goal and perhaps a greater understanding of why I was having trouble. However I do wish you luck and it has been sometime since I looked at this problem, changes may have occurred which make this easier, and I may have developed tunnel vision.
Ken
box-admins@lists.squeakfoundation.org