We now have access to a complete snapshot copy of SqueakSource. I am
in the process of retrieving the files to my personal PC (not yet complete,
I'm still trying to get a clean copy of one large 15GB file), and I will
now need a place to put them on squeak.org.
I need some help to proceed. We need a server with 20 to 30 GB of file space
available for the SqueakSource files, and with memory and CPU resources
similar to whatever we are currently doing for source.squeak.org.
Can someone help, either by giving me access to an existing server, or by
letting me know what is available? I will need a user account (lewis) similar
to my current account on build.squeak.org, and I will need sudo privilege
to set up an account for squeaksource and to set up the runtime environment.
In addition it would be helpful if I could get enough access to the
source.squeak.org server so that I can look at it and figure out how that
server is configured. Even better would be if the source.squeak.org
server has enough capacity (mainly disk space) to also host squeaksource,
in which case we can put both of them on the same server.
I would like to do this as soon as possible (tomorrow perhaps?) so any
assistance and pointers will be gratefully received.
Thanks,
Dave
----- Forwarded message from Andrea Caracciolo <caracciolo(a)iam.unibe.ch> -----
From: Andrea Caracciolo <caracciolo(a)iam.unibe.ch>
Date: Fri, 23 Aug 2013 09:41:33 +0200
Subject: Re: squeaksource.com transition to new server
To: "David T. Lewis" <lewis(a)mail.msen.com>
Cc: Fabrizio Perin <perin(a)iam.unibe.ch>,
Andrei Vasile Chis <chisvasileandrei(a)gmail.com>,
Bert Freudenberg <bert(a)freudenbergs.de>,
Andrea CARACCIOLO <caracciolo(a)iam.unibe.ch>,
Oscar Nierstrasz <oscar(a)iam.unibe.ch>,
Chris Muller <asqueaker(a)gmail.com>, denker(a)acm.org
Hi David,
We have made all the files available at the following url:
http://scg.unibe.ch/download/SSdump/
username:
password:
Let me know if you need anything else.
> we would switch DNS entries at some agreed time after the move has been
successfully completed.
I made a quick WHOIS lookup and it seems that the domain is owned by Marcus
Denker (in CC).
I will keep the site running until you manage to set everything up on your
side.
Please keep me informed.
Cheers
_____________________________
Andrea Caracciolo - caracciolo(a)iam.unibe.ch
Software Composition Group
University of Bern
On Mon, Aug 26, 2013 at 07:07:43PM +0000, Pape, Tobias wrote:
>
> Which SqS are you referring to? Fabrizio once send Andreas and Me a (then current) image of the squeaksource.com (SS.C) and
> IIRC, it just had VNC running, so you probably are referring to source.squeak.org (S.S.O)?
> Sorry, I also do not understand what you mean by the original design being ?hacked? for image-based persistence.
The image for squeaksource.com works as you say, with the VNC server running
entirely in the image. I a running a test copy right now on build.squeak.org:8888,
and you can connect to it with web browser, Monticello, or VNC client on display 1
with VNC password "Squeak99" (*).
> Am 26.08.2013 um 19:30 schrieb Chris Muller <ma.chris.m(a)gmail.com>:
>
> >
> > Such a misfortune could wipe out the entire community if we were unable to recover from a backup.
> >
Really?!? Wow, first Y2K and now this ;-)
Dave
(*) These messages are archived, so I will change the VNC password and/or take
the test image down within the next 24 hours.
>
> > As Dave mentioned, the immediate priority is simply ensuring continued
> operation of squeaksource.com. He is heading up the transition to a new
> server this week whether that's box3, box4 (Gandi) or a server hosted by
> Robert's group. If we cannot get it running on a new image and/or VM
> immediately, we'll just have to deal with the crashes/lockups until we can.
> >
> > While I think Squeak should host its own services, unfortunately what we
> have right now is unacceptably dangerous. It appears someone has hacked
> our SqueakSource implementation to bypass its original design for
> persistence to use "image-based" persistence. Image-based persistence, in
> itself, would be fine, except for the fact that our Linux-based SS system
> scripts carelessly KILL the SS process by PID as a means of opening the
> image under VNC -- potentially when the image could be in a half-saved
> state!
>
> Which SqS are you referring to? Fabrizio once send Andreas and Me a (then
> current) image of the squeaksource.com (SS.C) and
> IIRC, it just had VNC running, so you probably are referring to
> source.squeak.org (S.S.O)?
Sorry, I also do not understand what you mean by the original design
> being “hacked” for image-based persistence.
>
Yes, I was referring to SSO, which is running a slightly newer/forked
version of SqueakSource and using the save-the-image persistence.
SSO is also running on an old version of Squeak and old VM. Whatever is
good for the SS upgrade will be good for SSO and vice-versa.
> Such a misfortune could wipe out the entire community if we were unable
> to recover from a backup. Therefore, I fully support your idea to move to
> SqueakSource3 running on GemStone.
> >
> > But there is one more option. Last month, Tim Rowledge expressed
> interest in having a more-capable SqueakSource that could support what
> Magma-based MC repositories provide today:
> >
> >
> http://lists.squeakfoundation.org/pipermail/squeak-dev/2013-July/172243.html
> >
> > The advantage to this approach, besides having a working server example
> based on Squeak, is that all MCDefinitions across all Versions of each
> SSProject would be canonicalized and indexed inside the repository, so that
> the 'browse mc versions' and 'browse mc origin' functions which are
> afforded by Magma-based MCRepository's would be available on the trunk
> repository for everyone.
>
> I had the Idea (but not yet enough pressure) to provide similar features
> for SS3.
>
McModel is self-contained. Besides substituting an RcDictionary for the
MagmaPreallocatedDictionary, you might be able to lift it straight from
Magma. It's indexing is sufficient to provide the minimum MCRepository
subclass-responsibility API, plus the aforementioned (#historyOf:
aMCDefinition) and (#originOf: aMCDefinition) functions, all without MC
even knowing it exists.
> > I've already begun the work for this, but if there is no interest in
> this, I will simply bow out and support the idea that we go with the
> GemStone solution.
>
> SS3 already cracks open MCZs for determining diffs, browsing code, finding
> parents, reading
> version informations. It would be easy to just cache this information and
> provide links inbetween.
> Remember, however, that both, a Magma-Based solution or a SS3-feature
> would have to respect
> acls in the sense, that you shouldn't be able to browse into a
> non-readable project by means of
> 'browse mc origin'.
>
Yes, I'd planned for each SSProject to have its own McModel instance, so
access is allowed only through the existing Project level ACL.
We have ODBMS' we should use them to exceed what we had before with only
the filesystem.
- Chris
PS -- I attached McModel.st.gz in case you're interested.
When I try to ssh into box3 or box4 as "chrismuller" it's asking me for a
password. I also tried as "cmm", same result.
Could someone please help me gain access? My public key is attached.
Thanks.
As Dave mentioned, the immediate priority is simply ensuring continued
operation of squeaksource.com. He is heading up the transition to a new
server this week whether that's box3, box4 (Gandi) or a server hosted by
Robert's group. If we cannot get it running on a new image and/or VM
immediately, we'll just have to deal with the crashes/lockups until we can.
While I think Squeak should host its own services, unfortunately what we
have right now is unacceptably dangerous. It appears someone has hacked
our SqueakSource implementation to bypass its original design for
persistence to use "image-based" persistence. Image-based persistence, in
itself, would be fine, except for the fact that our Linux-based SS system
scripts carelessly KILL the SS process by PID as a means of opening the
image under VNC -- potentially when the image could be in a half-saved
state!
Such a misfortune could wipe out the entire community if we were unable to
recover from a backup. Therefore, I fully support your idea to move to
SqueakSource3 running on GemStone.
But there is one more option. Last month, Tim Rowledge expressed interest
in having a more-capable SqueakSource that could support what Magma-based
MC repositories provide today:
http://lists.squeakfoundation.org/pipermail/squeak-dev/2013-July/172243.html
The advantage to this approach, besides having a working server example
based on Squeak, is that all MCDefinitions across all Versions of each
SSProject would be canonicalized and indexed inside the repository, so that
the 'browse mc versions' and 'browse mc origin' functions which are
afforded by Magma-based MCRepository's would be available on the trunk
repository for everyone.
I've already begun the work for this, but if there is no interest in this,
I will simply bow out and support the idea that we go with the GemStone
solution.
On Mon, Aug 26, 2013 at 7:10 AM, Pape, Tobias <
Tobias.Pape(a)hpi.uni-potsdam.de> wrote:
> Am 24.08.2013 um 02:26 schrieb David T. Lewis <lewis(a)mail.msen.com>:
>
> > On Fri, Aug 23, 2013 at 03:18:19PM -0500, Chris Muller wrote:
> >> Hi! Ok so there are a couple of questions and potential answers
> brewing.
> >> How to host? Where to host?
> >>
> >> David Lewis expressed that we should recover the actual SS instance and
> >> stabilize it.
> >
> > My immediate concern is to move the existing squeaksource.com to a new
> > home that is stable, supportable, and sustainable for the long term. My
> > hope is to accomplish that migration within days, not weeks. Once that
> > has been completed, all options should be open as far as I'm concerned.
> > But for now my only objective is to achieve operational stability and a
> > sustainable support process for squeaksource.com.
> >
> > Just to check my understanding, the reference to hosting by Robert
> Hirschfeld's
> > group is for future consideration, and not something that I need to worry
> > about to the migration plan for the next few days, is that right?
>
> Well Oscar Nierstrasz is commit to us (Robert’s group) in a couple of weeks
> and would be able to bring a harddrive with all files… (image+mcz's)
>
> >
> >> This may include bringing it up to a more modern,
> >> Cog-supported Squeak image. I am working on doing this for
> >> source.squeak.org and so I hope that effort can overlap this one.
> >>
> >
> > Yes for sure, it is important to get source.squeak.org and
> squeaksource.com
> > onto a common code base and support structure in order to minimize the
> total
> > support requirement. One way or another, I am quite confident that this
> will
> > resolve the reliability and performance problems that have plagued
> squeaksource
> > in the past.
> >
> > But first things first, I'm still trying to copy files, including well
> over
> > 15 GB of compressed project files. Firefox says it might be done after
> another
> > 23 hours and 10 minutes or so ... then I'll need a box on squeak.org to
> unpack
> > them on.
>
> what files are you copying from where? Are you talking about
> source.squeak.org
> org squeaksource.com?
>
> All that said, I maintained an image-based Squeaksource-installation a
> couple
> of years ago here at HPI and we ran into lockup-troubles (I figure they
> are quite
> similar to those squeaksource.com experienced over the past year) at
> least once
> if not twice a day. I tracked the problem down to Socket-related problems
> (one
> end closed, the other not, image not responding henceforth), but they were
> not
> deterministically reproducible.
> That was why I started participating in migrating SqS to Gemstone,
> Initially
> creating SqueakSource2, but soon changed to Squeaksource3.
>
> From my experiences, I would _strongly_ advice not to setup a new
> SqueakSource
> based on a SqueakVM that still has this socket problem (no, it is not the
> wiggle-the-mouse-and-it-works-again bug).
>
> I am currently devoting my free time to bring GemStone's Monticello-support
> up to par with Squeaks (read: add missing support for ScriptDefinitions),
> which would make SS3 on GemStone feature-wise a superset of current
> squeaksource.com and source.squeak.org while at the same time providing
> a higher reliability (esp, compared to squeaksource.com).
>
> Best
> -Tobias
>
> --
> Tobias Pape
> Doktorand im Fachgebiet Software-Architekturen
> http://www.hpi.uni-potsdam.de/swa/
>
> Hasso-Plattner-Institut für Softwaresystemtechnik GmbH
> Universität Potsdam
> Prof.-Dr.-Helmert-Str. 2-3
> D-14482 Potsdam, Germany
> Amtsgericht Potsdam, HRB 12184
> Geschäftsführung: Prof. Dr. Christoph Meinel
>
>
Hi Tobias,
I was referring only to the existing squeaksource.com service, not to future
directions. The SCG group has kindly provided a full copy of the current system,
and this is being moved to squeak.org for operational support. Discussion is at:
http://lists.squeakfoundation.org/pipermail/box-admins/2013-August/date.html
I apologize if I mixed up some unrelated issues here. I am not involved in
future development directions and I did not intend to cause confusion.
Thanks for the tip regarding VM issues, I will make sure that we use an
up-to-date VM.
Dave
On Mon, Aug 26, 2013 at 12:10:36PM +0000, Pape, Tobias wrote:
> Am 24.08.2013 um 02:26 schrieb David T. Lewis <lewis(a)mail.msen.com>:
>
> > On Fri, Aug 23, 2013 at 03:18:19PM -0500, Chris Muller wrote:
> >> Hi! Ok so there are a couple of questions and potential answers brewing.
> >> How to host? Where to host?
> >>
> >> David Lewis expressed that we should recover the actual SS instance and
> >> stabilize it.
> >
> > My immediate concern is to move the existing squeaksource.com to a new
> > home that is stable, supportable, and sustainable for the long term. My
> > hope is to accomplish that migration within days, not weeks. Once that
> > has been completed, all options should be open as far as I'm concerned.
> > But for now my only objective is to achieve operational stability and a
> > sustainable support process for squeaksource.com.
> >
> > Just to check my understanding, the reference to hosting by Robert Hirschfeld's
> > group is for future consideration, and not something that I need to worry
> > about to the migration plan for the next few days, is that right?
>
> Well Oscar Nierstrasz is commit to us (Robert?s group) in a couple of weeks
> and would be able to bring a harddrive with all files? (image+mcz's)
>
> >
> >> This may include bringing it up to a more modern,
> >> Cog-supported Squeak image. I am working on doing this for
> >> source.squeak.org and so I hope that effort can overlap this one.
> >>
> >
> > Yes for sure, it is important to get source.squeak.org and squeaksource.com
> > onto a common code base and support structure in order to minimize the total
> > support requirement. One way or another, I am quite confident that this will
> > resolve the reliability and performance problems that have plagued squeaksource
> > in the past.
> >
> > But first things first, I'm still trying to copy files, including well over
> > 15 GB of compressed project files. Firefox says it might be done after another
> > 23 hours and 10 minutes or so ... then I'll need a box on squeak.org to unpack
> > them on.
>
> what files are you copying from where? Are you talking about source.squeak.org
> org squeaksource.com?
>
> All that said, I maintained an image-based Squeaksource-installation a couple
> of years ago here at HPI and we ran into lockup-troubles (I figure they are quite
> similar to those squeaksource.com experienced over the past year) at least once
> if not twice a day. I tracked the problem down to Socket-related problems (one
> end closed, the other not, image not responding henceforth), but they were not
> deterministically reproducible.
> That was why I started participating in migrating SqS to Gemstone, Initially
> creating SqueakSource2, but soon changed to Squeaksource3.
>
> From my experiences, I would _strongly_ advice not to setup a new SqueakSource
> based on a SqueakVM that still has this socket problem (no, it is not the
> wiggle-the-mouse-and-it-works-again bug).
>
> I am currently devoting my free time to bring GemStone's Monticello-support
> up to par with Squeaks (read: add missing support for ScriptDefinitions),
> which would make SS3 on GemStone feature-wise a superset of current
> squeaksource.com and source.squeak.org while at the same time providing
> a higher reliability (esp, compared to squeaksource.com).
>
> Best
> -Tobias
>
> --
> Tobias Pape
> Doktorand im Fachgebiet Software-Architekturen
> http://www.hpi.uni-potsdam.de/swa/
>
> Hasso-Plattner-Institut f?r Softwaresystemtechnik GmbH
> Universit?t Potsdam
> Prof.-Dr.-Helmert-Str. 2-3
> D-14482 Potsdam, Germany
> Amtsgericht Potsdam, HRB 12184
> Gesch?ftsf?hrung: Prof. Dr. Christoph Meinel
>
Hi! Ok so there are a couple of questions and potential answers brewing.
How to host? Where to host?
David Lewis expressed that we should recover the actual SS instance and
stabilize it. This may include bringing it up to a more modern,
Cog-supported Squeak image. I am working on doing this for
source.squeak.org and so I hope that effort can overlap this one.
This is why I will also be pressing you with questions about latest Seaside
-- right now the old SS is running on Seaside 2, not 3.
Based on this revitalization experience, we can decide whether we want to:
- make it totally read-only
- give up and port it to SqueakSource3 running on GemStone.
Then there is the question of hosting. It sounds like we may have a couple
of options between Robert Hirschfeld's group and the servers provided for
Squeak by the SFC. Wonderful!
Until the revitalization work is done, we may just have to endure
restarting it occasionally.
Everyone please chime in. Is this a good plan?
- Chris
On Fri, Aug 23, 2013 at 5:56 AM, Pape, Tobias <
Tobias.Pape(a)hpi.uni-potsdam.de> wrote:
> Hey Chris
>
> before getting back to Oscar, how should we proceed?
> The one big question probably is, where should the server
> be located?
> I bet Robert Hirschfeld would not mind hosting the server
> at our group, but I cannot promise that. However, he has
> expressed interest in it. We also should include Bert
> in the conversation at some point.
> The point why I asked Oscar is, that he will be at
> our site soon and could bring the data physically
> with him :)
>
> Best
> -Tobias
>
> PS: I received a Google Drive share invitation from you.
> What was the reason?
>
> --
> Tobias Pape
> Doktorand im Fachgebiet Software-Architekturen
> http://www.hpi.uni-potsdam.de/swa/
>
> Hasso-Plattner-Institut für Softwaresystemtechnik GmbH
> Universität Potsdam
> Prof.-Dr.-Helmert-Str. 2-3
> D-14482 Potsdam, Germany
> Amtsgericht Potsdam, HRB 12184
> Geschäftsführung: Prof. Dr. Christoph Meinel
>
>
>
On 14 May 2013 11:02, <squeak-dev-noreply(a)lists.squeakfoundation.org> wrote:
>
> Dave Lewis uploaded a new version of SystemTracing to project SystemTracing:
> http://www.squeaksource.com/SystemTracing/SystemTracing-dtl.24.mcz
>
> ==================== Summary ====================
We're seeing a whole lot of dups. What's wrong with the infrastructure?
frank