Box-Admins

box-admins@lists.squeakfoundation.org

958 discussions

Re: [Board] Reachability of source.squeak.org and "alan" (files.squeak.org) is probably full again...
by leves 04 Mar '24

04 Mar '24

Hi Marcel, This time alan is not full, there's 80 GB free space on its disk. IIRC either the deletion strategy has been implemented by Yannis or a manual round of deletions happened. However the mailing lists' cron jobs are consuming significant amount of CPU and RAM. Swap size is ~5GB, so that could be a reason why source.squeak.org is slow/down. The monthly job has already consumed 40 CPU hours. Since it's python, it's single threaded, so that's been running for 40 hours. The hourly job was struggling as well, it's been running for over 13 minutes. However, the source.squeak.org image is also consuming 100% cpu, so there's probably some other issue with it. Also, squeaksource.com works, so it's clearly the source.squeak.org image that needs to be investigated. Btw, source.squeak.org is started with parameter -memory 2048m. I suppose -maxoldspace would be a better option to use instead. Levente On 2024. 02. 02. 16:45, Taeumel, Marcel wrote: > Hi board, hi box-admins -- > > five months ago, I discussed a deletion strategy of our CI builds for > files.squeak.org with Yannis. Unfortunately, he has not had the time to > implement it yet. > > Now, it seems, "alan" is full again. And we are experiencing problems > with source.squeak.org and Monticello again. > > We should bump "alan" with another 5 GB and get on with our deletion > strategy. A lot of old stuff does not even need a fancy script. Here is > my prior take on this, which I discussed with Yannis: > > Ansonsten hier die alten Files, die *gelöscht *werden können: > > * http://files.squeak.org/5.1beta/ > <http://files.squeak.org/5.1beta/> alles bis auf 16514 (32-bit und > 64-bit) > * http://files.squeak.org/5.2alpha/ > <http://files.squeak.org/5.2alpha/> alles bis auf 18184 (32-bit und > 64-bit) > * http://files.squeak.org/5.2beta/ > <http://files.squeak.org/5.2beta/> alles bis auf 18199 (32-bit und > 64-bit) > * http://files.squeak.org/5.3alpha/ > <http://files.squeak.org/5.3alpha/> alles bis auf 19256 (32-bit und > 64-bit) > * http://files.squeak.org/5.3beta/ > <http://files.squeak.org/5.3beta/> alles bis auf 19412 (32-bit und > 64-bit) > * http://files.squeak.org/6.0alpha/ > <http://files.squeak.org/6.0alpha/> alles bis auf 21752 (32-bit und > 64-bit) > * http://files.squeak.org/6.0beta/ > <http://files.squeak.org/6.0beta/> alles bis auf 22075 (32-bit und > 64-bit) > > Und für 6.1alpha (bzw. der aktuelle Trunk) würde ich *behalten *wollen: > > * pro Monat drei Builds anfang/mitte/ende (32-bit und 64-bit) > * den aktuellen Monat komplett bzw. die letzten 30 Tage komplett > > Und bitte irgendwo eine Liste führen, wo man manuell Builds vom > aktuellen Trunk hinzufügen kann, die niemals gelöscht werden dürfen. > Quasi pinning. Aber keine Ahnung, ob wir sowas mal brauchen. > Ich denke, dass nur der aktuelle http://files.squeak.org/trunk/ > <http://files.squeak.org/trunk/> so eine automatische Lösch-Strategie > braucht, weil tägliche Builds. den Rest machen wir manuell. > > Best, > Marcel

3 6

Re: Deletion Strategy for files.squeak.org/trunk
by leves 15 Jun '23

15 Jun '23

Hi Marcel, On 2023. 06. 07. 13:00, Marcel Taeumel wrote: > Hi Levente, hi box-admins -- > > We need a deletion strategy for files.squeak.org/trunk. Our disk seems > to be full again bc. source.squeak.org is not working at the moment. > Squeak > Update fails bc. we cannot download Monticello versions. > > Yannis (CC) agreed to implement such a strategy to keep the disk space > used manageable. Yet, he needs instructions and access to the > corresponding box. > > Until then, Levente, would you delete Trunk bundles 22096 to 22532? :-) Instead of deleting those files, I decided to add another 5GB to the disk, because we have far older files to delete. Trunk builds are more likely to be useful than those older files. To give you an example, here's the disk usage of each alpha directory. 217M 3.10alpha 4.0K 3.11alpha 11M 3.3alpha 668M 3.9alpha 606M 4.0alpha 34M 4.2alpha 65M 4.3alpha 35M 4.4alpha 81M 4.5alpha 25M 4.6alpha 21M 5.0alpha 10M 5.1alpha 18G 5.2alpha 57G 5.3alpha 11G 6.0alpha 38G 6.1alpha From 5.2, up and including 6.0, the sizes seems to be unnecessarily large to keep around. Levente

1 0

Re: [Box-Admins] [squeak-dev] trunk update stream appears to have become inaccessible
by Chris Muller 27 Dec '22

27 Dec '22

We appear to be all fixed, but I don't know why! Did someone do something? I was getting ready to append the following fix that would limit the output of that one operation to the most recent 5000 items. In SSVersionListing>>#setProjects:. setProjects: aCollection projects := aCollection. associations := Array streamContents: [ :stream | projects do: [ :project | self appendProject: project to: stream ] ]. associations sort: [ :x :y | x value timestamp > y value timestamp ]. "Limit to the last 5000 versions." associations := associations copyFrom: 1 to: (associations size min: 5000) I've had the above typed into the browser in the production image, ready to save, when I decided I wanted to observe the "broken-before --> fixed-after" state change, broke-before was (is) no longer broken! Strange. So, I will hold off then. If this fixes trunk development for now, I'd rather invite the problem back to reveal itself again for better understanding than put in any arbitrary fixes to "hide" it. I'm not working in trunk right now, if the problem returns, please email me again. - Chris On Tue, Dec 27, 2022 at 3:23 PM Thiede, Christoph < Christoph.Thiede(a)student.hpi.uni-potsdam.de> wrote: > Hi Chris, > > > sorry for the misunderstanding! Some use case I was thinking of would be > this: Somewhere in the mailing list archive, I find a reference to a very > old Monticello version that I need to check out in order to understand why > something was built or changed. Usually, I would open a repository browser > on Trunk and scroll down until I find that really old version, just because > it is more intuitive than using the API via efficient #versionNamed:. > Will this still be possible through the UI with your intended change, > regardless of the number of versions in the package? > > > Apart from that, I can imagine that such a limitation would worsen the > explorability of the database, but given that this is just meant to be a > workaround and we have an urgent situation (of course, CI bundles > and smalltalkCI for Trunk are broken as well) I don't think that this > should be a real issue. In the long term, maybe add a small notice to the > UI to indicate that the list is truncated. > > > Best, > > Christoph > ------------------------------ > *Von:* Chris Muller <ma.chris.m(a)gmail.com> > *Gesendet:* Dienstag, 27. Dezember 2022 22:03:57 > *An:* Thiede, Christoph > *Cc:* The general-purpose Squeak developers list; > box-admins(a)lists.squeakfoundation.org > *Betreff:* Re: [squeak-dev] trunk update stream appears to have become > inaccessible > > Hi Christoph, > >> From the perspective of someone who has only gotten to know SqueakSource >> from the client side: The possibility to enumerate *all* trunk versions is >> important for me in several situations. >> > Would you mention or describe one or more of those situations, because... > > >> It would be quite a shame if one could no longer access the full ancestry >> of a version or browse an ancient version of any package through the UI. >> > ... this change would not affect the above use cases. They could still be > accomplished via the "History" button, because it accesses those very old > versions *directly*, by name or UUID. What we're talking about here is > just the "list of files", NOT the list of ancestors, which we still have. > The lists in the Repository browsers wouldn't show everything in the > ancestry, but the ancestry is a better place to review that anyway. I'm > not convinced there is _any_ use case that needs a list of *all* Repository > files (Versions) since the beginning of time. If there is one, it's on an > eternal linear degradation path, and some alternative approach should at > least be considered. > >> The release notes generator in the ReleaseBuilder could also be affected, >> I think. >> > It shouldn't be. As above, it's in the ancestry. > > >> With a view to modern REST and GraphQL APIs, would it be possible to add >> support for some parameters to the requests in question to enable >> pagination or filter the retrieved information vertically (i.e., leave out >> some metadata)? >> > I'm sure it is, but right now I'm just looking for TSTTCPW for getting > trunk development back online. I think the round-robin with date-based > balancing sounds like a good possibility for something that could be done > quickly. > > Best, > Chris > >> >

1 0

Re: [Box-Admins] [squeak-dev] trunk update stream appears to have become inaccessible
by Chris Muller 27 Dec '22

27 Dec '22

> browse an ancient version of any package through the UI This one could be affected. However, it could easily be fixed in the client by moving it to the ancestry list menu, where it could do the direct (by uuid) access. I also just realized, for the round-robin, it would need to make sure to always include ALL that are NOT an ancestor of the newest. That way, we can see old things that might still want to be merged... >

1 0

Re: [Box-Admins] [squeak-dev] trunk update stream appears to have become inaccessible
by Chris Muller 27 Dec '22

27 Dec '22

Hi Christoph, > From the perspective of someone who has only gotten to know SqueakSource > from the client side: The possibility to enumerate *all* trunk versions is > important for me in several situations. > Would you mention or describe one or more of those situations, because... > It would be quite a shame if one could no longer access the full ancestry > of a version or browse an ancient version of any package through the UI. > ... this change would not affect the above use cases. They could still be accomplished via the "History" button, because it accesses those very old versions *directly*, by name or UUID. What we're talking about here is just the "list of files", NOT the list of ancestors, which we still have. The lists in the Repository browsers wouldn't show everything in the ancestry, but the ancestry is a better place to review that anyway. I'm not convinced there is _any_ use case that needs a list of *all* Repository files (Versions) since the beginning of time. If there is one, it's on an eternal linear degradation path, and some alternative approach should at least be considered. > The release notes generator in the ReleaseBuilder could also be affected, > I think. > It shouldn't be. As above, it's in the ancestry. > With a view to modern REST and GraphQL APIs, would it be possible to add > support for some parameters to the requests in question to enable > pagination or filter the retrieved information vertically (i.e., leave out > some metadata)? > I'm sure it is, but right now I'm just looking for TSTTCPW for getting trunk development back online. I think the round-robin with date-based balancing sounds like a good possibility for something that could be done quickly. Best, Chris >

1 0

Re: [Box-Admins] [squeak-dev] trunk update stream appears to have become inaccessible
by Chris Muller 27 Dec '22

27 Dec '22

> > > So, I changed the production server code to only return the last 1000 > of those 14000, it's "working" again. > > Just in case this wasn't obvious, updating your image still doesn't work > because many versions are missing. Round-robin or some date-based heuristic > would surely be a better (yet more complex) workaround. :-) > Yup. And, it would alleviate our property of unsustainability. I'm curious about the proxy configuration, but this has me wanting to come up with something in the app code in any case. We're just sending that ever-growing list over the wire hundreds, possibly thousands of times, every day. I like your idea of having age be a factor. The impact of such a change would occur in the Repository browser. There would be fewer Versions in the right-hand pane for each package. However, I think it would be able to use the ancestry to diff anything older than what the Repository browser showed, so it may not be a big deal.. Does anyone see any holes in this idea? - Chris >

1 0

Re: [Box-Admins] [squeak-dev] trunk update stream appears to have become inaccessible
by David T. Lewis 27 Dec '22

27 Dec '22

Thanks Chris, ++box-admins I opened an item on the Slack box-admins channel to follow up on this. Dave On Mon, Dec 26, 2022 at 11:02:16PM -0600, Chris Muller wrote: > Hi all, > > The server appeared to be running fine. I was able to vnc into the > image and everything appeared normal. Nothing abnormal in the log. > The only anomaly was it was taking up a lot of RAM (7 GB) which is > really abnormal, so I restarted it. After coming back up it's now > taking between 1-2 GB which is normal for this sized repository. > > However, the trunk directory listing is still failing! > > After considerable live debugging, I could find no errors or > abnormalities occuring in the Squeak image. But, dog gone > it, Seaside apps with continuations and dynamic vars are really > difficult to debug, so I may have missed something, but I don't think so). > > So, it *appears* at this point to be some sort of HTTP response size > limitation, possibly within the proxy server. Levente? There are > currently over 14000 files in 'trunk'. All the other projects have > far fewer files (e.g., 'inbox', 'treated', etc.) and appear to be > working. So, I changed the production server code to only return the > last 1000 of those 14000, it's "working" again. > > I'm afraid I'm at a loss for what else the problem might be, or how to > solve it, so, here are my suggestions: > > - any HTTP / Proxy experts know of any server configuration > parameters that could be affecting this? > - If so, can we adjust it? Or break up trunk into smaller projects? > - change the /ss code to only return one from each project > round-robin until a total of 1000 have been retrieved. But then we > couldn't access old versions except directly -- which may or may not > be a problem...? > > Thoughts? We're going to have to put our heads together on this one. > > > - Chris > > On Mon, Dec 26, 2022 at 8:10 PM David T. Lewis <lewis(a)mail.msen.com> wrote: > > > > On Mon, Dec 26, 2022 at 08:13:11PM -0500, David T. Lewis wrote: > > > On Mon, Dec 26, 2022 at 04:30:52PM -0800, tim Rowledge wrote: > > > > I can't do a trunk update of any of my squeak 6 images right now. This is the case for several 6.0 release images on several platforms and a freshly downloaded 6.1alpha system. > > > > > > > > The problem is that the securesocketstream>receiveData method gets a connection closed response. This isn't new and 'm sure I should remember more about it but... in fact the only notes I have that relate to the problem date back to '04 and I certainly don't remember any details. > > > > > > > > The especially annoying thing is that using https://source.squeak.org/trunk/ in a web browser works fine, so it probably isn't that the server is dead. Is this another stupid SSL issue? > > > > > > > > > > Confirmed, I'm seeing the same. It seems specific to source.squeak.org/trunk as > > > opposed to source.squeak.org/inbox. Something has changed quite recently, I'm > > > pretty sure that updates were working earlier today. > > > > > > > Opening a repository browser on trunk gives the ConnectionClosed symptom > > as described. So I tried it with an older Squeak 4.5 image, and it fails > > with different symptoms, the progress bar stops part way through retrieving > > the list of file names. > > > > This is not SSL related, it seems to be something on the server that is > > specifically related to the trunk repository, or possibly something > > related to the (size of?) list of file names being retrieved. > > > > Dave > >

1 0

Re: [Box-Admins] [squeak-dev] Should I put my unofficial V3 versions of Squeak 5.2 and 5.3 on files.squeak.org?
by David T. Lewis 23 Dec '22

23 Dec '22

Moving this to box-admins for assistance. I have a couple of images that I would like to preserve for posterity on files.squeak.org. They can go in a new folder under "various_images" files.squeak.org/various_images/squeak_V3_images/ Can someone on box-admins help me make this happen? I do not have access to the server but I can put the files out on a google drive if that works. I zipped up the files and put them here: https://drive.google.com/file/d/1wUZa4IHEkaTqzgwR3K1Wcwa9ckb7iVo2/view?usp=… Thanks, Dave On Sat, Jul 25, 2020 at 10:35:22AM -0700, tim Rowledge wrote: > Hard to think of any reason *not* to have them available. > > > On 2020-07-25, at 10:04 AM, David T. Lewis <lewis(a)mail.msen.com> wrote: > > > > A couple of years ago, I put a copy of a Squeak 5.2 image in V3 object > > memory format on the squeakvm.org server, and I linked to it from the > > project page at http://www.squeaksource.com/TrunkUpdateStreamV3. That link > > has gone dead, and I think I recall someone noticing it a recently. > > > > I still have the files, should I put them somewhere on files.squeak.org? > > I was thinking of putting them under the "various_images" folder: > > > > files.squeak.org/various_images/squeak_V3_images/ > > > > I also have a V3 image that is up to date with the Squeak 5.3 release, > > so I could put that on line as well. > > > > These images will run under an up-to-date interpreter VM or a Cog VM. They > > may be of interest to people doing VM work, benchmarking, or as a reference > > to understand the image-side differences between Spur and V3. > > > > Dave > > > > > > > tim > -- > tim Rowledge; tim(a)rowledge.org; http://www.rowledge.org/tim > How was Thomas J. Watson buried? 9 edge down. > > >

3 6

Reset Selflanguage mailing list admin password?
by Russell Allen 24 Nov '22

24 Nov '22

Hi, I'm trying to get the admin to the mailman interface for the selflanguage email list reset for myself as admin Russell Allen mail(a)russell-allen.com Marcel suggested I try this email address, Thanks! Russell -------- Original Message -------- Subject: Re: Self mailing list archive down? Date: Wed Nov 23 2022 21:32:50 GMT+1100 (Australian Eastern Daylight Time) From: "Marcel Taeumel" <marcel.taeumel(a)hpi.de> To: "Russell Allen" <mail(a)russell-allen.com> Hi Russel -- Admin to which interface? http://lists.selflanguage.org/mailman/listinfo ? Hmm... maybe ask box-admins(a)lists.squeakfoundation.org ? Best, Marcel Am 22.11.2022 00:49:56 schrieb Russell Allen <mail(a)russell-allen.com>: thanks Marcel! I changed the DNS and it is now working. However what I thought was my admin password isn’t working anymore- is there a mechanism to reset it? best Russell Sent from my phone On 21 Nov 2022, at 8:08 pm, Marcel Taeumel <marcel.taeumel(a)hpi.de> wrote: Hi Russell -- Somebody should update the A record for lists.selflanguage.org to 116.203.28.174. We already did that for lists.squeakfoundation.org. Best, Marcel Am 21.11.2022 09:55:27 schrieb Marcel Taeumel <marcel.taeumel(a)hpi.de>: Hi Tobias -- Domain is at HPI, right? PS C:\Users\Marcel> nslookup Standardserver: ADS2.hpi.uni-potsdam.de Address: 10.10.10.12 > selflanguage.org Server: ADS2.hpi.uni-potsdam.de Address: 10.10.10.12 Nicht autorisierende Antwort: Name: selflanguage.org Address: 149.28.205.228 > set type=ns > selflanguage.org Server: ADS2.hpi.uni-potsdam.de Address: 10.10.10.12 Nicht autorisierende Antwort: selflanguage.org nameserver = dns102.register.com selflanguage.org nameserver = dns101.register.com dns102.register.com internet address = 162.159.25.158 dns101.register.com internet address = 162.159.24.117 > set type=mx > selflanguage.org Server: ADS2.hpi.uni-potsdam.de Address: 10.10.10.12 selflanguage.org primary name server = DNS101.REGISTER.COM responsible mail addr = root.REGISTER.COM serial = 122110921 refresh = 10800 (3 hours) retry = 3600 (1 hour) expire = 604800 (7 days) default TTL = 3600 (1 hour) > Best, Marcel Am 21.11.2022 06:24:38 schrieb Tobias Pape <das.linux(a)gmx.de>: Ah great -.- Squeak has been moving servers due to SFC and RackSpace not getting along very well. Marcel, can you include Russell in the conversation? I presume it's only necessary to update DNS, but there might be more. Best regards -Tobias > On 21. Nov 2022, at 00:10, Russell Allen wrote: > > Hi Tobias, Jecel, > the mailing list archive at lists.selflanguage.org seems to be down > Is there someone I should reach out to who is managing it? > Thanks, > Russell > >

1 0

Re: [Box-Admins] [squeak-dev] Request to manually move the following SqueakSource versions
by Chris Muller 09 Oct '22

09 Oct '22

Hi Christoph, > unfortunately, I have already uploaded Regex-Core-ct.78 which merges Regex-Core-ct.74, > i.e., references it as an ancestor. I only noticed afterward that this > version has an incorrect encoding. So we seem to have two options: > > > > 1. Move Regex-Core-ct.74 to the Trunk indeed and hope that it won't > make any further problems in the future. (You can still view it from the > image, just not from source.squeak.org.) > 2. Remove Regex-Core-ct.78 from the trunk (move it to treated) and > create a new merge commit that excludes Regex-Core-ct.74. But I don't know > how this will influence the update stream and the images of people who have > already installed the latest updates. > > Option 2. Make a new .74 as .79 with the correct encoding. Then make a .80 that merges it all together. We will get rid of .74 and .78. If you're worried about development images of an early alpha, then add a postScript to the .80 that fixes their MCWorkingCopy's ancestry (by removing .74 and .78). And, Christoph, _please_ understand the cost/benefit balance when cultivating our Monticello model. Remember what is stored where, and how, and accordingly *keep the commit notes as close to one sentence as possible*! Look at Marcel's commit notes as a guide. Please write your implementation notes and examples *in the code*, and NOT in the commit notes. .74 is a tome of text that would've forever lived in everyone's images, never to be read again after next week. And, please do NOT copy-and-paste the descriptions of the merged versions into the new .80! > I'm sorry for the trouble. I don't know what is the best option ... What > do you think? > Thank you so much for all the great work you do, and please forgive me for nit picking about lengthy commit notes. I really do think it's important, or I wouldn't. Best, Chris

3 2

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Box-Admins