I was able to get VNC access to the image and look at a process browser, see attached.

I think that a good thing to do would be:

1) connect via VNC

2) save the image under a different name for later evaluation

3) immediately exit the image and let systemd restart it automatically

I will be away for a couple of hours and cannot follow up now, but will check back later.

Dave

On 2024-02-02 17:25, lewis@mail.msen.com wrote:

Adding Chris in case he may be able to take a quick look at the source.squeak.org image before we restart it.

The fix will be to restart the squeaksource service on andreas.box.squeak.org. It is using 100% cpu right now. A VNC connection to the image will probably show something interesting.

I am away with limited access but I will try to check back in an hour or so.

Dave

On 2024-02-02 16:24, leves wrote:

Hi Marcel,

This time alan is not full, there's 80 GB free space on its disk. IIRC either the deletion strategy has been implemented by Yannis or a manual round of deletions happened.

However the mailing lists' cron jobs are consuming significant amount of CPU and RAM. Swap size is ~5GB, so that could be a reason why source.squeak.org is slow/down.
The monthly job has already consumed 40 CPU hours. Since it's python, it's single threaded, so that's been running for 40 hours.
The hourly job was struggling as well, it's been running for over 13 minutes.

However, the source.squeak.org image is also consuming 100% cpu, so there's probably some other issue with it. Also, squeaksource.com works, so it's clearly the source.squeak.org image that needs to be investigated.

Btw, source.squeak.org is started with parameter -memory 2048m.
I suppose -maxoldspace would be a better option to use instead.

Levente

On 2024. 02. 02. 16:45, Taeumel, Marcel wrote:
Hi board, hi box-admins --

five months ago, I discussed a deletion strategy of our CI builds for files.squeak.org with Yannis. Unfortunately, he has not had the time to implement it yet.

Now, it seems, "alan" is full again. And we are experiencing problems with source.squeak.org and Monticello again.

We should bump "alan" with another 5 GB and get on with our deletion strategy. A lot of old stuff does not even need a fancy script. Here is my prior take on this, which I discussed with Yannis:

Ansonsten hier die alten Files, die *gelöscht *werden können:

* http://files.squeak.org/5.1beta/
<http://files.squeak.org/5.1beta/> alles bis auf 16514 (32-bit und
64-bit)
* http://files.squeak.org/5.2alpha/
<http://files.squeak.org/5.2alpha/> alles bis auf 18184 (32-bit und
64-bit)
* http://files.squeak.org/5.2beta/
<http://files.squeak.org/5.2beta/> alles bis auf 18199 (32-bit und
64-bit)
* http://files.squeak.org/5.3alpha/
<http://files.squeak.org/5.3alpha/> alles bis auf 19256 (32-bit und
64-bit)
* http://files.squeak.org/5.3beta/
<http://files.squeak.org/5.3beta/> alles bis auf 19412 (32-bit und
64-bit)
* http://files.squeak.org/6.0alpha/
<http://files.squeak.org/6.0alpha/> alles bis auf 21752 (32-bit und
64-bit)
* http://files.squeak.org/6.0beta/
<http://files.squeak.org/6.0beta/> alles bis auf 22075 (32-bit und
64-bit)

Und für 6.1alpha (bzw. der aktuelle Trunk) würde ich *behalten *wollen:

* pro Monat drei Builds anfang/mitte/ende (32-bit und 64-bit)
* den aktuellen Monat komplett bzw. die letzten 30 Tage komplett

Und bitte irgendwo eine Liste führen, wo man manuell Builds vom aktuellen Trunk hinzufügen kann, die niemals gelöscht werden dürfen. Quasi pinning. Aber keine Ahnung, ob wir sowas mal brauchen.
Ich denke, dass nur der aktuelle http://files.squeak.org/trunk/ <http://files.squeak.org/trunk/> so eine automatische Lösch-Strategie braucht, weil tägliche Builds. den Rest machen wir manuell.

Best,
Marcel