After fixing a bug that let the new server grow to 2.5GB RAM, it is now stable at 500MB RAM. I also discovered I can't apply patches live and save the image, because that means its saved with the RFB server running.
This large bulk load uncovered a bug when using Magma in direct-mode since Eliot fixed the process-scheduling to not cycle through equal-level processes by default -- Magma's flush-to-disk process was at equal priority and so it wasn't getting flushed! Spur really needs a way to cap its memory allocation like the old VM's had, so something like that would not cause problems for other processes on the server..
Disk space is so tight, I'll have to log in to delete unneeded commit log files to avoid us running out. Hopefully Rackspace can afford us a bigger disk.
On Tue, 30 Aug 2016, Chris Muller wrote:
After fixing a bug that let the new server grow to 2.5GB RAM, it is now stable at 500MB RAM. I also discovered I can't apply patches live and save the image, because that means its saved with the RFB server running.
Is that a problem? Will you start the RFB server using the USR2 signal?
This large bulk load uncovered a bug when using Magma in direct-mode since Eliot fixed the process-scheduling to not cycle through equal-level processes by default -- Magma's flush-to-disk process was at equal priority and so it wasn't getting flushed! Spur really needs a way to cap its memory allocation like the old VM's had, so something like that would not cause problems for other processes on the server..
Disk space is so tight, I'll have to log in to delete unneeded commit log files to avoid us running out. Hopefully Rackspace can afford us a bigger disk.
There's still more than 16 GB of space left. Some servers have less than that. :)
Levente
Hi Levente,
After fixing a bug that let the new server grow to 2.5GB RAM, it is now stable at 500MB RAM. I also discovered I can't apply patches live and save the image, because that means its saved with the RFB server running.
Is that a problem? Will you start the RFB server using the USR2 signal?
The RFB server is started upon image start (accepting only local connections). I like your idea, but it starts to become a cumbersome process if one must 1) forward the port, then 2) send a kill signal, then 3) open their VNC viewer to do the work, THEN 4) remember to do some other step to ensure they stop the RFB server.. Probably, the last step would be forgotten and we're right back to it running 24x7..
This large bulk load uncovered a bug when using Magma in direct-mode since Eliot fixed the process-scheduling to not cycle through equal-level processes by default -- Magma's flush-to-disk process was at equal priority and so it wasn't getting flushed! Spur really needs a way to cap its memory allocation like the old VM's had, so something like that would not cause problems for other processes on the server..
Disk space is so tight, I'll have to log in to delete unneeded commit log files to avoid us running out. Hopefully Rackspace can afford us a bigger disk.
There's still more than 16 GB of space left. Some servers have less than that. :)
Our existing repository is currently 17GB worth of mcz's and the Magma copy may need about the same by the bulk-load is done..
On Wed, 31 Aug 2016, Chris Muller wrote:
Hi Levente,
After fixing a bug that let the new server grow to 2.5GB RAM, it is now stable at 500MB RAM. I also discovered I can't apply patches live and save the image, because that means its saved with the RFB server running.
Is that a problem? Will you start the RFB server using the USR2 signal?
The RFB server is started upon image start (accepting only local connections). I like your idea, but it starts to become a cumbersome process if one must 1) forward the port, then 2) send a kill signal, then 3) open their VNC viewer to do the work, THEN 4) remember to do some other step to ensure they stop the RFB server.. Probably, the last step would be forgotten and we're right back to it running 24x7..
What I tried to suggest was to keep it running. I don't use the signal trick, because I don't see any problem with letting RFB run.
This large bulk load uncovered a bug when using Magma in direct-mode since Eliot fixed the process-scheduling to not cycle through equal-level processes by default -- Magma's flush-to-disk process was at equal priority and so it wasn't getting flushed! Spur really needs a way to cap its memory allocation like the old VM's had, so something like that would not cause problems for other processes on the server..
Disk space is so tight, I'll have to log in to delete unneeded commit log files to avoid us running out. Hopefully Rackspace can afford us a bigger disk.
There's still more than 16 GB of space left. Some servers have less than that. :)
Our existing repository is currently 17GB worth of mcz's and the Magma copy may need about the same by the bulk-load is done..
We can make a few more GBs. Btw, I thought Magma wouldn't copy the mczs.
Levente
On 31.08.2016, at 23:16, Levente Uzonyi leves@caesar.elte.hu wrote:
On Wed, 31 Aug 2016, Chris Muller wrote:
Hi Levente,
After fixing a bug that let the new server grow to 2.5GB RAM, it is now stable at 500MB RAM. I also discovered I can't apply patches live and save the image, because that means its saved with the RFB server running.
Is that a problem? Will you start the RFB server using the USR2 signal?
The RFB server is started upon image start (accepting only local connections). I like your idea, but it starts to become a cumbersome process if one must 1) forward the port, then 2) send a kill signal, then 3) open their VNC viewer to do the work, THEN 4) remember to do some other step to ensure they stop the RFB server.. Probably, the last step would be forgotten and we're right back to it running 24x7..
What I tried to suggest was to keep it running. I don't use the signal trick, because I don't see any problem with letting RFB run.
+1
This large bulk load uncovered a bug when using Magma in direct-mode since Eliot fixed the process-scheduling to not cycle through equal-level processes by default -- Magma's flush-to-disk process was at equal priority and so it wasn't getting flushed! Spur really needs a way to cap its memory allocation like the old VM's had, so something like that would not cause problems for other processes on the server..
Disk space is so tight, I'll have to log in to delete unneeded commit log files to avoid us running out. Hopefully Rackspace can afford us a bigger disk.
There's still more than 16 GB of space left. Some servers have less than that. :)
Our existing repository is currently 17GB worth of mcz's and the Magma copy may need about the same by the bulk-load is done..
We can make a few more GBs. Btw, I thought Magma wouldn't copy the mczs.
Levente
The RFB server is started upon image start (accepting only local connections). I like your idea, but it starts to become a cumbersome process if one must 1) forward the port, then 2) send a kill signal, then 3) open their VNC viewer to do the work, THEN 4) remember to do some other step to ensure they stop the RFB server.. Probably, the last step would be forgotten and we're right back to it running 24x7..
What I tried to suggest was to keep it running. I don't use the signal trick, because I don't see any problem with letting RFB run.
Ah, good.
Our existing repository is currently 17GB worth of mcz's and the Magma copy may need about the same by the bulk-load is done..
We can make a few more GBs. Btw, I thought Magma wouldn't copy the mczs.
It doesn't copy mcz's, but it does open them up to get the MCVersion object inside, and adds it (along with all of its referenced MCDefinition objects) directly to a big Collection in the database. The MCDefinitions are canonicalized, so there is some space savings there, however, the source code is not compressed like in an MCZ, so the amount of space it will ultimately require is very hard to predict.
I'm keeping my eye on it, and since we don't need any of the files in the /home/squeaksource/webserver/ss.magma/commits directory, a lot of space is recoverable from there, so I think we'll be fine..
Best.
box-admins@lists.squeakfoundation.org
-
Chris Muller -
Chris Muller -
Levente Uzonyi -
Tobias Pape