[squeak-dev] All mczs have disappeared from source.squeak.org
rabbit at callistohouse.org
Fri Oct 7 15:45:16 UTC 2022
Great! I too look forward to hearing Tips and Guidance Welcome! I
totally and wholeheartedly agree! Welcome!
I started thinking about persistence. Kafka would excel at this, fault
tolerant, replicated and durable. I am unsure whether a directory of
files could be used as partition persistence...perhaps the Kafka events
sector block written are links that could be used by an Installer to
install a package in the squeaksource directory structure.
Further, I thought the way to approach ameliorating the shortage of FDs,
which max out per machine, is to use a redirector to load balance the
servicing machines and having a distributed image would be helpful, as
well as a multi-threaded vm running separate Vats on different native
threads, But the FD shortage issue is a big concern for my system as well.
I look to use Kafka replication with EventualLinda, yet without
persistence, and eventual references and whenReactors: to use
Capabilities-sending style. With remote promises it becomes distributed
amount Vats. As FDs are machine restricted, I'm interested in the many
Vat cluster experimentation. I wish to send replicated double-encrypted
traffic through third party bridging services and have more than 1
replicated PartitianSession separate stacks to different vats, talking
to the same unique receivers of services responding to messages from
senders. A Replicating Session would balance the PartitianSessions to
*ensure fault-tolerant* eventual message sends, responses and GC
traffic, for remote promises. The services would have multiple
connections to it from the same sender and using a distributed
EventualLinda the broadcast consumer service instances running as a
distributed fail-safe application. I.E. SqueakSource.
It's all going to take a minute! ^,^
Tips and guidance welcome.
Have a good one; keep it, light.
. .. … ‘…^,^
Sent from Callisto House :: decentralized mobile homeless solutions
On 10/7/22 11:06, David T. Lewis wrote:
> On Fri, Oct 07, 2022 at 10:09:50AM -0400, rabbit wrote:
>> Hi Levente,
>> Is this the same machine running SqueakSource? I get frequent intermittent
>> failures with accessing projects there (SqueakSource.com). If it were also
>> running out of FDs and at 100% CPU, that would explain it. It???s frustrating
>> me as I install a bunch of packages off SS, and it fails in the middle often.
> No, thehttps://squeaksource.com andhttps://source.squeak.org services
> are running on different machines.
> I suspect (but do know know for sure) that the intermittent issues
> on squeaksource.com are related to the server image exporting its
> repository data on data.obj file. The code that does this has changed
> in recent years and to be honest I don't know how it works. Specifically,
> I do not know what actually triggers the repeated "save" operations
> that seem to be causing problems. Tips and guidance welcome.
> With respect to exhausing file descriptors on source.squeak.org, I
> added a monitor process on squeaksource.com (the other service) that
> automatically restarts the image if it is getting too low on file
> descriptors. We could probably adapt this to work on source.squeak.org
> also. The code is inhttp://www.squeaksource.com/SSImageInit and I
> can work with Chris on it if the file descriptor issue turns out
> to be a recurring problem.
> Here is how the file descriptor monitor works on squeaksource.com.
> It would need to be slightly modified for source.squeak.org because
> on that server we never want to save the image file, so probably
> just change "SmalltalkImage current snapshot: true andQuit: true]"
> to "SmalltalkImage current snapshot: false andQuit: true]" in the
> method below.
> SSImageInit class>>startSocketMonitorProcess
> "Socket leak monitor process. If the number of open file descriptors in the
> VM process gets dangerously close to the per-process limit (normally 1024),
> we are at risk of putting the image into a state in which it cannot accept
> connections, and that may not allow a clean recovery even after an image
> restart. Thus if the file descriptor count exceeds a threshold of 800, save
> the image and exit. Assume that a supervisory script will detect the image
> exit and initiate a restart."
> | vmFileCount |
> self stopSocketLeakMonitorProcess.
> SocketLeakMonitorProcess :=
> [[vmFileCount := (FileDirectory on: '/proc/', OSProcess thisOSProcess pid asString, '/fd') entries size.
> "OSProcess trace: DateAndTime now asString, ' squeakvm has ', vmFileCount asString, ' open file descriptors'."
> vmFileCount > 800 ifTrue: [
> OSProcess trace: 'Too many open file handles, save image and exit'.
> "Save the image, exit and wait for the supervisory script to restart"
> WorldState addDeferredUIMessage: [SmalltalkImage current snapshot: true andQuit: true]].
> (Delay forSeconds: 3 * 3600) wait] repeat] fork name: 'the Socket leak monitor'.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Squeak-dev