[squeak-dev] All mczs have disappeared from source.squeak.org

rabbit rabbit at callistohouse.org
Fri Oct 7 19:29:16 UTC 2022


I spouted more design over in the Kafka channel of the Confluent slack workspace…YMMV! 🐇

—- message 1 —-
I just received this response and was told to discuss Kafka-core, here.

I have in mind something a little different. I’ll have to roll my own brokers and implement myself. I’ll explain. Tips and guidance welcome.

I want replicated comms (request, response, GC) such that a remote object can handle some network disruptions in my distributed capabilities system, by relying on replicated traffic. This is live active communications, so no store and forwards, no persistence, different flavor of replication. No elected leader … I’m trying to come up with an effective design.

As a sender contacts a remote receiver, by lookup or introduction, a number of connections (ChannelSessions) establish a Bridging) connection through a third-party BridgingBroker, by request to a DistributionBroker. The initiating client asks the DistributionBroker for a replicationFactors worth of ChannelSessions through different BridgingBrokers. When informed by the DistributionBroker (who will also send a hookUpRequest to the receiver) both will connect to the BridgingBroker and connect through a known nonce.

The BridgeService transfers traffic between its 2 ChannelSessions (sender/receiver). A second encryption negotiation occurs so these ChannelSessions are doubly encrypted through the third-party bridging intermediary.

So if the replicationFactor is 3, 3 separate ChannelSessions will be established through 3 BridgingBrokers. The sender and receiver each have 3 connections between them. Without persistence, no leader needed. Asks become asks that the other end received a particular message, no matter which bridge it went through. So no quorum needed, either. Each end will have a ReplicatedChannelSession to multiplex out/in the traffic.

The other aspect I’m thinking of is replicating the “Consumers” (Receiver ReplicatedChannelSession) using a leader and leader-election. Such that if primary Consumer’s rack falls over, it will activate a replicaConsumer.I’m thinking this requires each message to be processed in full before sending a response.

Andso clients receive comms ACKs when a Consumer/receiver gets the message and the response resolution message confirms successful processing, who’s resolution message also is multiplexed over the 3 ChannelSessions.

Fault-tolerant communications? DDOS attack resiliate? 

Tips and guidance welcome! 🐰

—- message 2 —-

Nevermind the Consumer replication confusion. As each Consumer is transactional in the sense that completion of processing a message, it remains state-less and all side effects are committed to storage (Hadoop…). 

So it is just the scale-out of concurrent Consumers to consume multiple senders. The only thing is there are no partitions so how does a sender connect to a particular consumer? It’s a naming service issue when the DistributionBroker is told to multiplex to a particular Consumer. The DistributionBrokers have client presence awareness. There needs to be a NamingBroker to do selection and load balancing and roundRobin or some other strategy for consumer selection by nameIdendity 

    camp := namingBroker connectToService: ‘/callistohouse.org: 2210/HomelessCamp/ZionOfBabelTowers#3’.
   (camp 
        scheduleResident: ‘rabbit’
        withReservationSpec: ‘dates: 12/23 - 12/26’ asTuple)
            then: [:confirmation | self reportConfirmation: confirmation].



Have a good one; keep it, light.
Kindly,
rabbit
. .. … ‘…^,^


Sent from Callisto House :: decentralized mobile homeless solutions



> On Oct 7, 2022, at 13:38, rabbit <rabbit at callistohouse.org> wrote:
> 
> I got off track, I was thinking about persistence….
> 
> 
> 
> 
> 
>>> On Oct 7, 2022, at 11:45, rabbit <rabbit at callistohouse.org> wrote:
>>> 
>> Great! I too look forward to hearing Tips and Guidance Welcome! I totally and wholeheartedly agree! Welcome!
>> 
> 
> +1!!!
>> I started thinking about persistence. Kafka would excel at this, fault tolerant, replicated and durable. I am unsure whether a directory of files could be used as partition persistence...perhaps the Kafka events sector block written are links that could be used by an Installer to install a package in the squeaksource directory structure. 
>> 
> Hadoop durable replicated partitions are spread across replicated brokers. I was thinking we would replicate the actual SS directory contents of all projects/packages into Hadoop. The SS brokers could be replicated and monitored for restart or replica switch. In this way I’m thinking of replicating Kafka Consumers talking to Hadoop, writes and reads of MCZ & MCM  files. This for good FD distribution as multiple web servers could also go wide. 
> 
> Go wide with connections, consumers and persistence.
> 
> 
>>>>>> Capabilities-sending style.
> This has zero to do with Kafka replication. It’s properly a semantics aspect of the core local promises…
> 
>   | ssDir sssEventual ssCryptoEventual ssPromisesEventual |
>   “In the machine serving ss files from it’s“ 
>   “projectDirectory/packageFile structure…”
>   ssDir := SqueakSourceDirectory on: ‘~/tribe/ssProjectDirectories/‘.
>   sssEventual := (SqueakSourceServer directory: ssDir) eventual.
>  
>   “In a remote Vat, given the above server behind a web interface”
>   “Get a remote eventual for the”
>   “SqueakSourceServer and ask for projects“
>   “from it and send retrieve: or publish:“
>   “messages remotely, to the projects.”
> 
>   (ssCryptoEventual := (RemoteServerInstaller on: ‘https://SqueakSource.com/squeakSourceServer/‘)
>       project: ‘Cryptography’)
>           then: [:project | project install: ‘RabbitZCryptoKitchenSink’]
>           then: [:project | self announce: ‘RabbitZCryptoKitchenSink installed!’]..
>           “Using #then:then: chaining convenience method.”
>           “As <
>               then: then1 then: then2
> 
>                   ^ (receiver then: then1) 
>                       then: then2
>           >
> 
> (ssPromisesEventual := (RemoteServerInstaller on: ‘https://SqueakSource.com/squeakSourceServer/‘)
>       project: ‘Promises’)
>           then: [:project | project allPackages]
>           then: [:packages | self announce: ‘Packages in Promises = ‘, packages asString’]..
> 
> Or something similar. Do you feel me?
>> It's all going to take a minute! ^,^
>> 
>> Tips and guidance welcome.
>> Have a good one; keep it, light.
>> Kindly,
>> rabbit
>> . .. … ‘…^,^
>> 
>> Sent from Callisto House :: decentralized mobile homeless solutions
>> 
>> 
>>> On 10/7/22 11:06, David T. Lewis wrote:
>>>> On Fri, Oct 07, 2022 at 10:09:50AM -0400, rabbit wrote:
>>>> Hi Levente,
>>>> 
>>>> Is this the same machine running SqueakSource? I get frequent intermittent
>>>> failures with accessing projects there (SqueakSource.com). If it were also
>>>> running out of FDs and at 100% CPU, that would explain it. It???s frustrating
>>>> me as I install a bunch of packages off SS, and it fails in the middle often.
>>>> 
>>> No, the https://squeaksource.com and https://source.squeak.org services
>>> are running on different machines.
>>> 
>>> I suspect (but do know know for sure) that the intermittent issues
>>> on squeaksource.com are related to the server image exporting its
>>> repository data on data.obj file. The code that does this has changed
>>> in recent years and to be honest I don't know how it works. Specifically,
>>> I do not know what actually triggers the repeated "save" operations
>>> that seem to be causing problems. Tips and guidance welcome.
>>> 
>>> With respect to exhausing file descriptors on source.squeak.org, I
>>> added a monitor process on squeaksource.com (the other service) that
>>> automatically restarts the image if it is getting too low on file
>>> descriptors. We could probably adapt this to work on source.squeak.org
>>> also. The code is in http://www.squeaksource.com/SSImageInit and I
>>> can work with Chris on it if the file descriptor issue turns out
>>> to be a recurring problem.
>>> 
>>> Here is how the file descriptor monitor works on squeaksource.com.
>>> It would need to be slightly modified for source.squeak.org because
>>> on that server we never want to save the image file, so probably
>>> just change "SmalltalkImage current snapshot: true andQuit: true]"
>>> to "SmalltalkImage current snapshot: false andQuit: true]" in the
>>> method below.
>>> 
>>> SSImageInit class>>startSocketMonitorProcess
>>> 	"Socket leak monitor process. If the number of open file descriptors in the
>>> 	VM process gets dangerously close to the per-process limit (normally 1024),
>>> 	we are at risk of putting the image into a state in which it cannot accept
>>> 	connections, and that may not allow a clean recovery even after an image
>>> 	restart. Thus if the file descriptor count exceeds a threshold of 800, save
>>> 	the image and exit. Assume that a supervisory script will detect the image
>>> 	exit and initiate a restart."
>>> 
>>> 	| vmFileCount |
>>> 	self stopSocketLeakMonitorProcess.
>>> 	SocketLeakMonitorProcess := 
>>> 	[[vmFileCount := (FileDirectory on: '/proc/', OSProcess thisOSProcess pid asString, '/fd') entries size.
>>> 	"OSProcess trace: DateAndTime now asString, ' squeakvm has ', vmFileCount asString, ' open file descriptors'."
>>> 	vmFileCount > 800 ifTrue: [
>>> 		OSProcess trace: 'Too many open file handles, save image and exit'.
>>> 		"Save the image, exit and wait for the supervisory script to restart"
>>> 		WorldState addDeferredUIMessage: [SmalltalkImage current snapshot: true andQuit: true]].
>>> 	(Delay forSeconds: 3 * 3600) wait] repeat] fork name: 'the Socket leak monitor'.
>>> 
>>> Dave
>>> 
>>> 
>> 
> 
> 
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20221007/eccd2c6c/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image0.jpeg
Type: image/jpeg
Size: 16094 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20221007/eccd2c6c/attachment-0002.jpeg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image1.jpeg
Type: image/jpeg
Size: 247097 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20221007/eccd2c6c/attachment-0003.jpeg>


More information about the Squeak-dev mailing list