David T. Lewis pravi:
Janko Mivšek wrote:
This image is running now for months, that's why its restart could help here. But finding the real cause of this trouble, well, this would be just perfect! Anyone has an idea what can be a problem?
Socket leak? Keep an eye on the contents of /proc/<squeakpid>/fd/ and see if open file descriptors are accumulating. If so, you will reach a limit (usually 1024) after which no new connections will be possible.
Yep, that's it, I just counted it: 1024 and dropped to 764 after Swazoo restart.
This problem is now solved simply after restarting the image. Now the open sockets vary from 20 to 300 and jump back to low every hour when the image is snapshoted.
But the problem will repeat after few months again, because we somehow leak sockets, as David said. I didn't manage to find the reason, not the offending sockets in image. But ok, restarting the image every few months is good enough solution for now.
Thanks David for help!
Best regards Janko
So, first question: how to close those remaining 764 open sockets without restarting the image,
The socket file descriptors that you see listed in /proc/<squeakpid>/fd/ are file descriptors that have been opened by the Squeak VM, and that have not been properly closed when no longer in use. If you can find the objects in the running image that were responsible for opening these descriptors (possibly this is "Socket allSubInstances"), then you can probably close them. Otherwise, if finalizers have been set up, a garbage collection may cause them to be closed.
second: how to avoid opening so many unclosed sockets (some kind of attack?),
It is not an attack. Most likely it is nothing more than some path in the code that allows a socket to become unreferenced in the image without having closed the socket first.
third: how to raise the 1024 limit?
It is best not to do that. If you are "leaking" sockets, it will eventually fail no matter what the limit is. But if you do want to control it, see the man page for bash and look for the ulimit command.
Dave