[squeak-dev] FileStreams Limit

Levente Uzonyi leves at caesar.elte.hu
Thu Feb 17 15:29:16 UTC 2022


Hi Jörg,

On Thu, 17 Feb 2022, Jörg Belger wrote:

> Seems not to work. I set the max semaphores to 8192 and opened only 4000 files. But the Squeak VM only created 242 files on hard disk and freezes with 100% cpu consumption

As others wrote, there are also limits on the maximum number of open file 
descriptors.
One is set by the OS per process. Typically 1024 on unix machines. E.g. on 
linux you can see the limit with the ulimit -n command. That can typically 
be changed with superuser privileges.
And there's another limit if the file descriptor sets are processed with 
the select() function, which currently happens on Windows and Mac.
Linux (and IIRC OpenIndiana too) uses epoll(), so there is no upper limit 
there.
If select() is used, the value of FD_SETSIZE decides the maximum number of file 
descriptors that can be opened. Its value is typically 1024, and you'd 
need to compile your own VM to be able to change that if the OS permits 
it (Windows and Mac do).


Levente

>
>       Am 17.02.2022 um 09:55 schrieb Levente Uzonyi <leves at caesar.elte.hu>:
> 
> Hi Jörg,
> 
> On Thu, 17 Feb 2022, Jörg Belger wrote:
>
>       Is there a limit with file streams? I cannot open more than 238 on my macOSX. The 239th fails and Squeak runs with 100% cpu power. I changed then the method StandardFileStream>>open:forWrite: and removed the #retryWithGC:until:forFileNamed:, because it looked to me, that it does not much things.
>       Now I get a <nil> back from the primitive.
>
>       | directory streams |
>       directory := FileDirectory default directoryNamed: 'Test'.
>       directory assureExistence.
>       streams := (1 to: 239)
>       collect: [:index | directory fileNamed: 'test', index printString].
>       streams do: #close
> 
> 
> There is a limit on the maximum number of external semaphores provided by the VM. Each socket uses 3 external semaphores and each file uses one.
> The default limit is 256 which is very conservative and I hope the next release will ship with at least 4096 as default.
> 
> Anyway, you can check the current value with
> 
> Smalltalk maxExternalSemaphores
> 
> and set it with
> 
> Smalltalk maxExternalSemaphores: 4096
> 
> 
> Some care must be taken not to change the limit while signals are expected to be processed (e.g. when you have activity on those sockets/files), because there's a short period during the resize while the VM may fail to signal the semaphores.
> 
> 
> Levente
> 
>
>       Jörg
> 
> 
> 
>


More information about the Squeak-dev mailing list