The least I expect is that I get a debugger in Smalltalk when the limit is reached and not a Squeak that is not reactive and consumes 100% cpu time. But anyway I get the hanging Squeak already with 242 file streams, independent if I set the #maxExternalSemaphores to 8192.

If I do the same in VisualWorks and open 4000 write streams I have no problems with limits, so I guess it is not an operating system limit, it is a Squeak VM limit. There is no reason why everything needs to be implemented in C code, the goal should be to implement as much as we can in Smalltalk.

The reason why I opened so much files is that the Squeak file API is very slow for my realtime streaming, so I simply leave the files open.
Here is an example:

VisualWorks needs 700ms
Time millisecondsToRun:
	[| filename |
	filename := 'test.txt' asFilename.
	10000 timesRepeat:
		[filename readWriteStream close]]

Squeak needs 4000ms
Time millisecondsToRun:
	[10000 timesRepeat: [(FileStream fileNamed: 'test,txt‘) close]]

If you can tell me a faster way to open/write/close a file it would be nice.

Jörg

> Am 17.02.2022 um 16:29 schrieb Levente Uzonyi <leves at caesar.elte.hu>:
> 
> Hi Jörg,
> 
> On Thu, 17 Feb 2022, Jörg Belger wrote:
> 
>> Seems not to work. I set the max semaphores to 8192 and opened only 4000 files. But the Squeak VM only created 242 files on hard disk and freezes with 100% cpu consumption
> 
> As others wrote, there are also limits on the maximum number of open file descriptors.
> One is set by the OS per process. Typically 1024 on unix machines. E.g. on linux you can see the limit with the ulimit -n command. That can typically be changed with superuser privileges.
> And there's another limit if the file descriptor sets are processed with the select() function, which currently happens on Windows and Mac.
> Linux (and IIRC OpenIndiana too) uses epoll(), so there is no upper limit there.
> If select() is used, the value of FD_SETSIZE decides the maximum number of file descriptors that can be opened. Its value is typically 1024, and you'd need to compile your own VM to be able to change that if the OS permits it (Windows and Mac do).
> 
> 
> Levente
> 
>> 
>>      Am 17.02.2022 um 09:55 schrieb Levente Uzonyi <leves at caesar.elte.hu>:
>> Hi Jörg,
>> On Thu, 17 Feb 2022, Jörg Belger wrote:
>> 
>>      Is there a limit with file streams? I cannot open more than 238 on my macOSX. The 239th fails and Squeak runs with 100% cpu power. I changed then the method StandardFileStream>>open:forWrite: and removed the #retryWithGC:until:forFileNamed:, because it looked to me, that it does not much things.
>>      Now I get a <nil> back from the primitive.
>> 
>>      | directory streams |
>>      directory := FileDirectory default directoryNamed: 'Test'.
>>      directory assureExistence.
>>      streams := (1 to: 239)
>>      collect: [:index | directory fileNamed: 'test', index printString].
>>      streams do: #close
>> There is a limit on the maximum number of external semaphores provided by the VM. Each socket uses 3 external semaphores and each file uses one.
>> The default limit is 256 which is very conservative and I hope the next release will ship with at least 4096 as default.
>> Anyway, you can check the current value with
>> Smalltalk maxExternalSemaphores
>> and set it with
>> Smalltalk maxExternalSemaphores: 4096
>> Some care must be taken not to change the limit while signals are expected to be processed (e.g. when you have activity on those sockets/files), because there's a short period during the resize while the VM may fail to signal the semaphores.
>> Levente
>> 
>>      Jörg
>