[squeak-dev] FileStreams Limit

Jörg Belger unique75 at web.de
Thu Feb 17 09:05:50 UTC 2022


My problem is simply that I need to leave the streams open coz reopening for every write is too slow. I have realtime data coming through a socket in nanosecond precision and the file handling must be very fast. Currently I have 120 nanosecond realtime streams and 2645 minute-based streams. 

As I use now binary format instead of the previous CSV format, I cannot read the plain data files anyway, so maybe I will give Magma a try. It does not matter if I can’t read binary files or can’t read Magma files with a text editor :-)

> Am 17.02.2022 um 09:55 schrieb Levente Uzonyi <leves at caesar.elte.hu>:
> 
> Hi Jörg,
> 
> On Thu, 17 Feb 2022, Jörg Belger wrote:
> 
>> Is there a limit with file streams? I cannot open more than 238 on my macOSX. The 239th fails and Squeak runs with 100% cpu power. I changed then the method StandardFileStream>>open:forWrite: and removed the #retryWithGC:until:forFileNamed:, because it looked to me, that it does not much things. Now I get a <nil> back from the primitive.
>> 
>> | directory streams |
>> directory := FileDirectory default directoryNamed: 'Test'.
>> directory assureExistence.
>> streams := (1 to: 239)
>> 	collect: [:index | directory fileNamed: 'test', index printString].
>> streams do: #close
> 
> There is a limit on the maximum number of external semaphores provided by the VM. Each socket uses 3 external semaphores and each file uses one.
> The default limit is 256 which is very conservative and I hope the next release will ship with at least 4096 as default.
> 
> Anyway, you can check the current value with
> 
> 	Smalltalk maxExternalSemaphores
> 
> and set it with
> 
> 	Smalltalk maxExternalSemaphores: 4096
> 
> 
> Some care must be taken not to change the limit while signals are expected to be processed (e.g. when you have activity on those sockets/files), because there's a short period during the resize while the VM may fail to signal the semaphores.
> 
> 
> Levente
> 
>> 
>> Jörg

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20220217/b828f4e9/attachment-0001.html>


More information about the Squeak-dev mailing list