Writing a large Collection of integers to a file fast

UZONYI Levente leves at elte.hu
Sat Jan 26 16:43:37 UTC 2008


Hi!

Try this:

writeIntegersFrom: aLargeCollection to: aFileName

	| file crLf buffer |
	crLf := String cr, String lf.
	buffer := WriteStream on: (String new: 65536).
	[ file := StandardFileStream fileNamed: aFileName.
	aLargeCollection do: [ :int |
		buffer
			nextPutAll: int printString;
			nextPutAll: crLf.
		buffer position > 64000 ifTrue: [
			file nextPutAll: buffer contents.
			buffer position: 0 ] ].
	file nextPutAll: buffer contents ]
		ensure: [
			file ifNotNil: [ file close ] ]

I'm sure there's a better solution, this is just fast and dirty one.
(30 secs for 6M integers on my machine)

Levente

On Sat, 26 Jan 2008, [UTF-8] André Wendt wrote:

> Lukas Renggli wrote:
> >> Oe possible optimization is to not use the #, message  because it
> >> create a copy of the String.
> >> So you can try:
> >>
> >> aFile := CrLfFileStream fileNamed: aFilename.
> >> aLargeCollection do: [ :int |
> >>   aFile nextPutAll: int printString; nextPut: Character cr.
> >> ].
> >> aFile close.
> >
> > Or even smarter:
> >
> >     aStream print: int; nextPut: Character cr
> >
> > Lukas
>
> Thanks for your suggestions! I've tried, but it didn't give the boost I
> was hoping for...
>
> André
>
> >> On Jan 26, 2008, at 12:37 PM, André Wendt wrote:
> >>
> >>> Hi,
> >>>
> >>> I'm looking for a fast way to write a large Collection of ints (over 6
> >>> million elements) to a file. My attempt was:
> >>>
> >>> aFile := CrLfFileStream fileNamed: aFilename.
> >>> aLargeCollection do: [ :int |
> >>>  aFile nextPutAll: int printString, String cr.
> >>> ].
> >>> aFile close.
> >>>
> >>> Unfortunately, this takes more than fifteen minutes. I suspect this is
> >>> due to my implementation.
> >>>
> >>> Is there any smart way I can do this faster?
> >>>
> >>> Thanks,
> >>> André
> >>>
> >>         Mth
>
>
>



More information about the Squeak-dev mailing list