[Newbies] Binary file I/O performance problems

[Yoshiki Ohshima]

At Fri, 5 Sep 2008 10:59:03 -0700,
David Finlayson wrote:
> 
> I re-wrote the test application to load the test file entirely into
> memory before parsing the data. The total time to parse the file
> decreased by about 50%. Now that I/O is removed from the picture, the
> new bottle neck is turning bytes into integers (and then integers into
> Floats).
> 
> I know that Smalltalk isn't the common language for number crunching,
> but if I can get acceptable performance out of it, then down the road
> I would like to tap into the Croquet environment. That is why I am
> trying to learn a way that will work.

  If the integers or floats are in the layout of C's int[] or float[],
there is a better chance to make it much faster.

  Look at the method Bitmap>>asByteArray and
Bitmap>>copyFromByteArray:.  You can convert a big array of non-pointer
words from/to a byte array.

  data := (1 to: 1000000) as: FloatArray.
  words := Bitmap new: data size.
  words replaceFrom: 1 to: data size with: data.
  bytes := words asByteArray.

  "and you write out the bytes into a binary file."

  "to get them back:"

  words copyFromByteArray: bytes.
  data replaceFrom: 1 to: words size with: words.

Obviously, you can recycle some of the intermediate buffer allocation
and that would speed it up.

  FloatArray has some vector arithmetic primitives, and the Kedama
system in OLPC Etoys image have more elaborated vector arithmetic
primitives on integers and floats including operations with masked
vectors.

-- Yoshiki