I think I have found the culprit.
If you listen carefully to the recorded sound, you will notice that frames are dropped at regular intervals. Since this pattern is quite obvious and the VM doesn't seem to screw the sound data, it should be something happening at the image level.
Looking for a clue in a 3.9 alpha version, I found a change made by "dgd" on April 4, 2006:
"workaround for OSS emulation on top on ALSA (on Linux environments)" (Delay forMilliseconds: 20) wait.
If the last line is commented out, the sound recorder just works as expected. The same code is present in the OLPC and Squeakland images and both work pretty well without this delay.
CoreAudio records the audio input in real time but SoundRecorder picks the data once in a while. The end result is that the sound data is stored in a different time frame that the sampling rate and the sound plays faster.
Now I am not sure what can be done to have a cross-platform solution, but as far I have seen the problem is not on the Mac VM side.