[squeak-dev] Sound mixing makes nasty noises

Eliot Miranda eliot.miranda at gmail.com
Fri Dec 18 16:54:47 UTC 2020


Hi Herbert,

_,,,^..^,,,_ (phone)

> On Dec 18, 2020, at 4:46 AM, Herbert König <herbertkoenig at gmx.net> wrote:
> 
> Hi all,
> 
> just noticed SoundBuffer is 16 bits so my suggestions are not practicable.

But adding a signed 32-bit, or even a signed 64-bit collection for mixing would be straight forward, except for ensuring it mixes down to 16 bits.  A signed 32-bit collection would allow 32k 16-bit sources to be additively mixed without clipping, right? So an underlying 64-bit collection is unnecessary.  So a MixedSound that adds into an underlying 32-bit signed sequence, and updates a count of added sounds, and does interpolation when adding sounds of different sample rates, and outputs 16-bit samples derived by dividing the 32-bit sums by the count of added sounds would be a solution right? 

> 
> Cheers,
> 
> Herbert
> 
>> Am 17.12.2020 um 14:14 schrieb Herbert König:
>> Hi Tim,
>> 
>> tl,dr Let's build a limiter.
>> 
>> I stopped using Squeak for Audio with 3.8 :-) With that in mind:
>> 
>> I believe that all sound goes out through SoundPlayer playing
>> SoundBuffers. The samples get created via mixSsampleCountInto.... which
>> seems to be implemented by many classes and there they all seem to do
>> clipping.
>> 
>> After running some of your examples I found several SoundBuffers
>> containing samples 32767 and -32768 which indicates clipping.  I didn't
>> find enough consecutive clipped samples (you shouldn't hear a single
>> one) but that's because I only found a few SoundBuffers.
>> BTW running your example labelled "OK because.." also clipped when run
>> several times.
>> 
>> I didn't find the place where float values are changed to Integers for
>> output.
>> 
>> I did not find any limiter. --> baaad!
>> 
>> So if the above is true and the problem is really the clipping (I had
>> the impression I also heard some wrapping where 32768 uses the top bit
>> as a sign) I suggest:
>> -No clipping during the mixing
>> -Find the place where the SoundBuffers go to the hardware
>> -Implement a limiter there
>> 
>> Basically you need a controlled gain. You look ahead in your SoundBuffer
>> for min and max. In case of overflow you calculate the necessary gain
>> and gradually reduce your actual gain in the time you have until that
>> sample really gets output. (attack phase). If no over/underflow is found
>> you (more) gradually raise your gain back to 1 (release phase). Maybe
>> introduce some delay before raising your gain again (hold phase).
>> 
>> For live real time audio you may not have the luxury to look ahead. You
>> still reduce your gain gradually (let's say in 10 ms) and clip until you
>> reached your final gain. Human's ears tend to tolerate this.
>> 
>> There's tons of compromises to evaluate with a limiter we should go with
>> a bit over TSTTCPW to save CPU. Maybe have another preference to choose
>> the system wide limiter. SCNR :-)))
>> 
>> I can help with rough implementations and audiowise discussions but have
>> no idea where to hook it in and cannot convert it to a primitive or
>> optimize it.
>> 
>> Let's wait until Stef chimes in, I'm sure he has solved that for muO
>> already.
>> 
>> Best regards,
>> 
>> Herbert
>> 
>> 
>>> Am 16.12.2020 um 23:52 schrieb tim Rowledge:
>>> I've discovered that playing two or more sounds simultaneously can
>>> make truly appalling noises. I'm hoping somebody has a good solution.
>>> 
>>> The problem started with playing a sound and using the
>>> pianokeyboardmorph at the same time. Ouch! Also, swizzling the mouse
>>> across the keyboard causes occasional quite audible 'scratchy-clicks'
>>> somehow. There can also be a quite noticeable sound between notes,
>>> not easily describable but sort of a 'dull thump'.
>>> 
>>> I thought it was some issue with how the sound was played, maybe the
>>> note-end process as each piano key was released and the next pressed,
>>> timing issues.. but none of that had very much effect.
>>> 
>>> The AbstractSound>>#stopGracefully method does seem to have a small
>>> bug, in that the decayInMs value surely ought not be the sum of the
>>> attack & decay times from the envelope? But carefully fudging the
>>> note shutdown to be faster didn't help a lot.
>>> 
>>> After too much messing around I discovered that at least some of the
>>> problem is the mixing of sounds.
>>> 
>>> Now, I know we did a small change to MixedSound>>#add:pan:volume:
>>> since 5.3, and that we made a small change to the
>>> SampledSound>>#mixSampleCount:into:startingAt:leftVol:rightVol:
>>> primitive. However, I'm running on a VM that has the prim change, and
>>> I've ported across the add:pan... change, with no real difference.
>>> This is, of course, on a Pi, but the same issue is audible on my iMac
>>> as well.
>>> 
>>> Eventually I cottoned on to the fact that I was mixing two sounds
>>> with volumes set to 100%. Changing to 50% for both actually results
>>> in a perfectly fine end result - Yay! This doesn't seem like a robust
>>> solution though. What if I have four sounds playing each set to 40%?
>>> Why does it seem to work perfectly well for
>>> AbstractSound>>#stereoBachFugue play, which uses four?
>>> 
>>> To make life even more fun, it doesn't seem like there is any volume
>>> control once a sound is playing. The
>>> AbstractSound>>#adjustVolumeTo:overMSecs: method appears to be the
>>> one to use but it has no audible effect. There is also #loudness:
>>> which does work, but produces pretty much exactly the glitch I hear
>>> from glissandoing(?) the keyboard.
>>> 
>>> So - some workspace code to illustrate
>>> 
>>> "awful noise when mixing"
>>> |snd|
>>> snd := FMSound organ1.
>>> snd setPitch: 440 dur: 10 loudness: 0.9;
>>>     play.
>>> 1 second wait.
>>> 
>>> FMSound brass1 setPitch: 470 dur:2 loudness: 0.9;
>>>     play.
>>> 1 second wait.
>>> snd stopGracefully
>>> 
>>> "OK, because both  sound defualt to quieter"
>>> |snd |
>>> snd := FMSound organ1.
>>> snd
>>>     duration: 5;
>>>     play.
>>> 1 second wait.
>>> 
>>> FMSound brass1
>>>     play.
>>> 
>>> 1 second wait.
>>> snd stopGracefully.
>>> 
>>> "nasty glitch using #loudness: plus mixing graunch"
>>> |snd |
>>> snd := FMSound organ1.
>>> snd duration: 5;
>>>     play.
>>> 1 second wait.
>>> snd loudness: 1.0 .
>>> 
>>> 1 second wait.
>>> FMSound brass1
>>>     play.
>>> 
>>> 1 second wait.
>>> snd stopGracefully.
>>> 
>>> 
>>> "no effect from adjustVol..."
>>> |snd |
>>> snd := FMSound organ1.
>>> snd
>>>     duration: 5;
>>>     play.
>>> 1 second wait.
>>> snd adjustVolumeTo: 1.0 overMSecs: 500.
>>> 1 second wait.
>>> FMSound brass1
>>>     play.
>>> 
>>> 1 second wait.
>>> snd stopGracefully.
>>> 
>>> Aargh!
>>> 
>>> tim
>>> --
>>> tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
>>> A computer program does what you tell it to do, not what you want it
>>> to do.
>>> 
>>> 
>>> 
>> 
>> 
> 
> 


More information about the Squeak-dev mailing list