Add Audio effects

[gamblor21]

Foundation to add audio effects to CircuitPython as discussed in issue #8974.

This PR aims to create initial modules, some base effects and utilities to help create future effects. The ones included will serve as a template for future creations.

To do:

• Create modules for filters, delays, dynamics and reverbs
• Create 2 or 3 effects (echo, chorus and TBD open to suggestions)
• Test effects against audio sources that are not signed, 16-bit
• Add "mix" setting for how much of the effect is applied.
• Look into adding dynamic range compression (like synthio)
• Look into BlockInput for inputs so they can be varied by the synth/time
• Consider a change to the processing logic to make it easier to understand

Looking for feedback on the API and anything else that comes up.

When I have time I plan to document how to create your own effects using the ones provided.

Sample code to run:

i2s_bclk, i2s_lclk, i2s_data = board.GP20, board.GP21, board.GP22
audio = audiobusio.I2SOut(bit_clock=i2s_bclk, word_select=i2s_lclk, data=i2s_data)
mixer = audiomixer.Mixer(voice_count=1, channel_count=1, sample_rate=44100, buffer_size=2048)
audio.play(mixer)

synth = synthio.Synthesizer(channel_count=CHANNELS, sample_rate=44100)
amp_env = synthio.Envelope(attack_time=0.02, attack_level=1, sustain_level=.9, release_time=0.1)
synth.envelope = amp_env
n1 = synthio.Note(261)

echo1 = audiodelays.Echo(delay_ms=500, decay=0.7, buffer_size=1024, channel_count=CHANNELS, sample_rate=44100)
echo1.play(synth)
mixer.voice[0].play(echo1)

synth.press(n1)
time.sleep(0.25)
synth.release(n1)

[dcooperdalrymple]

I'm still digging into the code, but my first instinct is to have the delay_ms property just be a float in seconds as delay instead (ie: rather than 500 ms use 0.5 s). The calculation that is being used to allocate the buffer uses floating point regardless.

Speaking of buffer sizes, it may be nice to explore having a fixed buffer size and altering the rate of playback of that buffer depending on the delay setting. This would be similar to a bucket-brigade style delay pedal and require some more advanced interpolation, etc. I think it be worth exploring an "Analog" delay effect of this nature.

[tannewt]

Thanks for the initial PR! This is very exciting.

[gamblor21]

I'm still digging into the code, but my first instinct is to have the delay_ms property just be a float in seconds as delay instead (ie: rather than 500 ms use 0.5 s). The calculation that is being used to allocate the buffer uses floating point regardless.

I'm not too worried at this point as I'm using Echo basically to test out ideas/concepts. Is there a reason you would want time in a float of seconds vs more precise?

Same thought on the BlockInput for the buffer size. You are right if it could change a maximum size/current size would have to be done to prevent constant reallocation.

Speaking of buffer sizes, it may be nice to explore having a fixed buffer size and altering the rate of playback of that buffer depending on the delay setting. This would be similar to a bucket-brigade style delay pedal and require some more advanced interpolation, etc. I think it be worth exploring an "Analog" delay effect of this nature.

I think this would be good for another effect. I have tossed about the idea of a generic delay buffer that other effects could use.

[gamblor21]

Update for anyone following along:

• Mix has been added
• Mix/Delay/Decay are all BlockInput now. Unlike most synth uses these values have max/min values that have to be checked during the buffer computation time to ensure they are within the acceptable range.
• You can set the max delay and then alter the delay without memory reallocations.
• After looking at what I modeled the first code after in Mixer, I think re-writing it to be clearer (and more like synth) still has the performance. The Mixer code came from when the controllers were significantly slower and in fact the first version was only for M4 processors. That's next on my list.

Getting a lot closer.

[gamblor21]

Word to the wise... 44.1Khz dual channel audio with asyncio without enough sleep in a loop causes weird stuttering that you will try to debug all night long, and turns out just add a small delay to the sleep loop and you are good. Maybe a sign a 0 asyncio.sleep isn't running audio tasks fully?

But it did prompt me to add double buffering in.

[tannewt]

Maybe a sign a 0 asyncio.sleep isn't running audio tasks fully?

This is very weird! I'd expect audio stuff to run in between VM byte codes. I'd appreciate it if you could dig into why it isn't working.

[gamblor21]

Latest updates:

• Simplified the processing code, hopefully making it easier for others to follow
• Tested again 16-bit signed and 8-bit unsigned sources. So far Synth, RawSample and WaveFile. MP3 still is not working.
• Added in dynamic range compression like in Synth for 16 bit echos.
• Added a lot of comments to the code

Things are close now. I still want to see if I can tell why MP3s are not working (they should be don't think it is a processing limit).
I do have to clean up the documentation still.

[gamblor21]

This is very weird! I'd expect audio stuff to run in between VM byte codes. I'd appreciate it if you could dig into why it isn't working.

Took a look, seemed between two channels of 44.1Khz audio and constantly checking a rotary encoder I hit the processing limit as close as I can tell. Or the rotary encoder blocked long enough (and I was checking it often enough) to slow the audio down. So at this point seems to be a non-issue.

[gamblor21]

Things are close now. I still want to see if I can tell why MP3s are not working (they should be don't think it is a processing limit).

MP3Decoder as a source works now. Was a bug I just hadn't ran into. Also samples of unusual lengths work now (wasn't update the remaining sample buffer correctly, and most samples use values that just happened to work).

[dcooperdalrymple]

I finally got around to testing out your audio_effects fork and playing around with audiodelays.Echo. I was hoping to recreate some form of chorus effect with this module, but I don't think that's possible with the current implementation. The decay parameter directly affects the level of the first repeat of the delay. Generally, I would assume that decay/feedback would only apply to additional repeats of the audio signal and the level of the initial copy would be controlled independently using the mix parameter.

For instance, if you wanted to create a strong "slapback" style delay, this is what I would expect to work in that scenario (I'm excluding buffer_size, channel_count, and sample_rate as they aren't relevant):

audiodelays.Echo(delay_ms=500, decay=0.0, mix=1.0)

The expected outcome would be a full volume repeat of the audio after 500ms and then no additional feedback/decay of that audio. Instead, there is no effect on the audio output because the decay parameter silences out all delayed audio including the first repeat.

The solution is relatively simple. Instead of applying decay as soon a sample is extracted from echo_buffer, only apply it when inserting the old sample back into echo_buffer. I have implemented this on my end by modifying /shared-module/audiodelays/Echo.c, but I'm a newbie when it comes to pull requests and I'm not certain how best to contribute here.

[dcooperdalrymple]

I've created a pull request on your fork for your consideration, @gamblor21. gamblor21#1

[gamblor21]

I've created a pull request on your fork for your consideration, @gamblor21. gamblor21#1

I will take a look but probably not until tomorrow. What you were saying makes sense.

One thing I do plan to do is create more effects then just this echo. This really is a proof-of-concept still. I was looking a chorus briefly last night, and want to consider others (reverb) as well.

If you are playing with it another thing you can try is chaining echos (since that is my only effect ha). But basically echo1.play(echo2.play(echo3.play(echo4.play(synth))))

[dcooperdalrymple]

Though the delay_ms parameter has been set up as a BlockInput object, it is not recalculating echo_buffer_len and subsequent read/write positions within audiodelays_echo_get_buffer. As a result, using a BlockInput parameter for delay_ms has no discernible affect beyond the initial value.

lfo1 = synthio.LFO(scale=250, offset=500, rate=0.5)
synth.blocks.append(lfo1)

echo1 = audiodelays.Echo(delay_ms=lfo1, decay=0.5, mix=1.0, buffer_size=4096, channel_count=CHANNELS, sample_rate=SAMPLES_RATE)
echo1.play(synth)
mixer.voice[0].play(echo1)

synth.press(60)
while True:
    print(chorus_lfo.value)
    time.sleep(0.25)

In this example, it is clear that the value of the LFO is changing over time, but there is no audible change in the output.

[gamblor21]

Though the delay_ms parameter has been set up as a BlockInput object, it is not recalculating echo_buffer_len and subsequent read/write positions within audiodelays_echo_get_buffer. As a result, using a BlockInput parameter for delay_ms has no discernible affect beyond the initial value.

Oh good catch! Thanks, I'll have a fix for that soon I know what is wrong.