Skip to content

Multicore Processing

Jump to bottom
Phil Schatzmann edited this page Nov 16, 2024 · 42 revisions

So far I never needed to use any of the ESP32 multicore functionality because all the examples are very simple and everything is fitting on a single core. However if you write some more complex scenarios which are e.g. updating a screen, process user input w/o impacting the audio, you should consider to use multiple tasks. Then you need to use a save way to communicate data between the tasks.

  • Task: a simple C++ class which represents a FreeRTPS task.
  • BufferRTOS: a FreeRTOS StreamBuffer implementation using the BaseBuffer API
  • SynchronizedNBuffer: a thread save NBuffer.
  • Mutex: with different implementations per platform.
  • LockGuard: a RAII class for managing the Mutex
  • SynchronizedBuffer: which allows you to use any BaseBuffer implementation in a save way
  • QueueRTOS You can also use the C++ API of an RTOS Queue to comminicate between processes.

You can use any of the buffers above in a QueueStream to use them as copy source and copy targets.

Task

If you use an processor that supports FreeRTOS, you can start multiple tasks

#include "AudioTools.h"
#include "AudioTools/Concurrency/All.h"

Task task("write", 3000, 10, 0);
void method(){}
task.begin(method);

BufferRTOS

If you use an ESP32, it is much more efficient to use a FreeRTOS StreamBuffer. Just write the data with writeArray() on one Task and read it back with readArray() from the other task. It is the same API you would use with any other class which is based on BaseBuffer

#include "AudioTools.h"
#include "AudioTools/Concurrency/All.h"

BufferRTOS<int16_t> buffer(2048, 512);

In my test sketch I was measuring a thruput of 36.57-37.93 Mbytes per second.

SynchronizedNBuffer

I have extended the NBuffer to use a FreeRTOS queue to manage the allocated buffers.

#include "AudioTools.h"
#include "AudioTools/Concurrency/All.h"

SynchronizedNBuffer<int16_t> buffer(2048, 512);

SynchronizedBuffer

Here is an example how to use the SynchronizedBuffer class. Just wrap one of the existing buffer classes. Here is an example of a thread-save Double Buffer:

#include "AudioTools.h"
#include "AudioTools/Concurrency/All.h"

audio_tools::Mutex mutex;
NBuffer<int16_t> nbuffer(512,2);
SynchronizedBuffer<int16_t> buffer(nbuffer, mutex);

In my test sketch this was giving a thuput of only 1.68 Mbytes per second.

Here is a thread-save RingBuffer

#include "AudioTools.h"
#include "AudioTools/Concurrency/All.h"

audio_tools::Mutex mutex;
RingBuffer<int16_t> nbuffer(512*4);
SynchronizedBuffer<int16_t> buffer(nbuffer, mutex);

and I was measuring 2.35 Mbytes per second.

Audio Tasks: A Basic Example

Usually it is good enought to just move the copy operation to a separate task, so that we can use the Arduino loop w/o impacting any audio.

In the following example we just decode the mp3 from an URL and send the output to I2S in a separate task.

#include "AudioTools.h"
#include "AudioTools/AudioCodecs/CodecMP3Helix.h"
#include "AudioTools/AudioLibs/AudioBoardStream.h"
#include "AudioTools/Concurrency/All.h"

URLStream url("ssid","password");  // or replace with ICYStream to get metadata
AudioBoardStream i2s(AudioKitEs8388V1); // final output of decoded stream
EncodedAudioStream dec(&i2s, new MP3DecoderHelix()); // Decoding stream
StreamCopy copier(dec, url); // copy url to decoder
Task task("mp3-copy", 10000, 1, 0);

void setup(){
  Serial.begin(115200);
  AudioLogger::instance().begin(Serial, AudioLogger::Info);  

  // setup i2s
  auto config = i2s.defaultConfig(TX_MODE);
  i2s.begin(config);

  // setup I2S based on sampling rate provided by decoder
  dec.begin();

  // mp3 radio
  url.begin("http://stream.srg-ssr.ch/m/rsj/mp3_128","audio/mp3");

  // start copy task
  task.begin([](){copier.copy();});
}

void loop(){
  delay(1000);
  Serial.println("ping...");
}

Please note that in this sceanrio the loop processing can still starve the audio task if you execute some long lasting operations that do not yield: in this case add some short delay() calls (to your looop porcess), to yield control to the audio task!

Audio Tasks: An Extended Example

Here is a maximum example where we use a separate task to generate the audio and a task to output the audio. Both tasks communicate via a queue that is created from a synchronized buffer implementation.

Both tasks can use a StreamCopy to manage the data copying so they just need to call the copy operation. Because this is quite short we do not write separate methods, but just implment this with the help of a lambda expression!

We still have the Arduino loop task available for other processing!

#include "AudioTools.h"
#include "AudioTools/AudioLibs/AudioBoardStream.h"
#include "AudioTools/Concurrency/All.h"

AudioInfo info(48000, 2, 16);
// source and sink
SineWaveGenerator<int16_t> sineWave(32000);               
GeneratedSoundStream<int16_t> sound(sineWave); 
AudioBoardStream out(AudioKitEs8388V1);
// queue
BufferRTOS<uint8_t> buffer(1024 * 10);
QueueStream<uint8_t> queue(buffer);
// copy
StreamCopy copierSource(queue, sound);                            
StreamCopy copierSink(out, queue);                            
// tasks
Task writeTask("write", 3000, 10, 0);
Task readTask("read", 3000, 10, 1);

void setup() {
  Serial.begin(115200);
  AudioLogger::instance().begin(Serial, AudioLogger::Warning);

  // start Queue
  queue.begin();

  // start I2S Output
  Serial.println("starting I2S...");
  auto config = out.defaultConfig(TX_MODE);
  config.copyFrom(info); 
  out.begin(config);

  // Setup sine wave Data Source
  sineWave.begin(info, N_B4);
  Serial.println("started...");

  // Start audio tasks
  writeTask.begin([]() {
    copierSource.copy();
  });

  readTask.begin([]() {
    copierSink.copy();
  });

  Serial.println("started...");

}

void loop() { delay(1000); }

Further Examples

Summary

I am providing the most important multi core functionality as C++ classes. Alternatively I recommend to use the arduino-freertos-addons which provides all the functionality you need in an elegant way: Just read the Readme of the project. However, if you prefer you can also use the official FreeRTOS C API.

Clone this wiki locally