sound engines and alternative outputs

[felixroos]

Let's use this discussion to talk about possible sound engines:

project	issue(s)	suggested by
web audio modules (web ports of open-source vsts)		@ojack
faust		@ojack
gibberish / genish		@ojack
webdirt	#121	@ojack
sunvox in the browser		@ojack
elementary audio		@ojack
enfer synth + sampler made to use with ORCA		@ojack
wags 🙂		@ojack
glicol	#29	@ojack
jssc3		@charlieroberts
csound wasm	#270	@Omicron666
assemblyscript wasmaudioworklet		@felixroos
jssc3 (supercollider on the web)
web-audio-rs		@chaosprint
cardinal wasm		@gogins
vult		@felixroos
monadic-synth + wasm_of_ocaml		@felixroos
cmajor (new dsp scripting lang from maker of juce)		thati.studio

non sound:

project	issue(s)	suggested by
hydra		@yaxu
webserial (done)	#80	@yaxu
dmx light control		@yaxu
webbluetooth		@yaxu
webusb		@Froos

With the current structure, we don't even have to decide one one, but can create a package that adds bindings for one or more.
If you know additional candidates, please keep em coming.

How to implement a custom output

Using a different sound output requires creating custom "bindings", which means creating a pattern method that tells the REPL how to output an event when it is triggered. This can be done like this:

Pattern.prototype.myOutput = function() {
  return this.onTrigger((time, event) => { // modify each event
      // time = scheduled audio context time for the event
      console.log('myOutput.onTrigger', time, event)
  });
}

"a b [c d]".myOutput() // open the console

The exact same approach is already used in .osc, .midi among others.

So if anybody is interested in trying out one of the above (or something else), this is the current way to go.
You can find project setup instructions here!

[eilseq]

If we have a server would be fairly straightforward to load SuperDirt in a docker instance of SuperCollider. But is quite hard to orchestrate in sessions instances (require probably Kubernetes) and it would rely on cloud computing.

[eilseq]

For a client solution I would say that a webAudio implementation of Dirt - starting from dktr0's WebDirt implementation - is very reasonable to ensure compatibility with tidal

[omicron321]

https://csound.com/wasm/

[nick-thompson]

Cool to see this! Great list, I'm eager to see the discussion.

What are the priorities and features that you need out of the engines you want to support?

I'm the author of Elementary Audio and would be happy to answer questions about whether or not it fits your needs. I think it could be a pretty cool integration; Elementary runs in the browser with a JavaScript API that speaks to a WASM engine running in a web audio worklet, and since you would interop just with that JavaScript API, you'll be able to easily run the same code on the other supported platforms of Elementary– cli, plugins, and file->file offline processing.

I'll leave it at that for now as I don't want to overstep my bounds, but I'm happy to chat further if it's helpful!

[felixroos]

Hey, thanks for tuning in! I definitely want to try out Elementary Audio with strudel. I think a really important requirement is that audio nodes can be created as cheaply as possible, because every aspect of the sound can be dynamic at any time. I will create a separate issue when I'm ready to dive in

[eilseq]

I used Nick's react-juce in a former position. If Elementary is nearly as awesome is worth trying. I saw the ADC talk about it. @nick-thompson congrats! Is really a concept like SuperCollider, very suited for this community in my opinion.

[jarmitage]

It seems like exploration of more than one of these options is warranted.

From that angle, maybe it would be good to discuss the bindings here as well, do you have anything in mind already? Or should "sound engine bindings" be a separate issue?

[felixroos]

I want to use the Glicol's Rust engine clock to do it, which should be accurate at the sample level.

In the end you'll have to use the audio context anyway i guess. so why should there be more accuracy?

[chaosprint]

I want to use the Glicol's Rust engine clock to do it, which should be accurate at the sample level.

In the end you'll have to use the audio context anyway i guess. so why should there be more accuracy?

You are right. It doesn't have to be that accurate if messages (such as the OSC usage for SuperDirt) are used to schedule in advance. But I am considering another way for communication, which is to precalculate the pattern and then send the info to the audio engine to avoid constantly sending messages. One pattern, one send_msg. So I need to know the duration that the pattern begins to repeat itself. It would be less robust for me to choose the duration of the query time span in advance. For example, if I query the pattern with time span 4, and users enter "[60 62 64 66]/8", then time span 4 is not enough.

[felixroos]

The problem with your approach is that a pattern can possibly have an infinite length. Also, there is no way of knowing how long it takes for it to repeat (if there is a repetition at all). So for example stack("c3", "e3".fast(Math.PI)) will create an infinite pattern, because pi is irrational, and the two notes will only perfectly line up at time zero.

[felixroos]

see https://strudel.tidalcycles.org/#c3RhY2soImMzIiwgImUzIi5mYXN0KE1hdGguUEkpKS5sZWdhdG8oLjUp

[chaosprint]

The problem with your approach is that a pattern can possibly have an infinite length. Also, there is no way of knowing how long it takes for it to repeat (if there is a repetition at all). So for example stack("c3", "e3".fast(Math.PI)) will create an infinite pattern, because pi is irrational, and the two notes will only perfectly line up at time zero.

I see, that's a really useful input, which makes it quite clear for me where to start. Thanks.

[chaosprint]

For those who are interested, here's a demo combining Strudel with Glicol language/audio engine:
https://glicol-strudel.netlify.app
I am also working on the scheduler style mentioned on the top.

[yaxu]

I thought it would be good to also think about non-sound possibilities:

• visual synths, e.g. hydra
• DMX light control https://www.npmjs.com/package/dmx-web
• webserial for patterning messages to arduino etc https://developer.mozilla.org/en-US/docs/Web/API/Web_Serial_API
• webbluetooth ?

I'm working on a webserial package for strudel at the moment, it should be quite easy.

[yaxu]

I think an important requirement is that the sound engine is really flexible in terms of being able to control effects and other synthesis parameters independently, to effect individual sounds.

For example it should be possible to play a sound sample, that is panned to a certain position, with a low pass filter set to a particular value, and a distortion effect applied. Slightly later another sound could then be triggered with different values, without the first one changing.

Then all these effects can be patterned. Ideally, we would go beyond trigger messages, by applying a pattern to the life of a sound, so that effect values can be modulated with the pattern while it plays.

Exceptions are 'global effects', like reverb.

[chaosprint]

Not difficult to implement that in Glicol. Just need some time to add event support for each node.

let p1 = mini("['bd' ['sn' 'hh']]").to_glicol({span: 2})
// let p2 = ...
// let p3 = ...
// ...

glicolEngine.run(`

~t1: psampler ${p1}
>> lpf ${p2} 1.0
>> distor ${p3}
>> pan ${p4}

~t2: psynth ${p5}
>> lpf ${p6} 1.0 >> mul 0.8

~t3: speed 4.0 >> seq 60 >> bd 0.1 >> mul 0.4
out: mix ~t.. >> plate 0.1

`)

[jarmitage]

New one: https://github.com/paolomarrone/Zampogna

cc @paolomarrone