Defining standards and SDKs via practical application

[Arlodotexe]

WASM allows us to run code natively, locally, in any language, using a downloadable payload file. As soon as we found that out, it upended our plans for any RPC or HTTP API.

These conversations have happened largely on Discord, so I'm archiving our musings here as notes.
As we define these better, this will be updated with more substance and less notes. Chat archives will be collapsed.

Discussion points and open questions (please read conversation below):

• Practicality of using WASM for portable, x-lang cores.
• What do terms like "SDK" and "Standard" exactly mean?
• Can we use the WASM bindings as the "standard"? What about using REST as the standard?
• Is the SDK a language specific implementation allowing usage of the standard?
• How does this affect the future REST and RPC APIs, and do we still need them?

---

08/02/2022 https://discord.com/channels/372137812037730304/547159933838163969/1004237169591857243

If I understand things correctly:

• We can compile the .NET Runtime (and with it, our code) to WASM
• The Bytecode Alliance, a nonprofit comprised of Amazon, Google, Intel, Microsoft, ARM, Mozilla, all the big names, is a responsible for building the WebAssembly standard.
• They also have the WASI standard, designed for non-web system languages. https://wasi.dev/
• A lot of languages have a WASM runtime package, thanks to the nonprofit mentioned above.

Maybe this could enable cross-platform and cross-language plugins and cores. My original plan for Cores involved compiling them, Process.Start, and stdin/out, but this is much better and could let plugins run natively in the browser, or in other languages if there's a Strix SDK available.

Except the wasmtime .NET nuget package needs .NET 6 or higher, meaning we'd need to use WinAppSDK or WebView2's WASM.

---

08/03/2022 https://discord.com/channels/372137812037730304/547159933838163969/1004462158450540584

The only downside is that I might need to re-implement the SDK in Rust, or something good at creating WASM. dotnet just compiles the entire .NET runtime and it makes massive binaries.

---

08/03/2022 https://discord.com/channels/372137812037730304/547159933838163969/1004497025741635656

Anyway, even if WinRT works in other places, it isn't the best solution for what Strix is trying to do
The barrier to entry for implementors and consumers is too high

Plus we really want this to function in the browser, as well as other languages

If someone wants to write a Strix Music Core in C++, Go, Rust, JS, etc., and consume it in anywhere, then WASM/WASI looks like our best option.

We're going to continue focusing on the .NET version of the Strix Music SDK for now, let WinAppSDK, WASI and WASM continue to grow and evolve a bit more, and probably drum up a prototype some time next year.
If the binary size if small enough, we could potentially reuse all existing Cores written in .NET in other languages

---

08/13/2022 https://discord.com/channels/372137812037730304/547159933838163969/1008079317768622140

Version management isn't done by the SDK, it's up to the consuming application to set that up

In order to make Wasi/Wasm happen on any language, we need to create bindings into WASM.

Those bindings are static methods iiuc, so managing instances needs to be done manually. It'll need a translation layer made of 2 parts - 1 that wraps around a normal core / plugin (any language) and provides static methods in WASM the consumer can call, and 1 in your SDK (any language) that wraps bindings around the binary and turns it back into normal core / plugin instances, in whatever language you're using.

That's the minimum we need to do for x-lang support.

This is more of an ecosystem concern. If people are using Nuget to publish actual binaries, then it might work in .NET using NuGet.Packaging, but it probably won't in other languages.

I think I would rather have people submit an IPNS address of their plugin (so they can update with new versions) and create an IPNS containing the other IPNS addresses.

A plugin gallery on IPFS, where a single address can get you a curated list of plugins, if you trust the source.
That would allow any Strix SDK implementation in any language to consume a plugin created in any other language. Anyone can create a plugin gallery, and nobody has to pay for hosting fees because it's all on IPFS.

---

08/18/2022 https://discord.com/channels/372137812037730304/547159933838163969/1010046006823493673

I bet it wouldn't be too hard to prototype this, once you figure out how to do WASM bindings...
We can already compile .NET to WASM, we could compile a Core and run it in another language.
CoreModels really are the bones of the Strix Music Standard, aren't they :hmm:
We'll want to get the standard well defined before we even attempt to go x-lang, and I'm not convinced that merging should be part of it.
But I think we need to keep stress testing the .NET SDK and make a ton of cores before we do that, too

[Arlodotexe]

Conversation archived from 8/19/2022 https://discord.com/channels/372137812037730304/547159933838163969/1010053785202208840

kermit the pog
Was cross-language support the only reason RemoteCore existed? I thought it could be used to keep multiple instances of Strix in sync

Arlo
Oh OwlCore.Remoting or AceRPC will still be used for that, those are app features
The only reason the OwlCoreRpc Core existed was to communicate with cores running out of process
Same with the HTTP API
That whole problem went away when I realized we can just compile a core to WASM in one language, and run it locally and natively with bindings into any other language.
Now I can focus on defining exactly what the standard is, so we can create those WASM bindings

kermit the pog
The HTTP API would still be useful for highly constrained systems, which are limited by what kinds of apps can run or the amount of processing powrr is available

Arlo
Yeah but that HTTP API, when we eventually create it, will be almost 1:1 based on the WASM bindings 😛

kermit the pog
What does that mean?

Arlo
because it's just the Strix Music Standard in HTTP form
Essentially it needs the same information needed to satisfy parameters on static method calls, like in the WASM bindings
so we can just translate it to a REST API

kermit the pog
So it's the Strix SDK, as it exists for WASM, exposed as a REST API?

Arlo
Well WASM doesn't know or care about the SDK
And it needs a whole custom API surface to translate static method calls into object instances, remember

kermit the pog
I think I'm just going to have to wait until there's a functioning prototype 😅

Arlo
The SDK in whatever language you're using is your "Strix SDK", the models and data that's needed to interop with WASM and HTTP are the actual Standard
Actually, I don't think merging would even be part of the Standard, it's a goodie you get for using the .NET version

kermit the pog
Merging as in MergedCore?

Arlo
Yeah

kermit the pog
How would you build an app supporting all the cores without it?

Arlo
Simultaneously? You don't, unless you have your own version of it in your language.
The .NET SDK is our incubation project, where we find out what works and what is feasible for a music standard. I don't think merging can realistically be part of the core standard, but it could be re-implemented in another language's SDK.