Pion WebRTC as A MCU

What is an MCU

For WebRTC a MCU (Multipoint Conferencing Unit) is a server that can modify/mix/transcode media and then send it to viewers.
These servers usually are doing heavy CPU/GPU based operations, and lots of flexibility is required. It is difficult to solve
everyones use cases with a single product.

Pion WebRTC is 'bottom up development'

Pion WebRTC is not a kitchen sink library. We try to solve a single problem at a time. We believe this gives developers more flexibility, and allows you to pick and choose components. Pion WebRTC does not come with any encoders/decoders. You are free to use something high level like
GStreamer or ffmpeg. You could also use x264 or libvpx directly, you know your problem best so you are free to choose the best tool for the job.

With Pion WebRTC the public API just accepts either bytes of media (H264, VP8, Opus) or RTP packets. You can consume/produce this data however you wish.

Solving MCU Problems with Pion

These are the problems that users have solved using Pion WebRTC in the MCU space.

Re-Broadcasting RTMP

This can be accomplished today using the gstreamer-send example. This example takes a GStreamer pipeline as a command line argument, so we can use an rtmpsrc like so.

echo $BROWSER_SDP | gstreamer-send -video-src "uridecodebin uri=rtmp://my-rtmp-server.com ! videoscale ! video/x-raw, width=320, height=240 ! queue " -audio-src "uridecodebin uri=rtmp://my-rtmp-server.com ! queue ! audioconvert"

Transcodes

gstreamer-receive shows how a user can feed audio/video into GStreamer. Once media is in GStreamer you can encode it at any bitrates/codecs you want.

These transcodes then can be distributed out to connected WebRTC peers like the gstreamer-send example.

Computer Vision/Video Analysis

gocv-receive demonstrates using the GoCV library to do CV on a server, but
upload/capture the video via WebRTC. This means you have weak clients (security cameras or phones) upload video via WebRTC and then do processing on your backend.

Compositing/Video Production