does pion have the ability to manage the buffer?

[LazarenkoA]

We are developing an app similar to a web messenger, and it’s extremely important for us that the video is synchronized in real-time or with minimal delay. Right now, there is a noticeable delay, and I suspect it might be due to WebRTC buffering (possibly jitter or another factor).

I added debug information to the console output before sending
videoTrack.WriteSample(media.Sample{....})

I can see that the data is being sent, but the video starts playing after some time.

Here's a video:
https://github.com/user-attachments/assets/12535cdb-f32e-4891-be24-ca5837b7c6bc

In the video, you can see that the data is sent immediately, but the video appears at the 5-second mark.

If I increase the data volume by increasing the FPS or using the following:

videoTrack.WriteSample(media.Sample{....})
videoTrack.WriteSample(media.Sample{....})
videoTrack.WriteSample(media.Sample{....})

The delay disappears, which aligns with the theory that there is some sort of buffer.
So the question is: how can I control this buffer?

[LazarenkoA]

perhaps this setting should be done on the recipient's side (javascript)?

[Sean-Der]

Hey @LazarenkoA

You can control this client and server side!

• Client Side - https://henbos.github.io/webrtc-timing/#dom-rtcrtpreceiver-playoutdelayhint lets you tune this
• Server Side - You can also adjust this value without an JS https://webrtc.googlesource.com/src/+/main/docs/native-code/rtp-hdrext/playout-delay/README.md

We should make an example that shows how to use https://github.com/pion/rtp/blob/a21194ecfb5362261a0dc4af1f68e4a8944df345/playoutdelayextension.go#L25 if you start a PR I would love to work on it together! Maybe something like https://github.com/pion/webrtc/tree/master/examples/swap-tracks

Instead of swapping between tracks you can dynamically adjust the delay and the server will change the header values? If you start that example would love to work on it togethe!

[LazarenkoA]

@Sean-Der
I added to sdp

a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=playout-delay:0 1 

but the delay remained
Unfortunately, I am not so familiar with webrtc to create a PR.