Accepting 'Big Changes' from people is VERY frustrating. These thoughts run through my head.
* Idea is usually good! Even if I don't understand it could help lots of others users.
* The contributor is very focused on just getting their feature in. The impact on the larger project isn't as much a concern.
* New contributors often don't have the grit to see it out. They will disappear before things are done. So I am left picking up the pieces (which is harder then doing it all myself)
----
What I try and remember is that their happiness/experience matters more then any code. I try to help the contributor learn/grow as much as possible and even see some career benefits out of it. Pion will cease to matter eventually, so I hope to help as many programmers with it as possible.
NLNet is a wonderful organization. They have supported two Pion projects!
I am grateful the code got written, but even better people got careers out of it/learned new stuff. If you are on the fence about taking on a project I encourage you to do it!
Amazing debugging, I loved reading that. HN doesn't get enough good posts like this anymore :)
If https://github.com/pion/sctp/issues/12 had happened (not just in Pion but across all implementations) this could have been fixed years ago. The hardcoding we all settle for is tragic.
I work on WHIP in OBS and an open source project Broadcast Box[0]. I made this site/video to talk about the reasons why https://webrtcforthestreamer.com
I hope this site can convince more people to check this stuff out. If you are curious or have feedback I would love to hear.
I have been working on making streaming cheaper/more private/lower latency (via WebRTC) in OBS. Been working on this site to get people excited https://webrtcforthestreamer.com/ after it is done want to record a YouTube video for it.
After that I want to spend the weekend just closing out Pion bugs/relaxing :)
You run one encoder for all the viewers. CPU usage won't scale up from 1 -> 15 viewers.
I could get it lower by encoding once and then syncing to keyframes. It would make the code more complicated though. If someone asks for it/gets excited would love to do it though :)
The 'Seek' is done server side. So if you go to `n` seconds it is done server side and done for everyone.
I should reword the README though. If someone is savy enough they could totally grab the video. For most users it is like a Google Meet though. If you click `Show Controls` you can pause the video that is it.
With things like Insertable Streams[0] you can totally grab the video.
1.) Latency vs quality doesn't come up enough to make people want to A/B test it unfortunately. At work I would say ~5 people care about WebRTC vs QUIC vs X. All effort is around the models (how can I provide tools to be support those doing that work)
2.) The model isn't processing just text anymore. Also taking into account breathing/emotion etc... not just spitting out big responses anymore. As it generates them it is taking into account the users response.
3.) It works with the LB setup today. Clients are sending ICE traffic, if it roams we lookup the ufrag and route appropriately.
4.) With DTLS 1.3 it is 1 RTT with SNAP[0] for WebRTC session. SCTP info goes in Offer/Answer, DTLS is packed into ICE. You are totally right about signaling though! [1] was my answer for doing WebRTC without signaling, couldn't get anyone to care though.
5.) I don't have anything that I need to tune. If I want to increase (or decrease) latency [3] is something I put into Transceiver. Otherwise I can't think of any 'change this WebRTC behavior' that has been asked by users/developers.
Responding to some technical points first, but then after that I do see a future that isn't WebRTC. I don't think it matches where WebTransport+WebCodecs etc is going though.
> …but as a user, I would much rather wait an extra 200ms for my slow/expensive prompt to be accurate
This is the opposite of the feedback I get. Users want instant responses. If you have delay in generating responses/interruptions it kills the magic. You also don't want to send faster than real-time. If the user interrupts the model you just wasted a bunch of bandwidth sending 3 minutes of audio (but only played 10 seconds)
You lose stuff like AEC. You also push complexity on clients. The simplicity of WebRTC (createOffer -> setRemoteDescription) is what lets people onboard easily. Lots of developers struggled with Realtime API + web sockets (lots of code and having to do stuff by hand)
----
I think if I had my choice I would pick Offer/Answer model and then doing QUIC instead of DTLS+SCTP. Maybe do RTP over QUIC? I personally don't feel strongly about the protocol itself. I don't know how to ship code to multiple clients (and customers clients) with a much large code footprint.
Socials: - github.com/Sean-Der - linkedin.com/in/sean-dubois - siobud.com - pion.ly
---