Odd thought: Matrix allows a number of types of messages, which include URLs et al. You need a TURN server for voice and video: Matrix doesn't carry these, and direct connection might be impossible, thus the TURN server.
I'm curious the practical implications—performance, latency, storage, etc.—of using an Ogg container and Opus encoding to stream through Matrix. Consider the protocol—essentially a system of relays like IRC or e-mail—with a message type amounting to an Ogg page.
Opus uses CELP for voice data, and can get voice down to 5kbit/s easy enough. You can flood a server with a ton of text messages; but Matrix servers support client rate limiting, and text messages are small.
If you're allowing stream containers, you have to allow 38 packets per second at 27 bytes of actual data per packet to carry 8kbit/s voice. For 64kbit/s (other audio), you're looking at 211 bytes of data per packet.
I think, at a 10kbit/s limit, 38 packets, per user, streaming voice is reasonable. With Matrix encrypted rooms, you can even store those packets on the server, time stamped, end-to-end encrypted, so a user can scroll back and play the on-screen audio (streaming audio history).
I'm curious the practical implications—performance, latency, storage, etc.—of using an Ogg container and Opus encoding to stream through Matrix. Consider the protocol—essentially a system of relays like IRC or e-mail—with a message type amounting to an Ogg page.
Opus uses CELP for voice data, and can get voice down to 5kbit/s easy enough. You can flood a server with a ton of text messages; but Matrix servers support client rate limiting, and text messages are small.
If you're allowing stream containers, you have to allow 38 packets per second at 27 bytes of actual data per packet to carry 8kbit/s voice. For 64kbit/s (other audio), you're looking at 211 bytes of data per packet.
I think, at a 10kbit/s limit, 38 packets, per user, streaming voice is reasonable. With Matrix encrypted rooms, you can even store those packets on the server, time stamped, end-to-end encrypted, so a user can scroll back and play the on-screen audio (streaming audio history).
Viable or stupid?