congratulations, looks cool!
I'm one of the persons behind https://laptopsinspace.de and it would probably be interesting to share some experiences.
Our tech stack is similar, yet different.
We also use mediasoup for the video, Svelte for the frontend and ThreeJS as the 3D framework!
shameless plug: we saw a similar problem, but our solution is spatial video calls in 3D instead of 2D: https://laptopsinspace.com
it's also built on top of the jitsi meet api but uses an extra backend for managing the game data.
that normally only happens if your webcam is already in use, but we surely need better user feedback for that...
I'd be happy to trouble shoot it with you on twitter, dm me at @neeshstudio
I already commented below, but that's pretty much what we are aiming at with https://laptopsinspace.de
It's not VR (on purpose), but instead it has facial expressions and some gaming feel to it
Thanks :) we have a couple of features lined up specifically targeted at (music) events, but are also putting some energy into making it suitable for artist talks and panels.