Yeah, awesome. Have to say this is the best version of something like this that I have tried. The packet capture didn't seem to be RTP, are you using libwebrtc under the hood?
The system seems to have the settings right for realtime collab. In fact I found it much better for chatting too if you can guarantee they have a headset and good mic - I hate current-gen AEC algorithms. Interested to see how it fares in a 3+ participant setting.
I had a go at jamming with my brother yesterday. We had to adjust down the delay to 2ms (I think default was 5ms) in order to counteract the "lagging" effect you get when you lock into the remote beat. Once we had that tuned it worked really well. We would occasionally suffer burst losses, but you can play through it and it's still synced afterward which is the best you can do with an unreliable network.
Some feature requests:
- Join room by name (we were confused that there was a different code than the name we chose)
- Auto register new audio devices when you plug them in/edit in settings.
- An easier way to join with multiple audio inputs. We had trouble setting up our instrument, but also adding a mic for chatting.
- Local recording of your end with some shared sync markers, that you could manually or automatically sync up post-hoc
The use case I use WebRTC for is slightly different, where quality >> latency , so we have it tuned to the other end and run the full gambit of FEC, ARQ, packet redundancy etc. Interesting to hear about other's solutions though!
Hey @weepy. Awesome project! As a webrtc dev - interested to hear about your choices around opus packet durations, FEC percentages, ARQ strategies, jitterbuffer length and packet redundancy.
The system seems to have the settings right for realtime collab. In fact I found it much better for chatting too if you can guarantee they have a headset and good mic - I hate current-gen AEC algorithms. Interested to see how it fares in a 3+ participant setting.
I had a go at jamming with my brother yesterday. We had to adjust down the delay to 2ms (I think default was 5ms) in order to counteract the "lagging" effect you get when you lock into the remote beat. Once we had that tuned it worked really well. We would occasionally suffer burst losses, but you can play through it and it's still synced afterward which is the best you can do with an unreliable network.
Some feature requests: - Join room by name (we were confused that there was a different code than the name we chose) - Auto register new audio devices when you plug them in/edit in settings. - An easier way to join with multiple audio inputs. We had trouble setting up our instrument, but also adding a mic for chatting. - Local recording of your end with some shared sync markers, that you could manually or automatically sync up post-hoc
The use case I use WebRTC for is slightly different, where quality >> latency , so we have it tuned to the other end and run the full gambit of FEC, ARQ, packet redundancy etc. Interesting to hear about other's solutions though!