Show HN: WebRTC signalling data using QR codes(github.com)
github.com
Show HN: WebRTC signalling data using QR codes
https://github.com/AquiGorka/webrtc-qr
54 comments
This is great, thanks for posting it. I've had the same idea myself and started and abandoned this exact same project a few times. Same down to cutting the SDP up into multiple QR codes. I am glad to see this brought to life.
I am also surprised to see @nilknarf has done that in 2014! (As per their comment.) Don't know how I didn't come across his post during my research into this, but checking it now, someone in the comments posted a link to a patent which captures this idea.
http://www.freepatentsonline.com/20160021148.pdf
The patent must be invalid, because it's from 2016 and there is clearly prior art, but it kind of sucks, because it basically means polishing something like e.g. @phiresky has done with the remote pen just means if you succeed in bringing down the time to connect and make it something usable enough for users, you're opening yourself up to a claim if you decide to package it up into an app.
I am also surprised to see @nilknarf has done that in 2014! (As per their comment.) Don't know how I didn't come across his post during my research into this, but checking it now, someone in the comments posted a link to a patent which captures this idea.
http://www.freepatentsonline.com/20160021148.pdf
The patent must be invalid, because it's from 2016 and there is clearly prior art, but it kind of sucks, because it basically means polishing something like e.g. @phiresky has done with the remote pen just means if you succeed in bringing down the time to connect and make it something usable enough for users, you're opening yourself up to a claim if you decide to package it up into an app.
> opening yourself up to a claim if you decide to package it up into an app
Even though there is public domain tech from 2014 using it? (@nilknarf's).
I was under the impression when something is part of the public domain there can't be a patent claim on it.
Even though there is public domain tech from 2014 using it? (@nilknarf's).
I was under the impression when something is part of the public domain there can't be a patent claim on it.
I think this patent has been mistakenly granted this year:
https://patents.google.com/patent/US20160021148A1 (see the "grant" status line)
I think it should not have been granted there being prior art, but for it to be turned over, I think it would have to be challenged in court. That's my limited understanding as a non-US person without any experience with patents whatsoever.
https://patents.google.com/patent/US20160021148A1 (see the "grant" status line)
I think it should not have been granted there being prior art, but for it to be turned over, I think it would have to be challenged in court. That's my limited understanding as a non-US person without any experience with patents whatsoever.
Oh wow, I am shocked to see the patent was granted and I'm very interested in understanding the repercussions. These might be big.
Nice!
I also now of this Android App that uses WebRTC Audio/Video without server (no Internet or DHCP needed): https://github.com/dakhnod/Meshenger
I made a submission, in case somebody likes to discuss the need for serverless WebRTC: https://news.ycombinator.com/item?id=18203495
(Disclaimer: I am not the author of the App, but I helped out a bit)
I also now of this Android App that uses WebRTC Audio/Video without server (no Internet or DHCP needed): https://github.com/dakhnod/Meshenger
I made a submission, in case somebody likes to discuss the need for serverless WebRTC: https://news.ycombinator.com/item?id=18203495
(Disclaimer: I am not the author of the App, but I helped out a bit)
> Contacts are found by taking the devices own IPv6 addresses and replacing the devices own MAC address by the contacts MAC address.
You require the devices to be on the same network for this to work right?
You require the devices to be on the same network for this to work right?
Yes, that is (mostly) true.
The App uses ipv6 link local addresses (fe80::.. + mac address). But it also tries to make use the received IPv6 network prefixes. Also, the last contact address is stored. These approaches improves the probability a bit to find the contacts in other networks.
As a side note, many mesh networks are emulating a layer 2 network. With this, everybody can build up their own decentralized telephone network.
The App uses ipv6 link local addresses (fe80::.. + mac address). But it also tries to make use the received IPv6 network prefixes. Also, the last contact address is stored. These approaches improves the probability a bit to find the contacts in other networks.
As a side note, many mesh networks are emulating a layer 2 network. With this, everybody can build up their own decentralized telephone network.
> everybody can build up their own decentralized telephone network
Nice. I'm very interested in being able to connect devices in areas where there is no Internet and using a mobile router, like in the mountains or in a bus trip through deserted areas.
Nice. I'm very interested in being able to connect devices in areas where there is no Internet and using a mobile router, like in the mountains or in a bus trip through deserted areas.
I've wondered about other signalling mechanisms. I figure you could use a DHT (js-ipfs/js-libp2p or webtorrent BT mainline) and put the offer directly on the DHT (AES'd?) or if it's too big to be a key, seed it as the stored value for its hash using those systems.
I've worked with `js-ipfs` and their `js-libp2p`, they use a signalling server (actually two types of them):
- https://github.com/libp2p/js-libp2p-webrtc-star
- https://github.com/libp2p/js-libp2p-websocket-star
That would be going back to using a server for the handshake.
In regards to using a DHT, that's something I have not researched. I am not sure how long an offer is valid, were you thinking of being able to update them as needed whenever you want to establish a connection?
- https://github.com/libp2p/js-libp2p-webrtc-star
- https://github.com/libp2p/js-libp2p-websocket-star
That would be going back to using a server for the handshake.
In regards to using a DHT, that's something I have not researched. I am not sure how long an offer is valid, were you thinking of being able to update them as needed whenever you want to establish a connection?
> In regards to using a DHT, that's something I have not researched. I am not sure how long an offer is valid, were you thinking of being able to update them as needed whenever you want to establish a connection?
Yes. I have toyed with js-libp2p to obtain "providers" based on a key, but I believe the part of js-libp2p that allows you to participate in the DHT isn't quite done [0]. I imagine you'd make a CID that would be a simple hash of some shared string. Then you'd tell the network you are a "provider" with that CID, and when requested, you give the offer (AES'd w/ shared string or other shared secret?). The client can just connect to the same DHT and give the hash and get the (encrypted?) offer. Not sure if IPFS has a way for me to prevent other people from serving/replicating the CID I am serving instead of forcing it to be a hash of the content. A similar approach can happen on BT mainline w/ an infohash maybe.
0 - https://github.com/ipfs/js-ipfs/pull/856
Yes. I have toyed with js-libp2p to obtain "providers" based on a key, but I believe the part of js-libp2p that allows you to participate in the DHT isn't quite done [0]. I imagine you'd make a CID that would be a simple hash of some shared string. Then you'd tell the network you are a "provider" with that CID, and when requested, you give the offer (AES'd w/ shared string or other shared secret?). The client can just connect to the same DHT and give the hash and get the (encrypted?) offer. Not sure if IPFS has a way for me to prevent other people from serving/replicating the CID I am serving instead of forcing it to be a hash of the content. A similar approach can happen on BT mainline w/ an infohash maybe.
0 - https://github.com/ipfs/js-ipfs/pull/856
I created swarm-peer-server [1] as a way to connect to specific peers over a DHT using a public key known ahead of time. I use this in a networked app I've built to perform WebRTC signaling between peers [2]. The networking and crypto works using modules popular in the Dat Project community.
The challenge is that multiple peers could announce under the same hash on the DHT. Thus I have code to perform a handshake to verify peers upon connecting.
Another project worth mentioning is hyperswarm [3], also created by the Dat Project folks. It's planned to be their new method of connecting peers on a DHT in Beaker Browser.
[1] https://www.npmjs.com/package/swarm-peer-server
[2] https://github.com/samuelmaddock/metastream/blob/master/app/...
[3] https://pfrazee.hashbase.io/blog/hyperswarm
The challenge is that multiple peers could announce under the same hash on the DHT. Thus I have code to perform a handshake to verify peers upon connecting.
Another project worth mentioning is hyperswarm [3], also created by the Dat Project folks. It's planned to be their new method of connecting peers on a DHT in Beaker Browser.
[1] https://www.npmjs.com/package/swarm-peer-server
[2] https://github.com/samuelmaddock/metastream/blob/master/app/...
[3] https://pfrazee.hashbase.io/blog/hyperswarm
Thank you for sharing. Metastream is very nice! I thought I was following Paul Frazee's blog but I wasn't (updated my feeds now) so, thanks for the link to the `hyperswarm` post too.
Thank you for sharing, will keep an eye out for that thread.
I did some research a year ago (https://aquigorka.net/post/how-to-find-peers-in-the-decentra...) and going back to it I found this repo that uses the DHT to connect peers: https://github.com/mafintosh/peer-network
The author is one of the main contributors to the Dat project so I'll do some catching up to see what they've come up with.
I did some research a year ago (https://aquigorka.net/post/how-to-find-peers-in-the-decentra...) and going back to it I found this repo that uses the DHT to connect peers: https://github.com/mafintosh/peer-network
The author is one of the main contributors to the Dat project so I'll do some catching up to see what they've come up with.
Although it uses Tor and is probably outdated, here is a PoC I wrote of bootstrapping a DHT network on the server side and using the client side to discover peers: https://github.com/cretz/tor-dht-poc
Cool. In case you are interested in a similar approach using signaling through audio (you might say an "audio QR code"): https://github.com/ggerganov/wave-share
Yeah! I was familiar with it, very cool.
Did you happen to see my post on this?
https://franklinta.com/2014/10/19/serverless-webrtc-using-qr...
(The demo unfortunately doesn't work anymore since it's from 2014)
The flow is very similar to yours, down to the flashing QR codes! https://youtu.be/rw4f9LfqqpI?t=8
The flow is very similar to yours, down to the flashing QR codes! https://youtu.be/rw4f9LfqqpI?t=8
Hey there, I wasn't aware you had worked on the same thing (and 4 years ago!). Amazing, my hat is off for you.
Did you continue to experiment with it? Improve the experience? Try out different approaches?
Did you continue to experiment with it? Improve the experience? Try out different approaches?
There were a few interesting ideas by others in the original comment thread. Mostly on how to compress the sdp so you can get away with just one large QR code without flashing (for example by presharing a dictionary or stripping the sdp down to the bare minimum: https://webrtchacks.com/the-minimum-viable-sdp/).
I lost interest in this approach because it was way more practical to give up on serverless in exchange for only scanning in one direction instead of both ways (e.g., scan a QR code once on mobile to both open the webpage and join a websocket channel, then use it to upgrade to a webrtc connection). It's a neat trick for use cases like AR.js where this is natural to do: https://github.com/jeromeetienne/AR.js
I lost interest in this approach because it was way more practical to give up on serverless in exchange for only scanning in one direction instead of both ways (e.g., scan a QR code once on mobile to both open the webpage and join a websocket channel, then use it to upgrade to a webrtc connection). It's a neat trick for use cases like AR.js where this is natural to do: https://github.com/jeromeetienne/AR.js
Doesn't work in Edge because createDataChannel isn't supported: https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConn...
Thank you for pointing that out I wasn't aware Edge has no support for sending data over WebRTC.
[deleted]
Cool idea, however I could not get it to work with Chrome on a laptop as a host (with the moving qr codes) and a S5 phone (join) also running Chrome pointing at the laptop (it could see the qr codes on the transparent video). It didn't connect.
I'm sorry to hear you could not get it to work, this means I have to improve the flow. I tested this using Chrome and both Android and iOS devices (chrome and safari).
The catch is that you gotta point the series of qr codes towards the center of the camera.
Seems like you can only host, not join, from Mobile Safari?
I think industries are ready to adopt real-time voice, texts, and video communication solutions to improve customer experience.
Why does the QR code constantly change?
I wasn't able to get all the info into one QR code, so I used a series of them (looping) to send all the data. I figure I can use this approach to send files too.
What is the capacity of those QR codes?
When I played with WebRTC, I was able to pack the relevant SDP data to just 80 bytes:
- 1 Type of the SDP - Offer or Answer
- 1 Packet size in bytes (not including ECC bytes)
- 4 IP address of the transmitting peer
- 2 Network port that will be used for the communication
- 32 SHA-256 fingerprint of the session data
- 40 ICE Credentials - 16 bytes username + 24 bytes password
When I played with WebRTC, I was able to pack the relevant SDP data to just 80 bytes:
- 1 Type of the SDP - Offer or Answer
- 1 Packet size in bytes (not including ECC bytes)
- 4 IP address of the transmitting peer
- 2 Network port that will be used for the communication
- 32 SHA-256 fingerprint of the session data
- 40 ICE Credentials - 16 bytes username + 24 bytes password
I found a similar limit (although I still want to try with different options for the qr codes). I send the data as is by splitting it into smaller segments and looping through them with more than one qr code.
Will this work across NATs?
This is still WebRTC, so the same limitations apply. The QR codes are used to share the `signalling data` for the WebRTC connection.
More detail: Yes, WebRTC contains several methods for routing through NATs -- ICE, STUN by default (using your browser's default STUN server), optionally TURN.
My browser manufacturer has a STUN server? Where can I find that? (e.g., for Chrome or Firefox)
Edit: AFAICT, this isn't correct. Your browser has no notion of a "default" (or any other) STUN/TURN server. The JavaScript doing WebRTC has to specify one.
That said Google runs one, at stun.l.google.com:19302; but it isn't clear if that's something third-parties could or should make use of (or if Google even permits such activity), though it seems like plenty of examples (and perhaps even real-world uses) make use of it.
Edit: AFAICT, this isn't correct. Your browser has no notion of a "default" (or any other) STUN/TURN server. The JavaScript doing WebRTC has to specify one.
That said Google runs one, at stun.l.google.com:19302; but it isn't clear if that's something third-parties could or should make use of (or if Google even permits such activity), though it seems like plenty of examples (and perhaps even real-world uses) make use of it.
In the past, I've seen this Google STUN server used on Chrome when none is supplied by the user JS. This may have changed in recent years.
https://www.youtube.com/watch?v=Gvsm84xL9Sk (https://github.com/phiresky/webrtc-remote-touch-pen-input)
Sadly it needs a (tiny) server component to exchange the initial connection information - using a QR code in both directions is smart, but also pretty annoying for the user.