Support HTTP over Unix domain sockets(bugzilla.mozilla.org)
bugzilla.mozilla.org
Support HTTP over Unix domain sockets
https://bugzilla.mozilla.org/show_bug.cgi?id=1688774
54 comments
curl can do this: https://curl.se/libcurl/c/CURLOPT_UNIX_SOCKET_PATH.html
It's pretty useful in test suites where you don't want to open a public TCP port (even on localhost), for obvious security reasons but also because it's hard to pick an unused port without conflicting with other tests running at the same time. Example test using this: https://gitlab.com/nbdkit/nbdkit/-/blob/master/tests/test-cu...
It's pretty useful in test suites where you don't want to open a public TCP port (even on localhost), for obvious security reasons but also because it's hard to pick an unused port without conflicting with other tests running at the same time. Example test using this: https://gitlab.com/nbdkit/nbdkit/-/blob/master/tests/test-cu...
> you don't want to open a public TCP port (even on localhost), for obvious security reasons
Can someone explain this a bit more? It's my understanding that all localhost traffic is confined to the host, and so only available to the host itself. If something malicious could access traffic on localhost, wouldn't that mean you've already lost?
Can someone explain this a bit more? It's my understanding that all localhost traffic is confined to the host, and so only available to the host itself. If something malicious could access traffic on localhost, wouldn't that mean you've already lost?
A control socket for a local daemon should only be accessible by authorised users or root. TCP sockets don’t help you with that, but Unix domain sockets are subject to regular file system access control.
On a multi-user machine, as a regular user, serving over TCP immediately allows any other logged-on users to access it, which can be unacceptable in some circumstances.
On a multi-user machine, as a regular user, serving over TCP immediately allows any other logged-on users to access it, which can be unacceptable in some circumstances.
It's true, but on a CI machine you may have other tests running in parallel. Even perhaps tests being run by other users/tenants.
Unix domain sockets can be confined to a randomly generated directory under /tmp and locked down with file permissions.
Unix domain sockets can be confined to a randomly generated directory under /tmp and locked down with file permissions.
It's possible to bind a server so that the OS chooses which port it listens on for you, and if you set up your test infrastructure to communicate that port to the things that need to connect to said server, you can happily run several tests in parallel. This is how Firefox's networking tests run in parallel, despite many of them requiring spawning an HTTP server to connect to.
What you say is only true if your firewall is configured to do so. By default (without any firewall), all traffic is exposed to every interface I believe.
EDIT: I stand corrected
EDIT: I stand corrected
This is not correct. Binding a socket to localhost (UDP, or TCP listener, or SCTP listener, ...) will prevent traffic from other network interfaces using it, even on the same machine.
virtus ~ # nft list ruleset
virtus ~ # nc -vvv -l -s 127.0.0.1 -p 1234
Listening on localhost 1234
virtus ~ # ip -4 addr show scope global dev enp3s0
2: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
inet 10.20.1.75/24 brd 10.20.1.255 scope global enp3s0
valid_lft forever preferred_lft forever
virtus ~ # nc -vvv 10.20.1.75 1234
nc: connect to 10.20.1.75 port 1234 (tcp) failed: Connection refused
If you use 0 for the port, the Linux kernel will assign you a random port.
It's more of a coordination problem. Even if the web server could be persuaded to use port 0 (most will barf at this), I now have to find out what port was assigned by the kernel and communicate that to the test client. Unix domain sockets just make this whole problem go away.
Also running out of ports is a thing! There are only 64K ports (with many not being usable) and a CI machine might be running many tests in parallel. Unix domain sockets give me an infinite space of ports, since they use filenames.
Also running out of ports is a thing! There are only 64K ports (with many not being usable) and a CI machine might be running many tests in parallel. Unix domain sockets give me an infinite space of ports, since they use filenames.
It isn't infinite space. The buffer for filenames is only 256 bytes, so 255 including the NUL terminator. But still, that is a ton more socket names than ports.
You can have arbitrarily complex names because filesystems are a tree, not a flat namespace. The limit you'll hit is either inodes or kernel memory, neither of which will run out all that easily.
That's not just Linux, that's a more global IP sockets thing.
This sounds great - and I wish UNIX namespaces were adopted more widespread for things like "local configurtion web UI".
As a workaround, (I believe, though only through a cursory web search and haven't tested) you can use network namespaces and socat to proxy traffic from Unix sockets to a private IP:port range only visible to your user - taking that part of containerization without necessarily having to execute things inside containers.
As a workaround, (I believe, though only through a cursory web search and haven't tested) you can use network namespaces and socat to proxy traffic from Unix sockets to a private IP:port range only visible to your user - taking that part of containerization without necessarily having to execute things inside containers.
The actually interesting discussion is here: https://github.com/whatwg/url/issues/577 with the more constructive and detailed proposition starting from here: https://github.com/whatwg/url/issues/577#issuecomment-118534...
I like the idea of specifying the socket in place of the port number in the URL.
I like the idea of specifying the socket in place of the port number in the URL.
... which of course won't happen because the port can only be digits, and changing the URL spec would affect the whole world
> Getting this to actually work would be a bit difficult, for very little benefit.
I wonder why an actually secure local connection to a web UI is considered to be of very little benefit. This sounds crazy to me, for all these years.
I wonder why an actually secure local connection to a web UI is considered to be of very little benefit. This sounds crazy to me, for all these years.
Yeah I don't think that comment aged too well. Not to mention the explosion of microservices and autoscaling temp services, this would be very useful.. I can appreciate the reluctancy though, it's not a trivial change to get right.
What would make that change non-trivial, to your mind?
Currently it can be achieved by a program as non-trivial as netcat
Also I don't share their concerns about SNI. To my mind, the Host header can just be unsupported for Unix sockets. SNI makes sense when the several domain names resolve to the same IP address. This should not be an important, case with Unix sockets. Not supporting it is a much better compromise than not supporting Unix sockets at all.
Currently it can be achieved by a program as non-trivial as netcat
Also I don't share their concerns about SNI. To my mind, the Host header can just be unsupported for Unix sockets. SNI makes sense when the several domain names resolve to the same IP address. This should not be an important, case with Unix sockets. Not supporting it is a much better compromise than not supporting Unix sockets at all.
I can't speak to the entire spec and quit frankly I don't know enough, but this comment seems to highlight some of the nuances that need to be considered
https://github.com/whatwg/url/issues/577#issuecomment-118534...
It's not insurmountable absolutely and I would appreciate it absolutely.
It's not insurmountable absolutely and I would appreciate it absolutely.
I see that all these problems are interesting details. But I find them just that, details. Many of them can be ignored in an initial implementation, in a way that would allow to fill them in later, if need be. Somehow the lack of handling of all these subtleties does not preclude the file:/ scheme from being implemented.
If there is a will, there is a way. I think the will is lacking here. The developers themselves do no care. I can see why the Chrome team is opposed to that: it's the Google's interest to eliminate all things offline. I didn't expect Mozilla folks to be so unreceptive though. Having the unix domain socket support would be a distinct advantage.
If there is a will, there is a way. I think the will is lacking here. The developers themselves do no care. I can see why the Chrome team is opposed to that: it's the Google's interest to eliminate all things offline. I didn't expect Mozilla folks to be so unreceptive though. Having the unix domain socket support would be a distinct advantage.
CUPS listening on a localhost port and asking you to authenticate with your username and actual password has long bothered me. (I know it’s a privileged port, I just don’t like the idea of ever typing my local password into any browser, ever.)
I haven’t seen this proposal yet in this thread: instead of convincing browser vendors to support connecting to Unix domain sockets, contrive a tiny adapter which listens on a localhost port and expects http with basic auth (and either generates a random username and password or accepts them by environment variables), prints out a url with the password like
You could do this with nginx really easily but you’d have to keep track of a config file for each service.
See https://stackoverflow.com/questions/17701420/bypassing-http-... for a similar idea (but that is about stripping auth, not adding it)
I haven’t seen this proposal yet in this thread: instead of convincing browser vendors to support connecting to Unix domain sockets, contrive a tiny adapter which listens on a localhost port and expects http with basic auth (and either generates a random username and password or accepts them by environment variables), prints out a url with the password like
Hi! To connect to your service at /tmp/whatever-service.unix, tell your browser to go to
http://fwahjiddbjko:derhhkiytdfbkifdx@localhost:45678/
You could do this with nginx really easily but you’d have to keep track of a config file for each service.
See https://stackoverflow.com/questions/17701420/bypassing-http-... for a similar idea (but that is about stripping auth, not adding it)
Especially that nowadays password authentication may fade away soon.
Windows already supports password-less authentication quite well. It's just a matter of time until there are good solutions for Linux too.
I have already some systems set up in a way that they ask for a TOTP when doing username/password login via SSH: https://github.com/google/google-authenticator-libpam
Windows already supports password-less authentication quite well. It's just a matter of time until there are good solutions for Linux too.
I have already some systems set up in a way that they ask for a TOTP when doing username/password login via SSH: https://github.com/google/google-authenticator-libpam
I use nginx for that. I would like to see more software that provides the option to listen to unix sockets. Most still won't work with unix sockets. Some prometheus exporters at least support systemd socket units.
I wrote an LD_PRELOAD library which can replace binds to a TCP port with binds to a UNIX socket: https://github.com/AGWA/ldpreload#ldpreload-unixbindso
NGINX solves half of the problem. It allows accessing UNIX socket services over TCP. This is great and I do it wherever possible (mostly because I can add my own access control and everything has to go through it, even local processes). But it doesn't allow local access without going over TCP.
So for example if CUPS or Syncthing listened on a UNIX socket you can use NGINX to expose this over TCP, but now you are back at the original problem of any local process can access this service and you need to worry about port conflicts. You can work around this by adding access control but this doesn't elegantly solve the core problem.
So it is valuable to have a client (like Firefox) that can directly access the socket. This way you get native UNIX permissions and a whole filesystem namespace to prevent conflicts.
So for example if CUPS or Syncthing listened on a UNIX socket you can use NGINX to expose this over TCP, but now you are back at the original problem of any local process can access this service and you need to worry about port conflicts. You can work around this by adding access control but this doesn't elegantly solve the core problem.
So it is valuable to have a client (like Firefox) that can directly access the socket. This way you get native UNIX permissions and a whole filesystem namespace to prevent conflicts.
Another LD_PRELOAD based solution can be found here: https://github.com/nixcloud/ip2unix
I use it to run a number of services under systemd, and haven’t had any issues.
I use it to run a number of services under systemd, and haven’t had any issues.
maybe systemd will come up with some kind of sandboxing that automaticaly converts TCP servers to unix sockets and unix sockets to TCP listeners available only in network namespace of individual users :-D
Actualy namespacing syncthing per user might be enough for every user at same system to run syncthing at tcp/8080 or whatever.
Actualy namespacing syncthing per user might be enough for every user at same system to run syncthing at tcp/8080 or whatever.
Ask and ye shall receive: https://www.freedesktop.org/software/systemd/man/latest/syst...
I use it for the usecase you describe.
I use it for the usecase you describe.
But, Docker doesn't even, and I'm on Windows, whoever needed access control, waah waah.
</s>
Unix domain sockets are great. I find it completely amazing that they are not used more widely.
Since when was it OK to open a TCP listener when you don't want to receive connections over the network? Did everyone forget about that during their Kuberenetes fling, am I too old, or what?
Unix domain sockets are great. I find it completely amazing that they are not used more widely.
Since when was it OK to open a TCP listener when you don't want to receive connections over the network? Did everyone forget about that during their Kuberenetes fling, am I too old, or what?
This would lead to a massive confused deputy vulnerability for unix domain sockets as already exists for localhost + port.
For a great example of this, see how Guile's live REPL was localhost + port... cool, only local users could access it, right? Except browsers could access localhost + port, and it turned out this was a path to being able to do arbitrary code execution in the browser https://lists.gnu.org/archive/html/guile-user/2016-10/msg000...
Switching to unix domain sockets was the recommended path, and that's only because browsers don't support them.
If you want to support unix domain sockets, you could, but it would have to be via object capability security discipline, and the poster explicitly talks about an ACL "protecting" things... it wouldn't.
Luckily this is 3 years old and hopefully will never make progress.
For a great example of this, see how Guile's live REPL was localhost + port... cool, only local users could access it, right? Except browsers could access localhost + port, and it turned out this was a path to being able to do arbitrary code execution in the browser https://lists.gnu.org/archive/html/guile-user/2016-10/msg000...
Switching to unix domain sockets was the recommended path, and that's only because browsers don't support them.
If you want to support unix domain sockets, you could, but it would have to be via object capability security discipline, and the poster explicitly talks about an ACL "protecting" things... it wouldn't.
Luckily this is 3 years old and hopefully will never make progress.
> Switching to unix domain sockets was the recommended path, and that's only because browsers don't support them.
Maybe the recommended path should have been to implement some actual security?
> https://lists.gnu.org/archive/html/guile-user/2016-10/msg000... > DNS rebinding attack
Can't DNS rebind to a "unix socket address" — feels like that by itself would considerably improve security for anything you're working on locally?
Maybe the recommended path should have been to implement some actual security?
> https://lists.gnu.org/archive/html/guile-user/2016-10/msg000... > DNS rebinding attack
Can't DNS rebind to a "unix socket address" — feels like that by itself would considerably improve security for anything you're working on locally?
The WHATWG discussion (linked from OP thread) is also interesting:
https://github.com/whatwg/url/issues/577
https://github.com/whatwg/url/issues/577
I'm not convinced that this is a good idea for a general purpose browser.
For web based desktop applications this might be a good solution. But there it is usually done with custom IPC. I think Electron can do it with custom protocols.
Listening on localhost is usually not a great solution. It's impossible to do HTTPS right this way (without HTTPS there is a danger of a MITM attack from an unprivileged process). Also authentication is an issue then, without authentication there is a possibility of an privilege escalation.
For web based desktop applications this might be a good solution. But there it is usually done with custom IPC. I think Electron can do it with custom protocols.
Listening on localhost is usually not a great solution. It's impossible to do HTTPS right this way (without HTTPS there is a danger of a MITM attack from an unprivileged process). Also authentication is an issue then, without authentication there is a possibility of an privilege escalation.
I don't understand how MITM could happen here? I'd assume that the server process would create the UNIX socket and then somehow open the path to the socket in the browser (or print a URL which a user can paste). If another process already has a UNIX socket at that path then surely the server process would just error?
I think using a UNIX socket instead of a TCP server for local HTTP development would be extremely useful. It solves most of the problems associated with creating a local TCP server (which everyone already does) without introducing new problems.
I think using a UNIX socket instead of a TCP server for local HTTP development would be extremely useful. It solves most of the problems associated with creating a local TCP server (which everyone already does) without introducing new problems.
When listening on localhost on an unprivileged TCP port any other process could open the port first and provide a similar UI. Opening the port may even trigger the original process to chose another free port and the malicious software could take the requests from the users browser on the usual port and forward them to the new port.
UNIX sockets would of course solve this issue.
UNIX sockets would of course solve this issue.
Unix domain sockets support peer authentication, which is way better than anything network sockets do, and the threat that you need HTTPS to protect against doesn't exist with them.
Yeah, that's why it's better than having a HTTP server listening on localhost. And why listening on localhost is often not a good idea.
But UNIX sockets are not available on all platforms, which defeats a bit the general idea of a browser. Windows got UNIX socket support a while ago, but I don't know how well it works.
But UNIX sockets are not available on all platforms, which defeats a bit the general idea of a browser. Windows got UNIX socket support a while ago, but I don't know how well it works.
What would be wrong with browsers only supporting UNIX domain sockets on platforms where they exist?
> I would argue that the main reason it seems niche is because of the lack of browser support.
This can't be said enough times. Browser vendors should never, ever say "we're not supporting feature X because it's not popular enough, and other browsers don't support it" when lack of browser support is what's keeping it from getting popular. We'd literally never get any new features if they always used that logic.
This can't be said enough times. Browser vendors should never, ever say "we're not supporting feature X because it's not popular enough, and other browsers don't support it" when lack of browser support is what's keeping it from getting popular. We'd literally never get any new features if they always used that logic.
No update in 3 years, not sure it will be implemented in foreseeable future
Trackers will be able to gain root access by fetching the user’s local Docker API socket, and by doing so provide better personalized ads. This is really good, sign me in!
I assume there are some clever gdb hacks out there to swap any http fd with another.
An LD_PRELOAD library would probably work. I wrote one which can make servers listen on UNIX domain sockets (https://github.com/AGWA/ldpreload#ldpreload-unixbindso) which would be a good starting point.
that's awesome
I saw a similar hack https://tldp.org/HOWTO/html_single/TCP-Keepalive-HOWTO/#libk...
I saw a similar hack https://tldp.org/HOWTO/html_single/TCP-Keepalive-HOWTO/#libk...
Not exactly what you mentioned, but you can do some fun tricks with https://github.com/cloudflare/tubular
why do i have to be exposed to these people who don't have the slightest clue about how the computer works? i want out. i don't want to use an ecosystem that includes these people to be part of what runs on my computer. why would i want arbitrary web pages to be able to connect to all my daemons that expected that outsiders can't connect to them including stuff like X and god knows what else. it's bad enough that browsers can already connect to localhost which has already single handedly enabled hundreds of thousands of vulnerabilities in the form of "attacker's page (even with js disabled) accesses your local daemon from some 10th level nested iframe ad crap"
i want out of this community. all these little people who have zero contextual understanding always requesting and implementing these features at all costs as long as the idea exists. this is a un*x problem at the core: if a feature conceptually exists, we must implement it at all costs. and another un*x problem here is having a global namespace (tcp port numbers) that things are just exposed to everything (either localhost or the network) by default when they could have easily just used an opaqaue handle that the machine operator can copy and paste into whatever application he wants to use it (no some openauth type shit that failed to be secure for 15 years is not what i have in mind)
i want out of this community. all these little people who have zero contextual understanding always requesting and implementing these features at all costs as long as the idea exists. this is a un*x problem at the core: if a feature conceptually exists, we must implement it at all costs. and another un*x problem here is having a global namespace (tcp port numbers) that things are just exposed to everything (either localhost or the network) by default when they could have easily just used an opaqaue handle that the machine operator can copy and paste into whatever application he wants to use it (no some openauth type shit that failed to be secure for 15 years is not what i have in mind)
> why would i want arbitrary web pages to be able to connect to all my daemons
Browsers implementing support for unix domain sockets would of course need to completely block such connection from tcp pages and only allow connection to a given socket provided it is the one currently in the url bar, that the current page has been loaded from.
If that's not enough, you can always use the file permission system to block your everyday browser running as your regular user to access the service sockets, and spawn a browser using a dedicated user account (www-sock? just like we have www-data for web servers?) that you only use for this.
> when they could have easily just used an opaque handle that the machine operator can copy and paste into whatever application he wants to use it
Then the point of failure would be the random number generator used to generate the "opaque handle". Security by obscurity never works.
Browsers implementing support for unix domain sockets would of course need to completely block such connection from tcp pages and only allow connection to a given socket provided it is the one currently in the url bar, that the current page has been loaded from.
If that's not enough, you can always use the file permission system to block your everyday browser running as your regular user to access the service sockets, and spawn a browser using a dedicated user account (www-sock? just like we have www-data for web servers?) that you only use for this.
> when they could have easily just used an opaque handle that the machine operator can copy and paste into whatever application he wants to use it
Then the point of failure would be the random number generator used to generate the "opaque handle". Security by obscurity never works.
H4ZB7(1)
I personally would like to see it but
I think there would need to be a solution for Windows too, since that is nearly 70% of desktop OS use vs. 20% for MacOS and 3% for Linux.[0]
Otherwise it would be a lot of work for the benefit.
[0]: https://gs.statcounter.com/os-market-share/desktop/worldwide...
[0]: https://gs.statcounter.com/os-market-share/desktop/worldwide...
Windows has had Unix sockets for quite a few years now.
https://devblogs.microsoft.com/commandline/af_unix-comes-to-...
https://devblogs.microsoft.com/commandline/af_unix-comes-to-...
I didn't know that, thanks. If this is news to anyone else - in the "What's next?" section of the article it mentions interop between Win32 and WSL using AF_UNIX sockets, which became possible in a later update[0]
[0]: https://devblogs.microsoft.com/commandline/windowswsl-intero...
[0]: https://devblogs.microsoft.com/commandline/windowswsl-intero...
In addition you can rely on Unix access control with the socket. Much better than granting access to all local users, such as is the case by default with e.g. Syncthing.
OpenSSH can also forward these sockets, so remote use can be safer that way and the port configuration issue is relevant here as well.
I've also hoped that NFS and its ilk would be able to transfer Unix domain sockets, but I haven't heard of a system that could do that. Then one could, in principle, just ssh /var/servers/gw over NFS!
edit: actually read the proposal and it mentions most of my points, including Syncthing :).