Some Differences Between macOS and Common Unix Systems(dyx.name)
dyx.name
Some Differences Between macOS and Common Unix Systems
https://www.dyx.name/posts/macunix.html
81 comments
It’s a no-brainer Mac OS has more Unix heritage than Linux, but I don’t think using certification by the company that happens to own the name is the way to show that.
I would think the BSDs have at least as much Unix heritage as macOS, but they aren’t certified (“Certified Unix” is a rare breed. https://www.opengroup.org/openbrand/register/ lists only 13 products)
(Nitpick: it isn’t Unix™, it’s UNIX®)
I would think the BSDs have at least as much Unix heritage as macOS, but they aren’t certified (“Certified Unix” is a rare breed. https://www.opengroup.org/openbrand/register/ lists only 13 products)
(Nitpick: it isn’t Unix™, it’s UNIX®)
I don't think the 'certified unix' moniker counts for very much at all. It is uncertain that any non-corporately maintained Linux distro could qualify (per the opengroup page) as the kernel is developed outside of the distro.
The BSD's on the other hand could probably apply as they are one system (kernel and userland) with one governing body. But I doubt they really care to play the corporate standard mark game. They are UNIX and have always been so.
The BSD's on the other hand could probably apply as they are one system (kernel and userland) with one governing body. But I doubt they really care to play the corporate standard mark game. They are UNIX and have always been so.
Certified Unix just means you've paid SCO enough money. There are Linux-based Certified UNIX products: EulerOS and Inspur K-UX.
> Certified Unix just means you've paid SCO enough money
Not SCO, the Open Group. The Open Group is a not-for-profit industry consortium that maintains various standards and conformance programs for those standards, among which POSIX and UNIX are probably the most widely-known.
Novell gave the UNIX trademark and test suites to The Open Group; all SCO actually got was the right to sublicense the copyright to the AT&T UNIX software (but not the actual copyright itself). The former is much more important than the later because there are operating systems which pass the test suite (and hence are allowed to use the UNIX trademark) but which aren't based on AT&T-derived code (e.g. macOS, z/OS, EulerOS)
Note that Inspur K-UX you mentioned is no longer a certified UNIX because the vendor failed to pay the renewal fees. The same is true of Solaris.
Not SCO, the Open Group. The Open Group is a not-for-profit industry consortium that maintains various standards and conformance programs for those standards, among which POSIX and UNIX are probably the most widely-known.
Novell gave the UNIX trademark and test suites to The Open Group; all SCO actually got was the right to sublicense the copyright to the AT&T UNIX software (but not the actual copyright itself). The former is much more important than the later because there are operating systems which pass the test suite (and hence are allowed to use the UNIX trademark) but which aren't based on AT&T-derived code (e.g. macOS, z/OS, EulerOS)
Note that Inspur K-UX you mentioned is no longer a certified UNIX because the vendor failed to pay the renewal fees. The same is true of Solaris.
Thanks. Google gave me https://www.opengroup.org/openbrand/register/brand3596.htm, which links to both http://www.opengroup.org/csq/search/t=XY1.html (Nice layout, BTW. Reminds me of the ‘90) and https://www.opengroup.org/openbrand/register/xy.htm.
Apparently, “conformance statements” are different from “registered products”. I guess one is self-reported and the other is externally verified?
Apparently, “conformance statements” are different from “registered products”. I guess one is self-reported and the other is externally verified?
> Apparently, “conformance statements” are different from “registered products”. I guess one is self-reported and the other is externally verified?
Products get dropped from the register if the vendor fails to pay renewal fees. The product no longer appears on the register page, but the page on the product is not taken down and can still be directly accessed. This is what has happened to both Inspur K-UX and Oracle Solaris (and I'm pretty sure a few more over the years).
Products get dropped from the register if the vendor fails to pay renewal fees. The product no longer appears on the register page, but the page on the product is not taken down and can still be directly accessed. This is what has happened to both Inspur K-UX and Oracle Solaris (and I'm pretty sure a few more over the years).
Also
> macOS is a great system. It’s based on Unix but if you use it as just another Unix it will be a huge waste.
I mean, I guess it's at least true that it's "based on Unix" through its BSD lineage, but still an odd turn of phrase as, as you note, macOS has been an actual, registered, UNIX 03 system since 10.5.
> macOS is a great system. It’s based on Unix but if you use it as just another Unix it will be a huge waste.
I mean, I guess it's at least true that it's "based on Unix" through its BSD lineage, but still an odd turn of phrase as, as you note, macOS has been an actual, registered, UNIX 03 system since 10.5.
Particularly as the reference UNIX seems to be a Linux with systemd, which isn’t UNIX.
De facto or de jure? Linux isn't technically one, sure, but given that Linux is where the OS hot-shit is now (e.g. eBPF, although I think FreeBSD has the basics - no idea whether it got merged but I saw a student project) is it not fair to say it's where Unix would've been had things been different?
Both, I would say, particularly with the GNU’s Not UNIX userland. I don’t think it matters much, though, as you can have a great non-UNIX OS, which Linux definitely is.
Also, we might forget it now, but in the past there were a lot of UNIX that were quite different in small and large ways. The differences between Linux and Darwin might be as great as the differences between SunOS and Xenix, or indeed NeXTSTEP, back in the day. Or maybe not, that was before my time, but my point is that UNIX was not a monolith and there were incompatibilities and different ways of doing things. So I guess yeah, Linux might be what one of these might have become.
I just find it ironic that so many people are keen to compare macOS to UNIX and just assume that macOS has to be the weird outsider, even though the reverse is actually true. The same people also often assume Linux-isms are the way UNIX works, like the now-dead comment about macOS’ commands not supporting GNU options.
Also, we might forget it now, but in the past there were a lot of UNIX that were quite different in small and large ways. The differences between Linux and Darwin might be as great as the differences between SunOS and Xenix, or indeed NeXTSTEP, back in the day. Or maybe not, that was before my time, but my point is that UNIX was not a monolith and there were incompatibilities and different ways of doing things. So I guess yeah, Linux might be what one of these might have become.
I just find it ironic that so many people are keen to compare macOS to UNIX and just assume that macOS has to be the weird outsider, even though the reverse is actually true. The same people also often assume Linux-isms are the way UNIX works, like the now-dead comment about macOS’ commands not supporting GNU options.
Outside of macOS, commercial Unix is basically dead. In the old days I used to see all sorts of systems. One company had Solaris 2.x, SunOS 4.x, Digital Unix, HPUX, AIX. Now you rarely see anything other than Linux, though I still know a couple of AIX shops that went all in on IBM hardware and still swear by it.
Like it or not, right or wrong, when people say "Unix" today, they mean Linux with GNU userland.
Like it or not, right or wrong, when people say "Unix" today, they mean Linux with GNU userland.
The IBM mainframe operating system z/OS (formerly known as MVS) is a certified UNIX 95. (That's an older UNIX standard, but still valid.)
There are still a lot of IBM mainframes out there. Far less than there used to be, but IBM mainframes remain (even in 2020) a sector with many billions of dollars of annual revenue (if you add up hardware and software and services across both IBM and the ISV ecosystem). And the vast majority of those mainframes run z/OS. (They also run other operating systems – z/Linux, z/VM, z/VSE and z/TPF – but many of the sites running one of those other operating systems run z/OS as well.)
The UNIX component of z/OS is not merely a compatibility layer, a lot of operating system components and applications now rely on it. For example, the REST-based z/OS management console Zowe that IBM is now promoting runs under z/OS UNIX. My understanding is that IBM prefers (where possible) to develop new OS components and applications to run under z/OS UNIX rather than the traditional z/OS APIs. Anything written in Java (or other newer languages, such as node.js) is running on z/OS UNIX. So even if a z/OS site isn't doing anything directly on UNIX, they are almost certainly using OS components and applications which directly rely on it to function.
Obviously z/OS is absolutely dwarfed by Linux and macOS, but it is probably the healthiest of all the commercial Unixes – Oracle has put Solaris in maintenance mode, same for HPE and HP/UX, HPE killed Digital Unix years ago; AIX is healthier than Solaris or HP/UX, but I think z/OS is even healthier than AIX. z/OS licensing fees are a lot higher than AIX licensing fees, and the non-UNIX parts of z/OS (JCL, VSAM, CICS, IMS, etc) make it far harder to migrate off than AIX is.
There are still a lot of IBM mainframes out there. Far less than there used to be, but IBM mainframes remain (even in 2020) a sector with many billions of dollars of annual revenue (if you add up hardware and software and services across both IBM and the ISV ecosystem). And the vast majority of those mainframes run z/OS. (They also run other operating systems – z/Linux, z/VM, z/VSE and z/TPF – but many of the sites running one of those other operating systems run z/OS as well.)
The UNIX component of z/OS is not merely a compatibility layer, a lot of operating system components and applications now rely on it. For example, the REST-based z/OS management console Zowe that IBM is now promoting runs under z/OS UNIX. My understanding is that IBM prefers (where possible) to develop new OS components and applications to run under z/OS UNIX rather than the traditional z/OS APIs. Anything written in Java (or other newer languages, such as node.js) is running on z/OS UNIX. So even if a z/OS site isn't doing anything directly on UNIX, they are almost certainly using OS components and applications which directly rely on it to function.
Obviously z/OS is absolutely dwarfed by Linux and macOS, but it is probably the healthiest of all the commercial Unixes – Oracle has put Solaris in maintenance mode, same for HPE and HP/UX, HPE killed Digital Unix years ago; AIX is healthier than Solaris or HP/UX, but I think z/OS is even healthier than AIX. z/OS licensing fees are a lot higher than AIX licensing fees, and the non-UNIX parts of z/OS (JCL, VSAM, CICS, IMS, etc) make it far harder to migrate off than AIX is.
Interesting. I had forgotten about z/OS. I don't know much about the mainframe world. Several years back, I did get it booted on the "Hercules" emulator, but making much real use of it was beyond my capabilities.
You just reminded me of this site, which I found incredibly useful back in the day when I was working closely with Solaris, Linux and OSX (as it was then known).
http://bhami.com/rosetta.html
http://bhami.com/rosetta.html
> Linux isn't technically one, sure, but given that Linux is where the OS hot-shit is now (...) is it not fair to say it's where Unix would've been had things been different?
No, it's not fair. Linux is just the OS kernel. UNIX covers interfaces, from APIs to shells and apps distributed by default, which are way out of the scope of Linux. That's the responsibility of Linux distributionsm.
And by the way, there are linux distributions that are certified UNIX
https://unix.stackexchange.com/questions/293396/is-there-a-l...
Moreover, UNIX is standardized. The drive for standardization was derived from the fact that all vendors of UNIX-like OSes were pushing their quirky vendor-specific things. In fact, even within Linux there is no widely established standard regarding basic interfaces, let alone UNIX.
With this in mind, do note that macOS's market share is significantly larger than the aggregate market share of Linux-based UNIX-like distributions.
No, it's not fair. Linux is just the OS kernel. UNIX covers interfaces, from APIs to shells and apps distributed by default, which are way out of the scope of Linux. That's the responsibility of Linux distributionsm.
And by the way, there are linux distributions that are certified UNIX
https://unix.stackexchange.com/questions/293396/is-there-a-l...
Moreover, UNIX is standardized. The drive for standardization was derived from the fact that all vendors of UNIX-like OSes were pushing their quirky vendor-specific things. In fact, even within Linux there is no widely established standard regarding basic interfaces, let alone UNIX.
With this in mind, do note that macOS's market share is significantly larger than the aggregate market share of Linux-based UNIX-like distributions.
> And by the way, there are linux distributions that are certified UNIX
How was I not aware of this? Thanks for the link! And you make a very good point regarding kernel vs OS. I assumed Linux == GNU/Linux in this context (so ignoring Android and others), but it certainly is a gross over simplification.
How was I not aware of this? Thanks for the link! And you make a very good point regarding kernel vs OS. I assumed Linux == GNU/Linux in this context (so ignoring Android and others), but it certainly is a gross over simplification.
> UNIX
I don't understand why UNIX is such a big deal in 2020. Linux has won, and at least one BSD has binary compatability with Linux. If anything, we should be taking about Linux compliance.
Having no strong package manager story, yet being UNIX compliant, seems like an extremely bizarre priority choice.
I don't understand why UNIX is such a big deal in 2020. Linux has won, and at least one BSD has binary compatability with Linux. If anything, we should be taking about Linux compliance.
Having no strong package manager story, yet being UNIX compliant, seems like an extremely bizarre priority choice.
> at least one BSD has binary compatability with Linux.
Two BSDs (NetBSD and FreeBSD, no OpenBSD, don't know about Dragonfly), and a handful of Illumos distros (who call the feature LX-branded zones).
> If anything, we should be taking about Linux compliance.
Please no; a standard defined by one implementation isn't a standard.
Two BSDs (NetBSD and FreeBSD, no OpenBSD, don't know about Dragonfly), and a handful of Illumos distros (who call the feature LX-branded zones).
> If anything, we should be taking about Linux compliance.
Please no; a standard defined by one implementation isn't a standard.
There is some value in standards such as UNIX and POSIX compared to whatever the maintainers of a GNU project decided this morning.
As for your second point, it just means that package management is irrelevant to UNIX (or POSIX) compliance. It did not prevent different package managers to appear over the years, because all the expected infrastructure was in place.
As for your second point, it just means that package management is irrelevant to UNIX (or POSIX) compliance. It did not prevent different package managers to appear over the years, because all the expected infrastructure was in place.
> It did not prevent different package managers
But did these UNIX package managers actually materialize? I guess my point is: is UNIX at all relevant? It had some really good ideas that were worth copying, but the world has moved on to greater ideas.
But did these UNIX package managers actually materialize? I guess my point is: is UNIX at all relevant? It had some really good ideas that were worth copying, but the world has moved on to greater ideas.
> I guess my point is: is UNIX at all relevant?
Obviously yes. I don't understand what is there so hard to understand.
I mean, just go learn about pretty much any field where you need to target a combination of platforms that don't agree on small arbitrary things, and how much wasted effort that consumes.
Obviously yes. I don't understand what is there so hard to understand.
I mean, just go learn about pretty much any field where you need to target a combination of platforms that don't agree on small arbitrary things, and how much wasted effort that consumes.
I'm still not buying it. We can all agree that UNIX should be relevant but, apart from Mac, when was the last time you used a bona-fide UNIX system?
You've assumed that your perspective is the correct one without justifying it. Sure, standards are important, but UNIX doesn't seem to be a successful (or modern) one.
You've assumed that your perspective is the correct one without justifying it. Sure, standards are important, but UNIX doesn't seem to be a successful (or modern) one.
> I don't understand why UNIX is such a big deal in 2020.
If you're interested in learning about why standards are "such a big deal" then you can start by discovering a phenomenon called linux fragmentation".
If you're interested in learning about why standards are "such a big deal" then you can start by discovering a phenomenon called linux fragmentation".
30% of Linux fragmentation comes from package managers. 30% of Linux fragmentation comes from the system process. 30% of fragmentation comes from choices concerning default installed packages.
UNIX covers exactly zero of those. If UNIX was worthwhile, we wouldn't have Linux fragmentation or, indeed, an article explaining how to run UNIX scripts on a UNIX-compliant OS (sure as shit ain't a standard if it doesn't actually work).
UNIX covers exactly zero of those. If UNIX was worthwhile, we wouldn't have Linux fragmentation or, indeed, an article explaining how to run UNIX scripts on a UNIX-compliant OS (sure as shit ain't a standard if it doesn't actually work).
[deleted]
It's true mac OS is the most common Unix, but that doesn't mean it's the only Unix which could be considered common.
macOS claims to be UNIX yet "XNU", the macOS kernel, is an abbreviation for "X is not UNIX". "Linux" refers to a kernel, inspired by Minix, which is an abbreviation for "mini-UNIX".
Linux is not based on Minix. What Linux does do is that it supports most everything required for POSIX compatibility.
He wrote inspired by, not based on - which is true. Linus Torvalds was inspired by Minix, to write Linux.
I think the comment I replied to was edited.
I worked at a Telco that ran over 100k Linux VMs in one department.
Not sure about unix certified os's, but mac os has a much smaller install base than linux.
Not sure about unix certified os's, but mac os has a much smaller install base than linux.
> Not sure about unix certified os's, but mac os has a much smaller install base than linux.
No, not really. macos's market share is about 4 times the aggregate market share of all Linux-based distributions.
No, not really. macos's market share is about 4 times the aggregate market share of all Linux-based distributions.
[deleted]
[deleted]
I keep hearing that `sudo` insecurity argument everywhere. I think that argument is fundamentally wrong.
In practice, a package manager that eschews `sudo` does not protect you any less than a package manager that works with `sudo`. If malware has obtained file system access, it’s already game over. The malicious code can now mess with your shell profile and fake the `sudo` command.
In practice, a package manager that eschews `sudo` does not protect you any less than a package manager that works with `sudo`. If malware has obtained file system access, it’s already game over. The malicious code can now mess with your shell profile and fake the `sudo` command.
The main problem isn't really that it doesn't require sudo.
If it installed as a user into your home dir, that would be fine.
Changing a shared system-wide path or installing shared system-wide files owned by a user is so fundamentally borked that you should immediately reject anything offered by anyone who suggested to do that.
But the masses of users who made it popular didn't know that and there were/are a lot of them, and the reason it's wrong won't bite many people because their macbooks only have one user account, and so you have a zillion people who didn't know they were getting bad advice and doing a broken thing, and if you tell them now, they don't see it because they've been doing it wrong for years and their house didn't burn down, and they vastly outnumber the people who unserstand the problem and would never have designed something so broken in the first place, and so it just stays both incorrect and the standard.
If it installed as a user into your home dir, that would be fine.
Changing a shared system-wide path or installing shared system-wide files owned by a user is so fundamentally borked that you should immediately reject anything offered by anyone who suggested to do that.
But the masses of users who made it popular didn't know that and there were/are a lot of them, and the reason it's wrong won't bite many people because their macbooks only have one user account, and so you have a zillion people who didn't know they were getting bad advice and doing a broken thing, and if you tell them now, they don't see it because they've been doing it wrong for years and their house didn't burn down, and they vastly outnumber the people who unserstand the problem and would never have designed something so broken in the first place, and so it just stays both incorrect and the standard.
Can you spoonfeed me here? If ~/bin is in my PATH and is writable to my user, what is the problem with /usr/local/bin being the same on my personal computer with a single human user?
Base case (those are true AFAIK):
- /usr/local/bin is in a "system-wide" default PATH, (and it is even in front of /bin !)
- There is an important service, like Time Machine, which needs root access for important actions -- for example, to erase previous backups
- There could be a vulnerability in an application which gets data from the internet -- for the sake of example, let's say it is "foo file viewer". This can be exploited for code execution.
---
Case 1: Packages in ~/bin, /usr/local/bin is read-only:
A user wants to look at a "foo file" from the internet. The file is malicious, it exploits "foo viewer" and gets local execution. It is a cryptolocker, so it encrypts all the user documents.
The malware has installed trojaned "~/bin/sudo" wrapper, but since the user does not use "sudo" that often, it did not have a change to get executed. And none of the system services look into user's "~/bin".
A week later, user notices that a document is encrypted. But the time machine backups are still OK. They reformat their machine, and then use time machine backup to restore the documents. Day is saved!
---
Case 2: Packages in usr/local/bin:
A user wants to look at a "foo file" from the internet. The file is malicious, it exploits "foo viewer" and gets local execution. It is a cryptolocker, so it encrypts all the user documents.
The malware has installed a bunch of trojaned binaries, including "/usr/local/bin/touch" binary. Unfortunately, there is a LaunchDaemon which runs the script periodically which contains the line "touch /some/file". That is run as root, so in a short time, the malware gets root access. And it immediately used this access to disable time machine and delete all the backups.
A week later, user notices that a document is encrypted. And the time machine backups are gone, too. Oh no! the documents are lost unless they pay the ransom!
---
You might notice that some people may consider the scenario unrealistic: What if the user uses command line a lot, and is used to running "sudo"? What if a malware social engineers user by popping up a fake dialog asking for user password? Are there other privilege escalation methods? How often does this happen anyway?
You should make your own decisions, but hopefully you at least see what the people are concerned about.
- /usr/local/bin is in a "system-wide" default PATH, (and it is even in front of /bin !)
- There is an important service, like Time Machine, which needs root access for important actions -- for example, to erase previous backups
- There could be a vulnerability in an application which gets data from the internet -- for the sake of example, let's say it is "foo file viewer". This can be exploited for code execution.
---
Case 1: Packages in ~/bin, /usr/local/bin is read-only:
A user wants to look at a "foo file" from the internet. The file is malicious, it exploits "foo viewer" and gets local execution. It is a cryptolocker, so it encrypts all the user documents.
The malware has installed trojaned "~/bin/sudo" wrapper, but since the user does not use "sudo" that often, it did not have a change to get executed. And none of the system services look into user's "~/bin".
A week later, user notices that a document is encrypted. But the time machine backups are still OK. They reformat their machine, and then use time machine backup to restore the documents. Day is saved!
---
Case 2: Packages in usr/local/bin:
A user wants to look at a "foo file" from the internet. The file is malicious, it exploits "foo viewer" and gets local execution. It is a cryptolocker, so it encrypts all the user documents.
The malware has installed a bunch of trojaned binaries, including "/usr/local/bin/touch" binary. Unfortunately, there is a LaunchDaemon which runs the script periodically which contains the line "touch /some/file". That is run as root, so in a short time, the malware gets root access. And it immediately used this access to disable time machine and delete all the backups.
A week later, user notices that a document is encrypted. And the time machine backups are gone, too. Oh no! the documents are lost unless they pay the ransom!
---
You might notice that some people may consider the scenario unrealistic: What if the user uses command line a lot, and is used to running "sudo"? What if a malware social engineers user by popping up a fake dialog asking for user password? Are there other privilege escalation methods? How often does this happen anyway?
You should make your own decisions, but hopefully you at least see what the people are concerned about.
Since Unix doesn't (traditionally at least) offer any access control mechanism more granular than a User, it was only natural that you'd get a lot of "system users" that are required for all sorts of operations but are not in any way associated with human users.
And now, since you have so many system users and almost always a single human user, you start wanting to install programs that will be used by both yourself and those system users, and it would be extremely problematic to go writing into 10 different bin/ and lib/ folders to install the same thing for 10 users.
Not to mention, a lot of software you want to install also runs some daemons or manages some system settings, and obviously requires root access to do so, but mag run into problems if it is not installed for root.
And now, since you have so many system users and almost always a single human user, you start wanting to install programs that will be used by both yourself and those system users, and it would be extremely problematic to go writing into 10 different bin/ and lib/ folders to install the same thing for 10 users.
Not to mention, a lot of software you want to install also runs some daemons or manages some system settings, and obviously requires root access to do so, but mag run into problems if it is not installed for root.
>Changing a shared system-wide path or installing shared system-wide files owned by a user is so fundamentally borked that you should immediately reject anything offered by anyone who suggested to do that.
I don't think that's fair to the homebrew developers. They chose a justifiable tradeoff - as you said most mac systems are single user systems and their design offers better protection to a far larger group of users. When you run `brew install ffmpeg` you are running an arbitrary ruby script from Homebrew git repo and it's probably better that it doesn't run as root. I can't see how that is bad advice; I'd argue that the opposite - sticking to "tradition" despite the changing the landscape of how personal devices have changed security is the broken mindset.
I'm sure most macOS users are far more concerned about an arbitrary script exfiltrating their keychain than some contrived multi-user setup where another user installs a keylogger in the system path.
I don't think that's fair to the homebrew developers. They chose a justifiable tradeoff - as you said most mac systems are single user systems and their design offers better protection to a far larger group of users. When you run `brew install ffmpeg` you are running an arbitrary ruby script from Homebrew git repo and it's probably better that it doesn't run as root. I can't see how that is bad advice; I'd argue that the opposite - sticking to "tradition" despite the changing the landscape of how personal devices have changed security is the broken mindset.
I'm sure most macOS users are far more concerned about an arbitrary script exfiltrating their keychain than some contrived multi-user setup where another user installs a keylogger in the system path.
"contrived" is not an excuse for doing something wrong.
All programming, heck all system design of any form, programming, mechanical hardware, legal process, is made of ensuring that situations are handled, including both the obvious ones you thought of and the ones you didn't think of.
All "corner cases" could be described as "contrived", yet they happen a million times an hour.
"Contrived" is a contrived rationale to ignore something you should not ignore for mere convenience.
"contrived" is just a silly and empty accusation, and no argument.
If you are in a conversation where the subject deals in probabilities or percentages, then you can talk about liklihood. Having say 1% of customers fail to navigate a subway map is better than having 8% fail. And some contrived example problem like the map needs to be only in shades of green because what if you're colorblind... the contrived nature of the problem matters because the problem deals in percentages where 99% is better than 92%.
But in system design (such as application programming or os design, especially the lowest most basic security aspects, things are only possible or not-possible. There is no "likely" and no "contrived". If you can "contrive" an example that invalidates a design, then you have invalidated the design, period.
If you cannot contrive an example, then you still have not proven the design, merely you have failed to IN-validate it, yet.
The /Applications directory is not equivalent to the /usr/local tree, or any other part of the system.
The /Applications dir itself is owned by root, and a user may not change it to be owned by theirself, and a user may only write within it because they are a member of the "admin" group. Until recently, the homebrew installer actually changed the ownership of /usr/local itself (or created it, I don't remember if it maybe didn't exist by default and was created by brew sometimes) rather than just some of the stuff under it. They only changed when forced to, when Apple started more actively protecting system components from modification a couple years ago. If it's ok for shared system components to be owned by joe random user, then why did Apple start protecting /usr/local? For that matter why can't joe random user replace /bin/cp with his own copy, or the kernel?
/usr/local/bin/mycp is (or rather should be, and is on sane sysyems) just as much a protected part of the system as /bin/cp. It's only different in that it's locally added to this machine and didn't come on the install cd, and may have different behavior than other parts of the system expect etc. But it's not "steve's" either. A "mycp" owned by "steve" should only exist in /home/steve (or /Users/steve on mac).
If brew installed everything in /Applications/Brew.app that would be ok too, because then it would be just doing what other apps do, and because /Applications is a special case which has extra handling by other means than merely the traditional unix file ownerships.
That would still be mildly broken because it would mean that all users could use the brew binaries and other assets, but only the installing user could modify them. That is fine for most apps which do not modify their internal contents, but brew is essentially a whole system itself, and it's contents are modified during normal use. A theoretical Brew.app is not like Calculator.app
So, the only sane way to do it is to install as the user somewhere within the users own domain (their home dir) or install in a system-wide shared place and require sudo for any "brew update" or "make install".
If it were all owned by a special user and/or group "brew" (not whatever random user happened to do the install), that would be sane too, but not necessary.
All programming, heck all system design of any form, programming, mechanical hardware, legal process, is made of ensuring that situations are handled, including both the obvious ones you thought of and the ones you didn't think of.
All "corner cases" could be described as "contrived", yet they happen a million times an hour.
"Contrived" is a contrived rationale to ignore something you should not ignore for mere convenience.
"contrived" is just a silly and empty accusation, and no argument.
If you are in a conversation where the subject deals in probabilities or percentages, then you can talk about liklihood. Having say 1% of customers fail to navigate a subway map is better than having 8% fail. And some contrived example problem like the map needs to be only in shades of green because what if you're colorblind... the contrived nature of the problem matters because the problem deals in percentages where 99% is better than 92%.
But in system design (such as application programming or os design, especially the lowest most basic security aspects, things are only possible or not-possible. There is no "likely" and no "contrived". If you can "contrive" an example that invalidates a design, then you have invalidated the design, period.
If you cannot contrive an example, then you still have not proven the design, merely you have failed to IN-validate it, yet.
The /Applications directory is not equivalent to the /usr/local tree, or any other part of the system.
The /Applications dir itself is owned by root, and a user may not change it to be owned by theirself, and a user may only write within it because they are a member of the "admin" group. Until recently, the homebrew installer actually changed the ownership of /usr/local itself (or created it, I don't remember if it maybe didn't exist by default and was created by brew sometimes) rather than just some of the stuff under it. They only changed when forced to, when Apple started more actively protecting system components from modification a couple years ago. If it's ok for shared system components to be owned by joe random user, then why did Apple start protecting /usr/local? For that matter why can't joe random user replace /bin/cp with his own copy, or the kernel?
/usr/local/bin/mycp is (or rather should be, and is on sane sysyems) just as much a protected part of the system as /bin/cp. It's only different in that it's locally added to this machine and didn't come on the install cd, and may have different behavior than other parts of the system expect etc. But it's not "steve's" either. A "mycp" owned by "steve" should only exist in /home/steve (or /Users/steve on mac).
If brew installed everything in /Applications/Brew.app that would be ok too, because then it would be just doing what other apps do, and because /Applications is a special case which has extra handling by other means than merely the traditional unix file ownerships.
That would still be mildly broken because it would mean that all users could use the brew binaries and other assets, but only the installing user could modify them. That is fine for most apps which do not modify their internal contents, but brew is essentially a whole system itself, and it's contents are modified during normal use. A theoretical Brew.app is not like Calculator.app
So, the only sane way to do it is to install as the user somewhere within the users own domain (their home dir) or install in a system-wide shared place and require sudo for any "brew update" or "make install".
If it were all owned by a special user and/or group "brew" (not whatever random user happened to do the install), that would be sane too, but not necessary.
That is exactly how installing an application in MacOS always works. When you run an installer for a native app, and it goes into /Applications, it's owned by the user who installed it. What Homebrew is doing isn't functionally different, other than /usr/local is in your PATH I guess?
Yes the only way around that is to have sudo activate a Secure Attention Key. Windows has had that for many many years ("press ctrl-alt-del to continue") though they seem to have ditched it in Windows 10. Presumably users didn't know what it meant or why it was there and just did it when asked by anyone.
I don't think it's really possible to make sudo use a SAK on Linux though because it requires tight integration of lots of components and Linux systems are cobbled together with glue and string.
I don't think it's really possible to make sudo use a SAK on Linux though because it requires tight integration of lots of components and Linux systems are cobbled together with glue and string.
Are you saying you want "sudo" to use a a password entry mechanism which cannot be spoofed or intercepted? Something very similar to this is pretty easy on Linux.
If you are willing to pay a little, requiring Yuibikey press for sudo is supported [0].
If you don't want extra hardware, you can use "pam_exec" PAM method to read second factor (like PIN) from "/dev/input/by-id/...". This'll read from specific input device directly, bypassing GUI session or any other processes controlled by non-root user. Or use a "chvt" call instead, and you can have a secure display output too.
But while this will prevent "evil sudo" from capturing your password, it won't prevent "evil sudo" modifying the commands you want executed (the same problem exists with Windows SAK / Escalation prompt: you know you are approving some action, but it is not clear which action exactly).
I guess Linux has a unique advantage in that "sudo" is designed for user-readable commands -- so in many cases, confirming the command on un-tamperable output will be enough for safety (for example, "sudo apt install foo" will be safe this way). But you'd still want to make sure you are vigilant at all times -- if you have a habit of typing "curl ... | sudo sh", then no type of SAK process will save you from evil sudo.
[0] https://support.yubico.com/hc/en-us/articles/360016649099-Ub...
If you are willing to pay a little, requiring Yuibikey press for sudo is supported [0].
If you don't want extra hardware, you can use "pam_exec" PAM method to read second factor (like PIN) from "/dev/input/by-id/...". This'll read from specific input device directly, bypassing GUI session or any other processes controlled by non-root user. Or use a "chvt" call instead, and you can have a secure display output too.
But while this will prevent "evil sudo" from capturing your password, it won't prevent "evil sudo" modifying the commands you want executed (the same problem exists with Windows SAK / Escalation prompt: you know you are approving some action, but it is not clear which action exactly).
I guess Linux has a unique advantage in that "sudo" is designed for user-readable commands -- so in many cases, confirming the command on un-tamperable output will be enough for safety (for example, "sudo apt install foo" will be safe this way). But you'd still want to make sure you are vigilant at all times -- if you have a habit of typing "curl ... | sudo sh", then no type of SAK process will save you from evil sudo.
[0] https://support.yubico.com/hc/en-us/articles/360016649099-Ub...
A SAK is some action that only the kernel can detect. On Windows user-level programs cannot receive the ctrl-alt-del key sequence, so if you do that and something happens you can be sure you are interacting with the real authentication GUI, not a fake one.
I don't think any of your solutions provide that capability.
I don't think any of your solutions provide that capability.
My solutions provide the capability to "be sure you are interacting with the real authentication GUI, not a fake one."
They do not provide "some action only kernel can detect" -- but I don't think you need this. Linux has strong enough user isolation that userspace programs running as root are as good as kernel is. Let me elaborate on that:
From security standpoint, they are equivalent. If you get kernel access, you can start a userspace root process. If you can get a root process access, you can modify kernel. So being "kernel only" does not really have that many advantages over privileged process.
And Linux kernel provides pretty strong user isolation. In particular, there are two subsystems in Linux which (AFAIK) are not present in windows, and which makes the safe userspace login possible:
- The system does not have a single "screen", instead it has multiple "virtual terminals". All the display/input devices are tied to single virtual terminal, and switching to a different virtual terminal completely deactivates the other one. So, for example, you might have your user's regular session on VT 7. If you somehow switch to the VT 2, then no matter what that user session does, it can no longer affect the display nor read from keyboard until you switch back.
- The kernel provides direct access to the input devices to the root user, via /dev/input/ file. This access happens before regular input layers, and cannot be affected by non-root users at all. So while a user's app can proxy the TTY input, or set up a global keygrab for the X session, or install a strange keyboard layout, this will not affect those /dev/input files at all.
So have a root process that monitors /dev/input/* and switches to a different tty when a designated SAK key is presses. And you have your "real GUI" solved, no kernel needed.
Agree?
They do not provide "some action only kernel can detect" -- but I don't think you need this. Linux has strong enough user isolation that userspace programs running as root are as good as kernel is. Let me elaborate on that:
From security standpoint, they are equivalent. If you get kernel access, you can start a userspace root process. If you can get a root process access, you can modify kernel. So being "kernel only" does not really have that many advantages over privileged process.
And Linux kernel provides pretty strong user isolation. In particular, there are two subsystems in Linux which (AFAIK) are not present in windows, and which makes the safe userspace login possible:
- The system does not have a single "screen", instead it has multiple "virtual terminals". All the display/input devices are tied to single virtual terminal, and switching to a different virtual terminal completely deactivates the other one. So, for example, you might have your user's regular session on VT 7. If you somehow switch to the VT 2, then no matter what that user session does, it can no longer affect the display nor read from keyboard until you switch back.
- The kernel provides direct access to the input devices to the root user, via /dev/input/ file. This access happens before regular input layers, and cannot be affected by non-root users at all. So while a user's app can proxy the TTY input, or set up a global keygrab for the X session, or install a strange keyboard layout, this will not affect those /dev/input files at all.
So have a root process that monitors /dev/input/* and switches to a different tty when a designated SAK key is presses. And you have your "real GUI" solved, no kernel needed.
Agree?
> though they seem to have ditched it in Windows 10
Nope, it's still there. It may require being joined to an AD Domain with the relevant setting toggled, though.
Nope, it's still there. It may require being joined to an AD Domain with the relevant setting toggled, though.
The magic sysrq key combination works, I don’t see why you couldn’t do something similar for a SAK?
> fake the `sudo` command.
I personally see that "su" and "sudo" are both fundamentally insecure. Compromising any sudoer, and root is essentially lost (e.g. keyloggers), thus the "wheel" group should only contain trusted accounts. It can be more secure if all root operations require a separate console, isolated from the rest of the world. For example, on historical Unix servers, it was a common practice to restrict root logins to a physical terminal. But nowadays it's unfeasible. One solution for servers is disabling "su" and "sudo", and to allow root logins via pubkeys. This goes against the best practices, but can be more secure when it's used properly. Alternatively, create an "admin" account with sudo privilege, and guard it as carefully as your root account, because a sudoer is root.
On desktop, a solution can be using Intel ME/SGX/TXT and allowing it to take over your screen and keyboard to create a root shell - but it is proprietary, unauditable, and cannot be trusted. But it's not strictly needed - pure software can do something similar, e.g. you run two X servers on two separate VTs with two users, only the second user has access to a root shell. I guess better integration at the graphics server/system level is possible, you just need to be careful about GUI spoofing. For example, on Windows NT, pressing Control+Alt+Delete opens an unspoofable & isolated secure login screen - perhaps we should implement a secure root shell like that!
A more practical, although much less secure alternative is requiring 2FA, e.g. pressing a physical button on a USB token to sudo, it's something you can use today. But it cannot solve the problem of malicious command injections.
But on second thought, these problems are not really a priority in infosec - user applications are insecure in general, and user data is accessible without root anyway... As xkcd said, "they can read my mails, take my money, and impersonate me, but at least they cannot install device drivers without my permission..." So doing all of the works can be a total waste of time. Also, even with a separate root console, one must also have strong discipline not to execute anything or opening any complex binary data from another user (potentially compromised), which makes the whole idea fragile.
Finally QubesOS's approach is an innovative workaround and worth mentioning - normal and sensitive applications are running in separate VMs, providing strong isolation. Since both system and user applications are already isolated, it's not really that dangerous to be "root", so by default, everyone can sudo without any authentication. But if you are concerned about someone attacking supervisor via root, you can use Dom0 to 2FA authenticate sudo in an application VM without any extra hardware.
I personally see that "su" and "sudo" are both fundamentally insecure. Compromising any sudoer, and root is essentially lost (e.g. keyloggers), thus the "wheel" group should only contain trusted accounts. It can be more secure if all root operations require a separate console, isolated from the rest of the world. For example, on historical Unix servers, it was a common practice to restrict root logins to a physical terminal. But nowadays it's unfeasible. One solution for servers is disabling "su" and "sudo", and to allow root logins via pubkeys. This goes against the best practices, but can be more secure when it's used properly. Alternatively, create an "admin" account with sudo privilege, and guard it as carefully as your root account, because a sudoer is root.
On desktop, a solution can be using Intel ME/SGX/TXT and allowing it to take over your screen and keyboard to create a root shell - but it is proprietary, unauditable, and cannot be trusted. But it's not strictly needed - pure software can do something similar, e.g. you run two X servers on two separate VTs with two users, only the second user has access to a root shell. I guess better integration at the graphics server/system level is possible, you just need to be careful about GUI spoofing. For example, on Windows NT, pressing Control+Alt+Delete opens an unspoofable & isolated secure login screen - perhaps we should implement a secure root shell like that!
A more practical, although much less secure alternative is requiring 2FA, e.g. pressing a physical button on a USB token to sudo, it's something you can use today. But it cannot solve the problem of malicious command injections.
But on second thought, these problems are not really a priority in infosec - user applications are insecure in general, and user data is accessible without root anyway... As xkcd said, "they can read my mails, take my money, and impersonate me, but at least they cannot install device drivers without my permission..." So doing all of the works can be a total waste of time. Also, even with a separate root console, one must also have strong discipline not to execute anything or opening any complex binary data from another user (potentially compromised), which makes the whole idea fragile.
Finally QubesOS's approach is an innovative workaround and worth mentioning - normal and sensitive applications are running in separate VMs, providing strong isolation. Since both system and user applications are already isolated, it's not really that dangerous to be "root", so by default, everyone can sudo without any authentication. But if you are concerned about someone attacking supervisor via root, you can use Dom0 to 2FA authenticate sudo in an application VM without any extra hardware.
I always thought that one of the big purposes of "sudo" is to prevent operator errors. For example, if I want to inspect the .deb file and accidentally type "dpkg -i" instead of "dpkg -I". Or if I am examining logs on a server, and mistype ">" and "<". Or if I typed "make install" and failed to set PREFIX to a local directory.
It also helps a lot as a voluntary "logging" solution -- if you collect syslog on your servers, this can record which commands got run and why things got worse. It is not security solution, you'll have to make a "social convention" to never do "sudo su" or invoke ! in your editor, but it helps to detect accidental mistakes.
As for secure console, you can do this today! Make sure "k" is enabled in magic sysrq flags, and then you can switch to VT1 (alt-ctrl-f1) and invoke SAK with alt-sysrq-k. This'll kill everything on the current VT, and give you a clean login prompt which you can safely log into.
Finally, even with a single user, the privilege separation can help a lot with data protection. Just set up a backup process to run as root, into local directory where only root can write to. Then at least your data won't get deleted.
It also helps a lot as a voluntary "logging" solution -- if you collect syslog on your servers, this can record which commands got run and why things got worse. It is not security solution, you'll have to make a "social convention" to never do "sudo su" or invoke ! in your editor, but it helps to detect accidental mistakes.
As for secure console, you can do this today! Make sure "k" is enabled in magic sysrq flags, and then you can switch to VT1 (alt-ctrl-f1) and invoke SAK with alt-sysrq-k. This'll kill everything on the current VT, and give you a clean login prompt which you can safely log into.
Finally, even with a single user, the privilege separation can help a lot with data protection. Just set up a backup process to run as root, into local directory where only root can write to. Then at least your data won't get deleted.
> I always thought that one of the big purposes of "sudo" is to prevent operator errors.
Good point.
> As for secure console, you can do this today! Make sure "k" is enabled in magic sysrq flags
Thanks. I thought I was SysRq-literate as I frequently use the key for OOM killer, emergency sync and reboot. But it appears I was too ignorant to know the SAK. Just learned something new today.
Good point.
> As for secure console, you can do this today! Make sure "k" is enabled in magic sysrq flags
Thanks. I thought I was SysRq-literate as I frequently use the key for OOM killer, emergency sync and reboot. But it appears I was too ignorant to know the SAK. Just learned something new today.
Interesting post; not sure I agree with all of it, though.
For example, I would not say /Library is equivalent to /usr/lib. /Library has lots of things that are not libraries as in .a or .so (actually .dylib) files and that would be in various different subdirectories somewhere on Linux.
Also, it’s a bit deceptive to show how you need 10 commands to add a user before saying you can do it all at once with a tool that was introduced 6 years ago.
And I’ve never had to use -isysroot with GCC from MacPorts, though I can believe that it is necessary in some special cases.
I think the post articulates well the issues with Homebrew, though.
For example, I would not say /Library is equivalent to /usr/lib. /Library has lots of things that are not libraries as in .a or .so (actually .dylib) files and that would be in various different subdirectories somewhere on Linux.
Also, it’s a bit deceptive to show how you need 10 commands to add a user before saying you can do it all at once with a tool that was introduced 6 years ago.
And I’ve never had to use -isysroot with GCC from MacPorts, though I can believe that it is necessary in some special cases.
I think the post articulates well the issues with Homebrew, though.
Consider using nix over homebrew / macports, it's pretty good on mac now. This is a reasonable intro doc https://wickedchicken.github.io/post/macos-nix-setup/
Is there any particular reason people would go out of their way to use gcc on macOS? It comes with clang/llvm, which seem adequate.
Some specific pieces of C++ support that Clang is missing:
Edit: newer link: https://clang.llvm.org/cxx_status.html
(older link, not applicable, was here)
Cross compiling for some embedded architectures.
Things like FORTRAN or Ada, calling routines built there from C (or vice versa), etc.
Edit: newer link: https://clang.llvm.org/cxx_status.html
(older link, not applicable, was here)
Cross compiling for some embedded architectures.
Things like FORTRAN or Ada, calling routines built there from C (or vice versa), etc.
I've found that in certain circumstances gcc makes faster binaries
gcc usually produces slightly faster O3 binaries
macOS does not come with llvm/clang, you have to download Xcode, which you can't do anonymously, you need to create an account with Apple. Ages ago, macOS did actually come with gcc.
You can install the toolchain with `xcode-select` which does not require an Apple ID.
And if I'm not mistaken, until clang is installed commands like "clang" and "gcc" redirect to xcode-select, so if the user tries to compile something the system will offer to install the toolchain for you.
Yep. That's how I setup a new Mac. I open a terminal and type `make` and it installs the world.
Thank you! Helps me in general, but in particular as I write documentation for some cross-platform software.
Not quite on this topic, but I have always felt Linux security seems lagging behind macOS.
Linux seems lacking on sufficient sandboxing. I am also increasingly uncomfortable with so many repositories, FOOS on GitHub and users sudoing left and right.
I am not sure if I am correct though.
Linux seems lacking on sufficient sandboxing. I am also increasingly uncomfortable with so many repositories, FOOS on GitHub and users sudoing left and right.
I am not sure if I am correct though.
Linux security is technically quite advanced. Namespaces and seccomp-bpf have no parallels in Mac OS land that I'm aware of. It is true that most programs are installed systemwide with no form of sandboxing beyond the usual Unix access controls, but that is starting to change with projects like Flatpak and Snap, which attempt to build a general purpose app sandboxing framework using said security primitives.
That depends on the distro. Fedora and RHEL and their derivatives have quite comprehensive SELinux policies.
Debian (and derivatives) and SUSE use AppArmor but IIRC with more permissive, looser policies. At least Debian does IIRC. SUSE might put in the extra effort to harden them a bit, I'm unfamiliar with their ecosystem.
Debian (and derivatives) and SUSE use AppArmor but IIRC with more permissive, looser policies. At least Debian does IIRC. SUSE might put in the extra effort to harden them a bit, I'm unfamiliar with their ecosystem.
MacOS is it's own security nightmare, it's just mangled and unrecognizable through so many layers of abstraction. Apple's understanding of security and privacy is fundamentally broken: they protect you against a boogyman, the idea of someone who wants to take your data for exploitative gain. In reality, the paradigm of computer security has completely changed. Apple had been leaking unencrypted user data to third party firms for years, and refused to yield when researchers warned them of the implications. Thunderbolt is rife with security vulnerabilities, some exploits being so advanced that they can give the attacker a raw dump of whatever was in memory (which is ironically even more dangerous thanks to MacOS' lackluster memory management).
At the end of the day, securing MacOS is a losing game. Big Sur introduced new, untraceable telemetry at the lowest levels of the system, basically forcing me out of the Mac game. Linux it a breath of fresh air by comparison: not only does my system never phone home, it gets security updates as a first-class citizen in the software world (one of the many perks of being the backbone of most computer infrastructure).
Security on Linux is entirely dependent on the user. One of the core Unix philosophies is that a true Unix system won't stop a dumb user from doing something dumb, because that impedes clever people trying to do something clever. The weakest link in the security chain is always the end user, which is why most "hacks" these days are just massive phishing campaigns: it's just an easier attack vector. It doesn't matter if you're on Windows, MacOS, Linux or TempleOS - if you put your credentials into a browser, someone's gonna pwn it.
P.S. - If sandboxing is your interest, you should check out Flatpak. I think it should clear up your repo concerns too ;)
At the end of the day, securing MacOS is a losing game. Big Sur introduced new, untraceable telemetry at the lowest levels of the system, basically forcing me out of the Mac game. Linux it a breath of fresh air by comparison: not only does my system never phone home, it gets security updates as a first-class citizen in the software world (one of the many perks of being the backbone of most computer infrastructure).
Security on Linux is entirely dependent on the user. One of the core Unix philosophies is that a true Unix system won't stop a dumb user from doing something dumb, because that impedes clever people trying to do something clever. The weakest link in the security chain is always the end user, which is why most "hacks" these days are just massive phishing campaigns: it's just an easier attack vector. It doesn't matter if you're on Windows, MacOS, Linux or TempleOS - if you put your credentials into a browser, someone's gonna pwn it.
P.S. - If sandboxing is your interest, you should check out Flatpak. I think it should clear up your repo concerns too ;)
“a boogyman, the idea of someone who wants to take your data for exploitative gain”
Are you seriously claiming this isn’t a real threat?
Are you seriously claiming this isn’t a real threat?
[deleted]
Some commands have different arguments.
ex: I can't do rm -rfd as i do usually in gnu/linux
Directory services seems more Unix than /etc/group because ‘everything is a file’.
I don't see any top-level comments mentioning POSIX, so I'm here to right that wrong:
POSIX.
POSIX.
Been saying the same thing about homebrew for years. It's ridiculous. No one should use homebrew, and macports is far more correct, but it's pointless to say it.
...by which is meant, "Some differences between macOS, a certified Unix™, and systemd/Linux". Which is fine and valuable! But macOS is the most common Unix™, so the title as written is... entertaining.