macOS 11: copies of dynamic libraries are no longer present on the file system(twitter.com)
twitter.com
macOS 11: copies of dynamic libraries are no longer present on the file system
https://twitter.com/mycoliza/status/1275305517066227712
163 comments
>not supported in non-Chromium browsers at the moment
Not supported in any-browser-that-follows-the-spec, really, considering scroll-to-text is a google made, unilaterally implemented, nonstandard extension (https://wicg.github.io/scroll-to-text-fragment/)
Not supported in any-browser-that-follows-the-spec, really, considering scroll-to-text is a google made, unilaterally implemented, nonstandard extension (https://wicg.github.io/scroll-to-text-fragment/)
I never said it's a standard. Anyway, I'm interested in providing information about dyld cache (and as an aside explaining why my best-effort link doesn't work in some browsers), not encouraging more tribalistic Chrome/anti-Chrome debate, so we can stop right here.
I agree with it, the title should be changed. I thought the same and it's the only reason I looked at the comments.
Ok, we've changed the title from "No more system shared libraries in macOS 11" to a representative phrase from the quoted documentation.
Not a macOS dev but maybe one can clarify: is the title "No more system shared libraries in macOS 11" actually true? The documentation reads to me as a restriction on how you read dynamic libraries (is using only dlopen controversial here? also just trying to understand), not on which system libraries are actually there. It seems dlopen with custom system dylibs would still work if there is also a means to get your dylibs in the system cache? Based on that piece of docs alone I see no reason to grab my pitchfork just yet.
Anyway, I was expecting a few shitstorms to start happening over macOS 11. Apple not announcing any major deprecations (edit: or removals) for this transition - even OpenGL/CL is still there, depecrated - was suspicious.
Anyway, I was expecting a few shitstorms to start happening over macOS 11. Apple not announcing any major deprecations (edit: or removals) for this transition - even OpenGL/CL is still there, depecrated - was suspicious.
You're correct. For 3rd-party provided dylibs, it also isn't necessary to get them into the system cache.
dlopen() will look for the dylib first (as today), and if the file doesn't exist, it'll look in the shared cache.
So the only people this actually impacts are those who stat() system-provided dylibs before calling dlopen(). They should just skip the stat(), and go straight to calling dlopen().
Certainly not worthy of the end-of-the-world vibes in the twitter thread...
dlopen() will look for the dylib first (as today), and if the file doesn't exist, it'll look in the shared cache.
So the only people this actually impacts are those who stat() system-provided dylibs before calling dlopen(). They should just skip the stat(), and go straight to calling dlopen().
Certainly not worthy of the end-of-the-world vibes in the twitter thread...
Thanks for the clarification. I'm guessing people are just looking for the inevitable pitchfork worthy big bad breaking change, but we haven't found one yet (although I'm slightly annoyed my Macbook will be out of official support with this release).
@dang - I flagged this for the editorialized title, in my view it's worth updating this to something like "Restrictions on reading dynamic libraries in macOS 11".
@dang - I flagged this for the editorialized title, in my view it's worth updating this to something like "Restrictions on reading dynamic libraries in macOS 11".
Classic linking uses a path to a library at link time. I suppose Apple may have patched ld to work around this choice, but it still begs the question — why change this? Who was it hurting?
The Apple linker has used `tbd` files, a textual description of a dynamic library's interface, rather than actual dynamic libraries at link time for iOS for several years now. Prior to that it was using stub dylibs, which were essentially symbol-table-only dylib files.
> why change this? Who was it hurting?
Performance.
Performance.
And security. If you sign a single cache across millions of systems you know that that is the one true non-tampered cache.
Can you elaborate beyond a single word? It is certainly not immediately obvious to me how this benefits performance, or I wouldn't have asked.
Not in every OS or programming language, it doesn't.
Linkers just need to somehow find the binary dependencies across the link path/staging.
Linkers just need to somehow find the binary dependencies across the link path/staging.
> They should just skip the stat(), and go straight to calling dlopen().
But `stat`ing it used to be fine on a Mac. And `stat`ing it is fine on other OSs. Why force software to change for no apparent reason just to appease the arbitrary whims of Apple? (And if it's not an arbitrary whim, you'd think they'd at least hint at the upsides of this, no?)
But `stat`ing it used to be fine on a Mac. And `stat`ing it is fine on other OSs. Why force software to change for no apparent reason just to appease the arbitrary whims of Apple? (And if it's not an arbitrary whim, you'd think they'd at least hint at the upsides of this, no?)
Stat then dlopen is a toctou (time of check/time of use) bug. There's a race condition between the check succeeding and the dlopen being called in which that library might be replaced by a malicious one.
Like all race condition bugs it is probabilistic, but it can be won. By contrast dlopen either succeeds or does not, and once you have the handle to the library you are good.
Actually windows also works this way, and has done since wow I think vista if not XP? There's a knowndlls registry key. Large parts of the windows api are contained in a few DLLs; these are loaded on system startup. When resolving shared libraries if one of these names is requested the normal DLL loading procedure of checking paths isn't even followed - it just links in the already loaded DLL from the cache.
It actually wasn't designed as a security feature, but for speed. No matter what the OS does there's little need to continually load these DLLs from disk when we know they are already loaded to even display the desktop.
Of course on windows you can still do the windows equivalent of stat'ing these DLLs but nobody ever does.
Like all race condition bugs it is probabilistic, but it can be won. By contrast dlopen either succeeds or does not, and once you have the handle to the library you are good.
Actually windows also works this way, and has done since wow I think vista if not XP? There's a knowndlls registry key. Large parts of the windows api are contained in a few DLLs; these are loaded on system startup. When resolving shared libraries if one of these names is requested the normal DLL loading procedure of checking paths isn't even followed - it just links in the already loaded DLL from the cache.
It actually wasn't designed as a security feature, but for speed. No matter what the OS does there's little need to continually load these DLLs from disk when we know they are already loaded to even display the desktop.
Of course on windows you can still do the windows equivalent of stat'ing these DLLs but nobody ever does.
> It actually wasn't designed as a security feature, but for speed. No matter what the OS does there's little need to continually load these DLLs from disk when we know they are already loaded to even display the desktop.
FWIW, this optimization is unnecessary on modern Unix systems with a unified buffer cache because libraries loaded from disk are already CoW'd into the user process, without the need for special semantics. It's a pertinent point, nonetheless, but I'm guessing Apple made the switch in macOS for other reasons, perhaps related to disk space, its less than ideal ASLR scheme for Mach-O, or something related to code signing and sandboxing.
I second the TOCTOU concern. It's a bad code smell and a poor habit to stat and then open a file, period. (Likewise for the more general pattern.) There are reasons to do it (some better than others), but they're exceptional. You should never be eager to do it, and when you do you should rule out all possible security exploits or bugs, which is usually more work than abstaining from the superfluous hack. Breaking such code patterns is worth the cost in backward compatibility, IMO, for all the exploits and usability headaches this coding pattern leads to.
FWIW, this optimization is unnecessary on modern Unix systems with a unified buffer cache because libraries loaded from disk are already CoW'd into the user process, without the need for special semantics. It's a pertinent point, nonetheless, but I'm guessing Apple made the switch in macOS for other reasons, perhaps related to disk space, its less than ideal ASLR scheme for Mach-O, or something related to code signing and sandboxing.
I second the TOCTOU concern. It's a bad code smell and a poor habit to stat and then open a file, period. (Likewise for the more general pattern.) There are reasons to do it (some better than others), but they're exceptional. You should never be eager to do it, and when you do you should rule out all possible security exploits or bugs, which is usually more work than abstaining from the superfluous hack. Breaking such code patterns is worth the cost in backward compatibility, IMO, for all the exploits and usability headaches this coding pattern leads to.
Even ignoring the race condition, has stating shared libraries ever been documented as a method to ascertain existence of a shared library?
My guess as to the reason: I would guess the new way is faster, more secure, or takes less memory. Leaving an unused copy of binaries around would be wasting disk space (and yes, disks are enormous nowadays, but SSDs aren’t)
It may cache data from Apple’s servers, but calling it a cache sounds incorrect, though.
My guess as to the reason: I would guess the new way is faster, more secure, or takes less memory. Leaving an unused copy of binaries around would be wasting disk space (and yes, disks are enormous nowadays, but SSDs aren’t)
It may cache data from Apple’s servers, but calling it a cache sounds incorrect, though.
I wouldn't think it's any of those reasons. I'd rather think it's to transparently serve up different libraries based on the caller, even if the name is the same, in the name of compatibility or something like it.
>Why force software to change for no apparent reason just to appease the arbitrary whims of Apple
Why assume there's no reason?
Why assume there's no reason?
To be fair, the reason is certainly not apparent.
The reason is immediately apparent. They're eliminating a redundant copy of the libraries to save disk space.
That's definitely not the reason, given that symlinks are still a thing.
You can't symlink into a byte range of a file. All of the system dylibs are stored in a single file.
Also, the dyld shared cache isn’t just some archive containing exact copies of the dylib files. Among other things, the cache builder pre-resolves references between different dylibs in the cache. This saves time on process startup, and also allows pages to be shared between multiple processes even if they contain relocations, saving memory. But it means that even if you could symlink to a byte range, you couldn’t just have the original dylib paths be symlinks to parts of the cache. (It might be possible to make this work with enough effort, but it would be very nontrivial.)
Note that the dyld shared cache itself has existed for a long time; the only change here is removing the original copy of the dylibs to save disk space.
Note that the dyld shared cache itself has existed for a long time; the only change here is removing the original copy of the dylibs to save disk space.
https://developer.apple.com/documentation/macos-release-note... has the text. dlopen will just work, but you can’t crawl through system directories anymore, hoping to find a shared library.
It wouldn’t surprise me if there’s no way to get your dynamic library there anymore, as this explicitly says it’s of the system provided ones.
It wouldn’t surprise me if there’s no way to get your dynamic library there anymore, as this explicitly says it’s of the system provided ones.
OpenGL was deprecated last year
Everyone thought it was deprecated last year so that they didn't have to support it on ARM Macs, but it appears (not 100% confirmed until people actually get these things) like it is still there and working on ARM Macs (albeit deprecated)
It is officially documented as being available in ARM Macs.
Yeah but prerelease documentation has been wrong before.
Isn't that why it's called "Pre" release? If there is a chance of changes between the release and what comes before that I'd say that makes sense.
Say a 'release' turns out to have a discrepancy between documentation and reality, that's when the problems start to mount.
Say a 'release' turns out to have a discrepancy between documentation and reality, that's when the problems start to mount.
Isn't this the model that has been encouraged on Windows forever with COM, WinRT and .NET etc? When you consume an API in one of those environments you don't have to worry about the location of the physical assets (library files).
I don't see the big deal as long as app local copies of e.g. curl can still be used
I don't see the big deal as long as app local copies of e.g. curl can still be used
FWIW it is also how you can use Win32 APIs introduced in recent versions of Windows when you want to also support older versions - call LoadLibrary with a system DLL name (no path) and use GetProcAddress to check if the call you want to perform is supported. AFAIK it is very rare for Windows applications to hardcode any DLL path.
On the flip side of that, Win32 also includes the local directory near the start of the LoadLibrary search path. The result being that a lot of applications bundle all of their own dependencies when there are already 15 copies installed on the system. In some cases I’m good with that, in others it’s a pretty big waste of space compared to having a system with an easily configurable versioned shared library repository.
One side effect of this is users can use patched DLLs for compatibility or to add new features (this is how tools like ReShade work).
Hmm I think shared lib loading on Mac OS isn't supposed to work like that. DYLD_LIBRARY_PATH is only a method of last resort; normally, an app should have any lib path baked in (where the baked in path can be relative, and can also contain variables as path steps). At link-time, the linker picks up the install name of any shared libs needed, and bakes these into the executable. I find this to be a saner model than on Linux where an executable can't have an RPATH (only a shared lib can). Actually, it seems the whole concept of shared libraries might be more trouble than worth, especially on F/OSS operating systems; on Linux, there's a whole cottage industry based on the problem of shared libs not being uniform across versions and distros (ie Docker).
Applications on Linux also do that often when they are released in binary form (especially closed source software) outside of some repository, they just use a shell script to add the current directory (or a subdirectory) to the library search path before starting the executable.
Maybe for new libraries, but I know that enough older Windows apps rely on expecting existing system libraries to exist as actual files that Wine has to create fake DLL files in order to keep them all happy.
Sounds like it to me. Similar to Windows WinSxS directory, or system32 redirection, or Program Files (x86).
Now we need a /System/Library/WOWARM to contain old x64 libraries and /System/Library/AMD64 to contain arm libraries and it will be perfect!
I know that you are ironic here, but for the uninformed, macOS supports "fat" binaries, where the machine code for multiple archetictures can be embedded in the same bundle
Yep, and for the sake of making sure everyone can appreciate the joke I suppose I should explain the other half. During the 32 -> 64 bit transition, Windows took the opposite approach: instead of packing both architectures into a single "fat" library file, they would separate them into different folders. 64 bit libraries would be "native" and go into the usual system library folder and legacy 32 bit versions would go into a new "Windows on Windows" emulation folder. Good so far. The problem is that the usual system library folder was /Windows/System32 which alludes to 32-bit-ness but will now hold 64 bit libraries. That's a bit facepalm, but sacrifices had to be made for compatibility, or something (I would assume). For the emulation folder they chose /Windows/SysWOW64 because "Windows on Windows" ran on 64 bit and emulated 32 bit. Fair enough, I suppose, but now a folder with a name alluding to 64-bit-ness will contain 32 bit libraries, so we have a double facepalm:
C:\Windows\System32 is where the 64 bit libraries live
C:\Windows\SysWOW64 is where the 32 bit libraries live
To be the "well, actually" guy, "Windows on Windows" was the name for 16-bit windows on win32. Because "Windows" meant Windows 3.1 in those days.
But "Windows on Win64" is WOW64, i.e., Win32 on Win64. Because by then "Windows" meant Win32.
so "SysWow64" is the "System directory for WOW64".
I've been on Windows too long because it all seems perfectly logical to me.
But "Windows on Win64" is WOW64, i.e., Win32 on Win64. Because by then "Windows" meant Win32.
so "SysWow64" is the "System directory for WOW64".
I've been on Windows too long because it all seems perfectly logical to me.
can we get a binary with 68k, powerpc, x86, and arm and call the binary .fatso
Sadly by tradition standalone dynamic libraries on macOS end in "lib", as in "dylib" or "jnilib", unless there is a good reason to not do so (pam, for example).
How does it work on mac? Do you have to specify the full path of the library to load it?
In practice it's transparent, like using a static library. The linker handles it all for you during compilation: it stores the paths at a special section in your executable, and a injects a dynamic linker in your code.
The paths can be either absolute (for system libs) or relative (useful for distributing Bundled apps).
The cool thing is that you can see those hardcoded paths using otool -L and you can modify them using install_name_tool. Useful for debugging misbehaving closed-source tools.
The paths can be either absolute (for system libs) or relative (useful for distributing Bundled apps).
The cool thing is that you can see those hardcoded paths using otool -L and you can modify them using install_name_tool. Useful for debugging misbehaving closed-source tools.
dyld, the dynamic linker on macOS, handles it for you. You tell it what you need it will handle mapping in the correct pages for you from the big file; both statically (based on your binary's load commands) and dynamically (using dlopen, etc.) The process is essentially transparent unless you look really carefully at what it's doing under the hood.
[deleted]
This is mildly interesting but not much more than that. Who cares if the set of system libraries are available in a series of separate dylib files, instead of being packed together in one mega archive?
Most comments here on HN seem to misunderstand what this is.
Most comments here on HN seem to misunderstand what this is.
Yeah, they’ve been doing the pre-linked mega archive thing for years. It’s in /var/db/dyld. The only difference here that I see is that they’re not shipping the originals and not rebuilding it all the time. And I’m guessing the resulting archive is now sitting on a protected partition.
This thing can be taken apart, but it’s a little tricky, since it’s pre-linked. I wrote something years ago to do it, not perfect, but enough to get symbols out. (I was trying to annotate iOS stack traces.) I don’t recall if I got it working well enough to open the libraries up in hopper and poke around.
This thing can be taken apart, but it’s a little tricky, since it’s pre-linked. I wrote something years ago to do it, not perfect, but enough to get symbols out. (I was trying to annotate iOS stack traces.) I don’t recall if I got it working well enough to open the libraries up in hopper and poke around.
People writing third party linkers or debuggers is one.
I often dig at shared libraries using tools like nm to see what functions are where.
I often dig at shared libraries using tools like nm to see what functions are where.
I'm sure there will be a way to unpack the mega archive and run nm (or maybe even run nm directly on the mega archive). Relax :)
Sadly:
$ nm /System/Library/dyld/dyld_shared_cache_x86_64h
/Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/nm: error: /System/Library/dyld/dyld_shared_cache_x86_64h: Invalid argument.
I never said I wasn't relaxed about it. I don't even use a Mac much.
But the platform does seem to have been in a multi-year process of technical decline, and much of that does speak to a "we know better than you, and everybody else" attitude.
But the platform does seem to have been in a multi-year process of technical decline, and much of that does speak to a "we know better than you, and everybody else" attitude.
A number of the changes that Apple has been making seem to be security related. One reason to just dlopen() rather than stat() and dlopen() is the race-condition, which an attacker could use to get a program to use the wrong library. Similarly macOS' locked-down root partition. Checking hashes of binaries is also for security, to make sure the binary you're running is the one the developer thought they were giving you. So on the one hand the platform is in "slow technical decline" yet we (HN) claim to value security. It seems to me that the platform is doing well technically. We all remember the mess Microsoft had 10 years ago with security vulnerability du jour; Apple seems to be pre-emptively protecting its users. I submit that the "technical decline" is really "macOS is changing from a freewheeling Unix system to a secure system" and people here want to be cowboys and artists with their own private filesystem canvas. I understand the frustration, but Linux still exists. Apple isn't in the business of making a dev playground, but a computer that Just Works for people. It seems to me there isn't a technical decline, it's that HN is revealing itself to value the plasticity of traditional Unix over security (now that security has gotten good enough). I expect that SELinux has similar chafing restrictions, the only difference is that Linux is more of a set of parts so you get to (must) choose how you put those tools together.
> One reason to just dlopen() rather than stat() and dlopen() is the race-condition
But dlopen() implies code execution all the time, and stat() does not.
A good way to close this race condition, by the way, is to have a dlopen-by-fd API [FreeBSD has fdlopen], and a single call to open(2) resolves the race condition inherent in referencing the filename multiple times.
But to your larger point... It seems there is a crowd that will kneejerk dismiss any criticism as "it's for security". And any implementation bugs (such as introducing massive delays when executing programmatically-generated shell scripts because they need to be hashed and sent to a server, as was an HN headline a while ago) are also justified. I think there needs to be greater skepticism than this about how secure these efforts really makes the system, or if they are just throwing buggy stuff at a wall and applying post hoc justifications.
But dlopen() implies code execution all the time, and stat() does not.
A good way to close this race condition, by the way, is to have a dlopen-by-fd API [FreeBSD has fdlopen], and a single call to open(2) resolves the race condition inherent in referencing the filename multiple times.
But to your larger point... It seems there is a crowd that will kneejerk dismiss any criticism as "it's for security". And any implementation bugs (such as introducing massive delays when executing programmatically-generated shell scripts because they need to be hashed and sent to a server, as was an HN headline a while ago) are also justified. I think there needs to be greater skepticism than this about how secure these efforts really makes the system, or if they are just throwing buggy stuff at a wall and applying post hoc justifications.
FFI developers and users.
autoconf developers and users. The whole lib probe ecosystem might need adjustment.
autoconf developers and users. The whole lib probe ecosystem might need adjustment.
>People writing third party linkers or debuggers is one.
All 10 of them?
All 10 of them?
Still not zero. And the number of people using them is higher than that.
If in your personal life you are needlessly a jerk to a small number of people... Do you turn around and say "yeah, 10 people. So what?" I suppose many people do.
If in your personal life you are needlessly a jerk to a small number of people... Do you turn around and say "yeah, 10 people. So what?" I suppose many people do.
Weasel words: needlessly and jerk.
Apple didn't do it to be jerks, and didn't do it needlessly (except in the sense "they could just leave it as is").
Apple didn't do it to be jerks, and didn't do it needlessly (except in the sense "they could just leave it as is").
Seems like you made my point in parentheses.
Jerkiness is subjective, but in my opinion there is a question of respect for the user, developer, or fellow human involved in the area of compatibility, where it is kind of a jerk move to fail to consider that you are breaking people, or someone might narcissistically expect the universe unconditionally to adapt to their whim, making them jump through hoops and live with the consequences of their questionable ideas. Apple does not have some of those values down well.
Jerkiness is subjective, but in my opinion there is a question of respect for the user, developer, or fellow human involved in the area of compatibility, where it is kind of a jerk move to fail to consider that you are breaking people, or someone might narcissistically expect the universe unconditionally to adapt to their whim, making them jump through hoops and live with the consequences of their questionable ideas. Apple does not have some of those values down well.
Apple swaps around stuff for seemingly no reason pretty much every update. Making any compiler or debugger writers life a pain.
The move seems to enforce memory-always, not file-based dyn linking. Whatever is sitting in the memory that responds to the established API.
On the first look it's not that different from how this works now. However, I'm thinking of the next step where the 'system cache' is not even local to the PC, like in a case of a netboot the whole or core set would get downloaded from the remote server be it Apple's or your friendly custom provider/employer.
MacBook Pro VT. It's a stretch, of course...
On the first look it's not that different from how this works now. However, I'm thinking of the next step where the 'system cache' is not even local to the PC, like in a case of a netboot the whole or core set would get downloaded from the remote server be it Apple's or your friendly custom provider/employer.
MacBook Pro VT. It's a stretch, of course...
If I understand correctly, this is just the same behavior as iOS's "dyld_shared_cache" which is basically all the system shared libraries stuffed into a single file.
https://iphonedevwiki.net/index.php/Dyld_shared_cache
https://iphonedevwiki.net/index.php/Dyld_shared_cache
Correct, all the system libraries can be now found in the dyld shared cache can be found in /System/Library/dyld/dyld_shared_cache_x86_64(h) (it's moved in Big Sur!).
Good. Apple is basically the only entity seriously looking at dynamic linking performance these days. We can do a lot better in the ELF world. We should not only copy dyld's closure cache, but also teach the kernel to do relocations on demand, like NT does. A relocated data-segment page should be a clean page, not a dirty page!
To think, the Amiga was doing something like this in the 80's. The "OpenLibrary" call would transparently access shared libraries either in ROM, already loaded into memory, or on disk.
Well ARM is a 1980s technology so seems apt.
AFAIK, they will be using AArch64, which is from 2011 (AArch64 is not an extension of the old 32-bit ARM ISA, it's a completely new ISA).
Compared to traditional linking and UNIX, which is an early 70s one?
x86 is 1980s. POWER is 1990s. MIPS is 1980s. Sparc is 1980s.
What would you prefer they use? Itanium? RISC-V?
What would you prefer they use? Itanium? RISC-V?
Itanium is 2000s. RISC-V is 2010s.
Apple could make a great leap forward by adopting the Mill, an ISA so novel that it hasn't even been released yet!
Apple could make a great leap forward by adopting the Mill, an ISA so novel that it hasn't even been released yet!
Too bad Alpha died. Those machines were like lightning (for the 90's, anyway.)
IIRC even Windows NT could run on Alpha. I wonder if computing would've been different had we gotten a modern 64-bit architecture with very little legacy become commonplace in the 90s.
Makes me think of the Tanenbaum quote from 1992: "Of course 5 years from now that will be different, but 5 years from now everyone will be running free GNU on their 200 MIPS, 64M SPARCstation-5".
Makes me think of the Tanenbaum quote from 1992: "Of course 5 years from now that will be different, but 5 years from now everyone will be running free GNU on their 200 MIPS, 64M SPARCstation-5".
Yes, Digital Unix, OpenVMS, and Windows NT all ran. You could also run Linux and FreeBSD. I worked at a DEC shop for a while in the late 90's. Alphas were speed demons. This was right before Compaq bought DEC. After that, I think things started going down hill...
The lead designer of one of the Alpha chips founded PA Semi (which is the company that Apple acquired to start designing their own chips).
x86 is 70s technology: the 8086 came out in 1978, and it is an extension of the 8080 which came out in 1974, itself an extension of the 8008 from 1972.
Funny how the modern world runs on the descendants of an overgrown calculator chip.
Funny how the modern world runs on the descendants of an overgrown calculator chip.
I wonder how many years we've got where you're still able to read the big mega archive...
Someday it may sit in some kind of "protected enclave", mapped into your process with "execute-only" permission for code, and maybe graded protections for its data as well, so you can't debug, trace, instrument, reverse engineer or otherwise get to really know the system libraries to see what they are doing and learn from them.
The trend has been that way for a while. For the moment, that kind of isolation is done at process boundaries, on devices that have enclaves.
There is no technical reason why that kind of isolation cannot extend inside processes, for a company that designs its own CPUs. For the same kind of "black boxes" as we already have inside mobile devices, but with faster API calls and data exchange than doing it across processes boundaries.
Someday it may sit in some kind of "protected enclave", mapped into your process with "execute-only" permission for code, and maybe graded protections for its data as well, so you can't debug, trace, instrument, reverse engineer or otherwise get to really know the system libraries to see what they are doing and learn from them.
The trend has been that way for a while. For the moment, that kind of isolation is done at process boundaries, on devices that have enclaves.
There is no technical reason why that kind of isolation cannot extend inside processes, for a company that designs its own CPUs. For the same kind of "black boxes" as we already have inside mobile devices, but with faster API calls and data exchange than doing it across processes boundaries.
Likely an unpopular opinion: not everything has to be a file. UNIX removed a lot of obstacles and helped usher in the era of computing accessible to individuals -- but we can't keep relying on old paradigms when it's now clear (with the benefit of hindsight) that it's not a be-all-end-all solution.
Filesystems are awful databases and I believe everyone who was a programmer for some time realizes this. The lack of ACID being at the top.
The sooner we move to specialized APIs for, well, everything, the better off we're going to be.
---
As one recent example, the PS 5 will just directly load 3D artefacts from the SSD storage to the GPU when the corresponding API is called. This gives Sony peace of mind that people won't be coming up with 100+ ways of doing it by themselves which Sony will then have to support for the entirety of the PS 5 lifecycle.
We the programmers have demonstrated to ourselves, time and again, that too much freedom can actually be a bad thing.
It's okay to move away from the "everything is a file" philosophy. If anything, I'd love to see a Linux-like OS that utilizes sqlite3 for, say, the entirety of the current `/etc` directory and uses sqlite3's backup/changeset APIs to do incremental and point-in-time backups. Sure it's little less transparent but you can't ever capture an inconsistent snapshot. That has to be an improvement, right?
---
EDIT: I suppose people can't find a link between my comment and the topic but my point was that moving to dedicated APIs -- and not relying on the filesystem -- for various services (like loading a dynamic library) is a good thing because the "everything is a file" paradigm isn't as bulletproof as it's made out to be (also because "stat" + "dlopen" one after another are race-condition prone as others pointed out).
I'd love to hear the downvoters elaborate their reasoning.
Filesystems are awful databases and I believe everyone who was a programmer for some time realizes this. The lack of ACID being at the top.
The sooner we move to specialized APIs for, well, everything, the better off we're going to be.
---
As one recent example, the PS 5 will just directly load 3D artefacts from the SSD storage to the GPU when the corresponding API is called. This gives Sony peace of mind that people won't be coming up with 100+ ways of doing it by themselves which Sony will then have to support for the entirety of the PS 5 lifecycle.
We the programmers have demonstrated to ourselves, time and again, that too much freedom can actually be a bad thing.
It's okay to move away from the "everything is a file" philosophy. If anything, I'd love to see a Linux-like OS that utilizes sqlite3 for, say, the entirety of the current `/etc` directory and uses sqlite3's backup/changeset APIs to do incremental and point-in-time backups. Sure it's little less transparent but you can't ever capture an inconsistent snapshot. That has to be an improvement, right?
---
EDIT: I suppose people can't find a link between my comment and the topic but my point was that moving to dedicated APIs -- and not relying on the filesystem -- for various services (like loading a dynamic library) is a good thing because the "everything is a file" paradigm isn't as bulletproof as it's made out to be (also because "stat" + "dlopen" one after another are race-condition prone as others pointed out).
I'd love to hear the downvoters elaborate their reasoning.
I think this is a good change. It's eliminates a potential avenue of accidentally doing the wrong thing but not noticing because it "works on my machine."
In what scenario does it eliminate this?
I have had to deal with bugs in the past caused by people using incorrect case for paths to dlopen(), etc.
Lots (I wouldn't be surprised if most) osx devs use case sensitive filesystems, but the default in case-insensitive, and so dlopen(path) fails if you have (say) "/system/..." instead of "/System/..."
Lots (I wouldn't be surprised if most) osx devs use case sensitive filesystems, but the default in case-insensitive, and so dlopen(path) fails if you have (say) "/system/..." instead of "/System/..."
Can anyone come up with a reason this would be preferable to having just the index cached for the linker and handing out mapped memory / open files instead? What's the benefit here over essentially ld.so.cache with bells and whistles?
It reduces disk space.
Historically the dyld_shared_cache has been a pre-bound copy of most system libraries that is mapped into memory once and then shared across all processes. It was updated automatically by the system at various times, and if a given system library was determined to be newer then the dynamic linker would simply load it from disk. The result of this was that for most system libraries you ended up with two copies on disk: one at the normal location, and one within the shared cache. Now only the latter copy exists, and Apple ships it directly rather than creating it on each machine from the copy of the system libraries that they ship.
Historically the dyld_shared_cache has been a pre-bound copy of most system libraries that is mapped into memory once and then shared across all processes. It was updated automatically by the system at various times, and if a given system library was determined to be newer then the dynamic linker would simply load it from disk. The result of this was that for most system libraries you ended up with two copies on disk: one at the normal location, and one within the shared cache. Now only the latter copy exists, and Apple ships it directly rather than creating it on each machine from the copy of the system libraries that they ship.
Disk is cheap, and we're not talking about a huge amount of space. Surely there's a better justification.
On lower end laptops it isn’t cheap. Also these savings add up - a few hundred MB here and there and you end up with a system that’s 10GB lighter on a laptop that has just 128/256GB of space.
Sorry, but the /usr model for compiling is out of date / flawed. It isn't about "me" it is about "you" the target for running the executable.
For many years, macOS has supported being a cross compilation environment for multiple platforms and multiple OS releases.
This means that include files and libraries for the target OS and platform are not stored at the "host filesystem" paths that were used in the 1960's, 1970's and 1980's... (an embarrassingly long list). Include files and references to libraries are available in the SDKs. One should use the SDK path as a prefix for the include and library paths - something that most GNU tools, and other open source projects have supported for a very long time, but sadly a little inconsistently. There are runtime tools and recipes that can be used to get these paths, including for "myself".
All compilation on macOS should be considered as cross compilation -- because macOS simultaneously supports multiple architectures and OS versions.
This really is a good abstracted model for all compilation.
This has benefits! Install time and disk space usage are reduced for most customers and developers. Developers on x86_64 do not need the aarch64 content in /System, /usr to compile. Install the SDKs for the targets you wish to compile for.
For many years, macOS has supported being a cross compilation environment for multiple platforms and multiple OS releases.
This means that include files and libraries for the target OS and platform are not stored at the "host filesystem" paths that were used in the 1960's, 1970's and 1980's... (an embarrassingly long list). Include files and references to libraries are available in the SDKs. One should use the SDK path as a prefix for the include and library paths - something that most GNU tools, and other open source projects have supported for a very long time, but sadly a little inconsistently. There are runtime tools and recipes that can be used to get these paths, including for "myself".
All compilation on macOS should be considered as cross compilation -- because macOS simultaneously supports multiple architectures and OS versions.
This really is a good abstracted model for all compilation.
This has benefits! Install time and disk space usage are reduced for most customers and developers. Developers on x86_64 do not need the aarch64 content in /System, /usr to compile. Install the SDKs for the targets you wish to compile for.
Frees up a gig or two of disk space. Good.
Frees up cheap disk space by moving the library contents to expensive RAM? That doesn't seem like a good thing. And it also doesn't seem possible — you want system libraries to survive reboot. Presumably these files exist somewhere on disk, just hidden.
The dylib cache is stored on disk. It's saving disk space by only having that copy of the libraries instead of two copies.
What would the second copy be?
Not hidden, just inside the ROSP partition at /System/Library/dyld.
two questions
Does this break pkg-config?
What about fishing for dylibs in .bundle directories that may be installed in multiple locations?
Does this break pkg-config?
What about fishing for dylibs in .bundle directories that may be installed in multiple locations?
So the title is wrong? The libs are just not directly accessible from the file system
I assume this is for privacy (an app can’t fingerprint you by looking at which libraries you have) but macOS is starting to become a terrible dev environment. Why are they locking down my device from me?
> I assume this is for privacy (an app can’t fingerprint you by looking at which libraries you have)
Well I mean it only applied to system libraries. And for those you can now just `dlopen` and check for failure, so it can't be about privacy.
> Why are they locking down my device from me?
I mean isn't this what Apple has been doing at least since the turn of the millenium?
Well I mean it only applied to system libraries. And for those you can now just `dlopen` and check for failure, so it can't be about privacy.
> Why are they locking down my device from me?
I mean isn't this what Apple has been doing at least since the turn of the millenium?
If nothing else, it's a size win. These libraries serve no purpose with a prebuilt dyld cache, so you'd be paying to download / would be paying to upload them to every customer for no good reason.
When you say "Starting To" I presume you mean from about 2015 on. It's clear Windows is trying to pick up slack and also be cloud developer friendly with WSL integration.
Apple sticking with Intel and now reverting to a terrible proprietary cpu architecture, locking software down more and more, releasing pro versions with inadequate options and keyboard all acts to ensure they lose their status as top developer environment
Before they dropped powerpc for intel, no one would have considered using a mac for dev work.
It's clear they don't understand the developer market at all or are only interested in Mobile because the revenue is insane.
They had a good ten years as the best dev environment but as they say "The King is dead, long live the King".
Apple sticking with Intel and now reverting to a terrible proprietary cpu architecture, locking software down more and more, releasing pro versions with inadequate options and keyboard all acts to ensure they lose their status as top developer environment
Before they dropped powerpc for intel, no one would have considered using a mac for dev work.
It's clear they don't understand the developer market at all or are only interested in Mobile because the revenue is insane.
They had a good ten years as the best dev environment but as they say "The King is dead, long live the King".
>terrible proprietary cpu architecture
Err, there are more cpus with ARM than x86 by an order of magnitude...
It's also less proprietary...
>Before they dropped powerpc for intel, no one would have considered using a mac for dev work.
Huh? Where did you get that from? OS X Macs where quite popular for dev work pre-Intel, popular for web devs, and with good Java support...
Err, there are more cpus with ARM than x86 by an order of magnitude...
It's also less proprietary...
>Before they dropped powerpc for intel, no one would have considered using a mac for dev work.
Huh? Where did you get that from? OS X Macs where quite popular for dev work pre-Intel, popular for web devs, and with good Java support...
I'm guessing when your parent said "cpu architecture" they meant precisely that and not "instruction set architecture". The ISA is ARM, sure, but the CPU architecture is Apple.
> Apple sticking with Intel and now reverting to a terrible proprietary cpu architecture
Are you referring to the ARM instruction set generally being more open for custom instructions? Going by the number of licensees and producers of ARM silicon, I'd say ARM is less proprietary compared to x86 which is basically just produced by Intel, AMD, and a Chinese licensor AFAIK.
Are you referring to the ARM instruction set generally being more open for custom instructions? Going by the number of licensees and producers of ARM silicon, I'd say ARM is less proprietary compared to x86 which is basically just produced by Intel, AMD, and a Chinese licensor AFAIK.
> Apple sticking with Intel and now reverting to a terrible proprietary cpu architecture
ARM is more open than x86. In fact, you could start your own ARM chip design company today. Which is something you cannot do with x86 at all, even if your were to do a complete takeover of AMD or Intel.
> Before they dropped powerpc for intel, no one would have considered using a mac for dev work.
DHH wrote Ruby on Rails on a Powerbook. Microsoft did all the development of the original XBox on PowerPC Macs. A lot of developers used macs before Intel, and they will use them after Intel.
> They had a good ten years as the best dev environment but as they say "The King is dead, long live the King".
Given the Frankenstein horror show that is Windows with WSL2 and the terminal lack of a polished working GUI in Linux, Mac is still the best dev environment unless you are doing some platform specific work.
ARM is more open than x86. In fact, you could start your own ARM chip design company today. Which is something you cannot do with x86 at all, even if your were to do a complete takeover of AMD or Intel.
> Before they dropped powerpc for intel, no one would have considered using a mac for dev work.
DHH wrote Ruby on Rails on a Powerbook. Microsoft did all the development of the original XBox on PowerPC Macs. A lot of developers used macs before Intel, and they will use them after Intel.
> They had a good ten years as the best dev environment but as they say "The King is dead, long live the King".
Given the Frankenstein horror show that is Windows with WSL2 and the terminal lack of a polished working GUI in Linux, Mac is still the best dev environment unless you are doing some platform specific work.
I was with you until this:
> Given the Frankenstein horror show that is Windows with WSL2
Here I will politely disagree. WSL2 is far from a monster. It is a quite nifty hack, and fast at that.
I'm normally working on a Lknux machine but I've used Windows with WSL2 day out and day in now and I am only aware of it twice or maybe three times a day, and even then it is not a horror show - quite the contrary - it is close to the definition of seamless.
> and the terminal lack of a polished working GUI in Linux,
To each their own. I won't get into enumerating everything I missed while I used Mac but I can say I tried very hard to make it work as some parts of the MacBook experience was really great.
> Mac is still the best dev environment unless you are doing some platform specific work.
As thousands of devs who are free to choose their own laptop and yet doesn't choose Mac can tell: it is a bit more complicated.
I'd appreciate if we could stop propagating this myth now.
Mac is different. Some enjoy it. Others cannot stand it.
> Given the Frankenstein horror show that is Windows with WSL2
Here I will politely disagree. WSL2 is far from a monster. It is a quite nifty hack, and fast at that.
I'm normally working on a Lknux machine but I've used Windows with WSL2 day out and day in now and I am only aware of it twice or maybe three times a day, and even then it is not a horror show - quite the contrary - it is close to the definition of seamless.
> and the terminal lack of a polished working GUI in Linux,
To each their own. I won't get into enumerating everything I missed while I used Mac but I can say I tried very hard to make it work as some parts of the MacBook experience was really great.
> Mac is still the best dev environment unless you are doing some platform specific work.
As thousands of devs who are free to choose their own laptop and yet doesn't choose Mac can tell: it is a bit more complicated.
I'd appreciate if we could stop propagating this myth now.
Mac is different. Some enjoy it. Others cannot stand it.
I used to enjoy it, cant bring myself to buy a new macbook pro with what they put out lately. Will go with some ryzen based book and run some apt-get or pacman distro of linux instead. Hell, I'd take w10 Pro over apple offerings of late.
Have you used Windows lately? It's going downhill.
Ads everywhere, surprise "you don't have a choice" updates, and just the other day after an update it locked the screen to force me through a slideshow that tried repeatedly to coerce me into using Edge ("let's import your bookmarks / contacts / whatever" and the "please not now" button was de-emphasized and kept changing names and moving between slides).
Ads everywhere, surprise "you don't have a choice" updates, and just the other day after an update it locked the screen to force me through a slideshow that tried repeatedly to coerce me into using Edge ("let's import your bookmarks / contacts / whatever" and the "please not now" button was de-emphasized and kept changing names and moving between slides).
> Ads everywhere, surprise
This is actually a good point.
At Microsoft guys around here: Please tell your folks that nothing says quality like when your paid business OS tries to upsell you like a really cheesy salesman /s
This is actually a good point.
At Microsoft guys around here: Please tell your folks that nothing says quality like when your paid business OS tries to upsell you like a really cheesy salesman /s
>Why are they locking down my device from me?
Because it belongs to the end users, and it should be as task focused, opaque, and easy to use as an appliance, not something you tinker with.
Because it belongs to the end users, and it should be as task focused, opaque, and easy to use as an appliance, not something you tinker with.
I am the end user
Not the typical one. As you complain about the Mac as a "dev environment"
This only effects the system libraries - not any of the libraries from other applications or tools.
How common is this technique for loading libraries? How about in the kind of POSIX tools that are available, but not exclusive to, on mac os (sucha as bash, termux, vim, git, apache etc)?
The old code checks for the filepath to be present before attempting to dlopen it. The new way requires you to dlopen the path. This is very meh because dlopen already executes code from the library, so if you didn't want that, good luck.
(glibc and some other core libs like openssl like to detect plugins by probing the paths for libs and only loading them when necessary)
(glibc and some other core libs like openssl like to detect plugins by probing the paths for libs and only loading them when necessary)
Yeah. So I understand it has been considered best practice to check the file before opening it? Does that mean we can assume that most well-written programs will follow the pattern and break on mac os?
I would have thought that even if Apple did make this change they could trap the fopen or whatever and return something useful based on a heuristic? It wouldn't be the first time that they put exceptions and heuristics in for software compatibility.
I would have thought that even if Apple did make this change they could trap the fopen or whatever and return something useful based on a heuristic? It wouldn't be the first time that they put exceptions and heuristics in for software compatibility.
It has never been good practice to check before open - open first, then check. And if you're creating a file, use exclusive mode, don't check first.
You're opening yourself up to a lot of race conditions otherwise, and this has been the source of many security vulnerabilities
You're opening yourself up to a lot of race conditions otherwise, and this has been the source of many security vulnerabilities
> So I understand it has been considered best practice to check the file before opening it?
Check it... for what?
Check it... for what?
For presence...
Why do you need to separately check for presence before you use dlopen? If it's not there dlopen will give you a nice error message - no need to check yourself.
Are you currently checking files are there before you dlopen them? How do you avoid a race between the stat and the dlopen? What happens if the file is removed between them? Seems broken and you shouldn't be doing this in the first place.
Are you currently checking files are there before you dlopen them? How do you avoid a race between the stat and the dlopen? What happens if the file is removed between them? Seems broken and you shouldn't be doing this in the first place.
It is usually unlikely that a file will be removed between stat and dlopen, such errors occur rarely. Even more rarely when the file happens to be a systems lib or plugin lib. Some errors occur rarely enough that it isn't worth worrying about.
NON-system libraries? Lots. e.g. reading through a directory of dynamically loadable plugins
System libraries (depending how Apple defines that)? Unusual.
System libraries (depending how Apple defines that)? Unusual.
I would expect this to impact mainly build systems, and I would expect to impact them really hard. But to be honest this is just a gut feeling. On the other hand, I am sure that Apple has thought this through, as we all know developer experience is of paramount importance for them.
Is there a good term which distinguishes a general-purpose OS (Unix, Windows, Plan 9, etc) from operating systems like Android, iOS, Windows RT, etc? The term "operating system" is being overloaded to ship systems which are really more of a graphical shell than a classical operating system. The moves to lock down operating systems, ostensibly made for "security" reasons but conveniently centralizing power and authority with the vendor at the same time, are being marketed to consumers as the same kind of product as a conventional operating system while in fact being profoundly different.
I think you're in the wrong thread.
Somebody asked on Twitter, "doesn't this break all of Brew?"
Does anybody know?
Does anybody know?
I don't see why, Brew just uses the standard compilation tools any other software on OSX uses, so it'll work the way any other tool works
Brew either uses the system libraries (which still work) or compiles its own version (which is not affected by this)
Brew either uses the system libraries (which still work) or compiles its own version (which is not affected by this)
Not to be paranoid, but with this change what stops Apple from silently shipping libraries with backdoors/surveillance if requested by law enforcement. How would a security researcher catch the change without being able to examine the library files?
If you're worried about those types of things you shouldn't be using proprietary operating systems, there's a million ways Apple can backdoor your system without you knowing
They don't need to ship a backdoor because all files in iCloud (photos, documents, etc.) aren't E2E encrypted anyway. According to Reuters they readily share that with the government, despite all their FBI-thwarted-yet-again posturing in the press.
Not to mention they could just backdoor the kernel, and since that's not open-source, good luck detecting that.
Not to mention they could just backdoor the kernel, and since that's not open-source, good luck detecting that.
And of course they do, surveillance is the law in the US. They are expected to comply.
It’s pretty trivial to re-extract the dylibs from the cache.
For example, jtool2 does it for iOS, where Apple has already purged system dylib files.
For example, jtool2 does it for iOS, where Apple has already purged system dylib files.
> How would a security researcher catch the change without being able to examine the library files?
How would you know that Apple was loading what was on disk, before this change?
How would you know that Apple was loading what was on disk, before this change?
After dlopen() you are free to examine the in-memory contents, including writing them out to disk if you like. Apple as an attacker could easily hide a backdoor in either model.
I don't see why it should be any different.
For end users, this transition will be fairly seamless. For developers, let the nightmares begin.
How would we write our own libraries?
I guess just as before, by using install_name_tool to burn the install location into a .dylib (with special vars), and then link against it? The question seems rather, how to link against system libs if these are no longer palpable files.
Clearly a change for privacy reasons.
You aren’t allowed to go fishing around the system anymore.
You aren’t allowed to go fishing around the system anymore.
> You aren’t allowed to go fishing around the system anymore.
Sure you can. You'll now `stat` around for everything you did before, but now you'll have to `dlopen` the system libraries instead. Surely no privacy win?
Sure you can. You'll now `stat` around for everything you did before, but now you'll have to `dlopen` the system libraries instead. Surely no privacy win?
If I read this correctly, this is absolutely bonkers. Basically the "System libraries" do not exist as files anymore? So they just deprecated normal dynamic linking for a subset of libraries? But as I know the Mac folks around there, they are going to swallow it. Because, uhm,..., It's Apple.
> this is absolutely bonkers
Why do you care if the files are on the disk or not though? What are you using that for?
Why do you care if the files are on the disk or not though? What are you using that for?
For linking.
Essentially breaks many fingerprinting strategies.
This is a very good change.
This is a very good change.
Can you elaborate on what fingerprinting attack is broken by this model?
Because now, you can only probe for the existence of a specific shared library. You can't just enumerate all the libraries that happen to be available. That said, this change appears to apply only to system libraries --- how much fingerprinting entropy is contained in the system library list that isn't already contained in the OS version information?
Zero, because the set of system libraries is fixed and doesn't vary between machines.
Fixed for this specific version. Probing tools are there to abstract different versions and changing API's.
Now you got a probing tool with executive side effects, before it was pure read-only. Not everyone will be happy with this idea to execute first, look later. dynamic libs execute init functions at dlopen(). Check the various ld security implications.
Now you got a probing tool with executive side effects, before it was pure read-only. Not everyone will be happy with this idea to execute first, look later. dynamic libs execute init functions at dlopen(). Check the various ld security implications.
Edit: Since this comment is currently at the top, here's a better link to the source: https://developer.apple.com/documentation/macos-release-note... (using scroll-to-text fragment[1] since there's unfortunately no usable anchor to the specific entry; not supported in non-Chromium browsers at the moment, in that case search for "built-in dynamic linker cache").
[1] https://caniuse.com/#feat=url-scroll-to-text-fragment