Show HN: macOS-cross-compiler – Compile binaries for macOS on Linux(github.com)
github.com
Show HN: macOS-cross-compiler – Compile binaries for macOS on Linux
https://github.com/shepherdjerred/macos-cross-compiler
75 comments
I feel like a reference to this should be somewhere in the macos-cross-compiler repo.
The documentation for this is unbelievably good. Thank you for sharing!
Speaking of cross compilers - I was able to use the zig toolchain in order to cross compile for macos apple silicon, arm Linux, windows x64, windows arm and x86_64 Linux musl (statically linked). It's super handy and works as a drop in.
I'm commenting to add a critical mass around Zig. If you are prisoner of CMake, autoconf, templates, makefiles, etc.. please try the Zig toolchain for your c/c++ projects. It is worth every second of investment
Zig build is somewhat poorly documented currently, which can lead to some annoyance (also some other annoyances with dependency hashes), but in my experimentation:
Dependencies in build.zig.zon are downloaded in to ~/.cache/zig/p/<hash> (incidentally, this means you need to mangle the hash if you are copying and pasting the hash for a dependency, at least currently. Dependency hashes are a sore spot tbh, and needs to be better)
Then when you are using said dependency in your build.zig, the function provided will refer to that source artifact in .cache.
At least this seems to be the case. I write a decent amount of zig, but haven’t dove too much in to the build system till recently, when I tried (and failed due to translate-c bugs), to get some C libraries added to zig using only zig build.
Dependencies in build.zig.zon are downloaded in to ~/.cache/zig/p/<hash> (incidentally, this means you need to mangle the hash if you are copying and pasting the hash for a dependency, at least currently. Dependency hashes are a sore spot tbh, and needs to be better)
Then when you are using said dependency in your build.zig, the function provided will refer to that source artifact in .cache.
At least this seems to be the case. I write a decent amount of zig, but haven’t dove too much in to the build system till recently, when I tried (and failed due to translate-c bugs), to get some C libraries added to zig using only zig build.
zig fetch fills the hash for you, or simply omitting .hash in zon and the compiler tells it for you. I'm writing backend in zig and I have zstd directly from zon as a dependency.
.dependencies = .{
.zstd = .{
.url = "git+https://github.com/facebook/zstd.git#v1.5.5",
.hash = "1220185ad79a437fd9f148d1422ff756287534c79a0712105039b4034031480e41a9",
},
},
I’m intrigued. Do you have a starting point? And how does it handle if I need to integrate with another language as well, like Swift?
I was researching this the other day, here is a non trivial example:
https://github.com/floooh/sokol-tools/blob/master/build.zig
Here is a tutorial: https://www.zvm.app/tutorials/zig-build-cpp.html
https://github.com/floooh/sokol-tools/blob/master/build.zig
Here is a tutorial: https://www.zvm.app/tutorials/zig-build-cpp.html
Interestingly you can use zig within your Rust build to more easily cross-compile:
https://github.com/rust-cross/cargo-zigbuild
I’m curious what the blockers are for rustc to cross-compile like zig does natively.
I’m curious what the blockers are for rustc to cross-compile like zig does natively.
AFAIK, there are no blockers really, it's just that Rust does not have its own linker, it delegates linkage to the system linker depending on the target. <https://rust-lang.github.io/rustup/cross-compilation.html?hi...>
You can specify your own linker if you want, mold is a very popular one, and cargo-zigbuild does the same behind the scenes with zig cc as the linker.
I did something similar a couple of months ago (or a year ago? I don't remember exactly). I managed to cross-compile to windows-msvc on Linux using Wine, there's a project that provides the scripts to make this easier, including the linker wrapper: <https://github.com/est31/msvc-wine-rust>. It was just for fun because Rust can already target windows-gnu and it'll use mingw64 linker.
Rust's approach to things is normally to provide the basic foundation and let the community build on top of it. I personally like this approach, but it also has this downside of people not knowing they may need an external/community built tool to accomplish what they want.
You can specify your own linker if you want, mold is a very popular one, and cargo-zigbuild does the same behind the scenes with zig cc as the linker.
I did something similar a couple of months ago (or a year ago? I don't remember exactly). I managed to cross-compile to windows-msvc on Linux using Wine, there's a project that provides the scripts to make this easier, including the linker wrapper: <https://github.com/est31/msvc-wine-rust>. It was just for fun because Rust can already target windows-gnu and it'll use mingw64 linker.
Rust's approach to things is normally to provide the basic foundation and let the community build on top of it. I personally like this approach, but it also has this downside of people not knowing they may need an external/community built tool to accomplish what they want.
> I’m curious what the blockers are for rustc to cross-compile like zig does natively.
Zig ships all the bits you need for a C/C++ toolchain including things like the C library in some cases (IIUC).
Rust could use libclang to make a "rustc cc" just like zig's. But I get the sense that it is probably not a goal of the project to have this functionality.
Zig ships all the bits you need for a C/C++ toolchain including things like the C library in some cases (IIUC).
Rust could use libclang to make a "rustc cc" just like zig's. But I get the sense that it is probably not a goal of the project to have this functionality.
Rust proc macros and the fact you can't really static link if something uses them made rust unsuitable for me.
There is cargo-zigbuild, which uses zig as a linker, but I have yet to try it out.
That's actually exactly how this cross-compiler works for Rust!
Just add another arch with rustup. Am I missing something?
That would solve the compilation problem, but there is more to create a working binary. For MacOS the official (and I believe only legal) way is to use a Mac with XCode installed for linking.
Now, you can download all the necessary files from a Mac and build a cross-compilation toolchain on your Linux system. I believe you could not legally distribute a project doing this and that is why these projects don't exist or are usually short-lived (with the notable exception of zig).
We will see how that goes for OP.
So, solutions like osxcross resort to shipping scripts that help you to acquire the necessary files and make that process easier.
The OP builds on osxcross, but is "batteries included".
zig has an even more difficult problem to solve because it tries to compile and link for many platforms. Shipping all the different requirements in their original form would make a zig distribution huge. So it does some clever magic I do not fully understand to create the necessary files from just the necessary subset of data. This reduces the size because there is overlap between platforms. It also means that they are not shipping the Mac files in their complete and original form and have gotten away with this legally so far.
At least that is what I believe is happening. I hope someone with more knowledge about zig could explain it better.
Now, you can download all the necessary files from a Mac and build a cross-compilation toolchain on your Linux system. I believe you could not legally distribute a project doing this and that is why these projects don't exist or are usually short-lived (with the notable exception of zig).
We will see how that goes for OP.
So, solutions like osxcross resort to shipping scripts that help you to acquire the necessary files and make that process easier.
The OP builds on osxcross, but is "batteries included".
zig has an even more difficult problem to solve because it tries to compile and link for many platforms. Shipping all the different requirements in their original form would make a zig distribution huge. So it does some clever magic I do not fully understand to create the necessary files from just the necessary subset of data. This reduces the size because there is overlap between platforms. It also means that they are not shipping the Mac files in their complete and original form and have gotten away with this legally so far.
At least that is what I believe is happening. I hope someone with more knowledge about zig could explain it better.
You don’t need the OSX SDK to make runnable Rust binaries from other OS: https://betterprogramming.pub/cross-compiling-rust-from-mac-...
The link is about making Linux binaries on MacOS, which works because almost everything to build for Linux is already on OSX. The other way around it's not true. A Linux system does come neither with the OSX SDK nor the Apple patched llvm linker. I believe at least part of the required files cannot be legally distributed with a Linux system, but I might be wrong on that.
It appears you’re right, thank you! I’m very surprised there isn’t a llvm linker available, minus all the foundation kits that rust doesn’t need.
The llvm linker is a cross-linker in principle. I think it is more about the specialities that Apple needs, like code signing.
Ah. For what it's worth, those have been reverse engineered and copied elsewhere. I maintain a project that does cross-compiled macOS builds and our own signing on Linux.
Rustup arch only adds support for static libraries (unliked objects), and craps out as soon as you try to build a binary.
Rust did not do the legwork that zig did to bundle libc and a working linker, and Cargo is exceptionally naive in its default configuration, so it won't even find a usable cross-linker on the system, nor even try the chronically-unfinished rustc-lld integration there is.
Rust did not do the legwork that zig did to bundle libc and a working linker, and Cargo is exceptionally naive in its default configuration, so it won't even find a usable cross-linker on the system, nor even try the chronically-unfinished rustc-lld integration there is.
I could be mistaken, but I believe that installs a completely parallel toolchain which is relatively large compared with zigs approach of having all the platforms generatable from a single toolchain.
Great to see all of this work brought together like this.
I looked into some of these solutions earlier and the licensing status of doing this is pretty grey and likely violates some of the toolchain licenses.
For personal use, that's not really an issue. It's far easier than trying to get a decently performing OSX runner somewhere, and I don't see Apple caring at all.
Use caution if you are bringing this into a commercial environment.
I looked into some of these solutions earlier and the licensing status of doing this is pretty grey and likely violates some of the toolchain licenses.
For personal use, that's not really an issue. It's far easier than trying to get a decently performing OSX runner somewhere, and I don't see Apple caring at all.
Use caution if you are bringing this into a commercial environment.
The tooling should be mostly fine, it's all the libraries that are the problem.
Theoretically this could be avoided by making a stub library, but the way namespacing works on OS X means that's tricky.
Theoretically this could be avoided by making a stub library, but the way namespacing works on OS X means that's tricky.
> it's all the libraries that are the problem
And also header files which are also part of macOS SDK, as the project claims to support C codes.
And also header files which are also part of macOS SDK, as the project claims to support C codes.
The problem with this is licensing: the macOS SDK license says that it's only to be used on an "Apple-branded computer".
That said I wonder what Apple would do if this is used outside of that case...
That said I wonder what Apple would do if this is used outside of that case...
Apple ships extra stickers with their logo in the box with a lot of their products, maybe slap one on your computer and see how far it gets you in court?
Which is why they changed the wording from "Apple-labeled" to "Apple-branded" around the time of Snow Leopard IIRC.
How much could a branding iron cost?
This place [0] sells custom design branding irons starting at $93.
0: https://brandingirons.com/products/basic-fire-heated-brandin...
0: https://brandingirons.com/products/basic-fire-heated-brandin...
It's cooler to use liquid nitrogen.
https://en.wikipedia.org/wiki/Freeze_brand
https://en.wikipedia.org/wiki/Freeze_brand
I don't think that works as well on computers.
I used a Hackintosh for a number of years and Apple could have easily shut down the entire community with enough cease and desists. In fact there were engineers that helped provide valuable information.
They don't mind what people do for personal use. Just don't try and turn it into a commercial product.
They don't mind what people do for personal use. Just don't try and turn it into a commercial product.
Any source or further information for what makes you say there were engineers that that provided information? I always suspected some of the people in these communities were engineers but never discovered anything for my thoughts to make it past just being suspicions.
They have shut that down by moving into their own CPUs.
Nothing unless they can know for sure your machine is not a mac running Linux.
Sounds like that would allow a Linux VM running on a macbook
I'll plug some work I've been doing to (attempt to) enable cross compilation of Python wheels. I put together a small example [1] that builds the zstandard wheel, and can build macos wheels on linux and linux wheels on macos using zig cc.
macos wheels must still be adhoc signed (codesign) and binary patched (install_name_tool), so I re-implemented those functions in Python [2].
[1] https://github.com/jvolkman/bazel-pycross-zstandard-example
[2] https://github.com/jvolkman/repairwheel/tree/main/src/repair...
macos wheels must still be adhoc signed (codesign) and binary patched (install_name_tool), so I re-implemented those functions in Python [2].
[1] https://github.com/jvolkman/bazel-pycross-zstandard-example
[2] https://github.com/jvolkman/repairwheel/tree/main/src/repair...
I think the project can easily add Go to the list as cross compiling is built-in:
GOOS=darwin GOARCH=arm64 go build -o bin/app-silicon-darwin app.goI've been using goreleaser-cross for cross-compiling Go apps with cgo
https://github.com/goreleaser/goreleaser-cross
Will this allow me to compile, let's say Crystal scripts to macOS from a Linux machine?
Because at the moment, "cross-compilation" with Crystal still requires you to be on the targeted system.
https://crystal-lang.org/reference/1.11/syntax_and_semantics...
Because at the moment, "cross-compilation" with Crystal still requires you to be on the targeted system.
https://crystal-lang.org/reference/1.11/syntax_and_semantics...
Currently I cross-compile the other way since the story is easier. Getting dependencies, etc. especially if you want something libssl is an annoying nightmare if you're trying to cross-compile targeting MacOS on a host on Linux. But if you're going the other way, very easy to use Docker.
The good part of this is that I can use a standard Linux host now to just build everything. Very nice!
The good part of this is that I can use a standard Linux host now to just build everything. Very nice!
Does this not need any xcode/macos license?
Why would it? You can cross compile for macos with the go toolchain on Linux too and that doesn't even require an additional compiler. Just GOOS=darwin GOARCH=arm64 and then the regular command. Apple can't regulate everything
The most obvious answer is you need the system frameworks to link against.
If you need the macOS SDK, then you need to obey the developer licensing rules. Does this need the SDK?
Only if you're using cgo. If you're building native Go code, the Go compiler is fully self-contained and can target any supported platform from anywhere -- macOS from Linux, Linux from Windows, Android from FreeBSD, you name it.
Are they allowed to redistribute the macOS SDK? I assume that Apple’s EULA doesn’t permit that.
> Code signing (but not notarizing) should be possible with this project, but it is untested
Code signing is required for aarch64 macs unless they have SIP disabled, so this is kind of important.
Code signing is required for aarch64 macs unless they have SIP disabled, so this is kind of important.
This vastly depends on how you intend to distribute your binaries.
I get away with distributing internal tools to my team completely ad-hoc signed, as long as I download them with cURL or SFTP. Downloading with a browser taints them, of course, and gatekeeper will prevent you from running them.
I get away with distributing internal tools to my team completely ad-hoc signed, as long as I download them with cURL or SFTP. Downloading with a browser taints them, of course, and gatekeeper will prevent you from running them.
To be clear, it doesn't depend on anything. If SIP is enabled, you cannot run executables on ARM Macs without codesigning (they may be ad hoc signed, sure, but they still need to be signed). The executable will get killed immediately if you don't.
You can avoid gatekeeper problems by clearing the xattrs on the file. If they're internal you can also just distribute over a network share and it's not an issue.
You can avoid gatekeeper problems by clearing the xattrs on the file. If they're internal you can also just distribute over a network share and it's not an issue.
Hm, need to give this* a try with NixOS cross compilation for macos. If I could pre-build nix packages for macOS developers on linux build hosts that'd be awesome.
*the building blocks
*the building blocks
This is really cool! I spent the time a couple years back to build a windows sysroot, but having a macOS one was on my mind for a while. Will definitley use
Great Project. I have 2 questions:
1. Can this be used to run in a github action?
2. Can this be used to compile ios apps?
great to see zig mentioned in the comments
What is happening with that first picture?? Is that an extra penguin arm coming out of the apple?
I actually didn't notice that!
I wanted to add some sort of image to the repository README, and it seemed like the simplest way to do so.
I wanted to add some sort of image to the repository README, and it seemed like the simplest way to do so.
The penguin is holding the apple, unfortunately rather than commissioning an actual artist they've seemingly gone for AI art, so the penguin also has three arms. The left foot with the third arm there is also a bit... weird and deformed.
It’s a project with 20 GitHub stars which wraps up existing tooling to make a fairly niche need easier. The expectation that they’d commission an artist to produce a fun hero image for the readme seems a little odd.
These are volunteers providing hundreds (at a guess) of hours of unpaid labor to a project they're giving away for free. You also expect them to pay an artist out of their own pockets?
Maybe an artist (for example, you) could donate his or her time and provide them with a better logo for free.
Maybe an artist (for example, you) could donate his or her time and provide them with a better logo for free.
Why is this a defense for them using the labor of non-consenting artists to generate a crappy header image?
Surely they could ask an artist to volunteer their time just as they volunteered theirs?
Surely they could ask an artist to volunteer their time just as they volunteered theirs?
Random generated images slapped up in a context where there’s clearly no competing commercial interest just seems like a really silly hill to have this fight on.
Even if you take a hard line on the debate on model training data, this is an utterly harmless case, and certainly not worth the gut punch threads like this cause to someone who is just trying to share their open source project with the community.
Also, you don’t in fact know that the model used to generate this was trained with art used without consent.
Even if you take a hard line on the debate on model training data, this is an utterly harmless case, and certainly not worth the gut punch threads like this cause to someone who is just trying to share their open source project with the community.
Also, you don’t in fact know that the model used to generate this was trained with art used without consent.
This is an internet forum, there's no hill here. It is the lowest possible stakes to have this discussion imaginable.
It is just as easy to grab some CC-licensed images and throw them together in an image editor, like nearly every other FOSS project does at this stage.
It is just as easy to grab some CC-licensed images and throw them together in an image editor, like nearly every other FOSS project does at this stage.
Please feel free to open a PR replacing the image with one from a volunteer.
I'm more than happy to make to make the switch and give credit to a proper artist if there's one willing to work for free.
I'm more than happy to make to make the switch and give credit to a proper artist if there's one willing to work for free.
> Surely they could ask an artist to volunteer their time just as they volunteered theirs?
Unfortunately "I spent a lot of time on this thing you have no interest in or relation to" is not a very convincing argument in the FOSS world.
Unfortunately "I spent a lot of time on this thing you have no interest in or relation to" is not a very convincing argument in the FOSS world.
> not a very convincing argument
Neither is "I'm not going to contribute a damned thing, but nonetheless the team behind this free product needs to spend more time and/or money to make me happy."
If you don't like the logo, well... that sounds a lot like a personal problem to me.
Neither is "I'm not going to contribute a damned thing, but nonetheless the team behind this free product needs to spend more time and/or money to make me happy."
If you don't like the logo, well... that sounds a lot like a personal problem to me.
I wonder if it will compile natively to arm instructions or rely Rosetta on host machine to perform translation
> A cross compiler is a compiler capable of creating executable code for a platform other than the one on which the compiler is running. For example, a compiler that runs on a PC but generates code that runs on an Android smartphone is a cross compiler.
https://en.m.wikipedia.org/wiki/Cross_compilation
So compile on Linux, run on macOS without Rosetta
https://en.m.wikipedia.org/wiki/Cross_compilation
So compile on Linux, run on macOS without Rosetta
No Obj-C????? WHYYYYYY
GCC has objective C/objective C++ enabled, but I have never tested it.
[0]: https://gregoryszorc.com/docs/apple-codesign/stable/