Public domain POSIX make(frippery.org)
frippery.org
Public domain POSIX make
https://frippery.org/make/
40 comments
> Extensions from a future POSIX standard have been added. Note that these are subject to change until the standard is finally published.
The extensions listed cover most of what I'm currently missing from posix make.
The extensions listed cover most of what I'm currently missing from posix make.
Even for those that do not assume GNU make, this may not be enough. GNU Autotools are not limited to running on GNU make, but this make does not implement everything used by Autotools. The first thing I noticed it does not handle is VPATH (https://www.gnu.org/software/make/manual/html_node/General-S...) which is used by Autotools for out-of-tree builds (mkdir build && cd build && ../configure && make as opposed to ./configure && make).
Yeah I'm left scratching my head at the purpose of this project.
I wrote 3 GNU makefiles from scratch (a few hundred lines each) starting in ~2016 for https://www.oilshell.org/, and regret it. I switched to Python + Ninja, and I should have just used that all along.
So I think GNU make is already pretty old and regretted, and POSIX make even more so.
CMake + Ninja seems be pretty common these days, but for my project Python works fine, and is a lot simpler. CMake is also a bad (shell-like) language, but I'm pretty sure it's better and more featureful than GNU make (i.e. you're less likely to need to switch build systems/languages due to a new requirement)
This thread has some interesting experiences ... it does seem like there needs to be a better high level language to generate Ninja
https://lobste.rs/s/7svvkz/using_bsd_make#c_bfwcyc
Also, for portability I just generate a shell script instead of Ninja (even though Ninja is extremely portable and has multiple implementations now). For tarball distributions you probably don't need incremental builds.
All the user should need to compile software is a shell, not ANY make!
I wrote 3 GNU makefiles from scratch (a few hundred lines each) starting in ~2016 for https://www.oilshell.org/, and regret it. I switched to Python + Ninja, and I should have just used that all along.
So I think GNU make is already pretty old and regretted, and POSIX make even more so.
CMake + Ninja seems be pretty common these days, but for my project Python works fine, and is a lot simpler. CMake is also a bad (shell-like) language, but I'm pretty sure it's better and more featureful than GNU make (i.e. you're less likely to need to switch build systems/languages due to a new requirement)
This thread has some interesting experiences ... it does seem like there needs to be a better high level language to generate Ninja
https://lobste.rs/s/7svvkz/using_bsd_make#c_bfwcyc
Also, for portability I just generate a shell script instead of Ninja (even though Ninja is extremely portable and has multiple implementations now). For tarball distributions you probably don't need incremental builds.
All the user should need to compile software is a shell, not ANY make!
My experience aligns well with your. For any moderately complex project makefiles are inadequate and trying to make them work is a frustrating experience even in GNU make is pretty capable.
Ninja is a fine tool but it requires an higher level language to generate it's build files.
Cmake is just horrible, I just cannot stand it. I choose Meson instead and I am quite happy with it. Meson developers focus a lot on doing the right design decisions. They keep the language very simple and offer mostly one logical way to do a thing and Meson ensure it works right in every circumstance. Yet it offers advanced options when they are required but they are always well thought out.
If you compare meson to make at first you may think make is more flexible but ultimately it leaves to you many problems it doesn't solve and it turns out to be inadequate.
With make, declare automatically dependencies from header files? You are alone, that's your problem. There are some tricks using GCC but they looks like hacks. Maintain projects build logic across subfolders? That's your problem. Make only provide a very rough, inadequate mechanism.
Want to do parallel build by respecting dependencies across folder? Sorry, that's your problem but it may work using some tricks.
Want to build out of tree? Hmmm not supported in a standard idiomatic way. You need to use tricks.
How you create a static library in a cross-platform way? That's your problem. The same for shared libraries.
Personally I think that make is an obsolete tool whose design is inadequate very much like CVS that was replaced by git.
To me the only downside of Meson is that it requires python but I can accommodate with that.
Ninja is a fine tool but it requires an higher level language to generate it's build files.
Cmake is just horrible, I just cannot stand it. I choose Meson instead and I am quite happy with it. Meson developers focus a lot on doing the right design decisions. They keep the language very simple and offer mostly one logical way to do a thing and Meson ensure it works right in every circumstance. Yet it offers advanced options when they are required but they are always well thought out.
If you compare meson to make at first you may think make is more flexible but ultimately it leaves to you many problems it doesn't solve and it turns out to be inadequate.
With make, declare automatically dependencies from header files? You are alone, that's your problem. There are some tricks using GCC but they looks like hacks. Maintain projects build logic across subfolders? That's your problem. Make only provide a very rough, inadequate mechanism.
Want to do parallel build by respecting dependencies across folder? Sorry, that's your problem but it may work using some tricks.
Want to build out of tree? Hmmm not supported in a standard idiomatic way. You need to use tricks.
How you create a static library in a cross-platform way? That's your problem. The same for shared libraries.
Personally I think that make is an obsolete tool whose design is inadequate very much like CVS that was replaced by git.
To me the only downside of Meson is that it requires python but I can accommodate with that.
> To me the only downside of Meson is that it requires python but I can accommodate with that.
Check out https://git.sr.ht/~lattis/muon
Check out https://git.sr.ht/~lattis/muon
Thank you, that looks interesting. I will check how it works with my projects.
https://xmake.io/ kinda is Meson but with Lua instead of Python. Lua is embedded so no external dependency.
Thank you, that looks interesting too. In addition I like Lua because is much more light weighted than python and being embedded is nice to reduce dependencies.
The only problem I see is fragmentation. People are already considering cmake as the de facto standard despite all its flaws. If the cmake alternatives are too fragmented it will play in favor of cmake.
On the other side projects like xmake are interesting propositions. I will have a look at it in any case, thank you.
The only problem I see is fragmentation. People are already considering cmake as the de facto standard despite all its flaws. If the cmake alternatives are too fragmented it will play in favor of cmake.
On the other side projects like xmake are interesting propositions. I will have a look at it in any case, thank you.
How do you do parallel builds with a shell script?
Right now I don't, because the project takes ~30 seconds to build serially... There are a few options:
1. Put the Ninja support in the tarball (which is trivial, but I guess I was paranoid about the extra dependency for a low level tool like shell). I think Ninja is supported everywhere now, including on Windows and BSDs. It's easier to build that GNU make itself, etc.
2. The last time I looked, the toybox project had some sort of "parallel job server" type thing in shell, to avoid GNU make.
3. Maybe do something simpler with "xargs -P" (non POSIX)
I'd probably go for the first option just because it's the least custom code to solve this particular problem. If the build was slower, it might be different ... although if speed is really an issue, then Ninja is the best option out of all of these, AND GNU make!
----
edit: I should also note that in practice I think many users and especially distros do NOT pass -j $N to GNU make because it's not reliable. If upstream didn't test with it, there can be bugs that result in incorrect builds. So distros often just use serial builds.
On the other hand, paralllel builds with Ninja are the default, so upstream will have tested in that mode.
1. Put the Ninja support in the tarball (which is trivial, but I guess I was paranoid about the extra dependency for a low level tool like shell). I think Ninja is supported everywhere now, including on Windows and BSDs. It's easier to build that GNU make itself, etc.
2. The last time I looked, the toybox project had some sort of "parallel job server" type thing in shell, to avoid GNU make.
3. Maybe do something simpler with "xargs -P" (non POSIX)
I'd probably go for the first option just because it's the least custom code to solve this particular problem. If the build was slower, it might be different ... although if speed is really an issue, then Ninja is the best option out of all of these, AND GNU make!
----
edit: I should also note that in practice I think many users and especially distros do NOT pass -j $N to GNU make because it's not reliable. If upstream didn't test with it, there can be bugs that result in incorrect builds. So distros often just use serial builds.
On the other hand, paralllel builds with Ninja are the default, so upstream will have tested in that mode.
That's pretty neat! But is there a link to the “future POSIX standard”? A few months ago I tried looking for it on https://pubs.opengroup.org/, but couldn't.
To answer my own question, if this[1] StackExchange to be believed, you need to join the mailing list of something called “The Austin Group”. Although some fairly informative meeting minutes are available publicly here[2].
[1]: https://unix.stackexchange.com/q/607333/105635
[2]: https://www.opengroup.org/austin/docreg.html
[1]: https://unix.stackexchange.com/q/607333/105635
[2]: https://www.opengroup.org/austin/docreg.html
And from the most recent minutes:
> Bug 805: Add Make conditionals Rejected
> https://austingroupbugs.net/view.php?id=805
> As the behavior described in the desired action hasn't actually been implemented, to our knowledge, and there is insufficient consensus amongst the variants that have been implemented to pick one as the de facto standard this request is being rejected. Should the industry come to a consensus a new bug reflecting that should be filed.
TL;DR: no way forward for adding if/else statements to make for now.
> Bug 805: Add Make conditionals Rejected
> https://austingroupbugs.net/view.php?id=805
> As the behavior described in the desired action hasn't actually been implemented, to our knowledge, and there is insufficient consensus amongst the variants that have been implemented to pick one as the de facto standard this request is being rejected. Should the industry come to a consensus a new bug reflecting that should be filed.
TL;DR: no way forward for adding if/else statements to make for now.
To be fair, if you need conditionals in target bodies, you're probably better off putting the body into a separate file as a shell script. And if you need conditionals in macros, there's the old macro-in-macro approach:
DEBUG = 0
DEBUG_FLAG_0 =
DEBUG_FLAG_1 = --debug
# …
main.bin: 1.o 2.o # …
build $(DEBUG_FLAG_$(DEBUG)) -o main.bin # …
Genuine question: what's wrong with GNU make?
Mostly nothing, but writing a makefile and only evaluating against it makes it difficult to guarantee that you will see the same behaviour on a non-GNU system (which has become more relevant these days as even systems based on the Linux kernel are no longer guaranteed to run a GNU userland). What makes matters worse is that even if you grab a BSD make implementation (there are several), there are no guarantees there either, although the number of non-POSIX extensions are far far fewer. Thus you are left to committing things to memory and it is not easy to get it right.
With a strict POSIX implementation, you could develop against it and if it works correctly on your system it should also work correctly with pretty much any other make implementation out there.
Personally, I was so frustrated recently about this that I was looking for exactly what pdpmake offers and could not find anything. Thus I am now very thankful that I do not need to write it myself. But there is still plenty of room for strict POSIX tools like this for: awk, sed, m4, etc.
With a strict POSIX implementation, you could develop against it and if it works correctly on your system it should also work correctly with pretty much any other make implementation out there.
Personally, I was so frustrated recently about this that I was looking for exactly what pdpmake offers and could not find anything. Thus I am now very thankful that I do not need to write it myself. But there is still plenty of room for strict POSIX tools like this for: awk, sed, m4, etc.
> but writing a makefile and only evaluating against it makes it difficult to guarantee that you will see the same behaviour on a non-GNU system
This would only be true if the entirety of thr GNU suite were necessary for GNU make to work. But it's not, so I always find this kind of argument really weird.
I mean it's not necessarily completely irrelevant criticism, but I somehow only see this level of criticism aimed at GNU tools.
I haven't seen many "you know let's make a variant of ninja because the current one is too ideological let's find another one that's still ninja but has slight differences that we 'all' agree are more basic or standardised somehow".
This would only be true if the entirety of thr GNU suite were necessary for GNU make to work. But it's not, so I always find this kind of argument really weird.
I mean it's not necessarily completely irrelevant criticism, but I somehow only see this level of criticism aimed at GNU tools.
I haven't seen many "you know let's make a variant of ninja because the current one is too ideological let's find another one that's still ninja but has slight differences that we 'all' agree are more basic or standardised somehow".
You are being overly defensive here. What I wrote had nothing to do with GNU in particular, as I hope I made clear in the very next sentence after the one you cited by stating that the same is true for BSD make implementations and portability. The parent asked about GNU make, thus I logically kept on talking about GNU make.
You're right, my apologies. :)
I misinterpreted the whole thread as "what's wrong with writing makefiles for GNU make specifically".
Which is subtly a different question (and to which the answer of course is "nothing, as long as you're happy to specifically use GNU make to run them").
I misinterpreted the whole thread as "what's wrong with writing makefiles for GNU make specifically".
Which is subtly a different question (and to which the answer of course is "nothing, as long as you're happy to specifically use GNU make to run them").
awk: nawk aka one true awk
sed: sbase sed
m4: https://github.com/ibara/m4
sed: sbase sed
m4: https://github.com/ibara/m4
Thank you! I was not aware of ibara’s port of OpenBSD’s m4, so now I can drop my own ugly 15-minute hack. Is there a good source for finding strict POSIX tools rather than just go digging?
Sad sidenote on m4, that even with strict POSIX [1], you still have no clue if `define` will turn into “” or “define” due to a split between AT&T on one hand and GNU and BSD (all of them?) on the other that POSIX did not rule one way or the other. It gets even worse when you realise that some m4 implementations have “constants” like `windows` so that “Stained glass windows” can turn into “Stained glass 0”. But I guess this can be expected when you are dealing with a tool that has evolved in multiple directions since 1977?
[1]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/m...
Sad sidenote on m4, that even with strict POSIX [1], you still have no clue if `define` will turn into “” or “define” due to a split between AT&T on one hand and GNU and BSD (all of them?) on the other that POSIX did not rule one way or the other. It gets even worse when you realise that some m4 implementations have “constants” like `windows` so that “Stained glass windows” can turn into “Stained glass 0”. But I guess this can be expected when you are dealing with a tool that has evolved in multiple directions since 1977?
[1]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/m...
Solaris had all the POSIX userland tools separated from the Sun mainline versions in the /usr/xpg4 and xpg6 directories. They can be a source for replacements.
Do those work on anything that isn't Solaris/Illumos?
> Is there a good source for finding strict POSIX tools rather than just go digging?
Some communities(iglunix, kisslinux, carbs linux, wyvertux, oasis linux, ...) are actively researching alternatives to the "standard" coreutils. You could check what they use, perhaps you'll find something interesting and strict. But general source is digging of course :)
> But I guess this can be expected when you are dealing with a tool that has evolved in multiple directions since 1977?
Exactly. 100% conformance isn't always possible, especially when standard is unclear on some things.
Some communities(iglunix, kisslinux, carbs linux, wyvertux, oasis linux, ...) are actively researching alternatives to the "standard" coreutils. You could check what they use, perhaps you'll find something interesting and strict. But general source is digging of course :)
> But I guess this can be expected when you are dealing with a tool that has evolved in multiple directions since 1977?
Exactly. 100% conformance isn't always possible, especially when standard is unclear on some things.
I didn't know about these projects; GNU-less Linux is something I've wanted for a long time. Thanks. Are there any other projects you know of (as indicated by your ellipsis)?
The wrong part in GNU make is that it always permit using non-posix extensions regardless of .POSIX special target, while pdpmake does not. This fact makes pdpmake kinda special since it can be used for posix compliance testing, which is crucial part of developing portable application that uses make to build itself.
That's not wrong. It's something that not everybody will like, but as long as the extensions don't conflict with anything specified by POSIX, that is permitted.
To me, the decision to let POSIX mode turn off extensions that are so widely available in other implementations that they have been approved for the next version of POSIX is an odd one. There is little reason not to already use those features today: using them is not going to seriously limit your portability. If you use them, you cannot also use pdpmake's POSIX mode. The obvious conclusion there, to me, is not to avoid using those features, but to avoid using pdpmake's POSIX mode.
To me, the decision to let POSIX mode turn off extensions that are so widely available in other implementations that they have been approved for the next version of POSIX is an odd one. There is little reason not to already use those features today: using them is not going to seriously limit your portability. If you use them, you cannot also use pdpmake's POSIX mode. The obvious conclusion there, to me, is not to avoid using those features, but to avoid using pdpmake's POSIX mode.
I agree that disabling will-be-posix extensions in POSIX mode sounds like an extreme idea, but seriously, such strictness is the main point of the pdpmake. It ensures that implementation conform to the current standard, not draft which may change quite often.
It turns out it's actually pdpmake-build-time-configurable. If you build with -DENABLE_FEATURE_MAKE_EXTENSIONS -DENABLE_FEATURE_MAKE_POSIX_202X, extensions are enabled by default, and .POSIX only flags non-POSIX-202x extensions. I don't know what, if anything, this says about the main point of pdpmake, but I'm glad with this mode.
>This version is also dedicated to the public domain
IANAL but under US copyright law this is a meaningless statement as it's not possible to disclaim ownership. You can only permissively license - though reading the LICENSE file, this is what they are doing in practice.
IANAL but under US copyright law this is a meaningless statement as it's not possible to disclaim ownership. You can only permissively license - though reading the LICENSE file, this is what they are doing in practice.
I thought that was a problem for other countries? In the US I’m pretty sure dedicating something to the public domain actually places it there.
Nope. I think people are misunderstanding the difference between courts interpreting "putting something in the public domain" as a permissive license and actually severing the copyright from your legal person. There's no current provision for severing those rights under the current law so it may be possible for them (or their heirs) to claw a work back from the public domain using copyright termination/relicensing provisions of the law [1].
The way copyright termination works is a big risk to opensource either way because in theory any open source license could be revoked after enough decades.
[1] https://www.techdirt.com/2015/01/23/why-we-still-cant-really...
The way copyright termination works is a big risk to opensource either way because in theory any open source license could be revoked after enough decades.
[1] https://www.techdirt.com/2015/01/23/why-we-still-cant-really...
I'm not sure I'd like to meet the court that wouldn't see a public domain dedication and round it up to a maximally permissive license if it ever came up anyway.
Afaik it's Germany (and maybe some othes) where that is an issue https://en.m.wikipedia.org/wiki/Public_domain#cite_ref-CC0_5...
Your mileage may vary.