A simple makefile is a unicorn(nibblestew.blogspot.com)
nibblestew.blogspot.com
A simple makefile is a unicorn
http://nibblestew.blogspot.com/2017/12/a-simple-makefile-is-unicorn.html
134 comments
Make sense. Then, it is because of the complexity we want to use the makefile. The include files is a mess for c/c++ because it poses no restrictions and people do abuse it. In my practice, I never put header files in the makefile. They are either stable or I need deliberately ponder on its design. When necessary, I simply touch the c source to trigger rebuild. So my makefile remain simple and I always need know roughly how my header files are used, which is a necessary responsibility.
Indeed, not all uses of make are as a "build tool" for "projects".
* https://news.ycombinator.com/item?id=11942498
* https://news.ycombinator.com/item?id=11942498
If the author of the article believes that the example Makefile isn’t simple, he needs to learn Make, for that example is TRIVIAL.
Learn UNIX tools people, they’re excellent tools developed by masters with the amount of knowledge, experience and insight very few of you will ever amass (don’t think “ah! But I’m better!” for that way lie pain, misery and a build process of nightmares), and they work the way they do for multiple reasons. When it comes to build engines, Make is the real deal: sufficiently simple to get started right away, infinitely flexible to do whatever one needs when it comes to automated / intelligent building, with lots of useful built-in rules “out of the box”. In this case the problem isn’t in the tool, but in one’s own capabilities as a programmer.
Learn UNIX tools people, they’re excellent tools developed by masters with the amount of knowledge, experience and insight very few of you will ever amass (don’t think “ah! But I’m better!” for that way lie pain, misery and a build process of nightmares), and they work the way they do for multiple reasons. When it comes to build engines, Make is the real deal: sufficiently simple to get started right away, infinitely flexible to do whatever one needs when it comes to automated / intelligent building, with lots of useful built-in rules “out of the box”. In this case the problem isn’t in the tool, but in one’s own capabilities as a programmer.
Here's my "simple" makefile for C. It's for building a simple executable and a static library. I don't use Make for more complex projects.
It has a bunch of extra features, like coverage and profiling and tags files. They are pretty easy to remove if you don't want them.
If you want to put object files to another directory, it's much better to run make in the output directory than to hack your Makefile to output objects to another directory. Make can look for source files elsewhere but not search for object files outside of $PWD.
https://github.com/rikusalminen/makefile-for-c
It has a bunch of extra features, like coverage and profiling and tags files. They are pretty easy to remove if you don't want them.
If you want to put object files to another directory, it's much better to run make in the output directory than to hack your Makefile to output objects to another directory. Make can look for source files elsewhere but not search for object files outside of $PWD.
https://github.com/rikusalminen/makefile-for-c
Gosh, programming is hard! Complaining about the syntax of make, or that it’s possible to write bad makefiles (and using bad examples to boot) is not, in and of itself, convincing. Almost the same text could be used to complain about C++, Java, HTML, Chinese, or any other subject.
And doesn’t work with visual studio??? Whose problem is that (hint: not make’s).
And doesn’t work with visual studio??? Whose problem is that (hint: not make’s).
In the C++ space modules would have been an option to clean this up and enforce more of an project structure, which in turn would allow smart tools to rely on that. Unfortunately the C++ committee can't agree to anything outside a compilation unit to exist and the abstract machine it's built upon to be quite limited ...
Just use cmake or scons. In 2018, you should not hand write a single makefile !
A simple cmake/sconcs build file will be as simple as the one shown (or even simpler !) and it will definitively will be easier to integrate external dependencies if you have to.
A simple cmake/sconcs build file will be as simple as the one shown (or even simpler !) and it will definitively will be easier to integrate external dependencies if you have to.
CMake has to be one of the worst documented projects I've come across.
I can't stand when people use a makefile for JavaScript projects. I mean, here we have this beautiful cross-platform engine and then you go and ruin it with your shitty non-cross-platform makefile. Yuck!
Lol, I just made the argument that ‘a simple makefile can do most of what people use grunt for’ on Twitter like 6 days ago. Worth nothing however everything in our makefile is a PHONY which makes things a ton simpler, though arguably we’re not using make to it’s full potential.
You may want to consider using 'stamp files'. These are just empty files created via 'touch' to serve as a timestamp that Make can use to determine if a target needs to be rebuilt if the target doesn't correspond to an actual file in the build directory. This is useful for build targets such as running docker-compose build.
See https://www.technovelty.org/tips/the-stamp-idiom-with-make.h...
See https://www.technovelty.org/tips/the-stamp-idiom-with-make.h...
I have to disagree with this, I consider stamp files a Makefile anti-pattern.
Stamp files are an often an indication that someone is trying to turn a Makefile into a shell script.
Rules should be generated that generate a target from its dependencies. The intermediate steps should be encapsulated as the rule commands.
Stamp files are an often an indication that someone is trying to turn a Makefile into a shell script.
Rules should be generated that generate a target from its dependencies. The intermediate steps should be encapsulated as the rule commands.
Sometimes it is necessary though. I have had to use this pattern in 2 scenarios:
Building a static website where the targets are wildly different from the source files. I could probably make it explicit but it will be overly verbose and brittle.
Check for and remove trailing whitespace. Because the dependency is actually the target, a stamp file is necessary.
Building a static website where the targets are wildly different from the source files. I could probably make it explicit but it will be overly verbose and brittle.
Check for and remove trailing whitespace. Because the dependency is actually the target, a stamp file is necessary.
Specifically in the second case: How do you get the timestamp on these stamp files to update, or do you just do it manually? And if you are doing it manually, why not just run a shell script instead?
The timestamp on the source files is updated by sed, when it strips the whitespace. The stamp file is then updated with touch. The advantage of using Make is that only modified files will be fed to sed and, consequently, only they will be caught by the rsync command, that I use to push the code out to the server(s).
Yeah, I think this is what people mean when they talk about simple Makefiles: running scripts + topological ordering. Building C is just hard.
I'd be inclined to agree if the author offered some alternative. Grunt is mentioned in the comments of course, but I have no experience with that.
In a simple C project packaged for Linux, FreeBSD anc Mac OS I seriously think a simple Makefile is possible.
And will continue to think that until I see an alternative that is easy to get going with on those systems.
In a simple C project packaged for Linux, FreeBSD anc Mac OS I seriously think a simple Makefile is possible.
And will continue to think that until I see an alternative that is easy to get going with on those systems.
Possible? It's everywhere!
My Makefiles usually have one line per directive, can I still call my Makefiles simple?
I mainly use them to replace scripts to run tests/linters/formatters/containers/etc
I mainly use them to replace scripts to run tests/linters/formatters/containers/etc
I think the simplest makefile is not to create one.
echo 'int main(){}' > a.c && make a
[deleted]
When seeing "simple" as an absolute measure, the author surely has a point. As for a relative measure - I think the argument is unjust, as you can easily come across much more complex makefiles. So my take on this is this: explain the width of the scale you're using when you measure in terms relative to it.
The Makefile shown in the article is simultaneously not complicated and also over-complicated. I feel like most Make users don't understand Make very well (especially the ones that complain really loudly about it).
Here's a couple examples:
1. The "%.o: %.c" line is a huge code smell for Makefiles. You shouldn't ever write that line—it's built in!
2. Mentioning .h files in the Makefile. Don't. Go look up the `-MMD` flag in your C compiler. Clang, gcc and many other C compilers support it. Add that and a "-include *.d" line in the Makefile and you'll have great dependency support.
Everything else in that makefile is just setting variables. It's made overly complicated by the directory junk, but I'm failing to see the problem.
Here's a couple examples:
1. The "%.o: %.c" line is a huge code smell for Makefiles. You shouldn't ever write that line—it's built in!
2. Mentioning .h files in the Makefile. Don't. Go look up the `-MMD` flag in your C compiler. Clang, gcc and many other C compilers support it. Add that and a "-include *.d" line in the Makefile and you'll have great dependency support.
Everything else in that makefile is just setting variables. It's made overly complicated by the directory junk, but I'm failing to see the problem.
> I feel like most Make users don't understand Make very well (especially the ones that complain really loudly about it).
This is the very same narrative that vim users proclaim: "It's totally not complex if you took some years to learn your tools" — while simply not realizing that the whole Make syntax is a pile of clunks and hacks and stopgaps and bandaids that has a super steep learning curve and looks like incomprehensive gibberish to the uninitated eye.
And most of that needlessly so IMHO. It shouldn't be hard to find a better, well designed syntax with sane defaults, stricter rules with typing and linting, less surprises (tabs, I'm looking at you) that's overall much less scary for newcomers — as well as less taxing for "expert casual" users like me. (I still go for my coworkers and Stack Overflow regularly if I have to fix one and I'm a fucking CTO with 10 years of dev experience)
Make's only reason for survival is its ubiquity. No matter which terminal you boot up, it's just there, that's why we use it too, and I hate it every day.
I think none of the alternatives (cmake, Jam, Gradle, Maven, grunt etc. etc.) has really succeeded as a cross-platform, cross-ecosystem solution that's both an approachable, simple and usable version of a DAG execution engine that's usable for lots of situation.
So the situation will probably stay like with vim and the QWERTY keyboard, the status quo pretty bad, but everyone's too used to it to go for something better.
This is the very same narrative that vim users proclaim: "It's totally not complex if you took some years to learn your tools" — while simply not realizing that the whole Make syntax is a pile of clunks and hacks and stopgaps and bandaids that has a super steep learning curve and looks like incomprehensive gibberish to the uninitated eye.
And most of that needlessly so IMHO. It shouldn't be hard to find a better, well designed syntax with sane defaults, stricter rules with typing and linting, less surprises (tabs, I'm looking at you) that's overall much less scary for newcomers — as well as less taxing for "expert casual" users like me. (I still go for my coworkers and Stack Overflow regularly if I have to fix one and I'm a fucking CTO with 10 years of dev experience)
Make's only reason for survival is its ubiquity. No matter which terminal you boot up, it's just there, that's why we use it too, and I hate it every day.
I think none of the alternatives (cmake, Jam, Gradle, Maven, grunt etc. etc.) has really succeeded as a cross-platform, cross-ecosystem solution that's both an approachable, simple and usable version of a DAG execution engine that's usable for lots of situation.
So the situation will probably stay like with vim and the QWERTY keyboard, the status quo pretty bad, but everyone's too used to it to go for something better.
> This is the very same narrative that vim users proclaim: "It's totally not complex if you took some years to learn your tools"
Something interesting I see more often than not when someone I know decides to try out vim:
They know I'm a vim user so they ask me for good tutorials and I tell them there are lots out there but they should start by running vimtutor (included with vim). Takes around 15-20 minutes to work through the interactive lessons and gives you all the basics of movement and editing.
A day or so later I ask how it's going and the answer is usually that they're struggling with vim, so I ask if they ran vimtutor. Usually nope, or they started it but quit after a minute or two.
Why would one switch to a tool with a different paradigm, ask for and get a tutorial recommendation, and then ignore the advice of spending less than 1/2 hour on a tutorial that ships with the tool? sigh
Something interesting I see more often than not when someone I know decides to try out vim:
They know I'm a vim user so they ask me for good tutorials and I tell them there are lots out there but they should start by running vimtutor (included with vim). Takes around 15-20 minutes to work through the interactive lessons and gives you all the basics of movement and editing.
A day or so later I ask how it's going and the answer is usually that they're struggling with vim, so I ask if they ran vimtutor. Usually nope, or they started it but quit after a minute or two.
Why would one switch to a tool with a different paradigm, ask for and get a tutorial recommendation, and then ignore the advice of spending less than 1/2 hour on a tutorial that ships with the tool? sigh
> They know I'm a vim user so they ask me for good tutorials and I tell them there are lots out there but they should start by running vimtutor (included with vim). Takes around 15-20 minutes to work through the interactive lessons and gives you all the basics of movement and editing.
I think that I am rather good at memorizing lots of obscure raw data - but in my experience it takes a lot longer to remember all the "obscure" commands of vim. In this sense I would not say it takes 15-20 minutes, but at least a day if at the end you really want to have the commands in your brain.
In this sense the mistake in your argumentation is the timescale that you are talking about.
I think that I am rather good at memorizing lots of obscure raw data - but in my experience it takes a lot longer to remember all the "obscure" commands of vim. In this sense I would not say it takes 15-20 minutes, but at least a day if at the end you really want to have the commands in your brain.
In this sense the mistake in your argumentation is the timescale that you are talking about.
I never meant to imply that 15-20 minutes in vimtutor would make you a vim wizard, or that you'd remember everything that vimtutor went through. There's no quick and easy answer to switching to something that does things in a fundamentally different way.
Also, I hold some fairly controversial opinions: you don't need dozens of vim plugins; you don't need to compete with vim golf champions; you don't need an extensive configuration.
What you do need is to fully grok the idea of normal mode as an editing language with verbs, objects, and modifiers. Once you "get it" then you have a working knowledge of how to construct "sentences", and learning a new text object allows you to apply pretty much all verbs to it. Same for learning a new verb; you can apply it to lots of things.
Over the years I've learned a lot of interesting vim things and I've forgotten most of them. Because I don't need them every day or even every week. That's ok. The things I remember are things I use often, because they're useful in everyday editing.
Also, I hold some fairly controversial opinions: you don't need dozens of vim plugins; you don't need to compete with vim golf champions; you don't need an extensive configuration.
What you do need is to fully grok the idea of normal mode as an editing language with verbs, objects, and modifiers. Once you "get it" then you have a working knowledge of how to construct "sentences", and learning a new text object allows you to apply pretty much all verbs to it. Same for learning a new verb; you can apply it to lots of things.
Over the years I've learned a lot of interesting vim things and I've forgotten most of them. Because I don't need them every day or even every week. That's ok. The things I remember are things I use often, because they're useful in everyday editing.
Because in the windows world- every ui-interface holds its contract, everything explains itself and does what it supossed to do.
There is no disk-icon that doesent save.
There is no set of shortkeys ignoring controll s.
Design is meeting the users expectations without additional instructions with functionality. VIM is not designed very well.
Just because Vim was not designed for the mouse clicking UI environment does not mean that it's not designed well. If it weren't designed well, Vim would not be in widespread use 40 years after its inception.
Sure, but if you're trying to carry over Windows expectations over to Linux you're sure to be disappointed.
In Vim there are modes that describe sets of keybindings and there are commands that are bound to them. The commands themselves are are straightforward for anyone who's fluent in Linux vocabulary.
'Saving' is a Windows word. We :write to files or :w for short.
In Vim there are modes that describe sets of keybindings and there are commands that are bound to them. The commands themselves are are straightforward for anyone who's fluent in Linux vocabulary.
'Saving' is a Windows word. We :write to files or :w for short.
Maybe you should point to a better editor(or design one if you can) in vim case than simply claiming everyone is using it because they are used to it.
>This is the very same narrative that vim users proclaim
Sounds like a solid reason why makefiles are great to me then.
Vim is so good it finally is letting emacs-OS have a real editor.
Sounds like a solid reason why makefiles are great to me then.
Vim is so good it finally is letting emacs-OS have a real editor.
> I feel like most Make users don't understand Make very well (especially the ones that complain really loudly about it).
But the question is whether this is Make's fault for being incomprehensible, or the User's fault for being lazy.
The thing is virtually any system or tool can be understood with enough practice, but just because there is someone out there who can use it well does not mean it's not poorly designed.
Both C and Make are decades-old technologies, but most people can read a C file without much experience and intuitively understand what is going on. The same is not true for Makefiles.
But the question is whether this is Make's fault for being incomprehensible, or the User's fault for being lazy.
The thing is virtually any system or tool can be understood with enough practice, but just because there is someone out there who can use it well does not mean it's not poorly designed.
Both C and Make are decades-old technologies, but most people can read a C file without much experience and intuitively understand what is going on. The same is not true for Makefiles.
> Both C and Make are decades-old technologies, but most people can read a C file without much experience and intuitively understand what is going on.
While they might think they know what's going on, I'm not convinced they actually do. C can be an incredibly subtle language to get right...
http://blog.llvm.org/2011/05/what-every-c-programmer-should-...
Regarding make, the thing to realize there is that it's just a DSL for specifying a dependency graph (and the way for satisfying dependencies). It's not a hard concept, but unless you come to the field with a bit of computer science background, it's not likely something you've been directly exposed to at the level required to understand make. But I don't know that this means make's choice of abstraction is necessarily a bad thing for its intended use cases. I also think this tends to put the lie to the notion that the difficulty is mainly around tabs... tabs may obscure the details, but switching to a clearer syntax won't solve the more fundamental gaps in understanding the mechanics of make.
I'll also argue that this is a potential problem with DSL's in general... because DSL's often don't use traditional imperative language semantics, there's always a bit of a learning curve that needs to be ascended. However, for make at least, there are a useful debugging tools to help understand what it's actually doing.
For me, at least, 'make -n' is great in that it instructs make to resolve the dependency graph based on the current state, and then just print the commands it'll run without executing them. If you know what the commands do, and know what needs to run, this can be a huge help understanding what's going on. (And there are other more powerful debugging capabilities as well for dumping variables, etc.)
While they might think they know what's going on, I'm not convinced they actually do. C can be an incredibly subtle language to get right...
http://blog.llvm.org/2011/05/what-every-c-programmer-should-...
Regarding make, the thing to realize there is that it's just a DSL for specifying a dependency graph (and the way for satisfying dependencies). It's not a hard concept, but unless you come to the field with a bit of computer science background, it's not likely something you've been directly exposed to at the level required to understand make. But I don't know that this means make's choice of abstraction is necessarily a bad thing for its intended use cases. I also think this tends to put the lie to the notion that the difficulty is mainly around tabs... tabs may obscure the details, but switching to a clearer syntax won't solve the more fundamental gaps in understanding the mechanics of make.
I'll also argue that this is a potential problem with DSL's in general... because DSL's often don't use traditional imperative language semantics, there's always a bit of a learning curve that needs to be ascended. However, for make at least, there are a useful debugging tools to help understand what it's actually doing.
For me, at least, 'make -n' is great in that it instructs make to resolve the dependency graph based on the current state, and then just print the commands it'll run without executing them. If you know what the commands do, and know what needs to run, this can be a huge help understanding what's going on. (And there are other more powerful debugging capabilities as well for dumping variables, etc.)
DSLs are most suited to constrained execution spaces - where you know that there are relatively hard limits on what it is that the resultant program will have to do. The problem is that build systems are not really a constrained execution space - there's too often a lot more required than just specifying a dependency tree and just saying 'make it happen'.
As with many 'simplified' DSLs that are often unsuited to the task at hand (e.g. C++ templating), make (or versions of it, anyway), evolved to accommodate these use cases and then morphed into something that accidentally became turing complete: https://stackoverflow.com/questions/3480950/are-makefiles-tu...
As with many 'simplified' DSLs that are often unsuited to the task at hand (e.g. C++ templating), make (or versions of it, anyway), evolved to accommodate these use cases and then morphed into something that accidentally became turing complete: https://stackoverflow.com/questions/3480950/are-makefiles-tu...
> As with many 'simplified' DSLs that are often unsuited to the task at hand (e.g. C++ templating)
Isn't a C++ template the exact opposite of 'Domain Specific'?
Isn't a C++ template the exact opposite of 'Domain Specific'?
...Nor does it mean that just because it has a steep learning curve it’s badly designed.
Make is not a needlessly complex tool (unlike most garbage tools these days); it’s complex because it needs to be flexible enough to build any programmer’s software. Since ways to program and build software are infinite, the tool itself cannot be simple, otherwise trade-offs have to be made.
Make is not a needlessly complex tool (unlike most garbage tools these days); it’s complex because it needs to be flexible enough to build any programmer’s software. Since ways to program and build software are infinite, the tool itself cannot be simple, otherwise trade-offs have to be made.
You can also create a point and click gui mechanism and brag about how wonderful it is, too, but it wouldn't be 'make' or be as capable, well-rounded, flexible, encompassing, etc.
'make' exists for what it's capable of doing and it must be so as it's used in every piece of software and operating sytem everywhere.
'make' exists for what it's capable of doing and it must be so as it's used in every piece of software and operating sytem everywhere.
>'make' exists for what it's capable of doing and it must be so as it's used in every piece of software and operating sytem everywhere.
So it's extremely popular/widespread, therefore it must be good, and therefore you should use it (thereby making it even more popular/widespread)?
I really hope that you poorly worded that/I'm grossly misunderstanding you because otherwise, your logic is batshit insane.
So it's extremely popular/widespread, therefore it must be good, and therefore you should use it (thereby making it even more popular/widespread)?
I really hope that you poorly worded that/I'm grossly misunderstanding you because otherwise, your logic is batshit insane.
> So it's extremely popular/widespread, therefore it must be good, and therefore you should use it (thereby making it even more popular/widespread)?
My experience has been that that people that truly understand and appreciate technology for its own sake are also likely to underestimate the value of a tool being popular and widespread. What 'popular' really means is that the tool is more likely to be where you need it to be and the people you need to use it will already know how. Those things outweigh a 'perfectly' designed tool that you don't have or know how to use.
My experience has been that that people that truly understand and appreciate technology for its own sake are also likely to underestimate the value of a tool being popular and widespread. What 'popular' really means is that the tool is more likely to be where you need it to be and the people you need to use it will already know how. Those things outweigh a 'perfectly' designed tool that you don't have or know how to use.
> What 'popular' really means is that the tool is more likely to be where you need it to be and the people you need to use it will already know how.
There's a balance point, for sure. For instance, the pace of "innovation" in front-end javascript libraries throws out a lot of babies with the bathwater, and we might be well-served by accepting a paradigm for 10 minutes and building on it rather than always chasing after the next hot thing.
But at the other extreme, there was a time when PHP was ubiquitous for back-end development, and it stuck around largely because of inertia: the obvious flaws were accepted just because it was so accepted as the tool for that job. But I don't think many people are lamenting the fact that we've moved on to better tools for back-end development. In that case inertia was a bad thing because it delayed innovation.
For me an example of a "good enough" tool is GIT. It's not perfect, it's got a non-trivial learning curve, and surely some innovations could be made to improve version control. Some would already argue that Mercurial is the Betamax to GIT's VHS - but it does the job well enough, and everybody uses it, so it's good enough.
I'd argue Make is more like PHP than it is GIT.
There's a balance point, for sure. For instance, the pace of "innovation" in front-end javascript libraries throws out a lot of babies with the bathwater, and we might be well-served by accepting a paradigm for 10 minutes and building on it rather than always chasing after the next hot thing.
But at the other extreme, there was a time when PHP was ubiquitous for back-end development, and it stuck around largely because of inertia: the obvious flaws were accepted just because it was so accepted as the tool for that job. But I don't think many people are lamenting the fact that we've moved on to better tools for back-end development. In that case inertia was a bad thing because it delayed innovation.
For me an example of a "good enough" tool is GIT. It's not perfect, it's got a non-trivial learning curve, and surely some innovations could be made to improve version control. Some would already argue that Mercurial is the Betamax to GIT's VHS - but it does the job well enough, and everybody uses it, so it's good enough.
I'd argue Make is more like PHP than it is GIT.
> we might be well-served by accepting a paradigm for 10 minutes and building on it rather than always chasing after the next hot thing.
Exactly this.
> I'd argue Make is more like PHP than it is GIT.
What would you want a make replacement to look like?
Make is pretty 'roll your own' in places, but I've found it to have a nice balance between expressiveness and terseness. For simple projects, there isn't much to do in make, and for the more complex projects I've been involved in (embedded work, a self-hosting compiler, etc.) make can still generally express the build process within the confines of Makefile syntax. (In contrast to more modern tools that might require plugins in a different language, etc.)
For the compiler project I mention, the build process includes building several executables in C, running them against a pre-built VM image to compile another VM image, translating that image into C code that is then compiled and linked into the VM to produce another executable, after which there's a unit test suite. Root rebuild command is 'make clean && make tested', and there are only a couple hundred lines of Makefile code to get there. (There's also cross-compile support, etc.)
I'll admit that there was a certain amount of effort involved in writing those Makefiles, but I've also implemented the same build process using a modern GUI tool and found it to be much more difficult.
Exactly this.
> I'd argue Make is more like PHP than it is GIT.
What would you want a make replacement to look like?
Make is pretty 'roll your own' in places, but I've found it to have a nice balance between expressiveness and terseness. For simple projects, there isn't much to do in make, and for the more complex projects I've been involved in (embedded work, a self-hosting compiler, etc.) make can still generally express the build process within the confines of Makefile syntax. (In contrast to more modern tools that might require plugins in a different language, etc.)
For the compiler project I mention, the build process includes building several executables in C, running them against a pre-built VM image to compile another VM image, translating that image into C code that is then compiled and linked into the VM to produce another executable, after which there's a unit test suite. Root rebuild command is 'make clean && make tested', and there are only a couple hundred lines of Makefile code to get there. (There's also cross-compile support, etc.)
I'll admit that there was a certain amount of effort involved in writing those Makefiles, but I've also implemented the same build process using a modern GUI tool and found it to be much more difficult.
> What would you want a make replacement to look like?
Admittedly I haven't given a ton of thought to this specific problem, but here goes:
I actually like Make from a conceptual standpoint. I think it's a good idea to codify build orchestration in code rather than non-human-readable output from a GUI application. My main gripe is that it seems to be very much a 1.0 tool: it was build to solve a specific problem, and then features were added as needed. I would love to have an iteration on that concept which benefits from what we've learned from Make while rounding off the rough edges of what feels like the result of a largely ad-hoc design process.
As far as what that would look like: I would like to see a concise, low-level API for doing the kinds of things Makefiles do (i.e. defining a dependency graph, setting up build parameters etc), and better tools for composition.
Also I would sacrifice a bit of terseness for the benefit of readability. For instance an arbitrary keyword like `Phony` could be replaced by a more extensible name-spacing syntax which lets you know exactly what's going on.
Admittedly I haven't given a ton of thought to this specific problem, but here goes:
I actually like Make from a conceptual standpoint. I think it's a good idea to codify build orchestration in code rather than non-human-readable output from a GUI application. My main gripe is that it seems to be very much a 1.0 tool: it was build to solve a specific problem, and then features were added as needed. I would love to have an iteration on that concept which benefits from what we've learned from Make while rounding off the rough edges of what feels like the result of a largely ad-hoc design process.
As far as what that would look like: I would like to see a concise, low-level API for doing the kinds of things Makefiles do (i.e. defining a dependency graph, setting up build parameters etc), and better tools for composition.
Also I would sacrifice a bit of terseness for the benefit of readability. For instance an arbitrary keyword like `Phony` could be replaced by a more extensible name-spacing syntax which lets you know exactly what's going on.
You are then suggesting that make is not popular and widespread because it's "good". I am saying that, when given the choice, most operating systems and software use 'make' as evidence.
Lets blame the users, it lost us the desktop wars- what more damage can this paradigm do?
Sure but if all things were kept intuitive, well-specified and simple, no one would have paid us six-figs to maintain it. It's part of the guild protection.
The Make problems I know [0] are:
* Recursive Make Considered Harmful
* Changes to the Makefile do not implicitly trigger rebuilds
* Dependencies on directories is not really possible
* Lots of output by default
* Rules with multiple output files are not really possible
* can't handle spaces in pathnames
[0] https://github.com/qznc/annoying-build-systems#make-alone
* Recursive Make Considered Harmful
* Changes to the Makefile do not implicitly trigger rebuilds
* Dependencies on directories is not really possible
* Lots of output by default
* Rules with multiple output files are not really possible
* can't handle spaces in pathnames
[0] https://github.com/qznc/annoying-build-systems#make-alone
Some more:
* Change tracking entirely depends on modified timestamps instead of something more robust like checksums.
* Makefiles are always platform specific. Make executes commands based on a dependency graph, but it has no commands of its own. You either write multiple makefiles or you build everything in a bash environment.
* Change tracking entirely depends on modified timestamps instead of something more robust like checksums.
* Makefiles are always platform specific. Make executes commands based on a dependency graph, but it has no commands of its own. You either write multiple makefiles or you build everything in a bash environment.
Do you have a good link/example, why timestamps are bad?
I think in the context of Make it is ok. If you rely on checksums, you must implicitly assume that all commands you execute are idempotent. That assumption easily breaks.
I think in the context of Make it is ok. If you rely on checksums, you must implicitly assume that all commands you execute are idempotent. That assumption easily breaks.
When I worked with Clearcase (which is terrible in many ways, this is just one), one of its nice features was that it changed file timestamp to the timestamp of commit, not the timestamp it was changed on developer's machine. The commit timestamp may be after previous commit but also before the compiled object file. Because of this, the only way to do a reliable build was to start from scratch. Which took 45 minutes.
Ceterum censeo, Clearcase must be destroyed.
Ceterum censeo, Clearcase must be destroyed.
I don't have a link, but one reason to prefer checksums is if you have intermediary steps that produce artifacts, possibly binary input to another step, that may or may not need to actually be built.
Very nice. It appears to be high time to create my own simple make.
"The "%.o: %.c" line is a huge code smell for Makefiles. You shouldn't ever write that line—it's built in!"
Does it hurt to write it though? It makes the makefile more readable for people who don't know it's built-in. Besides, it's only built-in for ".c". I prefer to keep it in order to have a generic Makefile I can adapt to other programming languages.
Does it hurt to write it though? It makes the makefile more readable for people who don't know it's built-in. Besides, it's only built-in for ".c". I prefer to keep it in order to have a generic Makefile I can adapt to other programming languages.
Either you should not write it, or disable all built-in rules of Make with a flag. Doing both is asking for trouble.
Make has lots of built-in rules for lots of languages, C, C++, Fortran, Lex, Yacc, etc.
Make has lots of built-in rules for lots of languages, C, C++, Fortran, Lex, Yacc, etc.
Could you provide an equivalent Makefile to the one written in the article to demonstrate best practices?
I am by no means a make expert but this is how I would write it:
The first target, hellomake, is our executable. This is the default target, so it's the one that'll be built if we run make with no arguments. It runs the C compiler with the flags we specified earlier. Its dependencies are the object files, which means they'll be built automatically and rebuilt when their dependencies change. As noted, there is a built-in rule to build the object files from the C source using our CC and CFLAGS. We don't need to tell make how to do that.
The next target, clean, is a "phony" target—it doesn't refer to a file. Normally a make target tells make how to build a file. A phony target, in contrast, is really just a shell script that doesn't produce any file output. This one deletes all the build products.
The remaining lines set up the dependency tracking for headers. This makes it so that a change in a header file will trigger a rebuild of any source that includes it. We begin by making a list of files by copying the list of object files and swapping in ".d" for the ".o" file extension. Next is a rule that tells how to build a .d file from a .c file. This rule runs the C compiler and tells it to output a dependency tracking file. The last line says to include each of the .d files in the Makefile. (The .d files contain Makefile syntax to specify the header dependencies.)
I omitted the IDIR and ODIR and such because in a project of this size I wouldn't bother putting includes and object files in separate directories from the source.
I hope that helps. I know make can seem outdated and weird but I think it really can be simple once you learn a few things about how it works. Hopefully this will get you started in that direction!
CC = clang
CFLAGS += -Wall -Wextra
libs = -lm
objs = hellomake.o
hellomake: $(objs)
$(CC) $(CFLAGS) -o $@ $(objs) $(libs)
.PHONY: clean
clean:
rm -f hellomake *.o *.d
deps := $(objs:.o=.d)
%.d: %.c
$(CC) $(CFLAGS) -MM -MF $@ $<
-include $(deps)
The first target, hellomake, is our executable. This is the default target, so it's the one that'll be built if we run make with no arguments. It runs the C compiler with the flags we specified earlier. Its dependencies are the object files, which means they'll be built automatically and rebuilt when their dependencies change. As noted, there is a built-in rule to build the object files from the C source using our CC and CFLAGS. We don't need to tell make how to do that.
The next target, clean, is a "phony" target—it doesn't refer to a file. Normally a make target tells make how to build a file. A phony target, in contrast, is really just a shell script that doesn't produce any file output. This one deletes all the build products.
The remaining lines set up the dependency tracking for headers. This makes it so that a change in a header file will trigger a rebuild of any source that includes it. We begin by making a list of files by copying the list of object files and swapping in ".d" for the ".o" file extension. Next is a rule that tells how to build a .d file from a .c file. This rule runs the C compiler and tells it to output a dependency tracking file. The last line says to include each of the .d files in the Makefile. (The .d files contain Makefile syntax to specify the header dependencies.)
I omitted the IDIR and ODIR and such because in a project of this size I wouldn't bother putting includes and object files in separate directories from the source.
I hope that helps. I know make can seem outdated and weird but I think it really can be simple once you learn a few things about how it works. Hopefully this will get you started in that direction!
Put -MMD to CFLAGS and you can drop the rules to build dependency files. The compiler will then generate the deps files when compiling.
Don't use wildcards in clean target. Use $(RM) $(OBJS) $(DEPS) instead.
Don't use wildcards in clean target. Use $(RM) $(OBJS) $(DEPS) instead.
> $(CC) $(CFLAGS) -o $@ $(objs) $(libs)
Please honour LDFLAGS and LDLIBS here.
> $(CC) $(CFLAGS) -MM -MF $@ $<
Please honour CPPFLAGS here.
Please honour LDFLAGS and LDLIBS here.
> $(CC) $(CFLAGS) -MM -MF $@ $<
Please honour CPPFLAGS here.
> Clang, gcc and many other C compilers support it. Add that and a "-include *.d" line in the Makefile and you'll have great dependency support.
Doesn't that sacrifice portability, though? Is this method portable between GNU make and FreeBSD make and OpenBSD make?
Doesn't that sacrifice portability, though? Is this method portable between GNU make and FreeBSD make and OpenBSD make?
Why wouldn't it? the *.d files are file snippets in regular make syntax.
The special handling is in the compilers: they emit make file parts!
The special handling is in the compilers: they emit make file parts!
Because "-include" is a non-standard[1,2] extension. "include" is standard POSIX behavior[1] but (1) does not seem to support wildcards and (2) errors if the file does not exist, which it never will on initial build. From a glance at the man pages, OpenBSD make does support -include[2], but FreeBSD make does not[3].
[1] cf. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ma...
[2] https://man.openbsd.org/make
[3] https://www.freebsd.org/cgi/man.cgi?make
[1] cf. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ma...
[2] https://man.openbsd.org/make
[3] https://www.freebsd.org/cgi/man.cgi?make
Frankly at this point in time, when people say "Make" they mean GNU Make. Unix distributions that wish to remain stuck in the 1980s with their original BSD Make are welcome to do so, but no sane developer is going to use BSD Make for a build system written any time in the last 25 years unless they have to.
Re -MMD: without automated dependency discovery, make was half a tool: if a dependency was omitted, it might do a less-than minimal job of rebuilding, introducing regressions into the build output that did not exist in the source. For this reason, make was quite often used in an unconditional manner, and if you are doing that, you can get the same result more simply and clearly (and without surprises) with a procedural shell script. It is surprising how long it took for that gap in capabilities to be closed.
It is not completely closed.
* https://news.ycombinator.com/item?id=15060146
* https://news.ycombinator.com/item?id=15060146
> I feel like most Make users don't understand Make very well
This indicates that make is hard to understand. I've been using it for a while and I still struggle to grok concepts like order-only prerequisites.
This indicates that make is hard to understand. I've been using it for a while and I still struggle to grok concepts like order-only prerequisites.
The more build scripts I write the more convinced I become that an ideal build system needs to be:
* Using declarative code as much as possible but still written in a good general purpose turing complete language (maybe python - not make).
* Built upon a foundation of well designed libraries, installable from a package manager that abstract common build workflows.
Make meets neither of these standards.
* Using declarative code as much as possible but still written in a good general purpose turing complete language (maybe python - not make).
* Built upon a foundation of well designed libraries, installable from a package manager that abstract common build workflows.
Make meets neither of these standards.
I agree, and recommend Biomake:
https://github.com/evoldoers/biomake
It uses the declarative programming language Prolog to write flexible rules.
A logic programming language like Prolog is very well suited for expressing rule-based dependencies.
https://github.com/evoldoers/biomake
It uses the declarative programming language Prolog to write flexible rules.
A logic programming language like Prolog is very well suited for expressing rule-based dependencies.
I've come to the same conclusion after many years of writing and modifying build scripts for Java, .NET, Ruby, JavaScript and C.
Of course, if you only care about POSIX, a lot of the make issues go away. Not all, but many.
Of course, if you only care about POSIX, a lot of the make issues go away. Not all, but many.
Gradle is pretty much like this I think, although it's still a bit too tied to the Java build flow to be truly general purpose.
Sounds like you may want something like Scons with better support for common build patterns.
Something more along those lines, yes, although I'm really not that convinced about some of their design decisions and though the documentation is at least very detailed, it's rather obtuse.
I think it's possible to do better.
I think it's possible to do better.
There are any number of technologies that I've seen described as "simple" that I'm inexperienced in...and they look overly complex to me. When I learn to use them, they don't.
I've used make for a long time. I've never considered it difficult to learn or use. It has a limited number of things that it does for you, and leaves the rest to you. I like that paradigm a lot more than systems that seem to operate magically.
I've used make for a long time. I've never considered it difficult to learn or use. It has a limited number of things that it does for you, and leaves the rest to you. I like that paradigm a lot more than systems that seem to operate magically.
the only bad sin I've seen with makefiles is when organizations distribute complex ones that every project must use.
make should be simple, custom, and predictable for each piece of a project. I sometimes have a file for each dir. having a central invisible one is awful.
make should be simple, custom, and predictable for each piece of a project. I sometimes have a file for each dir. having a central invisible one is awful.
Compiling projects is inherently complicated, so the solution must also be complicated. You can solve the problem on a spectrum of easy but lack of configurability, or complex/verbose but very flexible. Make is the latter.
The author has shown that he does not understand the Make language very well since half of his final bullet points are wrong.
The author has shown that he does not understand the Make language very well since half of his final bullet points are wrong.
This is simple vs magic.
Makefiles are simple in that you can see every directive and very little is hidden from you. There is no magic - just dependency ordering using mtime as a very simple heuristic.
Makefiles are simple in that you can see every directive and very little is hidden from you. There is no magic - just dependency ordering using mtime as a very simple heuristic.
Except that there is a lot of magic [1] and it does cause bugs. For example the implicit `%: %.o` rule has to be cancelled if you have a rule like `$(SPECIFIC_OBJ_FILES): %.c.o: %.c`
[1]: https://www.gnu.org/software/make/manual/html_node/Catalogue...
[1]: https://www.gnu.org/software/make/manual/html_node/Catalogue...
I like to put `MAKEFLAGS += --no-builtin-rules` near the top of my makefiles. This way, all predefined rules are cancelled and I won't be caught off guard by a rule that I didn't specify explicitly being a match.
I also usually put `.SUFFIXES:`, but I guess that's redundant: "The `-r` or `--no-builtin-rules` flag causes the default list of suffixes to be empty." [1]
[1]: https://www.gnu.org/software/make/manual/make.html#Suffix-Ru...
I also usually put `.SUFFIXES:`, but I guess that's redundant: "The `-r` or `--no-builtin-rules` flag causes the default list of suffixes to be empty." [1]
[1]: https://www.gnu.org/software/make/manual/make.html#Suffix-Ru...
I use makefile to build my javascript and css. It works brilliantly. I initially used Gulp, but started running into problems when there were build errors that would crash Gulp so I had to use a plugin called Gulp-plumber just to keep the watcher going. Then when I started having a somewhat more complex chain of dependencies I didn't enjoy having to manually specify the build order. The final nail in the coffin for Gulp is when I wanted to dynamically alter flags based on certain conditions, whether it was a debug or prod build for example, it just started getting messier and messier.
I am completely satisfied with Make as a replacement. Took me a couple days of hacking to figure out how to set it up and it's been problem-free ever since.
By far, my favourite aspect is that I no longer need to depend on flimsy gulp-specific wrappers. I just call my build tools natively.
I am completely satisfied with Make as a replacement. Took me a couple days of hacking to figure out how to set it up and it's been problem-free ever since.
By far, my favourite aspect is that I no longer need to depend on flimsy gulp-specific wrappers. I just call my build tools natively.
I also use Makefile for all my node/docker/frontend needs.
It is an EXCELLENT entry point for all of the project scripts, needs etc.
Check it out: https://gist.github.com/molszanski/56de03b54f66885ef415af84e...
npm scrips are also awesome. But editing and maintaining them inside a package.json is not cool
Check it out: https://gist.github.com/molszanski/56de03b54f66885ef415af84e...
npm scrips are also awesome. But editing and maintaining them inside a package.json is not cool
I used Make for my javascript for a while, but having to update the makefile everytime dependencies changed got annoying. Now I just `webpack --watch`; the startup time isn't great, but it's a one-time cost and the recompile happens faster than I can switch to the browser.
I write my makefile once at the beginning of a project and never touch it again. If you need to update yours with every new dependency, you might have your makefile set up sub-optimally. I make sure to have something like the following to include all my js/css dependencies.
JS = $(shell find . -name "*.js" -o -name "*.json")
CSS = $(shell find . -name "*.scss" -o -name "*.css")
I'm writing for the browser so imports have to be bundled into the final product; I'm not seeing how that will rebuild a.js (depends on b.js) when b.js changes.
You could do something as follows assuming all source files are in src/, your entry point is src/a.js and your output is build/scripts.js. I use browserify for my compilation but this would work similarly with webpack.
JS_FILES = $(shell find ./src -name "*.js")
JS_ENTRY = ./src/a.js
JS_OUT = ./build/scripts.js
$(JS_OUT): $(JS_FILES)
browserify $(JS_ENTRY) > $(JS_OUT)
Meh. The author uses a silly example of Makefiles. People get the wrong idea about make. They think it's a program for compiling things; it's not, it's a very early, slim, clean version of chef or ansible. Personally I only got to know how cool make really is when I started to use it for stuff other than compilation, like file transfers or service run control (at one point my toy distro's rc system was "/bin/make -f /etc/rc.mk", where /bin/make was a static version.)
Even for the example he uses, since he seems to be on GNU Make, it's a pretty perfect case to use Guile integration to handle that metaprogramming he's trying to do by hand.
Even for the example he uses, since he seems to be on GNU Make, it's a pretty perfect case to use Guile integration to handle that metaprogramming he's trying to do by hand.
Early yes, slim no, clean debatable.
I certainly think whatever incarnation of make that you would summon to do some sort of multi-machine configuration management (per your chef/ansible comparison) is something I would do everything in my power to avoid.
I certainly think whatever incarnation of make that you would summon to do some sort of multi-machine configuration management (per your chef/ansible comparison) is something I would do everything in my power to avoid.
Love me some make -- here is a session:
: fred@dejah ~ $; cat >hello.c
#include <stdio.h>
int main() {printf("hello\n"); return 0;}
^D
: fred@dejah ~ $; make hello
gcc hello.c -o hello
: fred@dejah ~ $;
I've programmed in the early heyday when Make was my ONLY option and I'm primarily a C++ developer. These days? No thanks. I don't want to return to that hell. Any technology is "slim" if you cherry-pick just one feature. By any reasonable definition, make is filled with warts and all sorts of tricks that are REQUIRED for any non-trivial project that supports multiple platforms to work. It is old crufty technology and I'll take my chances with other options thanks.
You're free to love make all you want, but I've been there, was effectively held hostage by it for lack of better options, and hate it. Automake and asociated tools was the first attempt at getting us out of that hell, and it didn't succeed but cmake/premake/bazel are doing a whole lot better.
You're free to love make all you want, but I've been there, was effectively held hostage by it for lack of better options, and hate it. Automake and asociated tools was the first attempt at getting us out of that hell, and it didn't succeed but cmake/premake/bazel are doing a whole lot better.
> slim no, clean debatable
Many a time I see unslim and unclean Makefiles where the problem is mostly that someone started to use make then tried hard to use make as a single unifying tool for too many things, some clearly not belonging to make.
There is nothing wrong with writing wrapper and helper scripts or programs alongside a Makefile, so that make is only bothered with its dependency graph building.
Many a time I see unslim and unclean Makefiles where the problem is mostly that someone started to use make then tried hard to use make as a single unifying tool for too many things, some clearly not belonging to make.
There is nothing wrong with writing wrapper and helper scripts or programs alongside a Makefile, so that make is only bothered with its dependency graph building.
Yes but personally, I blame the tool for being neither opinionated enough, or enforcing more "structure" so that those weaknesses don't come to bear.
History has shown that having sensible defaults for common use cases and workflows goes a long way to reducing complexity and ammunition for self-afflicted foot shooting should be isolated and explicit.
History has shown that having sensible defaults for common use cases and workflows goes a long way to reducing complexity and ammunition for self-afflicted foot shooting should be isolated and explicit.
You cannot even properly escape a left parenthesis, for crying out loud!
So if you invoke a tool that expects some arguments wrapped in parentheses, you need to do stuff like
So if you invoke a tool that expects some arguments wrapped in parentheses, you need to do stuff like
LEFTPAREN:=(
RIGHTPAREN:=)
$(addprefix $(LEFTPAREN), $(addsuffix $(RIGHTPAREN), $(PARAMS)))
> So if you invoke a tool that expects some arguments wrapped in parentheses, you need to do stuff like
...like $(patsubst)?
...like $(patsubst)?
$(patsubst %,(%),$(PARAMS))
If you're interested in build systems, you may find djb's redo concept interesting, as implemented in https://github.com/apenwarr/redo/ for example.
Can't upvote this enough; I to create a Makefile for my app (https://zenaud.io) and came to the exact same conclusion. Once you're using the -MD clang option to generate makefile directives for include dependencies, your Makefile is no longer simple -- in fact it starts looking horrible. It's best to avoid IMHO.
I think the author would get a heart attack immediately if he heard about https://github.com/ninenines/erlang.mk ... :)
I for one find erlang.mk abhorrent as well. I usually write my own makefiles (~ less than 20 lines) to utilize eunit, edoc and rebar3.
My own makefiles are less than 20 lines, too, but they use erlang.mk instead of rebar3.
You don't get to claim simplicity if you use rebar.
You don't get to claim simplicity if you use rebar.
>> You don't get to claim simplicity if you use rebar.
Yea, thats true. But good to see that I am not the only one who creates simplified makefiles to utilize other build tools.
My critique on erlang.mk is, that its far simpler to remember the 10-20ish commands of the rebar toolchain than thousands (or hundreds, but that it what it feels like) of erlang.mk options for every erlang tool that has ever been invented.
Yea, thats true. But good to see that I am not the only one who creates simplified makefiles to utilize other build tools.
My critique on erlang.mk is, that its far simpler to remember the 10-20ish commands of the rebar toolchain than thousands (or hundreds, but that it what it feels like) of erlang.mk options for every erlang tool that has ever been invented.
The reverse is also true: it's far simpler to remember the 10-20ish options of
the erlang.mk plugins you use than hundreds options of various rebar plugins.
There is no significant difference between the two.
There is no significant difference between the two.
Is this not a simple Makefile?
https://github.com/apex/apex/blob/master/Makefile
https://github.com/apex/apex/blob/master/Makefile
I wouldn't expect `make clean` to delete stuff it didn't create itself. This does though as it just executes `git clean -f`!
Arguably the simplest Makefile is not having any Makefile and just doing
make <executable name>
i find it always starts simple, and usually one maintainer; after a year, it’s a nightmare of hack on hack on hack with three+ contributors each knowing a slice. makefiles on projects that are active for two+ years, a year+ later might as well be chinese. (if it was chinese i could at least ask someone fluent to translate.)
I agree with the author - for C/C++ projects. The module system in C and C++ is pushed out of the compiler and into the build tool. This means make (which is simple) is insufficient to easily correctly implement the logic to build C/C++ projects of more than a few files.
However, a lot of other commenters in this thread have mentioned using make to manage their JS builds. JS build tools tend to work on whole directories of files together, and as such its fairly easy to compose these tools without a lot of logic in the build tool, and here make excells because its ubiquitous and simple. We use a makefile for building crystal and its simple and works great.
Make isnt useless, its just useless in the world it was created in.
However, a lot of other commenters in this thread have mentioned using make to manage their JS builds. JS build tools tend to work on whole directories of files together, and as such its fairly easy to compose these tools without a lot of logic in the build tool, and here make excells because its ubiquitous and simple. We use a makefile for building crystal and its simple and works great.
Make isnt useless, its just useless in the world it was created in.
Then why does your Linux/BSD/Unix operating system use it, including every piece of software installed on it?
Not every piece of software installed on my BSD/Linux operating systems uses make. (-:
> Then why does your Linux/BSD/Unix operating system use it
Because many old programmers don't want to change their habit.
> including every piece of software installed on it?
Look up ninja.
Because many old programmers don't want to change their habit.
> including every piece of software installed on it?
Look up ninja.
That’s right, I don’t want to change something powerful which works really well because someone finds it hard to understand (lacking the prerequisite knowledge) or even worse, wants instant gratification and isn’t willing to master it. That’s like wanting to design a CPU accelerator without wanting to learn how to design electrical circuits. Of course I’m going to say no!, I don’t want to change the tool! If learning this stuff is too hard or not interesting to one, then one ought to consider doing something else for a living.
We have tools like Maven and Ant and Scons and CMake because someone didn’t get Make and shell but they thought they did. Now I’m stuck with toy build tools which use unnecessary dependencies (like Java and .NET and Python) and only do as much as their programmer could imagine, limited by that programmer’s imagination (and experience). You bet your patootie I’m going to say NO! I don’t want to be stuck with a toy, someone’s misunderstanding of Make.
We have tools like Maven and Ant and Scons and CMake because someone didn’t get Make and shell but they thought they did. Now I’m stuck with toy build tools which use unnecessary dependencies (like Java and .NET and Python) and only do as much as their programmer could imagine, limited by that programmer’s imagination (and experience). You bet your patootie I’m going to say NO! I don’t want to be stuck with a toy, someone’s misunderstanding of Make.
Yeah, let's better create another abstraction layer and release several frameworks each year to make things simpler. /s
Make has its use. It's a great tool for building small/medium projects that don't have many dependencies. You can setup a new project in minutes with nothing more than a text editor and compiler. If you know *sh scripting, it's not really complicated. For bigger projects I prefer Bitbake to avoid "dependency hell".
Make has its use. It's a great tool for building small/medium projects that don't have many dependencies. You can setup a new project in minutes with nothing more than a text editor and compiler. If you know *sh scripting, it's not really complicated. For bigger projects I prefer Bitbake to avoid "dependency hell".
Isn't the main problem with build tools that no one spends much time learning them? You set them up, they work, then forget about them. Skip forward six months and waste hours on SO figuring out how your build works.
I think that means, the cognitive complexity is not in scale with the business requirements. If I had a service I needed to set up once and maybe update a couple of times a year, but took multiple days to sucessfully update, I'd spend some effort on making iteasier to make changes (either by better understanding, simplifying the system, or rewriting it to remove overhead).
This is one of the problems with text based tools, the interface isn't very discoverable.
People in this thread might be able to offer me advice on a project I have open.
I know C but am not experienced working on large C codebases. Below I will describe its rough build structure. I hope for strong tool or re-framing advice.
I have an existing python codebase. I want to make it easy to create native-code plugins. The method I have worked out: I compile shared-libraries. Python then calls into these via python-ctypes.
Each plugin is a directory project/ext/plugin,
* Python code: __init__.py and ext.py
* C code, api.c
* A rough build script, build.py. This accesses global config (e.g. path information) held outside this directory. It compiles api.c into a .so or .dll. It does this by generating a shell script or batch file, and then shelling out to it.
A python script in the root directory walks through this structure and forks a process for each plugin build.py. It is multiplatform (currently linux gcc and windows mingw32). So I have if statements saying "if windows, generate this big string, otherwise generate that big string."
The shared-object is compiled to a separate directory, res/ext/plugin. I did this to create separation between my source and build artifacts separate. (Is this sensible?)
This naive build structure was easy to create, and allowed me to spend my focus elsewhere while I was bootstrapping. Now that is done, I am interested in how I could improve the build so that it could scale to a large number of plugins.
Would I be well-served to use Make or Cmake or something else? Can they access configuration from outside of their own directory? Maybe I should communicate configuration via environment variables?
I know C but am not experienced working on large C codebases. Below I will describe its rough build structure. I hope for strong tool or re-framing advice.
I have an existing python codebase. I want to make it easy to create native-code plugins. The method I have worked out: I compile shared-libraries. Python then calls into these via python-ctypes.
Each plugin is a directory project/ext/plugin,
* Python code: __init__.py and ext.py
* C code, api.c
* A rough build script, build.py. This accesses global config (e.g. path information) held outside this directory. It compiles api.c into a .so or .dll. It does this by generating a shell script or batch file, and then shelling out to it.
A python script in the root directory walks through this structure and forks a process for each plugin build.py. It is multiplatform (currently linux gcc and windows mingw32). So I have if statements saying "if windows, generate this big string, otherwise generate that big string."
The shared-object is compiled to a separate directory, res/ext/plugin. I did this to create separation between my source and build artifacts separate. (Is this sensible?)
This naive build structure was easy to create, and allowed me to spend my focus elsewhere while I was bootstrapping. Now that is done, I am interested in how I could improve the build so that it could scale to a large number of plugins.
Would I be well-served to use Make or Cmake or something else? Can they access configuration from outside of their own directory? Maybe I should communicate configuration via environment variables?
It sounds like your build process is not well designed. Why would a build script need to generate a shell script to call? And why would a python script fork a process for each plugin?
I think waf, which I've plugged before, would be a good choice for your project. Without knowing the details, your build script would look something like:
I think waf, which I've plugged before, would be a good choice for your project. Without knowing the details, your build script would look something like:
def configure(ctx):
# Load C compiler support
ctx.load('compiler_c compiler_cxx')
if ctx.env.DEST_OS == 'win32':
# Special thing for Windows
def build(ctx):
for plugin in ['a', 'b', ...]:
ctx.shlib(source=ctx.path.ant_glob('ext/'+plugin+'/*.c'), target=plugin)
> how I could improve the build so that it could scale to a large number of plugins
Is your existing parallel builder too slow? What's the problem with it? Make can do parallel builds as well, but it won't walk a directory so you'd have to write a million targets.
I think you should post this on the SO site for programming design/advice though.
Is your existing parallel builder too slow? What's the problem with it? Make can do parallel builds as well, but it won't walk a directory so you'd have to write a million targets.
I think you should post this on the SO site for programming design/advice though.
Based on the headline, I expected this to be a rant about unicorn valuations. Was very pleasantly surprised!
The example Makefile in this article is contrived, here's an example of a simple Makefile: https://git.suckless.org/dmenu/tree/Makefile. I can find plenty of other such examples.
Make is good at traversing and resolving a dependency graph; the problem is that we are forced to write said dependency graph. It's no hardship for small projects, but as projects grow and the dependency graph becomes annoying and/or difficult to manage by hand, we turn to automatic solutions -- and we should. Except that pattern substitution doesn't cut it, we really need a way to programmatically generate dependencies given a source file.
Fortunately, compiler writers have noticed this problem, and C/C++ compilers will spit out makefile snippets which define the dependencies for a given file. See the sample makefile above by danaliv.
They are, however, insufficient for the whole task.
* https://news.ycombinator.com/item?id=15060146
* https://news.ycombinator.com/item?id=15060146
Yeah, the comments by '__david__ and 'danaliv would be welcome news if I ever wrote C/C++ again.
For programming languages, it should be builtin in some form for you(build systems) to extract the dependency relations. Unfortunately C/C++ are not designed for this. But some programming languages do make it possible to be automated.
For other use cases, there is no generic way to automate this process.
For other use cases, there is no generic way to automate this process.
Well, while I agree that it is nearly impossible to write a useful Makefile for a more than minimal C project, I found that if you are willing to structure your project according to the best practices of CMake you can write fairly simple and useful CMake file.
But be aware, that as soon as you want to structure your project in any other way than it is the CMake default you are starting to write your own Makefile again ;-)
But be aware, that as soon as you want to structure your project in any other way than it is the CMake default you are starting to write your own Makefile again ;-)
gnumake is a scripting language based on rules. It is not a simple language (bash is not simple either), but it allows to write simple scripts that work fine to perform simple tasks. It works reasonably well for project compilation which is not a so simple task. It becomes a excessively complex when it tries to become overly generic. On one of my projects, gnumake handles double compilation of sources for shared and static libraries, compilation of test programs written in C++/java/Ada often using corba. It handles dependencies in the whole source tree and works perfectly fine. The single limitation I have encountered is the handling of spaces in file or directory names. It is very difficult in bash scripts. It is almost impossible in complex makefiles.
Make shares a fundamental property with C and C++: they are not meant to be simple; they are meant to be powerful!
In the purest Unix tradition, Make was designed to be tool-agnostic: you can choose which editor to code in, which (if any) preprocessor to use, which compiler and which linker to invoke and with which arguments, and what else to do aside.
And, all of this is controlled by a bunch of text files, which means grep and sed are your friends when you want to change a linker option for several projects.
Make comes from a philosophy which is the opposite of the IDE's ground. Not that the IDE concept is wrong per se, but if you are only used to have the same monolithic behemoth from code typing to binary output, your critics of make sound the same as an industrial fast food cook complaining that artisan patisserie is too complex.
In the purest Unix tradition, Make was designed to be tool-agnostic: you can choose which editor to code in, which (if any) preprocessor to use, which compiler and which linker to invoke and with which arguments, and what else to do aside.
And, all of this is controlled by a bunch of text files, which means grep and sed are your friends when you want to change a linker option for several projects.
Make comes from a philosophy which is the opposite of the IDE's ground. Not that the IDE concept is wrong per se, but if you are only used to have the same monolithic behemoth from code typing to binary output, your critics of make sound the same as an industrial fast food cook complaining that artisan patisserie is too complex.
I think the problem here is the complexity of C, C++ and the building of such, not the makefile itself. Make can make simple things simple, but it can't take away genuine, real complexity as if it never existed.
I have lots of makefiles I use and am super-happy with. But none of them are for C-based projects. When you take away the complexity that C/C++ brings, you no longer need to bring that along to your makefile.
And you know what you have then? Apparently what the author refers to as a "unicorn": A nice and super-simple makefile. The unicorn is suddenly real.