Taskfile: A Modern Alternative to Makefile(cloudnativeengineer.substack.com)
cloudnativeengineer.substack.com
Taskfile: A Modern Alternative to Makefile
https://cloudnativeengineer.substack.com/p/ep-5-taskfile-a-modern-alternative
220 comments
That is not the argument though. The argument is: After being unhappy with Makefile for years now
As to why being unhappy is probably not because it haven't been proof tested. I get your sentiment but feel the remark is out of place. I thought the rationale for picking other tools was straightforward and motivated.
Though the simplicity of make has been lost and I'm not sure the overhead is worth it. I wouldn't want to be the one introducing taskenv, direnv and derenv to a project.
As to why being unhappy is probably not because it haven't been proof tested. I get your sentiment but feel the remark is out of place. I thought the rationale for picking other tools was straightforward and motivated.
Though the simplicity of make has been lost and I'm not sure the overhead is worth it. I wouldn't want to be the one introducing taskenv, direnv and derenv to a project.
> That is not the argument though.
But it was; one of his arguments. Yes, he opens with the argument that he was unhappy with the Make for years, but he later clarifies why:
> I am still stuck with a tool that (according to Wikipedia) has been written 47 years ago.
You probably haven't read his article to the end? Those were his words I copy-pasted from the blog. So one of the reasons he is unhappy is because Make is 47 years old. For some reason, it stopped to be good enough 20 years ago, according to his opinion.
I can give him that we can always have a better tool, but it is a tautology. Unfortunately, his offered alternative is not better IMO.
But it was; one of his arguments. Yes, he opens with the argument that he was unhappy with the Make for years, but he later clarifies why:
> I am still stuck with a tool that (according to Wikipedia) has been written 47 years ago.
You probably haven't read his article to the end? Those were his words I copy-pasted from the blog. So one of the reasons he is unhappy is because Make is 47 years old. For some reason, it stopped to be good enough 20 years ago, according to his opinion.
I can give him that we can always have a better tool, but it is a tautology. Unfortunately, his offered alternative is not better IMO.
Software build has essential complexity. Dealing with peculiarities in config syntax is a mere drop in the vast lake of unhappiness wept from having to deal with build at all. It's the tools that are salient, so folks may perceive that it's the tools at fault and must be reworked, but it isn't really: complexity in build tools is an emergent property of the domain, and any sufficiently developed reinvention will asymptotically correspond to makefiles.
[deleted]
Can you use spaces in filenames with GNU make yet? Seems like a 47yo vestige when spaces in filenames weren’t really a thing…
Does this suffer from all the common problems with yaml (like the "Norway problem", but many others)?
What is the "Norway problem"?
“The Norway Problem” YAML has: when you abbreviate Norway to its ISO 3166-1 ALPHA-2 form NO, YAML will return false when parsing list of countries f.e ‘[GB, IN, NO,…]’
It's a good example of how you can overengineer a relatively simple format.
The reason is that they specify a boolean value as either "true/false" or "yes/no".
The last alternative causes problems not only for the Norwegian country code (no), but also in cases where you need to specify the actual word "yes"
You could of course end up with the same problem if you need to specify the literal "true" and "false", but by overengineering the acceptable boolean values, they increase the potential problem areas.
The reason is that they specify a boolean value as either "true/false" or "yes/no".
The last alternative causes problems not only for the Norwegian country code (no), but also in cases where you need to specify the actual word "yes"
You could of course end up with the same problem if you need to specify the literal "true" and "false", but by overengineering the acceptable boolean values, they increase the potential problem areas.
Claiming more common words as keywords basically. It's just not well considered. true and false are ordinary words, and it would probably be better if sometime way back someone had come up with special keywords that didn't conflict with ordinary words, but for good or bad, at least by now it's practically universal in most languages that true & false are keywords and everyone knows it. Claiming yes and no as keywords also was just thoughtless and inconsiderate (ok that was redundant), doubly so when they don't even do a new job but simply double the consumed namespace to do the same job that was already done by true & false.
Eh, it's fine if they're keywords as long as not everything is a string. If `"NO"` and `NO` are distinct types, then they're not equal and there's no Norway problem. YAML treats strings, numbers, null, and booleans as all the same type (scalars). The problem isn't keywords clashing with common words, it's not having distinct types!
There's even more options apparently :)
A Boolean represents a true/false value. Booleans are formatted as English words (“true”/“false”, “yes”/“no” or “on”/“off”) for readability and may be abbreviated as a single character “y”/“n” or “Y”/“N”.
Regexp: y|Y|yes|Yes|YES|n|N|no|No|NO|true|True|TRUE|false|False|FALSE|on|On|ON|off|Off|OFF
(https://yaml.org/type/bool.html)
A Boolean represents a true/false value. Booleans are formatted as English words (“true”/“false”, “yes”/“no” or “on”/“off”) for readability and may be abbreviated as a single character “y”/“n” or “Y”/“N”.
Regexp: y|Y|yes|Yes|YES|n|N|no|No|NO|true|True|TRUE|false|False|FALSE|on|On|ON|off|Off|OFF
(https://yaml.org/type/bool.html)
To be fair, yaml has a form of inheritance, I wouldn't define it a relatively simple format
To be fair, YAML 1.2 does not suffer from the "Norway Problem", but YAML is still a rather complex format.
I feel like the Makefile syntax has some pesky warts, like not breaking when there are spaces in filenames, requiring tabs[1], the ceremony around PHONY targets... If you take the legacy stuff out of make,
[1] Apparently it can be fixed with a complex .RECIPEPREFIX incantation which behaves differently in different versions of GNU make, and was conveniently added just one minor version after Apple has graciously decided to stop updating all their GNU software versions.
Redo, ninja and tup seems like trying to do that, with their own twists, but none of them really managed to nudge make out;
I feel like the Makefile syntax has some pesky warts, like not breaking when there are spaces in filenames, requiring tabs[1], the ceremony around PHONY targets... If you take the legacy stuff out of make,
[1] Apparently it can be fixed with a complex .RECIPEPREFIX incantation which behaves differently in different versions of GNU make, and was conveniently added just one minor version after Apple has graciously decided to stop updating all their GNU software versions.
Redo, ninja and tup seems like trying to do that, with their own twists, but none of them really managed to nudge make out;
I use make for (among other things) transforming files; batch compression being the simplest example, so all one requires is a rule saying %.log.xz : %.log and all is well.
But if people have colons in filenames, for example for timestamps, all bets are off, and make chokes spectacularly.
But if people have colons in filenames, for example for timestamps, all bets are off, and make chokes spectacularly.
And that really sucks in Windows because every absolute path contains a colon. Not that absolute paths are common in makefiles, but it's not something that should break on a major platform.
In GNU make? Actually, yes:
$ cat Makefile
a\ file: ; echo 123 > '$@'
$ make
echo 123 > 'a file'
$ cat a\ file
123AFAIK this does work but it’s a limited workaround. Make in general just does not support file names with spaces well.
http://savannah.gnu.org/bugs/?712
That being said… In projects I just avoid filenames with spaces in general
http://savannah.gnu.org/bugs/?712
That being said… In projects I just avoid filenames with spaces in general
Or just don't use spaces in filenames. It just causes trouble for no benefit.
Ah yes - the user should serve the computer, not vice versa, clearly.
/s, in case it's necessary.
/s, in case it's necessary.
It's more that doing the 80% solution lets us spend time developing other things. And then we can have a bunch of other really cool stuff as long as the users are willing to work with it.
Which seems like a pretty reasonable trade-off for something that's free.
Which seems like a pretty reasonable trade-off for something that's free.
It's the user serving themselves by naming files in a way that makes them easy to handle everywhere. There is also sorting and completion to consider in commonly-traversed directories.
rather fond of leading hyphens and nonprint ascii myself
[deleted]
Oh yeah, tell that to Dropbox.
Don't use dropbox. That's not even just a thing to say, reclaim your autonomy and stop using a proprietary tool.
No doubt sent from a device with fully open-source hardware stack, right?
Is it conceivable to you that steps are possible, and getting rid of dropbox is a step in a direction ? That perfect is the enemy of good ? That you can't wait to find a perfectly pure plan before starting moving ?
/Τĥιs ñåmè įß ą váĺîδ POSIX /paτĥ
Do we know that taskfile does any better than make on this front? It isn't mentioned in the article. Sounds like taskfile is runs on raw "command line string interpolation" just like good old make.
devenv.sh uses Nix, not Guile.
I thought Nix used as Guix Guile under the hoos, but looked up now, and they seem to use custom language, in which case is it even worse. I should have looked up that before the hand, my bad there, I appologize.
Op complained about that make requires a bunch of extra tools to get running, and suggests another bunch of extra tools, as well as of make DSL which is more or less shell on steroids, but suggests a mix of two different languages instead, and does not even seem to be aware he can use Guile Scheme to extend Make.
Op complained about that make requires a bunch of extra tools to get running, and suggests another bunch of extra tools, as well as of make DSL which is more or less shell on steroids, but suggests a mix of two different languages instead, and does not even seem to be aware he can use Guile Scheme to extend Make.
It’s proof tested yes. But it still falls over if you put in a space versus a tab, and the error message is cryptic and frustrating to beginners. Make is such a huge impediment to C adoption, long time users have no idea.
Even the fact makefiles have no extension is confusing to beginners. Everything about it screams outdated, idiosyncratic computing artifact and students pick up on that.
Even the fact makefiles have no extension is confusing to beginners. Everything about it screams outdated, idiosyncratic computing artifact and students pick up on that.
Having hurdles to jump over is perhaps a good thing? After all Torvalds built git to have a hierarchy of trust in the development of Linux.
If someone can't understand how to use make perhaps they shouldn't be fiddling with C.
If someone can't understand how to use make perhaps they shouldn't be fiddling with C.
Gatekeeping the entry to learn something is never good.
There’s a big difference in developing trust to contribute to a project and developing knowledge to start with a new language.
And I’ve worked with tons and tons of very amazing developers who aren’t comfortable with Make or even git because their careers have been with Visual Studio and other VCS. Therefore I don’t think it’s an adequate judge of skill.
There’s a big difference in developing trust to contribute to a project and developing knowledge to start with a new language.
And I’ve worked with tons and tons of very amazing developers who aren’t comfortable with Make or even git because their careers have been with Visual Studio and other VCS. Therefore I don’t think it’s an adequate judge of skill.
I'm pretty glad they gatekeep pilots and surgeons and all kinds of other occupations.
Reading a manual, or failing to, is not gatekeeping. I learned how to write a Makefile in a few minutes, for free, with no one telling me I wasn't allowed to.
There is no "gatekeeping" argument here, just whingy babies that no one should waste 10 seconds caring about.
Reading a manual, or failing to, is not gatekeeping. I learned how to write a Makefile in a few minutes, for free, with no one telling me I wasn't allowed to.
There is no "gatekeeping" argument here, just whingy babies that no one should waste 10 seconds caring about.
FWIW, both piloting and surgery have easy on ramps in guided settings, often with simulations. So even as a strawman analogies, they’re bad ones.
Nevermind that you ignored the part where I delineated between learning and using it in important projects.
I’m not even going to address the rest of your comment which is frankly a depressing outlook on life.
Nevermind that you ignored the part where I delineated between learning and using it in important projects.
I’m not even going to address the rest of your comment which is frankly a depressing outlook on life.
So, the 'easy on ramp guided path' to being allowed to pilot a plane or practice medecine, is as easy or even in the same universe with writing a makefile? Do you hear yourself? Talk about invalid arguments holy shit!
The easy on ramp was it was totally free and permissionless to learn how to write a Makefile, or read a bunch of existing ones, or countless howtos, and that a basic makefile is basic and you don't need to figure out the arcane bits until you want to, and at no point ever do you need a license no matter how easy or guided the path.
You can be a poor 10 year old in a 3rd world country and write a Makefile. No gates kept.
It's a depressing outlook on life to see something as simple a useful as make as some kind of problem where the supposedly superior answer is something ridiculous like python.
The fact that anyone even for one second thinks these are reasonable thoughts is what is truly depressing.
The easy on ramp was it was totally free and permissionless to learn how to write a Makefile, or read a bunch of existing ones, or countless howtos, and that a basic makefile is basic and you don't need to figure out the arcane bits until you want to, and at no point ever do you need a license no matter how easy or guided the path.
You can be a poor 10 year old in a 3rd world country and write a Makefile. No gates kept.
It's a depressing outlook on life to see something as simple a useful as make as some kind of problem where the supposedly superior answer is something ridiculous like python.
The fact that anyone even for one second thinks these are reasonable thoughts is what is truly depressing.
Is anger and derision your default state of discourse? Both your responses so far have been just completely over the top in both regards.
But putting that aside, you don’t think Make has a ton of footguns for someone starting out?
Spaces vs tabs for different bits because of decades of legacy? Oh but most editors default to tabs as spaces and suddenly no matter what you do, following tutorials will still fail.
Or you try and mess around with an existing open source project to learn from it. You want to list the targets? Oh wait, there’s no standard way to examine the makefile without understanding Make first.
If people consider Make easy to learn, imho they haven’t really reflected on how much more streamlined the rest of the software engineering ecosystem is, and how it benefits everyone from new programmers to experienced devs. I’m not the one saying it should be Python, I’m just saying Make is not a good solution in this day and age, when so many other ecosystems have shown they can have better on-boarding experiences.
Anyway I won’t respond further. Feel free to rage reply as you want.
But putting that aside, you don’t think Make has a ton of footguns for someone starting out?
Spaces vs tabs for different bits because of decades of legacy? Oh but most editors default to tabs as spaces and suddenly no matter what you do, following tutorials will still fail.
Or you try and mess around with an existing open source project to learn from it. You want to list the targets? Oh wait, there’s no standard way to examine the makefile without understanding Make first.
If people consider Make easy to learn, imho they haven’t really reflected on how much more streamlined the rest of the software engineering ecosystem is, and how it benefits everyone from new programmers to experienced devs. I’m not the one saying it should be Python, I’m just saying Make is not a good solution in this day and age, when so many other ecosystems have shown they can have better on-boarding experiences.
Anyway I won’t respond further. Feel free to rage reply as you want.
Would anyone allow people without a drivers license onto the streets in a car? Is that gatekeeping or learning the rules of how a societal form of cooperation is practiced?
Now whether learning how make works is the same as driving a car is left to the reader to decide. Either way make is also a societal form of cooperation with its rules and traditions.
Where does gatekeeping start and where does societal norms and cooperation begin?
Now whether learning how make works is the same as driving a car is left to the reader to decide. Either way make is also a societal form of cooperation with its rules and traditions.
Where does gatekeeping start and where does societal norms and cooperation begin?
Are people just going to ignore that I have a clear delineation in my comment about the difference of learning and contributing to a project, and instead throw unrelated strawman arguments at me? Driving and programming are different, and learning to program and contributing code to important codebases is different. The stakes are different, the entire paradigms are different.
The on ramp to learn to drive is easy relative to actually being trusted/licensed to drive. But do I need to understand how an engine works to learn to drive? No. Maybe I needed to learn how gear changes worked in the day of manual transmission, but today with EVs and CVTs, why should learning stick be a gate to pass? That’s the equivalence of make to C
Regardless, all these analogies fall down because the crux of the issue is that if there are better alternatives that prevent issues, it isn’t sensible to keep using the old one.
Take your driving analogy. Should we force people to disable proximity sensors? After all you can drive without them but they make driving safer and easier for everyone.
Make is much the same way. It gets the job done, that doesn’t mean it’s good at getting the job done when better alternatives exist.
The on ramp to learn to drive is easy relative to actually being trusted/licensed to drive. But do I need to understand how an engine works to learn to drive? No. Maybe I needed to learn how gear changes worked in the day of manual transmission, but today with EVs and CVTs, why should learning stick be a gate to pass? That’s the equivalence of make to C
Regardless, all these analogies fall down because the crux of the issue is that if there are better alternatives that prevent issues, it isn’t sensible to keep using the old one.
Take your driving analogy. Should we force people to disable proximity sensors? After all you can drive without them but they make driving safer and easier for everyone.
Make is much the same way. It gets the job done, that doesn’t mean it’s good at getting the job done when better alternatives exist.
Point taken but why is learning yaml easier then learning make? It seems that if its about the amount that needs to be learnt then we'll get nowhere.
Make is easy for certain folks, yaml for others. If a project has been using make for years, should they be forced to replace it since someone believes they are gatekeeping? What happens if said project rejects a pull request that replaces said makefile with task file requiring python and yaml?
I'm pragmatic and also say if it gets the job done OK, but must I replace make with task if someone suggests I'm gatekeeping just because I have a pragmatic point of view?
Make is easy for certain folks, yaml for others. If a project has been using make for years, should they be forced to replace it since someone believes they are gatekeeping? What happens if said project rejects a pull request that replaces said makefile with task file requiring python and yaml?
I'm pragmatic and also say if it gets the job done OK, but must I replace make with task if someone suggests I'm gatekeeping just because I have a pragmatic point of view?
the point isn’t to replace existing use. To take your analogy of cars, we aren’t retrofitting cars with new tech.
But maybe it means that new projects use a more accessible build system. I’m not saying Task is the answer, I’m just saying Make is an issue for a lot of reasons like accessibility and portability to multiple toolchains and systems.
If you’re just coding for yourself, it’s not important. Nobody is saying you should change. But if you’re coding for an ecosystem, it’s always worth evaluating where changes can be made and then evaluating case by case if the pros outweigh the cons.
But maybe it means that new projects use a more accessible build system. I’m not saying Task is the answer, I’m just saying Make is an issue for a lot of reasons like accessibility and portability to multiple toolchains and systems.
If you’re just coding for yourself, it’s not important. Nobody is saying you should change. But if you’re coding for an ecosystem, it’s always worth evaluating where changes can be made and then evaluating case by case if the pros outweigh the cons.
> Should we force people to disable proximity sensors? After all you can drive without them but they make driving safer and easier for everyone.
I think your analogy is quite bad here. You ain't giving people proximity sensors or parking cameras or automatic parking. You are exchanging a good, comfortable vehicle we all know how to drive for a new type of vehicle with clearly worse ergonomics and forcing everyone to re-learn how to drive.
Once part of designing cars is that we can sit down in any car and just drive it. Perhaps we need to look up some minor and less important details, but we are guaranteed some very basic operation: how the blinking lights work, how to operate gas, brakes, steering wheel and manual transmission. I do think that they have messed up with short/long lights and back gears. That should have also been the same on all cars. But the basics are the same.
Makefile uses shell scripts for the recipes and shell variables, which makes Make an extension to the shell. There are differences, of course, but basics are very, very simple to get to learn. I learned 30+ years ago by just looking at other makefile and basically copying the details for my projects. I am still not even a very advanced makefile writer, I still lookup what automatic variables do, which one do I want and which functions I can use when I need them.
I think your analogy is quite bad here. You ain't giving people proximity sensors or parking cameras or automatic parking. You are exchanging a good, comfortable vehicle we all know how to drive for a new type of vehicle with clearly worse ergonomics and forcing everyone to re-learn how to drive.
Once part of designing cars is that we can sit down in any car and just drive it. Perhaps we need to look up some minor and less important details, but we are guaranteed some very basic operation: how the blinking lights work, how to operate gas, brakes, steering wheel and manual transmission. I do think that they have messed up with short/long lights and back gears. That should have also been the same on all cars. But the basics are the same.
Makefile uses shell scripts for the recipes and shell variables, which makes Make an extension to the shell. There are differences, of course, but basics are very, very simple to get to learn. I learned 30+ years ago by just looking at other makefile and basically copying the details for my projects. I am still not even a very advanced makefile writer, I still lookup what automatic variables do, which one do I want and which functions I can use when I need them.
Honestly, I’m closer to ditching C as the language for my systems course and moving to Rust entirely. It’s much easier than trying to rationalize the state of affairs through this gatekeeping in the name of safety lens.
I thought we all decided that norms and just imploring coders to hold the tools the right way isn’t scalable. I thought the industry had moved to safety through tooling, not safety through peer pressure and hoop jumping.
I mean, if we’re talking about cortical systems and safety, why is anyone using C at all?
I thought we all decided that norms and just imploring coders to hold the tools the right way isn’t scalable. I thought the industry had moved to safety through tooling, not safety through peer pressure and hoop jumping.
I mean, if we’re talking about cortical systems and safety, why is anyone using C at all?
This exactly.
the level of work needed to add a dependency in C+Make vs Rust+Cargo is huge.
the level of work needed to add a dependency in C+Make vs Rust+Cargo is huge.
Don't forget, banks still use cobol... Should be replaced... If only someone could!
Herein lies the issue for me: should existing projects switch because they are deemed to be gatekeeping when in fact either they are fine with their technology choices or they have not the resources or knowledge to change but they aren't gatekeeping. The external appearance is the same whichever reason might be the truth.
Herein lies the issue for me: should existing projects switch because they are deemed to be gatekeeping when in fact either they are fine with their technology choices or they have not the resources or knowledge to change but they aren't gatekeeping. The external appearance is the same whichever reason might be the truth.
> If people consider Make easy to learn, imho they haven’t really reflected on how much more streamlined the rest of the software engineering ecosystem is
This right here. It goes back to my other comment about how those who are proficient with Make don’t understand how esoteric and arcane it is to beginners.
The attitude of “well it should be hard, the shouldn’t even learn C if they can’t handle make” is explains why my students love Rust so much. Rust has a sane and standardized build system they prefer to use. Make is the Stone Age to them.
And don’t get me wrong, they learn Make. They learn how to use it proficiently and we get through it. But at the end of the class, they will throw Make away and use another language like Rust. They simply do not want to deal with it.
This right here. It goes back to my other comment about how those who are proficient with Make don’t understand how esoteric and arcane it is to beginners.
The attitude of “well it should be hard, the shouldn’t even learn C if they can’t handle make” is explains why my students love Rust so much. Rust has a sane and standardized build system they prefer to use. Make is the Stone Age to them.
And don’t get me wrong, they learn Make. They learn how to use it proficiently and we get through it. But at the end of the class, they will throw Make away and use another language like Rust. They simply do not want to deal with it.
Whingy babies, freshmen students… tomato tomato.
Problem is if you care about the future, someone has to help the whingy babies become competent adults. Not everyone is a polymath autodidact that can absorb everything about computers like the HN crowd. Some rockstars start as whingy babies. If you don’t help them at that crucial time, the stay whingy babies, or worse, go on to become project managers. You don’t want the whingy babies managing your project.
Problem is if you care about the future, someone has to help the whingy babies become competent adults. Not everyone is a polymath autodidact that can absorb everything about computers like the HN crowd. Some rockstars start as whingy babies. If you don’t help them at that crucial time, the stay whingy babies, or worse, go on to become project managers. You don’t want the whingy babies managing your project.
And yet there are actually whingy babies and, and no not all developers were once a whingy baby, in fact only the shit end of the bell curve ever were. Most people I even know at all never were, because it's not an automatic part of merely being new.
There is no value in catering to them, since that generally requires blunt rather than sharp tools, and everyone else requires sharp tools to get actual work done. They can play with their toys, but they are toys thoight up by people who care about the wrong things. Anyone else should ignore and disregard them and their ignorant ideas.
Make is probably not the final evolution of a build sysyem for eternity, merely the arguments given for python of all the ridiculous possibilities, have all been garbage so far.
Now if a proposition included some sort of canonical flavor of python (or whatever language) that can be counted on to:
- exist on any system, and "any system" does not mean "any linux intall from today forward"
- could at least be added to any system if not already, ie, plain c source that compiles even on obscure old systems, or built out of something else equally ubiquitous like shell or awk.
- single simple binary
- all features included (no libraries of plugins and modules)
- spec advances only slowly and thoughtufully and without breaking compatibility
Make already exists, meets all of these points, and provides the means to express complex jobs or simple jobs without any special built-in magic support for any one language or framework of the month, especially not at the expense of any others.
Although, even a mythical "canonical python subset" would be a strange thing to propose with it's meaninful whitespace, when one of the main complaints about make is the tabs.
There is no value in catering to them, since that generally requires blunt rather than sharp tools, and everyone else requires sharp tools to get actual work done. They can play with their toys, but they are toys thoight up by people who care about the wrong things. Anyone else should ignore and disregard them and their ignorant ideas.
Make is probably not the final evolution of a build sysyem for eternity, merely the arguments given for python of all the ridiculous possibilities, have all been garbage so far.
Now if a proposition included some sort of canonical flavor of python (or whatever language) that can be counted on to:
- exist on any system, and "any system" does not mean "any linux intall from today forward"
- could at least be added to any system if not already, ie, plain c source that compiles even on obscure old systems, or built out of something else equally ubiquitous like shell or awk.
- single simple binary
- all features included (no libraries of plugins and modules)
- spec advances only slowly and thoughtufully and without breaking compatibility
Make already exists, meets all of these points, and provides the means to express complex jobs or simple jobs without any special built-in magic support for any one language or framework of the month, especially not at the expense of any others.
Although, even a mythical "canonical python subset" would be a strange thing to propose with it's meaninful whitespace, when one of the main complaints about make is the tabs.
Pilots and surgeons are following rules to avoid killing people.
Make wasn't designed with any of that in mind. It was just one or a few people deciding the easiest path to code, or a style that they preferred.
It turns out most people don't find their decisions to be intuitive or explicable even decades later.
Make wasn't designed with any of that in mind. It was just one or a few people deciding the easiest path to code, or a style that they preferred.
It turns out most people don't find their decisions to be intuitive or explicable even decades later.
How is it different to learn Make from any other building tool like Gradle, Scons, Cmake, you name it?
Actually, basic is very simple, since the syntax is pretty much like the shell, or I mean, it is more or less shell scripts on steroids.
Actually, basic is very simple, since the syntax is pretty much like the shell, or I mean, it is more or less shell scripts on steroids.
> But it still falls over if you put in a space versus a tab
It fails only in a certain part of the script where tab is part of the syntax. It is like saying Python language fails if I add a wrong number of spaces, or replace a space with a tab, since white spaces are part of the Python syntax.
> Even the fact makefiles have no extension is confusing
How is that confusing, the entire filename (makefile) is "the extension" :-)
Put .mk at the end of the file name if that makes you happy and less "outdated" and "idiosyncratic". C/C++ does not require extension either, it is just a convention. Ever seen <vector>, <algorithm>, <iostream> etc?
It fails only in a certain part of the script where tab is part of the syntax. It is like saying Python language fails if I add a wrong number of spaces, or replace a space with a tab, since white spaces are part of the Python syntax.
> Even the fact makefiles have no extension is confusing
How is that confusing, the entire filename (makefile) is "the extension" :-)
Put .mk at the end of the file name if that makes you happy and less "outdated" and "idiosyncratic". C/C++ does not require extension either, it is just a convention. Ever seen <vector>, <algorithm>, <iostream> etc?
Sorry for the plug, but I can't help myself everytime I read about task managers to notice you have to learn yet another DSL.
DSL are the plague of our field, for one that is useful like SQL, you have a hundred that turn your life into hell. Make's DSL is already full of gotchas, but templated YAML based DSL are just insanity. They have so many footguns, and so little flexibility and ability to be debugged.
After trying a lot of task runners and builders, I finally settled on doit:
https://www.bitecode.dev/p/doit-the-goodest-python-task-runn...
- It makes the simple things simple, and the hard things possible.
- It scales up (you get targets, dependencies, manual scripting, etc), but it also scales down (you can define a simple list of shell commands to run and that's it).
- Instead of a DSL, you have regular Python. Which for all it's faults, is an excellent scripting language. It has great tooling support, and comes with a debugger.
- To avoid people abusing the expressiveness of Python, the default syntax to create tasks encourages a declarative style.
I've been using doit for years, and every single project I introduced it to was improved by it.
Of course, it comes with it's own problems (non python dev can dislike having to install that, and it doesn't yet provide a stand alone executable), but the ROI is way better than the alternatives I tried.
DSL are the plague of our field, for one that is useful like SQL, you have a hundred that turn your life into hell. Make's DSL is already full of gotchas, but templated YAML based DSL are just insanity. They have so many footguns, and so little flexibility and ability to be debugged.
After trying a lot of task runners and builders, I finally settled on doit:
https://www.bitecode.dev/p/doit-the-goodest-python-task-runn...
- It makes the simple things simple, and the hard things possible.
- It scales up (you get targets, dependencies, manual scripting, etc), but it also scales down (you can define a simple list of shell commands to run and that's it).
- Instead of a DSL, you have regular Python. Which for all it's faults, is an excellent scripting language. It has great tooling support, and comes with a debugger.
- To avoid people abusing the expressiveness of Python, the default syntax to create tasks encourages a declarative style.
I've been using doit for years, and every single project I introduced it to was improved by it.
Of course, it comes with it's own problems (non python dev can dislike having to install that, and it doesn't yet provide a stand alone executable), but the ROI is way better than the alternatives I tried.
Python is such a massive dependency though, people who code in python rarely realize how big of a barrier it is for people not in that ecosystem. Package management is insane and every project has its own way of dealing with it. Installing python by itself is a chore on many operating systems.
Agreed.
I wish more projects would compile their Python tool into a stand alone executable so that the users wouldn't have to deal with the installation process.
I will at some point write about how to do that with Python, so that the knowledge will more broadly available and maybe the community will grow the habit of doing it.
Meanwhile, I don't have more to offer than the "Relieving your Python packaging pain" procedure: https://www.bitecode.dev/p/relieving-your-python-packaging-p...
It's not perfect, but it's an improvement for any people that don't do a lot of Python but has to.
I wish more projects would compile their Python tool into a stand alone executable so that the users wouldn't have to deal with the installation process.
I will at some point write about how to do that with Python, so that the knowledge will more broadly available and maybe the community will grow the habit of doing it.
Meanwhile, I don't have more to offer than the "Relieving your Python packaging pain" procedure: https://www.bitecode.dev/p/relieving-your-python-packaging-p...
It's not perfect, but it's an improvement for any people that don't do a lot of Python but has to.
Excellent, one of the problems with this is that you can't google it. Well, of course you can but you'll get a million contradicting resources and all that float to the top are too basic anyway. I really enjoyed your post, saved and will pass it along.
The mentioned "follow-up" article: https://www.bitecode.dev/p/back-to-basics-with-pip-and-venv (if you find the time, please add a link to it to make it even more complete)
The mentioned "follow-up" article: https://www.bitecode.dev/p/back-to-basics-with-pip-and-venv (if you find the time, please add a link to it to make it even more complete)
Thanks a lot, I planned to do it and I forgot. I've just added the link and it's indeed way better like this.
Substack doesn't really have any way to make a series of articles, or to bundle stuff together, so linking manually is the only thing you can do that will give a little structure.
One more reason to move off it when I find the time.
Substack doesn't really have any way to make a series of articles, or to bundle stuff together, so linking manually is the only thing you can do that will give a little structure.
One more reason to move off it when I find the time.
And it breaks over time. Absolutely terrible choice for makefile. It would be far better to just use bash, even though it would mean coding your own dependecy and update-detecting logic.
I wouldn't write anything in python that I still wanted to work a month later or on even one other system.
Could you elaborate on what you mean by it breaks over time?
Python has no standard and the language implemented by the reference compiler is a moving target. (E.g. every script in Python 2 that used print without () broke in Python 3. There were other breaking changes. Some software still depends on installing Python 2.). The ecosystem in general is similarly dynamic. It's common to install a set of specific versioned packages and Python interpreter to get a package running.
It's not really a language where you can write something and be confident it'll still work in ten years with the latest release of Python that'll be shipping with distros.
It's not really a language where you can write something and be confident it'll still work in ten years with the latest release of Python that'll be shipping with distros.
> every script in Python 2 that used print without () broke in Python 3. There were other breaking changes. Some software still depends on installing Python 2
People had 15 years to move from 2 to 3.
I had to port dodo files from 2 to 3, it's a 10 minutes job. Most of the time, running 2to3 on it does the job automatically.
After all, you only use the stlib for task runners, delegating a lot of work to bash, and they are quite short.
> It's not really a language where you can write something and be confident it'll still work in ten years
I still have Python 2.7 projects running fine, and reinstalled one serving half million users a few month ago.
But even without this, Python was created in 1991, and Python 3 was released in 2008, that's 17 years between first release and the first big migration. To compare, go is 14 years old. In total.
Plus your build file will have been edited so many times by the end 10 years anyway.
Make has a lot going for it, but this is a terrible argument.
People had 15 years to move from 2 to 3.
I had to port dodo files from 2 to 3, it's a 10 minutes job. Most of the time, running 2to3 on it does the job automatically.
After all, you only use the stlib for task runners, delegating a lot of work to bash, and they are quite short.
> It's not really a language where you can write something and be confident it'll still work in ten years
I still have Python 2.7 projects running fine, and reinstalled one serving half million users a few month ago.
But even without this, Python was created in 1991, and Python 3 was released in 2008, that's 17 years between first release and the first big migration. To compare, go is 14 years old. In total.
Plus your build file will have been edited so many times by the end 10 years anyway.
Make has a lot going for it, but this is a terrible argument.
And yet, almost every random python script I come across does not work on my system, either because of modules or installation layout differences between the authors system and mine, or the passage of a mere few months or a year of time.
They are usually fixable of course. Assuming the thing worked on the authors system at all, it's possible to diagnose and fix it up for the current environment. But that has to be done, and sometimes takes more than a few minutes to hunt everything down.
And then one day I started seeing these pyc directories appearing out of the blue, right in whatever directory any script happened to be in. The same scripts that didn't do that yesterday. What an awesome thoughtfully engineered system!
The argument was not terrible, it was plain lived experience fact.
They are usually fixable of course. Assuming the thing worked on the authors system at all, it's possible to diagnose and fix it up for the current environment. But that has to be done, and sometimes takes more than a few minutes to hunt everything down.
And then one day I started seeing these pyc directories appearing out of the blue, right in whatever directory any script happened to be in. The same scripts that didn't do that yesterday. What an awesome thoughtfully engineered system!
The argument was not terrible, it was plain lived experience fact.
I don’t disagree with you, but I also don’t think that’s a bad thing. Things certainly don’t break quickly, there are years of warnings for deprecated functionality. I also think it’s not a bad thing to drop old functionality. It may not be warranted that code that worked 10/20 years ago should still work today. Languages evolve, and by pruning old features, footguns, or bad patterns, the language can evolve without carrying the weight of every mistake or legacy feature.
You can't run linux without python these days, and it's not hard to stick to packages available in the repositories.
It's very rare, most distros ship Python, and even mac does it.
But:
- Half of the devs in the world are on Windows.
- Installing doit with pip is not trivial. Hence the "Relieving python packaging pain" procedure: https://www.bitecode.dev/p/relieving-your-python-packaging-p...
And most people will never ever encounter this article, so they will hit a missing command, a missing package, a PATH error, a PYHONPATH error, a version error or something like that. They may even try to solve it by copy/pasting many commands from the web, some of them with admin right, and make a mess of their setup.
That's why, for end user tools like doit, providing a stand alone executable is still the best solution.
But:
- Half of the devs in the world are on Windows.
- Installing doit with pip is not trivial. Hence the "Relieving python packaging pain" procedure: https://www.bitecode.dev/p/relieving-your-python-packaging-p...
And most people will never ever encounter this article, so they will hit a missing command, a missing package, a PATH error, a PYHONPATH error, a version error or something like that. They may even try to solve it by copy/pasting many commands from the web, some of them with admin right, and make a mess of their setup.
That's why, for end user tools like doit, providing a stand alone executable is still the best solution.
Thankfully, every major distribution packages doit, so you don't need to touch pip at all
https://repology.org/project/doit/information
https://repology.org/project/doit/information
Incorrect.
Ruby is much better on the package management front and may be better suited to this task overall.
Having try both, I would say quite the opposite.
Ruby package management is lacking in many ways. E.G: there is not good build in solution to manage projects and multiple versions of ruby on windows, it's all 3rd party. And C-extensions are still terrible to install
The apparent simplicity comes from the fact the ruby ecosystem is much smaller than the Python one, with 95% of it revolving around ror, so you are more often on the happy path and see less errors.
Ruby package management is lacking in many ways. E.G: there is not good build in solution to manage projects and multiple versions of ruby on windows, it's all 3rd party. And C-extensions are still terrible to install
The apparent simplicity comes from the fact the ruby ecosystem is much smaller than the Python one, with 95% of it revolving around ror, so you are more often on the happy path and see less errors.
[deleted]
> templated YAML based DSL are just insanity
The truest statement of all of HN
It started off with configuration files, but then we needed logic, control flow, or variables; and for some reason nobody goes "ok, maybe we need an actual progamming language now" and instead we've created horrors beyond our comprehension.
Azure Pipelines is the worst I've used: three types of variables, conditional statements return string "True" and "False", and of course it's mostly undebuggable.
The truest statement of all of HN
It started off with configuration files, but then we needed logic, control flow, or variables; and for some reason nobody goes "ok, maybe we need an actual progamming language now" and instead we've created horrors beyond our comprehension.
Azure Pipelines is the worst I've used: three types of variables, conditional statements return string "True" and "False", and of course it's mostly undebuggable.
I have hopes that one day something like CUElang (https://cuelang.org) will become popular to be the configuration language of everything. Just enough logic to get by, but not turing complete, plus strong typing and embedded schemas.
However, for now there is just a single implementation in GO.
However, for now there is just a single implementation in GO.
Oh interesting.
I came up with basically the same thing as CUE (as described in [1]) on my own, except for the heavy schema stuff.
I think the schema stuff is unfortunately what's going to keep it from adoption.
I mean, I didn't add schema stuff explicitly in, but you can define schemas in mine, which makes it easier.
But making it explicit raises the bar for an implementation and may scare people away from writing one.
[1]: https://cuelang.org/docs/tutorials/tour/intro/json/
I came up with basically the same thing as CUE (as described in [1]) on my own, except for the heavy schema stuff.
I think the schema stuff is unfortunately what's going to keep it from adoption.
I mean, I didn't add schema stuff explicitly in, but you can define schemas in mine, which makes it easier.
But making it explicit raises the bar for an implementation and may scare people away from writing one.
[1]: https://cuelang.org/docs/tutorials/tour/intro/json/
Templated YAML just means that something somebody thought was configuration either wasn't or wasnt under every circumstance. It's not really a problem with YAML.
In most cases I think it's the fault of the creators of the YAML DSL (e.g. ansible) but with CI pipelines people really just should be writing scripts 100% in other languages and wiring them up to events with YAML. I dont have any sympathy for people using them as programming languages.
I'm sure Microsoft is happy to indulge it though because hey, vendor lock-in.
In most cases I think it's the fault of the creators of the YAML DSL (e.g. ansible) but with CI pipelines people really just should be writing scripts 100% in other languages and wiring them up to events with YAML. I dont have any sympathy for people using them as programming languages.
I'm sure Microsoft is happy to indulge it though because hey, vendor lock-in.
That's pretty much my strat: put everything in doit, and call doit from the yml file.
Anything else is shenanigans.
Anything else is shenanigans.
And yet you are here complaining about Taskfile and YAML overall. Nobody forces you to write full scripts in the YAML file. Just call scripts places somewhere next to it from it
Just because you have strats to minimize your exposure to something terrible doesn't mean the thing is less terrible.
But I'd do exactly the same with .phony targets in a Makefile. Keep the targets short and move larger chunks of code to individual script files that are just called from the Makefile
I don't disagree with any of that.
I just prefer Python to make, it's easier for me and most colleagues I met to write and to debug. And making a custom Python function when I need more than bash inside the dodo file is great.
But for someone used to make, why not used it?
I just prefer Python to make, it's easier for me and most colleagues I met to write and to debug. And making a custom Python function when I need more than bash inside the dodo file is great.
But for someone used to make, why not used it?
This is the exact reason why I moved to magefiles[0] in my Go projects. I had a semi-working Makefile base I spent days honing years ago, but then I ran into a few walls (can't remember which).
Enter magefiles, I could use the same language in them as I could with my other code and stuff just worked.
[0] https://magefile.org
Enter magefiles, I could use the same language in them as I could with my other code and stuff just worked.
[0] https://magefile.org
No need to apologise, I don't like YAML, I like Python, and I agree: I'd rather have a domain specific library than a DSL anyday.
The best of both worlds is a good programming language that allows expressing domain specific libraries almost as if they were domain specific languages, without going so declarative that people don't know how to script anything anymore.
I don't know if Python/doit really achieved that, but its task definitions seem almost as readable as the YAML, in a real language where I can do anything I want without asking permission. So I would always choose doit over Taskfile.
But a better comparison is between doit and make. I much prefer make's basic syntax, although I dislike newer features which inevitably have stretched the syntax beyond its original simplicity, but conceivably you can achieve something similar to make in a language but as a library rather than a DSL, and it would be immediately friendlier and more powerful than make. Depending on what I was doing I would use doit, but for something simple I've still got make (and it would be easy to convert if desired, in fact I have some projects in mind that could do with this!).
The best of both worlds is a good programming language that allows expressing domain specific libraries almost as if they were domain specific languages, without going so declarative that people don't know how to script anything anymore.
I don't know if Python/doit really achieved that, but its task definitions seem almost as readable as the YAML, in a real language where I can do anything I want without asking permission. So I would always choose doit over Taskfile.
But a better comparison is between doit and make. I much prefer make's basic syntax, although I dislike newer features which inevitably have stretched the syntax beyond its original simplicity, but conceivably you can achieve something similar to make in a language but as a library rather than a DSL, and it would be immediately friendlier and more powerful than make. Depending on what I was doing I would use doit, but for something simple I've still got make (and it would be easy to convert if desired, in fact I have some projects in mind that could do with this!).
Yes. I said it in another comment as well, but I'm rooting for a solid non turing complete conf language like CUElang (https://cuelang.org) to become popular. For now we can already use it to check and generate regular JSON or YAML, which is a good compat story.
But imagine having this in CI instead of YAML files.
But imagine having this in CI instead of YAML files.
I prefer Jsonnet for JSON and YAML
There is also Dhall.
I've tried the 3 of them, eventually I found CUElang was the best balance between expressiveness, ability to enforce and check for constraints, and easy of use.
But I would welcome any of them replacing the YAML mess we currently have in sysadmin.
I've tried the 3 of them, eventually I found CUElang was the best balance between expressiveness, ability to enforce and check for constraints, and easy of use.
But I would welcome any of them replacing the YAML mess we currently have in sysadmin.
For Pythonistas or people interested in Python, I can't plug the commenter's newsletter enough: it is an absolute treasure trove of good advice for new and old users of Python. They did a whole series on the right way to install and use Python [1]. If you follow their advice you'll never struggle with any of the problems people complain of encountering with Python.
[1] https://www.bitecode.dev/p/installing-python-the-bare-minimu...
[1] https://www.bitecode.dev/p/installing-python-the-bare-minimu...
Cheers :)
> DSL are the plague of our field, for one that is useful like SQL, you have a hundred that turn your life into hell.
The same can be said about using general purpose programming languages to specify stuff declaratively, but instead of implementation the problems happen in each and every single use of it. It only takes a single user that either does not comply with standards or feels they are particularly clever to mess everything for everyone.
The same can be said about using general purpose programming languages to specify stuff declaratively, but instead of implementation the problems happen in each and every single use of it. It only takes a single user that either does not comply with standards or feels they are particularly clever to mess everything for everyone.
The dose makes the poison
I guess that could work for a Python project, where you already have Python installed. But still you would need to set up a venv or similar to keep your system Python clean. Then you cannot run the tasks without creating that venv or similar. How to create the venv and install all the correct dev dependencies into it? You can't use the task runner yet, because you still need to install that ... Maybe a Makefile?
That's why I'm advocating for compiling tools stand alone in another comments.
Having to install doit first is indeed going to be a barrier of entry for a lot of people.
It's still worth it in my experience, but I get how it can deter users.
Having to install doit first is indeed going to be a barrier of entry for a lot of people.
It's still worth it in my experience, but I get how it can deter users.
Rather than downvote, since it's valid to state an opinion, I'll instead just do the same: Ugh. No.
I am thinking there must be a doit kind of tool but using bash shell instead. Anyone aware of one?
No guarantees, but I use https://github.com/svandragt/Taskfile
Unfortunately it’s named the same.
Unfortunately it’s named the same.
People that love bash usually are comfortable with make.
I have coworkers who use this. They tried to make me use this too. I failed to see the point. It offers no benefits of a build system, no benefits of existing automation tools, comes with "bespoke" syntax and requires installing an extra program to run it.
I have never tested it enough to discover things it does wrong, but given how new and not exactly popular this tool is, I bet there are lots of things it doesn't do right...
So, I have no idea why would anyone want this tool... beside just using one extra infra tool to appear as if you know something others don't (and that's exactly the kind of people who use it at my workplace).
I have never tested it enough to discover things it does wrong, but given how new and not exactly popular this tool is, I bet there are lots of things it doesn't do right...
So, I have no idea why would anyone want this tool... beside just using one extra infra tool to appear as if you know something others don't (and that's exactly the kind of people who use it at my workplace).
I remember how I promoted Git over a decade ago, and I heard from the lead guy that “SVN does the same” and “there are no benefits of using branches” (he never used them). Please listen to your coworkers and don’t be that guy. Technology moves forward, make it great, but it’s time to move on.
I didn't like the transition from SVN to Git. My biggest problem was the commit numbering. While SVN assigned consecutive numbers to commits, Git assigned hashes.
I still think that most people working with VCS today would've been better off using SVN. Most companies use Git in a centralized way anyways -- they don't need the distributed feature of the system. Most people using Git today don't know how to go back in history and wouldn't be physically able to carry out this task if requested because most Git repositories I've seen in my life were, essentially, write-only, just a glorified rsync.
I needed many years of working in infra to come to terms with Git and become efficient at using it. Most developers using it today will not have a multiple-year rotation in infra position, and are terrible at using the tool.
But this is very different from Taskfile vs Makefile. Git came with many attractive features. Some, perhaps inconvenient or demanding in terms of learning, but it was a genuine new tool that solved a problem other tools at the time didn't (branches were very expensive in SVN, sharding repositories across large distances was very expensive in SVN, and Git solved both).
Taskfile doesn't do anything better than any of the existing tools. It doesn't even promise anything like that... It's just "like make" but worse... so, what's the point exactly?
> Technology moves forward
Oh you sweet summer child... no, it doesn't. Sometimes it moves backwards, sometimes it jumps in place pretending to do useful work... there's no rule to say it should, and the evidence abounds of how this statement isn't true.
I still think that most people working with VCS today would've been better off using SVN. Most companies use Git in a centralized way anyways -- they don't need the distributed feature of the system. Most people using Git today don't know how to go back in history and wouldn't be physically able to carry out this task if requested because most Git repositories I've seen in my life were, essentially, write-only, just a glorified rsync.
I needed many years of working in infra to come to terms with Git and become efficient at using it. Most developers using it today will not have a multiple-year rotation in infra position, and are terrible at using the tool.
But this is very different from Taskfile vs Makefile. Git came with many attractive features. Some, perhaps inconvenient or demanding in terms of learning, but it was a genuine new tool that solved a problem other tools at the time didn't (branches were very expensive in SVN, sharding repositories across large distances was very expensive in SVN, and Git solved both).
Taskfile doesn't do anything better than any of the existing tools. It doesn't even promise anything like that... It's just "like make" but worse... so, what's the point exactly?
> Technology moves forward
Oh you sweet summer child... no, it doesn't. Sometimes it moves backwards, sometimes it jumps in place pretending to do useful work... there's no rule to say it should, and the evidence abounds of how this statement isn't true.
> Most companies use Git in a centralized way anyways -- they don't need the distributed feature of the system.
Even if you have a single centralized remote, git being distributed lets you work offline, because you have a full copy of the repository, and you can commit locally.
> Most people using Git today don't know how to go back in history and wouldn't be physically able to carry out this task if requested
Google exists and can help if someone doesn’t know but needs to do that.
> because most Git repositories I've seen in my life were, essentially, write-only, just a glorified rsync.
[citation needed]. If you have more than one person, how can a repository be write-only?
Even if you have a single centralized remote, git being distributed lets you work offline, because you have a full copy of the repository, and you can commit locally.
> Most people using Git today don't know how to go back in history and wouldn't be physically able to carry out this task if requested
Google exists and can help if someone doesn’t know but needs to do that.
> because most Git repositories I've seen in my life were, essentially, write-only, just a glorified rsync.
[citation needed]. If you have more than one person, how can a repository be write-only?
> Google exists and can help
Google isn't going to help if you were doing merges for a year, and then discovered you need to go back in history, and now you realize you should've been doing rebases instead. Google all you want -- it will be of no use, because your repository is a pile of garbage at this point, and it's not humanly possible to go back in time. Like I wrote in the post you replied to -- most programmers use Git where rsync would do.
> git being distributed lets you work offline
You could do that with SVN too. It was actually my other complaint about Git. Both SVN and Git use the word "commit", but those two mean different things. SVN "commit" is more like Git "push". Perhaps, you'd have to work harder in SVN having to reconcile the changes you haven't committed for a while, esp. because the tradition was for many developers to work on the same branch, but not impossible.
> If you have more than one person, how can a repository be write-only?
Do merges instead of rebases. When I say "write-only", I mean that the history of the repository is worthless -- it's written, but it's not actionable. This isn't apparent with smaller teams who have very few branches and work on a relatively small project. With larger teams merges in a repository create a ball-of-yarn history. I mean, you can read it, technically, but even using it as a log isn't feasible anymore. You could just throw it away, and nothing will change in your workflow / ability to recover from bad code changes / ability to transfer patches across versions etc.
Google isn't going to help if you were doing merges for a year, and then discovered you need to go back in history, and now you realize you should've been doing rebases instead. Google all you want -- it will be of no use, because your repository is a pile of garbage at this point, and it's not humanly possible to go back in time. Like I wrote in the post you replied to -- most programmers use Git where rsync would do.
> git being distributed lets you work offline
You could do that with SVN too. It was actually my other complaint about Git. Both SVN and Git use the word "commit", but those two mean different things. SVN "commit" is more like Git "push". Perhaps, you'd have to work harder in SVN having to reconcile the changes you haven't committed for a while, esp. because the tradition was for many developers to work on the same branch, but not impossible.
> If you have more than one person, how can a repository be write-only?
Do merges instead of rebases. When I say "write-only", I mean that the history of the repository is worthless -- it's written, but it's not actionable. This isn't apparent with smaller teams who have very few branches and work on a relatively small project. With larger teams merges in a repository create a ball-of-yarn history. I mean, you can read it, technically, but even using it as a log isn't feasible anymore. You could just throw it away, and nothing will change in your workflow / ability to recover from bad code changes / ability to transfer patches across versions etc.
This is the sort of logic that replaced make with cmake. New can definitely be worse than old. For example, cmake is terrible.
> I promoted Git over a decade ago
And from the response you got you failed at it. Were you promoting it because technology was "moving forward" and it was "time to move on"--whatever that means, instead of its merits? Didnt work for git, certainly won't work for Taskfile.
So, don't be what guy? The idiot that says branches have no benefits? The lesson is that there are dummies on both sides of the table.
Evaluate technology on merit. And, anything that is to dislodge a ubiquitous tool that's been around for 47 years has a high bar to cross.
And from the response you got you failed at it. Were you promoting it because technology was "moving forward" and it was "time to move on"--whatever that means, instead of its merits? Didnt work for git, certainly won't work for Taskfile.
So, don't be what guy? The idiot that says branches have no benefits? The lesson is that there are dummies on both sides of the table.
Evaluate technology on merit. And, anything that is to dislodge a ubiquitous tool that's been around for 47 years has a high bar to cross.
This is a poor take. MAKE files have existed for decades.
Every makefile I've used in the last decade has had every target marked as phony. When you're disabling one of the primary features of a tool to continue using it because you know how to use it, it's time to find a new tool.
Explicit is better than implicit? If everything was implicitly PHONY then of course people would complain about that as well. This is a bit like typing in function-signatures. But what's the proposal here, acknowledgement that your use-case is the best / most normal for default behaviour? Should function sigs that are missing type-hints always default to strings because that works well with your project code?
It's not about being explicit it's about using an abstraction that doesn't fit what you're abstracting.
Make operates on the abstraction of targets are files. If you don't confirm to that abstraction, make is the wrong tool. To use the hammer analogy, just because you're good at hammering something, doesn't mean you should use it for a screw.
Tools like docker and terraform don't use conventional files for state. Running tests in e.g. golang doesn't generate output files. Logging into the GitHub cli or aws don't have input and output files to compare for whether the task needs to be re-run.
Make operates on the abstraction of targets are files. If you don't confirm to that abstraction, make is the wrong tool. To use the hammer analogy, just because you're good at hammering something, doesn't mean you should use it for a screw.
Tools like docker and terraform don't use conventional files for state. Running tests in e.g. golang doesn't generate output files. Logging into the GitHub cli or aws don't have input and output files to compare for whether the task needs to be re-run.
To be pedantic: in the default mode, Terraform _does_ use a normal file for state, and for locks, and Make dependency tracking works fine with it.
That's a fair point. Funnily enough, I have the same complaints about terraform - the out of the box defaults (local state/lock files, and how backends work) are nonsense and incompatible with how the tool is actually used.
Make is not a task runner. Let me quote from it's manpage:
The purpose of the make utility is to determine automatically which pieces of a large program need to be recompiled, and issue the commands to recompile them.
The purpose of the make utility is to determine automatically which pieces of a large program need to be recompiled, and issue the commands to recompile them.
And yet of course it runs tasks all the time in thousands of projects for decades, so there's the possibility that you're being overly literal and pedantic about the whole thing. Make specifies DAGs and helps resolve paths in them. A build is just a "program" where "recompile" is "re-execute task" and dealing with file-based dependencies is a special case.
sometimes there is great leap forward in build tools!
this is not one of those times
this is not one of those times
[deleted]
git's CLI is famously inconsistent, but git merge was obviously much more reliable than svnmerge.py ever had been.
(I hear svn may have improved merge tracking, but I'm not going to spend more time on it.)
(I hear svn may have improved merge tracking, but I'm not going to spend more time on it.)
Git's CLI has improved over the years. E.g. git switch is a thing now. And it's quite consistent regarding order of arguments.
What bugs me is stuff like
git branch -c [old] new
git checkout -b new [old]
git switch -c new [old]
git rebase old new # if new only exists upstream?
Because Make is too stable and well documented, because it's just one tool and not 3, because it isn't written in Go and because you don't have to scratch you head at another YAML hell. Because 47 years of proven track record is nothing compared to "modern". Need I go on?
If you think there’s nothing to fix with make, you’re severely mistaken. The differences between GNU Make and BSD Make are for me a big reason to be extra careful when using it. Make also relies a lot on cleverness so it comes with lots of idiosyncrasies (like having to add .PHONY to all the targets not representing files, which in itself proves Make was not thought to be used like we use it). Some basic conventions we use today are also hard to do correctly with a Makefile (using dotenv files for example)
I used Task on a new project and the explicitness of it all was a breath of fresh air. I mean which is clearer between:
I used Task on a new project and the explicitness of it all was a breath of fresh air. I mean which is clearer between:
BINARY = ./foo
GO_FILES = $(shell find . -type f -name '*.go')
# Rebuild BINARY only if GO_FILES have been modified
$(BINARY): $(GO_FILES)
go build -o $@
.PHONY: build
build: $(BINARY)
env:
BINARY: ./foo
tasks:
build:
cmds:
- go build -o $BINARY
sources:
- **/*.go
generates:
- $BINARY
?But.... It is not build... It is running one binary one time, without any inter-dependencies. This binary (go in case of golang and cargo in case of Rust) IS build tool in these cases.
Looking at your example I think I understood why there is two schools for thoughts about this (and similar ones): because different ecosystems (go/rust vs C/C++/ObjC, mostly in these days, but also Fortran, etc) have completely different build needs.
Golang and Rust (and node.js, AFAIK) almost doesn't need external build tool, as they have their own (go, cargo, npm). It is very easy to make receipt for such build in any, even very limited, DSL, with any "task running" software.
On the other hand, how will Taskfile look for complex C/C++ project? You want separate on-demand compilation of each source file separately ("${CC} ${ALL-SOURCES}" is baaaaad idea), you need dependency extraction (ugly part of any Makefile, I admit, but receipts are well-known), you need linking, you often need some resource processing, or yacc/bison generation.
I don't think, Task file with all these features will be less ugly than typical Makefile. And many tricks are non-existing for Taskfile (and it is not clear, are they possible at all without extending DSL), and well-known for Makefiles.
This difference leads to polar views on such software.
Thank you for example!
Looking at your example I think I understood why there is two schools for thoughts about this (and similar ones): because different ecosystems (go/rust vs C/C++/ObjC, mostly in these days, but also Fortran, etc) have completely different build needs.
Golang and Rust (and node.js, AFAIK) almost doesn't need external build tool, as they have their own (go, cargo, npm). It is very easy to make receipt for such build in any, even very limited, DSL, with any "task running" software.
On the other hand, how will Taskfile look for complex C/C++ project? You want separate on-demand compilation of each source file separately ("${CC} ${ALL-SOURCES}" is baaaaad idea), you need dependency extraction (ugly part of any Makefile, I admit, but receipts are well-known), you need linking, you often need some resource processing, or yacc/bison generation.
I don't think, Task file with all these features will be less ugly than typical Makefile. And many tricks are non-existing for Taskfile (and it is not clear, are they possible at all without extending DSL), and well-known for Makefiles.
This difference leads to polar views on such software.
Thank you for example!
What if you use `embed`? Why rebuild the binary if you've only touched test files? Make at least theoretically gives you tools to deal with these (but between Make and Go they're quite hard to use), but Task gives you no chance.
> What if you use `embed`
You can add the embedded files to the "sources" list, no issues
> Why rebuild the binary if you've only touched test files
Yep, this is a thing not currently supported by Task, but they're considering extended globing iirc (with negation, like in gitignores). I think you could do something like that though:
You can add the embedded files to the "sources" list, no issues
> Why rebuild the binary if you've only touched test files
Yep, this is a thing not currently supported by Task, but they're considering extended globing iirc (with negation, like in gitignores). I think you could do something like that though:
env:
SOURCES:
sh:
find . -name *.go | grep -v _test.go
Keep digging this hole and you will just reinvent make, only worse.
It's like Greenspun's tenth rule, only for build tools.
It's like Greenspun's tenth rule, only for build tools.
I don’t see how this is worse than Make? This just shows that Taskfiles offers nicer syntax for some cases, but let’s you degrade to… something that looks just like Make actually, a variable initialized from a shell call
If your needs aren't trivial, just use GNU make if you're using make.
> which is clearer (..) ?
the makefile
the makefile
In bad faith, yeah probably.
$@, .PHONY, and the implicit target/dependency syntax is clever, terse, but NOT clear.
$@, .PHONY, and the implicit target/dependency syntax is clever, terse, but NOT clear.
[deleted]
It might be good to take a step back, and realize that the tool which is most "clear" is almost always the one which the user is most familiar with. It doesn't follow that the "clear" tool is actually more simple or more expressive.
To find out which tool is more simple, we could just count core concepts or use docs-length as a proxy. But 50 years of makefile's will muddy the water here.. is it super complicated or just well-documented? At minimum, when you reach for makefile then you know the concepts it brings are stable if complicated, and the tool is available by default almost everywhere, and the tool-versioning is almost totally safe to ignore. Looking at taskfile, I don't know the schema for the config-yaml, and more importantly I have no real confidence it's stable over time. Worrying about that sort of thing for project libs is enough work, so people don't want this kind of trouble in their build system. I also know taskfile isn't available by default until I install it.
Build-system yak-shaving mostly comes down to the same stuff. A group of people who don't want to learn tools try to convince everyone to learn some different tools, failing to realize that most of us are lazy in the same ways. Don't want to learn makefile tips and tricks? No one does really, but no one cares about taskfile tips and tricks either. Given the basic equivalence between these two situations.. there's definitely some wisdom here in preferring the standard rather than the startup.
To find out which tool is more simple, we could just count core concepts or use docs-length as a proxy. But 50 years of makefile's will muddy the water here.. is it super complicated or just well-documented? At minimum, when you reach for makefile then you know the concepts it brings are stable if complicated, and the tool is available by default almost everywhere, and the tool-versioning is almost totally safe to ignore. Looking at taskfile, I don't know the schema for the config-yaml, and more importantly I have no real confidence it's stable over time. Worrying about that sort of thing for project libs is enough work, so people don't want this kind of trouble in their build system. I also know taskfile isn't available by default until I install it.
Build-system yak-shaving mostly comes down to the same stuff. A group of people who don't want to learn tools try to convince everyone to learn some different tools, failing to realize that most of us are lazy in the same ways. Don't want to learn makefile tips and tricks? No one does really, but no one cares about taskfile tips and tricks either. Given the basic equivalence between these two situations.. there's definitely some wisdom here in preferring the standard rather than the startup.
[deleted]
[deleted]
some tasks a developer might need to do besides just building the application:
- copying a database from production to local for bug reproduction/fixes - inserting seed-data into the local database - creating a migration after changing records - applying the migration - rolling back the migration - running a tool to copy configuration from a keyvault - running the end to end tests - generate certificates - set up an ssh tunnel to a remote server or database
All of these things are part of the development process, and cannot be part of the build system. Each of them has their own commands and tools.
All developers need to easily be able to do these things, without asking help each time, or learning the syntax of each command.
One easy way is to stuff all of them in a task runner with simple aliases. Another could be writing a document that you can copy/paste from. I prefer a task runner, since you can memorize the alias and work from the terminal without looking things up.
- copying a database from production to local for bug reproduction/fixes - inserting seed-data into the local database - creating a migration after changing records - applying the migration - rolling back the migration - running a tool to copy configuration from a keyvault - running the end to end tests - generate certificates - set up an ssh tunnel to a remote server or database
All of these things are part of the development process, and cannot be part of the build system. Each of them has their own commands and tools.
All developers need to easily be able to do these things, without asking help each time, or learning the syntax of each command.
One easy way is to stuff all of them in a task runner with simple aliases. Another could be writing a document that you can copy/paste from. I prefer a task runner, since you can memorize the alias and work from the terminal without looking things up.
So, what's your point exactly? There are plenty of tools for this that do it a lot better than Taskfile? Like, Ansible for example?
Also, what makes you think a build system doesn't or shouldn't do those things?
Also, you could just write a Shell script to do all of those -- what's the difference?
Also, what makes you think a build system doesn't or shouldn't do those things?
Also, you could just write a Shell script to do all of those -- what's the difference?
> stuff all of them in a task runner with simple aliases
$ mkdir bin
> easily be able to do these things, without asking help each time
$ ls bin
> generate certificates
$ bin/mkcert.sh
What you didn't need: yet another tool and config syntax as a dumping bucket for your one-liners and command aliases.
$ mkdir bin
> easily be able to do these things, without asking help each time
$ ls bin
> generate certificates
$ bin/mkcert.sh
What you didn't need: yet another tool and config syntax as a dumping bucket for your one-liners and command aliases.
Make doesn’t have any builtin —help for the list of user defined targets, and has a slightly wonky bespoke syntax of it’s own. So it’s quite natural to look for something that has those properties.
But Taskfile didn't improve anything about Make? Make has a huge user base, it's battle-tested in many environments, it has a ton of infrastructure around it...
Taskfile brings nothing to the table. It cannot compete with existing automation tools. It doesn't even promise to be better than existing automation tools...
I mean, give me at least some reason to believe it could be better than what I already have... to me it looks like the people who wrote it simply never saw what's already there. Kind of like a university project where students write a simplistic compiler for a subset of C language... It's nice that they tried their hand, but they have a long way before they actually produce something useful.
Taskfile brings nothing to the table. It cannot compete with existing automation tools. It doesn't even promise to be better than existing automation tools...
I mean, give me at least some reason to believe it could be better than what I already have... to me it looks like the people who wrote it simply never saw what's already there. Kind of like a university project where students write a simplistic compiler for a subset of C language... It's nice that they tried their hand, but they have a long way before they actually produce something useful.
This is the most real criticism of makefiles I've seen in this thread. There's fixes for it, but it usually involves parsing `--print-data-base` output, which is pretty awkward without reaching for a helper language like python/perl/ruby, and that then reduces portability and the batteries-included stuff that makefiles enjoy. It really does need a fix of some kind in make-core.
At least on debians using bash, tab completion will include the user defined targets.
Targets can be conditionally defined, so a built-in help may be inaccurate.
Given that targets may be conditionally defined, having built-in documentation of existing targets is _more_ critical - and it should take into account such conditions.
How does this compare with `just`[0]? I've never written a Makefile but I'm a big fan of `just`.
[0] https://github.com/casey/just
[0] https://github.com/casey/just
Taskfile looks heavily inspired by terraform and other "cloud native tooling (read: lots of yaml)
Yep, `just` is much more truthful to the bare bones `make` legacy while `task` offers a bit more structured approach coming from inclusion of yaml as a specification language and many convenience things like handling env, run directories etc. I have worked with `just` for quite a bit but this looks very good honestly and I will consider switching.
I, too, use just a lot in a very off handed way. My memory isn't great and I just toss commands in a projects justfile which I occasionally need.
Yes, a readme would work to a degree too, but a justfile is runnable code. That beats dead documentation everytime.
Just from reading the title i figured their idea of 'modern' was using something like yaml or json.
How Yaml of all things became so popular is a mystery to me.
I am guessing the primary selling point is that it's sort of human readable similar to Markdown but even that (in fact, especially that) falls short when a configuration is more than just 5 lines, in my opinion. Trying to figure out which bullet this sub-bullet belongs to by holding my finger on the screen while scrolling is not an improved experience. And don't even get me started on the clever type system.
I am guessing the primary selling point is that it's sort of human readable similar to Markdown but even that (in fact, especially that) falls short when a configuration is more than just 5 lines, in my opinion. Trying to figure out which bullet this sub-bullet belongs to by holding my finger on the screen while scrolling is not an improved experience. And don't even get me started on the clever type system.
> Trying to figure out which bullet this sub-bullet belongs to by holding my finger on the screen while scrolling is not an improved experience
I don't see this as any worse than figuring out where nested parentheses are closed: tooling helps. I'm almost certain there's a whitespace plugin equivalent of Rainbow Parentheses for your IDE/editor of choice
I don't see this as any worse than figuring out where nested parentheses are closed: tooling helps. I'm almost certain there's a whitespace plugin equivalent of Rainbow Parentheses for your IDE/editor of choice
Yes, tooling helps, though I am not always in my IDE. And I agree that it shouldn't be worse than, say, JSON, but for one reason or another my brain seems to struggle with it just a bit more. It certainly does not seem like an improvement, which I feel is implied.
Indentation guide are available
on most editors.
> How Yaml of all things became so popular is a mystery to me.
Well . . . the Perl folks, I think, came up with something less pointy than XML. And we all appreciated that because we'd all poked ourselves with XML too many times.
https://en.wikipedia.org/wiki/YAML
Well . . . the Perl folks, I think, came up with something less pointy than XML. And we all appreciated that because we'd all poked ourselves with XML too many times.
https://en.wikipedia.org/wiki/YAML
Yelling At My Laptop
I found it amusing how the main features advertised for Task were the programming language used to write it, how it's distributed as a single executable, and how it's easy to install.
None of them are relevant, or made any case for using it instead of just using the thing described in an international standard that is shipped by default in UNIX and Unix-like OSes for decades.
It's like their selling point is to talk crap about other tools without making any case for their own offering.
None of them are relevant, or made any case for using it instead of just using the thing described in an international standard that is shipped by default in UNIX and Unix-like OSes for decades.
It's like their selling point is to talk crap about other tools without making any case for their own offering.
UNIX might be standard but it doesn't make it cross platform. You might not like it but a lot of people develop on windows. Build tools that assume a unix environment is present aren't a solution.
Posix is cross platform. There's a reason it's easy to build stuff on mac os, free bsd and linux.
Just because Windows decided to do everything differently to Unix and you decided to use Windows doesn't mean everything else has to adapt to be "cross platform".
FWIW, GNU Make runs fine in Windows if you're allergic to WSL
Just because Windows decided to do everything differently to Unix and you decided to use Windows doesn't mean everything else has to adapt to be "cross platform".
FWIW, GNU Make runs fine in Windows if you're allergic to WSL
> UNIX might be standard but it doesn't make it cross platform. You might not like it but a lot of people develop on windows.
I fail to understand what point you tried to make. There are already plenty of Make implementations for Windows. If you think that installing a third party tool like Task is ok, I don't see how doing the same with a Make implementation would exclude it as a valid alternative.
I fail to understand what point you tried to make. There are already plenty of Make implementations for Windows. If you think that installing a third party tool like Task is ok, I don't see how doing the same with a Make implementation would exclude it as a valid alternative.
I don't think using Task is better either. I was just replying to OP saying to just use unix tools. Unix tools aren't cross platform so they aren't a good build tool either.
> Unix tools aren't cross platform (...)
I just pointed out there are Make implementations for Windows.
And I'm not even mentioning WSL or MinGW.
I don't know what else to say.
I just pointed out there are Make implementations for Windows.
And I'm not even mentioning WSL or MinGW.
I don't know what else to say.
> Build tools that assume a unix environment is present aren't a solution.
Ut is a solution for the vast majority of cases. Even most Windows environments use WSL.
Ut is a solution for the vast majority of cases. Even most Windows environments use WSL.
nmake has been part of MS Visual Studio and former incarnations such as MSVC since basically forever (1988?). And while the shell language recognized by command.com limits nmake's usefulness, tar and curl have been added to standard Win installations only recently.
We use this to trigger local tasks (like build, codegen, db migrations etc.) in our monorepo. We moved to it because our npm scripts were getting messier in absense of task dependencies, lack of multiline support, comments in json etc. Being an easy to use cross platform binary that can be managed through asdf made adoption very easy.
We have few colleagues working on react-native who use windows, and it is easy to support them because it uses mvdan/sh for script execution.
Being able to organize tasks into multiple module specific taskfiles which can be merged (with auto-prefixing) into the top level taskfile is quite nice. The support for watch mode, timestamp checking, environment propagation with built in dotfile support are all nice features that we found very handy.
It would have been nicer if it used hcl but I can live with yaml. I am glad that it doesn't use a real programming language - it makes it easy to look at a task in isolation (we have hundreds) and figure out what it depends on and what it does.
It is easy to be dismissive of such tools when glancing at the homepage, but it is one of those tools that grow on you when you give it a chance. The author has also been addressing a lot of common pain points over the years for which we are thankful - it is nice to be able to trigger tasks from any subdir (similar to npm run ...) and the new editor integration is quite too.
I see people saying that you can do all that this tool does using a few simple scripts. But these scripts only start off as simple, once you need a watch mode, cross platform support, timestamp change detection, selective argument propagation, need to exec dependencies in right directories etc. they tend to get buggier and messier.
We have few colleagues working on react-native who use windows, and it is easy to support them because it uses mvdan/sh for script execution.
Being able to organize tasks into multiple module specific taskfiles which can be merged (with auto-prefixing) into the top level taskfile is quite nice. The support for watch mode, timestamp checking, environment propagation with built in dotfile support are all nice features that we found very handy.
It would have been nicer if it used hcl but I can live with yaml. I am glad that it doesn't use a real programming language - it makes it easy to look at a task in isolation (we have hundreds) and figure out what it depends on and what it does.
It is easy to be dismissive of such tools when glancing at the homepage, but it is one of those tools that grow on you when you give it a chance. The author has also been addressing a lot of common pain points over the years for which we are thankful - it is nice to be able to trigger tasks from any subdir (similar to npm run ...) and the new editor integration is quite too.
I see people saying that you can do all that this tool does using a few simple scripts. But these scripts only start off as simple, once you need a watch mode, cross platform support, timestamp change detection, selective argument propagation, need to exec dependencies in right directories etc. they tend to get buggier and messier.
One day after writing a complex Makefile and getting frustrated because 1) it was hard to read and reason about because of make weird syntax, 2) it didn't run on CI because it uses GNU Make instead of BSD Make on my MacBook, I thought of writing a "modern" alternative:
I wanted it easy to read, write, parse and use, so I'd use YAML. You guys all shit on YAML because it's extremely badly used by, say, Helm charts, but it's a good markup language in itself. StrictYAML is anyway.
So I started writing a toy Makefile in YAML with these requirements: I need to run a command, with variables, it needs to depend on another command, and shouldn't run if the "output" file is already present and newer than the "source" files. So something like this:
So yeah, Taskfiles are just Makefiles with a sane syntax and better tooling (JSON schema, auto generated doc, real default target...). I have no idea why the comments are so hostile towards it, because clearly Makefiles are not perfect (it's even a pretty bad format)
I wanted it easy to read, write, parse and use, so I'd use YAML. You guys all shit on YAML because it's extremely badly used by, say, Helm charts, but it's a good markup language in itself. StrictYAML is anyway.
So I started writing a toy Makefile in YAML with these requirements: I need to run a command, with variables, it needs to depend on another command, and shouldn't run if the "output" file is already present and newer than the "source" files. So something like this:
env:
APPNAME: app
clean:
run: rm -rf ./dist
build:
depends:
- clean
input:
- **/*.go
run:
- go build -o dist/$APPNAME
output:
- dist/$APPNAME
So yeah, Taskfiles are just Makefiles with a sane syntax and better tooling (JSON schema, auto generated doc, real default target...). I have no idea why the comments are so hostile towards it, because clearly Makefiles are not perfect (it's even a pretty bad format)
I'm the creator of StrictYAML and honestly, I would love to see something like this built with it.
I hate make and I dont like the idea of using non-typesafe YAML to build this (which presumably this tool does, since it's done in golang).
I've been using a "standardized" bash script with a huge case statement as a justfile/this/makefile replacement but it's not ideal.
I hate make and I dont like the idea of using non-typesafe YAML to build this (which presumably this tool does, since it's done in golang).
I've been using a "standardized" bash script with a huge case statement as a justfile/this/makefile replacement but it's not ideal.
Thank you for StrictYAML! I wish it was the standard
it didn't run on CI because it uses GNU Make instead of BSD Make on my MacBook
$ /usr/bin/make --version
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
This program built for i386-apple-darwin11.3.0My memories must fail me, I thought it was a BSD make vs GNU make issue but probably not! Maybe a version issue between the bundled make binaries on each platform, I don't remember the details to be honest
I wanted to see what was wrong with Makefile and how Taskfile solved the issue, but the post doesn't explain; it's more a tutorial on how to use Taskfile. If the author reads this, I think we'd love to know what's wrong with make.
Also, a lot of those look like artificial complexity over a shell script sourcing another one containing only variables export, and running the required scripts.
Also, a lot of those look like artificial complexity over a shell script sourcing another one containing only variables export, and running the required scripts.
If you don't know what's wrong with make, just use make. But it probably means you probably don't USE make.
If your Makefiles look like this:
If your Makefiles look like this:
build:
go build -o dist/app
I don't use make, so I wanted to see what was wrong with it, and the author's argument is that it's old. Not very convincing.
Your description starts to be more interesting, and I feel like mk (the successor of make from the Plan 9 people: https://plan9.io/sys/doc/mk.html) might address those, but again, I don't have a need for it.
Your description starts to be more interesting, and I feel like mk (the successor of make from the Plan 9 people: https://plan9.io/sys/doc/mk.html) might address those, but again, I don't have a need for it.
> Taskfile is just a dialect of YAML format with a specific syntax
One of the few formats in common use that's actually worse than Makefile, plus you need to go and find another binary to run the thing.
One of the few formats in common use that's actually worse than Makefile, plus you need to go and find another binary to run the thing.
So what does make (the took is not called Makefile, btw) "wrong". Where could one improve on it?
1. Syntax. Makefiles aren't parseable and have some at least unexpected syntactic properties.
2. Dependency detection: AFAIK any make will always use a timestamp and the precision varies a lot. A checksum or at least guaranteed resolution would be great. Also, you can't depend on directories.
3. Dynamics (this is going to be contested): Changing dependencies and tasks at runtime is possible with GNU make, but not ideal.
Taskfile improves on problem 1 but probably worsens everything else.
1. Syntax. Makefiles aren't parseable and have some at least unexpected syntactic properties.
2. Dependency detection: AFAIK any make will always use a timestamp and the precision varies a lot. A checksum or at least guaranteed resolution would be great. Also, you can't depend on directories.
3. Dynamics (this is going to be contested): Changing dependencies and tasks at runtime is possible with GNU make, but not ideal.
Taskfile improves on problem 1 but probably worsens everything else.
- Multiple out file dependencies are also difficult to describe in make (GNU Make). There's a dedicated section for how to solve it in the manual[^1] but it's a crutch.
- Taking environment variables into account as a dependency (they may affect Makefile logic and also code generation by tools that interpret them).
[^1]: https://www.gnu.org/software/automake/manual/html_node/Multi...
- Taking environment variables into account as a dependency (they may affect Makefile logic and also code generation by tools that interpret them).
[^1]: https://www.gnu.org/software/automake/manual/html_node/Multi...
Multiple outfile dependencies (aka "grouped targets") can be specified in GNU Make via `&:` since 4.3.
It's a shame, IMO, that make hasn't really evolved.
I've been at a few places that insist on still using them. They mostly work fine, but having PHONY littered everywhere gets a bit annoying.
I don't think a successor can exist. Makefiles seem like the last of a dying breed - not because make is bad, but because there are better ways to accomplish this already.
The market for people who've outgrown Makefiles but haven't embraced the alternatives has to be pretty small...
I've been at a few places that insist on still using them. They mostly work fine, but having PHONY littered everywhere gets a bit annoying.
I don't think a successor can exist. Makefiles seem like the last of a dying breed - not because make is bad, but because there are better ways to accomplish this already.
The market for people who've outgrown Makefiles but haven't embraced the alternatives has to be pretty small...
The problem with Make is people abuse it with phony targets. If there's discipline in writing Makefiles and most targets are files, Make works quite well due to the basically simple timestamp/dependency stuff (vs say cmake which I honestly find harder to debug). There are obviously a lot of patchwork fixes and the absence of logical operators drives me nuts but overall I have yet to find a better, more battle tested tool.
Why do you think they haven't evolved? Latest versions give you access to Guile scheme as an extension language, not to mention that they extended over time for quite a bit. But it is all backwards compatible so old makefiles still run.
Make has no way to opt into lots of greybeard "best practices" by default, and the syntax is still difficult to deal with.
I want to keep the solver, which is pretty good. I want to keep the idea of targets, rules, recipes, which is what tools like Taskfile throw out that makes them not very attractive to me. I even want to keep a good chunk of the syntax for specifying these. But Make needs some kind of way to invoke it that switches off all the dumb shit.
- parse any reasonable whitespace indentation, for chrissake
- -r by default, or some way to opt into relevant sets of rules (e.g. `import "c"` to get CC, CFLAGS, %.o, etc.)
- Separate def syntax and namespacing for proper file-generating rules vs. phony rules, no more .PHONY or fake dependencies; also move stuff like .INTERMEDIATE or .PRECIOUS to the rule def.
- .DELETE_ON_ERROR, .ONESHELL by default, probably others I forgot
- better semantics for double-colon rules; once you know what the phony rules are you can make them operate in this mode by default and give better syntax for pre/post hooks, priorities, run-on-failure, etc.
- something solving the same problem as but less dumb than .SECONDEXPANSION (guile helps somewhat here but give me some help for the most common cases)
- stamp files implemented by some out-of-tree cache, they're just an optimization and I'd rather lose them now and then and have to rebuild, than have to keep adding them to every other tool's ignore lists.
As well as some actually new features to help manage targets without local inputs, e.g. smarter connectors for fetching dependencies from network resources.
I want to keep the solver, which is pretty good. I want to keep the idea of targets, rules, recipes, which is what tools like Taskfile throw out that makes them not very attractive to me. I even want to keep a good chunk of the syntax for specifying these. But Make needs some kind of way to invoke it that switches off all the dumb shit.
- parse any reasonable whitespace indentation, for chrissake
- -r by default, or some way to opt into relevant sets of rules (e.g. `import "c"` to get CC, CFLAGS, %.o, etc.)
- Separate def syntax and namespacing for proper file-generating rules vs. phony rules, no more .PHONY or fake dependencies; also move stuff like .INTERMEDIATE or .PRECIOUS to the rule def.
- .DELETE_ON_ERROR, .ONESHELL by default, probably others I forgot
- better semantics for double-colon rules; once you know what the phony rules are you can make them operate in this mode by default and give better syntax for pre/post hooks, priorities, run-on-failure, etc.
- something solving the same problem as but less dumb than .SECONDEXPANSION (guile helps somewhat here but give me some help for the most common cases)
- stamp files implemented by some out-of-tree cache, they're just an optimization and I'd rather lose them now and then and have to rebuild, than have to keep adding them to every other tool's ignore lists.
As well as some actually new features to help manage targets without local inputs, e.g. smarter connectors for fetching dependencies from network resources.
Plan 9 has introduced mk, a successor to make:
https://plan9.io/sys/doc/mk.html
It's basically the same with some niceties and good defaults. One of them is the introduction of attributes, the most prominent being virtual: a target can be virtual meaning mk will never check the existence of a file and always run the recipe.
https://plan9.io/sys/doc/mk.html
It's basically the same with some niceties and good defaults. One of them is the introduction of attributes, the most prominent being virtual: a target can be virtual meaning mk will never check the existence of a file and always run the recipe.
I've had the pleasure of using mk on a single project once and it was nice, fixing about 80% of what I think is broken in Make.
However it's an extremely hard sell to bring into a corporate tech stack today for either brown or green field projects.
However it's an extremely hard sell to bring into a corporate tech stack today for either brown or green field projects.
It's not a serious alternative to make, but I had a go at writing a sort of "new make" which tried to solve most of the problems:
https://people.redhat.com/rjones/goals/ http://oirase.annexia.org/rwmj.wp.com/2020-01-goals-preview....
https://people.redhat.com/rjones/goals/ http://oirase.annexia.org/rwmj.wp.com/2020-01-goals-preview....
Instead of a DSL backed by some external tool, it's possible to write build scripts in a normal programming language. You only really need to implement "spawn process and give it these arguments" and "this depends on that" to reproduce most of C++ build scripts.
I've gone with lua and libuv because the concurrency model is so trivial. Each process spawn goes in a coroutine, stderr and stdout go into lua strings, use yield when the process is still running. However that comes with pretty minimal linting or error detection on the input, so maybe use some other language if you want that.
Or rephrasing, instead of a DSL for describing a graph and transforms to apply to the nodes, you can use a library in your favourite language instead. You might like that more than debugging cmake.
I've gone with lua and libuv because the concurrency model is so trivial. Each process spawn goes in a coroutine, stderr and stdout go into lua strings, use yield when the process is still running. However that comes with pretty minimal linting or error detection on the input, so maybe use some other language if you want that.
Or rephrasing, instead of a DSL for describing a graph and transforms to apply to the nodes, you can use a library in your favourite language instead. You might like that more than debugging cmake.
Make is ubiquitous, it’s simple. Everybody knows it. That’s the reason to use it.
Just add a makefile to your repo and anybody is able it get started. I would have no clue what todo with a project in the structure proposed.
Note: I just makefiles as an entry point to the underlying build tool such as gradle or npm or go build
Just add a makefile to your repo and anybody is able it get started. I would have no clue what todo with a project in the structure proposed.
Note: I just makefiles as an entry point to the underlying build tool such as gradle or npm or go build
The article doesn't sell the tool very well. The flaws of make are well known, let's see some example of how this new tool outshines make? To me it looks like something very similar to make but with a different (slightly verbose) syntax.
We’ve been using Task for our (Drupal) web projects for a few years now and are really pleased with it.
Why not Make? We actually used that on a few projects previously, but found it challenging to use with both our teams and with typical tools used in the PHP and JavaScript stack.
1. Most of our newer hires have never used Make before, or anything like it. It seems to have fallen out of many curriculums in favour of other build tools, presumably due to decreasing focus on C / C++ in fundamental computer science classes. The idea that a build tool is inherently files based is a completely foreign concept. 2. File-based dependency tracking being the exception, and not the rule, simply works better with the majority of the types of tasks we run. Those include building docker containers, pulling down CMS databases for local development, and running CLI tasks like database migrations. 3. For those tasks that can be tracked with files, Task supports both timestamps and hashes.
One big win of Task over other tools is that it ships it’s own built-in sh shell. If you have tasks that run on a host and not in a container, it significantly reduces your host dependencies. Being a typical single-binary go app makes deployments easy too.
I’ll also say the maintainers have been very responsive to our few bug reports and feature requests, and responsive to our contributions too.
The worst part is yes, it’s another YAML DSL to learn. And personally, I wish Make had gained more traction in the web and docker world. But given all of the above, the trade offs of switching to Task have been worth it for us.
Why not Make? We actually used that on a few projects previously, but found it challenging to use with both our teams and with typical tools used in the PHP and JavaScript stack.
1. Most of our newer hires have never used Make before, or anything like it. It seems to have fallen out of many curriculums in favour of other build tools, presumably due to decreasing focus on C / C++ in fundamental computer science classes. The idea that a build tool is inherently files based is a completely foreign concept. 2. File-based dependency tracking being the exception, and not the rule, simply works better with the majority of the types of tasks we run. Those include building docker containers, pulling down CMS databases for local development, and running CLI tasks like database migrations. 3. For those tasks that can be tracked with files, Task supports both timestamps and hashes.
One big win of Task over other tools is that it ships it’s own built-in sh shell. If you have tasks that run on a host and not in a container, it significantly reduces your host dependencies. Being a typical single-binary go app makes deployments easy too.
I’ll also say the maintainers have been very responsive to our few bug reports and feature requests, and responsive to our contributions too.
The worst part is yes, it’s another YAML DSL to learn. And personally, I wish Make had gained more traction in the web and docker world. But given all of the above, the trade offs of switching to Task have been worth it for us.
I use Makefiles and native Bash in 2023 because both come on just about any system that will build and package my software.
Also, I disagree with Make being a bad task runner. In fact, I find it quite versatile. The fact that it is not a DSL with a meta programming language or, in the case of Taskfile, a YAML document that gets convoluted quickly (see also: Ansible) is a feature IMO.
Also, I disagree with Make being a bad task runner. In fact, I find it quite versatile. The fact that it is not a DSL with a meta programming language or, in the case of Taskfile, a YAML document that gets convoluted quickly (see also: Ansible) is a feature IMO.
Heartening to realise that that my venerable and slightly moth-eaten ThinkGeek t-shirt (ca.2001) emblazoned with "Go away or I will replace you with a very small shell script" remains relevant today.
Taskfile is great! I used it to write a NixOS deployment tool [1] which I have been using ever since
[1] https://github.com/pinpox/lollypops
[1] https://github.com/pinpox/lollypops
Taskfile is a wonderful little tool.
Used with an alias it can be a wonderful way to manage scripts.
I use mine as a sort of global bookmark for shell commands I can't be bothered to memorize all the params for.
Handling env files is also a huge help.
Used with an alias it can be a wonderful way to manage scripts.
I use mine as a sort of global bookmark for shell commands I can't be bothered to memorize all the params for.
Handling env files is also a huge help.
Ok cool, let's see what this is about
> Taskfile is just a dialect of YAML format with a specific syntax
I imagined that. Ok bye
Just write a js or python script. Or even a shell script
Makefile syntax is bad but frankly YAML was a bad idea. (slightly less than XML sure)
> Taskfile is just a dialect of YAML format with a specific syntax
I imagined that. Ok bye
Just write a js or python script. Or even a shell script
Makefile syntax is bad but frankly YAML was a bad idea. (slightly less than XML sure)
I can understand people not wanting to write Makefile and most of them just ends up being .phony and a bunch of shell code in many cases.
Taskfile might be great for many use cases, but yes, why not just write a shell script. Almost every time I use a CI system, I end up falling back to just having it run a shell script, rather than relying on many of the build in features, because they are almost always a bit contrived. The same can be said for Makefile replacements, they try to predict what people need, and in the end it ends up being a fancy way of running a script.
Taskfile might be great for many use cases, but yes, why not just write a shell script. Almost every time I use a CI system, I end up falling back to just having it run a shell script, rather than relying on many of the build in features, because they are almost always a bit contrived. The same can be said for Makefile replacements, they try to predict what people need, and in the end it ends up being a fancy way of running a script.
You don't have to choose one. M approach is to keep most "tasks" in separate script files and use Task (or just, or make, or doit, or...) to call and glue them together.
XML is verbose. That’s the problem.
YAML is ridiculously complex, with a bunch of surprising behaviour.
I would take XML over YAML any day.
YAML is ridiculously complex, with a bunch of surprising behaviour.
I would take XML over YAML any day.
Basic XML is fine, maybe a bit verbose
But when you start getting into why are some things properties, or values, XSD or what not, then no.
Nobody has time for those thick XML books
But when you start getting into why are some things properties, or values, XSD or what not, then no.
Nobody has time for those thick XML books
You can choose to use XML without XSD. That's entirely up to you.
You cannot choose to use YAML without the string 'no' becoming boolean false.
Plenty of people complain about the complexity with XML, and there is a valid argument there. Using it as an argument in favour of YAML is kind of ridiculous.
You cannot choose to use YAML without the string 'no' becoming boolean false.
Plenty of people complain about the complexity with XML, and there is a valid argument there. Using it as an argument in favour of YAML is kind of ridiculous.
If I had the choice of yaml and XML I'd probably choose XML.
Scons using Pure Python Lang.
No more DSL.
https://scons.org/
It has cache facility to speed up re-builds. scons --implicit-cache I heard people say, Scons is slow, but it need not be if we use cache facility.
Some people say Meson (https://mesonbuild.com/) would be its successor, but I use scons for building my C/Python files. (ex: Python to .mpy files, asciidoc to pdf, html etc)
It has cache facility to speed up re-builds. scons --implicit-cache I heard people say, Scons is slow, but it need not be if we use cache facility.
Some people say Meson (https://mesonbuild.com/) would be its successor, but I use scons for building my C/Python files. (ex: Python to .mpy files, asciidoc to pdf, html etc)
Here I found an opinionated, but still very descriptive comparison of build tools the authors of SCons have experienced.
https://github.com/SCons/scons/wiki/FromMakeToScons
I used to swear by scons but it really collapses under its own weight when projects become larger.
At my work we used it for a fairly large source tree and no-change incremental builds would take a good minute for scons to figure out there's nothing to do.
These days we use something based on google blueprint which generates ninja build files.
At my work we used it for a fairly large source tree and no-change incremental builds would take a good minute for scons to figure out there's nothing to do.
These days we use something based on google blueprint which generates ninja build files.
My task runner is bash. After 15 years I've yet to find a system it's not installed on.
Bonus: you write your shell commands in a file that's aware of what's correct/incorrect.
Bonus: you write your shell commands in a file that's aware of what's correct/incorrect.
The command line entry points for your tooling need to be in the same language environment as everything else.
If your module in app/constants.py defines PORT which app/server.py serves and app/client.py connects to, how do you drop that same constant into your Makefile or docker-compose.yaml?
For me, literally everything including my docker, docker-compose, caddy, nginx, and of course the application code and all auxiliary scripts are presented as Python modules to solve this exact problem.
If your module in app/constants.py defines PORT which app/server.py serves and app/client.py connects to, how do you drop that same constant into your Makefile or docker-compose.yaml?
For me, literally everything including my docker, docker-compose, caddy, nginx, and of course the application code and all auxiliary scripts are presented as Python modules to solve this exact problem.
I use autoenv and a `./scripts` directory.
So my .env file just “includes” (automatically via autoenv) what’s in my scripts directory along with setting any environment variables when I’m jumping around my shell. Therefore I can just write a function like “dbup” or “devstart” and have those locally aliased to whatever they summon in the current project
So my .env file just “includes” (automatically via autoenv) what’s in my scripts directory along with setting any environment variables when I’m jumping around my shell. Therefore I can just write a function like “dbup” or “devstart” and have those locally aliased to whatever they summon in the current project
That's basically what I use Taskfile for. And ensuring that things are always executed starting in the root of the repository.
[deleted]
Rake[0] is still the best ‘make-like’ build tool/task runner I’ve used for general purpose stuff. The syntax is nice and it’s just Ruby which is a delight. I briefly used Mage (similar, but Go) and it was fine too.
[0]: https://github.com/ruby/rake
[0]: https://github.com/ruby/rake
Whenever I read "modern alternative", it's nearly always certain that the older tech is superior on multiple fronts:
Better documented, better designed and architected (taking more use-cases / edge-cases into account), composable, easier to debug, fewer dependencies.
Better documented, better designed and architected (taking more use-cases / edge-cases into account), composable, easier to debug, fewer dependencies.
and usually also far less bloated
Correction:
> Taskfile: It’s PHONY Makefile in Yaml
> Taskfile: It’s PHONY Makefile in Yaml
[deleted]
Seems less file-centric than gmake. You have to repeat filenames in task commands, and depend on tasks by name rather than depending on generated files.
Naming isn’t consistent with what make does and why it’s called Makefile.
Makefile is for make, which “makes files”.
You don’t “task files”
*file in general is wrong. Of course it’s a file
Makefile is for make, which “makes files”.
You don’t “task files”
*file in general is wrong. Of course it’s a file
There are weirder choices in the realm of build tools, such as CMakeLists.txt.
Looking good. But no thanks, I'm sticking to my awesome tup.
The syntax of Taskfile looks more verbose to me than Makefile, which is basically a shell script in disguise. To note also is that you praise Devenv because of it being based on Guix (Guile scheme), while you ditch Makefile which also embeds Guile scheme as the extension language (in latest versions).
Also to note, the opening argument, writing Readme files on how to build and use the software, has nothing to do with Makefiles at all. You could write a simple skeleton codegen that generates a Readme file skeleton or a template with those steps autogenerated in any language preferred, shell included, and run it as a Makefile target.