An Introduction to Modern CMake(cliutils.gitlab.io)
cliutils.gitlab.io
An Introduction to Modern CMake
https://cliutils.gitlab.io/modern-cmake/
42 comments
Do you know of anything worth reading for CMake?
The official docs are too sparse to be useful to me and the results of googling mostly give stale advice.
I also have being easing into CMake recently - I was initially impressed as getting a "hello world" example was simple and the CMakeLists.txt was clear. As the project has progressed - simple additions like unit tests, external dependencies, specific aspects for debug builds, handling basic cross platform differences, etc. have made the CMakeLists.txt almost unreadable and really quite brittle.
I'm not quite at the stage of abandoning it but I have wondered whether an old fashioned Makefile wouldn't be simpler. At least there is a logic, if convoluted, to make and the documentation is top class. CMake seems to lack a clear design and just seems like hack on hack but this could be my ignorance.
I've no real idea of what I'm dealing with when working with CMake - it's wears declarative clothing but actually it isn't at all. The ordering of CMakeLists.txt is significant and you have to be aware of whether what you add will be "executed" during cmake or gets executed with cmake --build.
The official docs are too sparse to be useful to me and the results of googling mostly give stale advice.
I also have being easing into CMake recently - I was initially impressed as getting a "hello world" example was simple and the CMakeLists.txt was clear. As the project has progressed - simple additions like unit tests, external dependencies, specific aspects for debug builds, handling basic cross platform differences, etc. have made the CMakeLists.txt almost unreadable and really quite brittle.
I'm not quite at the stage of abandoning it but I have wondered whether an old fashioned Makefile wouldn't be simpler. At least there is a logic, if convoluted, to make and the documentation is top class. CMake seems to lack a clear design and just seems like hack on hack but this could be my ignorance.
I've no real idea of what I'm dealing with when working with CMake - it's wears declarative clothing but actually it isn't at all. The ordering of CMakeLists.txt is significant and you have to be aware of whether what you add will be "executed" during cmake or gets executed with cmake --build.
I didn't say these partial treatments are not worth reading/watching. On the contrary - they're pretty nice. You just have the mentally prepare yourself of "expanding your consciousness" gradually, that's all.
Those things you mention - cross-platform differences, unit tests, debug builds etc. - require a bit of getting used to, but once you do, the syntax is not bad and I don't feel I'm suffering. Certainly not feeling the brittleness.
And - no, an old-fashioned Makefile can't even _begin_ to cut it. Not by a long-shot. (And that's not even mentioning the fact that it's mostly irrelevant on Windows.)
Those things you mention - cross-platform differences, unit tests, debug builds etc. - require a bit of getting used to, but once you do, the syntax is not bad and I don't feel I'm suffering. Certainly not feeling the brittleness.
And - no, an old-fashioned Makefile can't even _begin_ to cut it. Not by a long-shot. (And that's not even mentioning the fact that it's mostly irrelevant on Windows.)
This. A 1000 times.
Nobody really wants to focus too much on build configurations when working on things, they should 'just work'.
And CMake, although I am using for a long time already, is not the offering this 'just works' experience I am afraid. It is so easy to introduce side-effects, and figuring out the intended way to include a project can be a big pain when documentation is sparse. Sometimes I wished I had a CMake debugger. And with modern CMake I feel like this got worse to some degree.
Nobody really wants to focus too much on build configurations when working on things, they should 'just work'.
And CMake, although I am using for a long time already, is not the offering this 'just works' experience I am afraid. It is so easy to introduce side-effects, and figuring out the intended way to include a project can be a big pain when documentation is sparse. Sometimes I wished I had a CMake debugger. And with modern CMake I feel like this got worse to some degree.
CMake may "just work" without you investing much/any effort - in the future.
For now, the situation is that if you invest in your CMake - blood sweat and tears - then the build itself will "just work" on many/most/all platforms. Which is already pretty good - compared to being stuck with Makefiles.
Also, CMake is essentially its own debugger, because almost all state is just a bunch of string variables.
For now, the situation is that if you invest in your CMake - blood sweat and tears - then the build itself will "just work" on many/most/all platforms. Which is already pretty good - compared to being stuck with Makefiles.
Also, CMake is essentially its own debugger, because almost all state is just a bunch of string variables.
> * There are a half-dozen different ways to have your project depend on on other code...
And this "tutorial" skips mentioning this and bluntly suggests the dummiest - git submodules.
then what's the point of cmake? won't legacy gnu make suffice? nothing is perfect including Make, but it seems gnu Make can put all its doc into one html manual at least. KISS.
https://www.gnu.org/software/make/manual/make.html
https://www.gnu.org/software/make/manual/make.html
Writing Makefiles by hand is great for small projects. But make becomes unmanageable for large projects, and is bad at dealing with system and compiler differences.
It is not "great" for small projects, because:
1. Small projects also have dependencies
2. Small projects need to be build on multiple platforms too
3. It is more verbose, and at the same time more obscure, than writing a CMakeLists.txt (typically).
1. Small projects also have dependencies
2. Small projects need to be build on multiple platforms too
3. It is more verbose, and at the same time more obscure, than writing a CMakeLists.txt (typically).
CMake is not a replacement for Make; it generates Makefiles for you (or ninja files, or whatever-Microsoft-uses files). In that sense it's like a more modern and less crufty GNU autotools which can have different kinds of output build systems.
cross platform mostly works.
What's wrong with just a real language as a build system? Speaking for Scala, everyone just uses sbt, which is just Scala. Nothing special about it. I'm not saying you should use Scala to compile your C project, but maybe we could settle on some Lisp dialect or something for compilation, regardless of the language? That would seem ideal to me. Programmers in general have this problem of creating non-turing complete shit to solve problems that require turing completeness. And sometimes it gets worse: first a non turing complete language is created and then when it finally becomes necessary, some loops and shit are grafted on (or a shell script is called in the middle). It's a mess.
Eventually those scripts will turn into full fledged programs that no one knows what they do, specially since most of these build programming languages lack debugging features.
Don't use cmake for new projects. You'll regret it. Try meson instead:
https://mesonbuild.com/
https://mesonbuild.com/
Why would I want to use Meson over CMake? Why would I regret using CMake?
I had a look at the Meson FAQ and it says:
> What is the correct way to use threads (such as pthreads)? > > thread_dep = dependency('threads') > > People coming from Autotools or CMake want to do this by looking for libpthread.so manually.
... which is false. We do:
> find_package(Threads REQUIRED) > target_link_library(my_app Threads::Threads)
and Bob's your uncle.
I had a look at the Meson FAQ and it says:
> What is the correct way to use threads (such as pthreads)? > > thread_dep = dependency('threads') > > People coming from Autotools or CMake want to do this by looking for libpthread.so manually.
... which is false. We do:
> find_package(Threads REQUIRED) > target_link_library(my_app Threads::Threads)
and Bob's your uncle.
Doesn't meson bring in all of python3 as a dependency for a project? Similar to scons.
(It's been a while since I did semi-serious C++, maybe nowadays we're playing a bit more loose with deps and polyglot setups)
(It's been a while since I did semi-serious C++, maybe nowadays we're playing a bit more loose with deps and polyglot setups)
Does that matter? CPython comes preinstalled on basically every Linux distribution as a dependency for some other program these days.
Unfortunately even if you're starting a new project chances are that your dependencies are using cmake and they expect their dependencies to use cmake... in the end of the day you spend a lot of time fixing the upstream projects build files.
More than that CMake at this point is the ""standard"" and there are many others with a similar share of the market and selling points, it makes it harder to pick one that is not CMake.
More than that CMake at this point is the ""standard"" and there are many others with a similar share of the market and selling points, it makes it harder to pick one that is not CMake.
Meson supports CMake subprojects:
https://mesonbuild.com/CMake-module.html#cmake-subprojects
CMake subprojects was far from reliable last time I tested it. But this might also have been due to meson targeting unreleased CMake versions...
If your dependencies use CMake or not doesn't really matter, you still end up writing some FindX.cmake file.
(Which is a whole other thing with this "modern CMake" by the way: only half of them that come with CMake are "upgraded" to modern and if you don't want to require the absolute latest version, you end up having to check what the FindX script did or redefine stuff manually)
(Which is a whole other thing with this "modern CMake" by the way: only half of them that come with CMake are "upgraded" to modern and if you don't want to require the absolute latest version, you end up having to check what the FindX script did or redefine stuff manually)
I will consider it when QtCreator, Visual Studio, Eclipse CDT, Clion give the same support as they offer currently for CMake based projects.
Recent versions of CMake have also been growing the support for CUDA. There's still a number of edge cases (e.g. passing arguments to the device linker) that have open tickets on their bugtracker, but there's more than enough stable functionality now for CUDA projects and generally a number of relevant improvements with each new release.
We started using it about 6 months ago for a cross platform CUDA project, and have found it to be a big improvement over how we previously managed similar CUDA projects (e.g. manually). I'm especially a fan of it's ability to download external CMake project dependencies. As others have mentioned, some of the documentation is a little unclear for particular edge cases though so it takes some familiarisation.
We started using it about 6 months ago for a cross platform CUDA project, and have found it to be a big improvement over how we previously managed similar CUDA projects (e.g. manually). I'm especially a fan of it's ability to download external CMake project dependencies. As others have mentioned, some of the documentation is a little unclear for particular edge cases though so it takes some familiarisation.
I always wondering why CMake can't properly support filenames with white space. Is there a reason why no one can fix it rather than recommend CMake users to avoid white space?
For full-fledged CMake project following best practices, see C++ Weeklys CMake tutorial:
https://www.youtube.com/watch?v=YbgH7yat-Jo
For C++ starter project, see: https://github.com/lefticus/cpp_starter_project
(I agree with nearly everything except generating the conanfile)
https://www.youtube.com/watch?v=YbgH7yat-Jo
For C++ starter project, see: https://github.com/lefticus/cpp_starter_project
(I agree with nearly everything except generating the conanfile)
Or we can also try xmake. https://xmake.io/
The dev branch version is already as fast as ninja now, and there is no additional step to generate a makefile/build.ninja.
I sure wish Qmake had won over CMake.
Nice to see a CMake guide here, but I don't see any discussion of generators at a cursory glance, even in the "IDEs" section.
CMake is used a lot in embedded development with large or cross-platform projects, like an RTOS or the famous ESP-IDF.
Unfortunately, the embedded development space is a messy hodge-podge of IDEs. There are shiny commercial ones like Keil which support most kinds of devices, but most vendors also provide a free Eclipse-based option which targets their chips, and there are usually flavors of GCC for the adventurous.
CMake does have a nice way to add support for a new IDE or compiler in a modular way; you write a "generator":
https://cmake.org/cmake/help/latest/manual/cmake-generators....
My pet peeve is, it seems like nobody does that. Instead, they try to re-write all of the logic which generates build files either using CMake's internal logic, or one-off scripts which get called from CMake.
If (for example) Espressif provided a CMake generator for their style of `xtensa-elf-gcc` Makefiles, everybody could use that to quickly add build support for ESP32s to their CMake systems. What they use now doesn't seem to be very easy to re-use. Ditto for companies like ST/TI/Microchip/etc; if they provided CMake generators for their Eclipse IDEs, people wouldn't have to keep re-inventing the wheel and making it square-shaped.
It makes me dread using embedded libraries that use CMake-based build systems, even though I really like CMake as a cross-platform build system.
CMake is used a lot in embedded development with large or cross-platform projects, like an RTOS or the famous ESP-IDF.
Unfortunately, the embedded development space is a messy hodge-podge of IDEs. There are shiny commercial ones like Keil which support most kinds of devices, but most vendors also provide a free Eclipse-based option which targets their chips, and there are usually flavors of GCC for the adventurous.
CMake does have a nice way to add support for a new IDE or compiler in a modular way; you write a "generator":
https://cmake.org/cmake/help/latest/manual/cmake-generators....
My pet peeve is, it seems like nobody does that. Instead, they try to re-write all of the logic which generates build files either using CMake's internal logic, or one-off scripts which get called from CMake.
If (for example) Espressif provided a CMake generator for their style of `xtensa-elf-gcc` Makefiles, everybody could use that to quickly add build support for ESP32s to their CMake systems. What they use now doesn't seem to be very easy to re-use. Ditto for companies like ST/TI/Microchip/etc; if they provided CMake generators for their Eclipse IDEs, people wouldn't have to keep re-inventing the wheel and making it square-shaped.
It makes me dread using embedded libraries that use CMake-based build systems, even though I really like CMake as a cross-platform build system.
I've built and maintained a generator for one of those vendor-specific eclipse IDEs. It was an absolute nightmare I don't recommend to anyone, but not for any of the reasons you'd imagine. CMake itself is easy to build / read. Implement a few functions and you'll quickly have something workable. The issues came from elsewhere:
* None of the graphical CMake tools allow you to select the CMake binary, so you'd get users using the wrong binary and not seeing the generator mentioned in docs.
* When things break elsewhere, it's "your responsibility" to fix it in the generator. People put meaningless, nonsensical junk in their CMake files and expect it to output something that builds. Doesn't matter that the build script or the IDE are fundamentally broken, people just want the output so they can get back to their work. They don't want to yak shave.
* Vendor tooling code quality is somehow even worse than the actual product they provide. Dropping every 4096th character from stdin? Why not? Not generating debug info for files with uppercase letters in the filename? Sure. I won't name names, but they're a billion dollar EDA vendor whose initials are MG.
Avoid CMake generators like the plague unless you want to spend the next bit of your life debugging your vendor tools and supporting every workflow quirk your coworkers have.
* None of the graphical CMake tools allow you to select the CMake binary, so you'd get users using the wrong binary and not seeing the generator mentioned in docs.
* When things break elsewhere, it's "your responsibility" to fix it in the generator. People put meaningless, nonsensical junk in their CMake files and expect it to output something that builds. Doesn't matter that the build script or the IDE are fundamentally broken, people just want the output so they can get back to their work. They don't want to yak shave.
* Vendor tooling code quality is somehow even worse than the actual product they provide. Dropping every 4096th character from stdin? Why not? Not generating debug info for files with uppercase letters in the filename? Sure. I won't name names, but they're a billion dollar EDA vendor whose initials are MG.
Avoid CMake generators like the plague unless you want to spend the next bit of your life debugging your vendor tools and supporting every workflow quirk your coworkers have.
Interesting perspective, thanks for sharing your experience! That doesn't sound too much worse than working with modern CMake build systems, but it also doesn't sound mucn better.
>People put meaningless, nonsensical junk in their CMake files and expect it to output something that builds. Doesn't matter that the build script or the IDE are fundamentally broken, people just want the output so they can get back to their work.
When you put it that way, maybe that's my real pet peeve with CMake in big projects :)
>People put meaningless, nonsensical junk in their CMake files and expect it to output something that builds. Doesn't matter that the build script or the IDE are fundamentally broken, people just want the output so they can get back to their work.
When you put it that way, maybe that's my real pet peeve with CMake in big projects :)
I think the kind of embedded development you are doing is not the same as I am doing.
Where I work we were looking at CMake and were "forced" to start learning it when we embarked on a pilot project that needed to compile WebKit for several embedded platforms (the kind that can usefully use WebKit to display something). There were at least 3 ways to build WebKit but someone before me chose CMake.
Fast-forward to today, and our main build system is actually a wrapper around buildroot, which can use CMake projects for the packages if desired. We use CMake for one of our Linux kernel modules and for our middleware. We don't use generators, or to be precise we only use the Unix Makefile generator. For the CMake projects we adapt the compilers of our various platforms by using toolchain files, which we write ourselves. Toolchain files are chosen by an option to the initial CMake command and primarily describe the location and preferred settings of the toolchain's tools.
There is a temptation, a very practical temptation, to merge toolchain considerations with the ways targets vary, and handle both subjects with the toolchain file. We have resisted that temptation across over a dozen target platforms. We create platform files separately and we have devised a convention for specifying which one to use.
Our developers all have their own IDE/non-IDE ways of working with the source code and the build system has not asked to integrate in any way with that.
A lot of our developers work primarily within the middleware, and we have a documented way to make that work easy. Others have to cope with buildroot as much as with the build system of the buildroot package they are working on, and that is not refined yet.
Where I work we were looking at CMake and were "forced" to start learning it when we embarked on a pilot project that needed to compile WebKit for several embedded platforms (the kind that can usefully use WebKit to display something). There were at least 3 ways to build WebKit but someone before me chose CMake.
Fast-forward to today, and our main build system is actually a wrapper around buildroot, which can use CMake projects for the packages if desired. We use CMake for one of our Linux kernel modules and for our middleware. We don't use generators, or to be precise we only use the Unix Makefile generator. For the CMake projects we adapt the compilers of our various platforms by using toolchain files, which we write ourselves. Toolchain files are chosen by an option to the initial CMake command and primarily describe the location and preferred settings of the toolchain's tools.
There is a temptation, a very practical temptation, to merge toolchain considerations with the ways targets vary, and handle both subjects with the toolchain file. We have resisted that temptation across over a dozen target platforms. We create platform files separately and we have devised a convention for specifying which one to use.
Our developers all have their own IDE/non-IDE ways of working with the source code and the build system has not asked to integrate in any way with that.
A lot of our developers work primarily within the middleware, and we have a documented way to make that work easy. Others have to cope with buildroot as much as with the build system of the buildroot package they are working on, and that is not refined yet.
Not sure you want to tie yourself to a buggy inconsistent IDE and force that upon all developers.
Why wouldn't you do it with CMakes internal logic? Isn't that the point, especially since it is cross-platform.
Why wouldn't you do it with CMakes internal logic? Isn't that the point, especially since it is cross-platform.
I don't think I understand the question. Generators are CMake's way of creating IDE-specific build files, and I wish that people used them instead of writing their own scripts to create IDE-specific build files from CMake.
Generators are there so that you don't get tied to any one IDE; you select a different generator when you want to use a different IDE or compiler, but your CMake logic can largely stay the same besides that.
If I were a cynic, I would say that the vendors probably don't mind it being difficult to interoperate with their competitors' products, so there's little incentive to support things like that.
Generators are there so that you don't get tied to any one IDE; you select a different generator when you want to use a different IDE or compiler, but your CMake logic can largely stay the same besides that.
If I were a cynic, I would say that the vendors probably don't mind it being difficult to interoperate with their competitors' products, so there's little incentive to support things like that.
Related from 2018: https://news.ycombinator.com/item?id=17897685
This has the same kind of infuriating quality that most cmake documentation does. It explains in detail how to do obvious stuff but has nothing to say about what the execution model behind the syntax is.
I mean, if you learn classic make, one of the very first things you learn is "this is a target; it's just a file; the stuff on the right are dependencies; the commands below will be run whenever the dependencies are newer than the target". Now, there are implications to learn, but that is the model, and it's obvious: make runs programs to create files when the stuff needed to create the file changes.
Now, how do I do that in cmake? It's almost not even here! It's all about C++ files and libraries and how to define variables and write functions and include other cmake files. But what is it going to do with that stuff? It's all just magic. Type it like this and it'll work. Cmake understands your compiler better than you do.
What docs there are on creating a custom target are buried in two paragraphs inside a subsection named "Running a command at build time". And it barely tells you how to do this at all (this being a topic I've actually struggled with in the past). In particular it fails to mention that a target is is not the same thing as a file in cmake, and that this has implications that are likely to be surprising to make experts.
I mean, if you learn classic make, one of the very first things you learn is "this is a target; it's just a file; the stuff on the right are dependencies; the commands below will be run whenever the dependencies are newer than the target". Now, there are implications to learn, but that is the model, and it's obvious: make runs programs to create files when the stuff needed to create the file changes.
Now, how do I do that in cmake? It's almost not even here! It's all about C++ files and libraries and how to define variables and write functions and include other cmake files. But what is it going to do with that stuff? It's all just magic. Type it like this and it'll work. Cmake understands your compiler better than you do.
What docs there are on creating a custom target are buried in two paragraphs inside a subsection named "Running a command at build time". And it barely tells you how to do this at all (this being a topic I've actually struggled with in the past). In particular it fails to mention that a target is is not the same thing as a file in cmake, and that this has implications that are likely to be surprising to make experts.
Thank you! I cannot count the number of times I have had a CMake problem and had the usual inclination to try to understand the framework behind it, only to be stonewalled by the seemlingly complete lack of context in the documentation. I have given up on it time and time again. Somehow my project manages to barely get over every CMake hurdle but I still dream about the day when I would actually understand it, and have the same feeling of control that I usually strive for.
I'm glad I'm not the only one finding cmake's "magic" approach to software building totally unhelpful, and more on the problem rather than solution side of things. You know there's gone something out of hand if your very meta build tool's build itself takes much longer than the project you wanted to build in the first place.
Excerpt:
"The --trace option will print every line of CMake that is run. Since this is very verbose, CMake 3.7 added --trace-source="filename", which will print out every executed line of just the file you are interested in when it runs. If you select the name of the file you are interested in debugging (usually by selecting the parent directory when debugging a CMakeLists.txt, since all of those have the same name), you can just see the lines that run in that file. Very useful!"
"The --trace option will print every line of CMake that is run. Since this is very verbose, CMake 3.7 added --trace-source="filename", which will print out every executed line of just the file you are interested in when it runs. If you select the name of the file you are interested in debugging (usually by selecting the parent directory when debugging a CMakeLists.txt, since all of those have the same name), you can just see the lines that run in that file. Very useful!"
Something I've noticed is that there is no such thing as a single good introduction:
* CMake itself is changing. While the "revolution" of modern CMake is upon us already from version 3.0 (or a little earlier), a bunch of thing change with versions 3.12, 3.13 etc. See: https://youtu.be/y7ndUhdQuU8
* Different treatments - a mini-website, PDF presentation or conference video - focus on different aspects of work with CMake, but you probably need most or all of them in focus in order to actually do what you need done. That's how you can see videos entitled "More Modern CMake" and "Oh No! More Modern CMake"...
* Setting up the CMake machinery to have your repository/project installed is the opposite of straightforward, and very easy to get confused about.
* There are a half-dozen different ways to have your project depend on on other code: External projects, package management CMake modules, relying on other projects being installed in a CMake-idiomatic way and then find_package working, and more. Regardless of which you think is the right one, you'll need to live with others doing something else.
So, do follow the link, but don't expect it be sufficient.