Qb – Zero-configuration build system to quickly build C/C++ projects(github.com)
github.com
Qb – Zero-configuration build system to quickly build C/C++ projects
https://github.com/codecat/qb
29 comments
> It seems aimed at making the trivial use cases easy, the normal use cases hard, and the hard use cases impossible.
I'd add that mainstream build systems already covered this sort of usecase. For instance, qmake used templates[1]. With CMake anyone can put together their own macro, and have config-free cmake projects. In fact, I've already worked on projects that followed this approach. I guess it's all fine and dandy when all you do is follow the happy path, but that's hardly the case.
[1] https://doc.qt.io/qt-5/qmake-variable-reference.html#templat...
I'd add that mainstream build systems already covered this sort of usecase. For instance, qmake used templates[1]. With CMake anyone can put together their own macro, and have config-free cmake projects. In fact, I've already worked on projects that followed this approach. I guess it's all fine and dandy when all you do is follow the happy path, but that's hardly the case.
[1] https://doc.qt.io/qt-5/qmake-variable-reference.html#templat...
As much as I love zero-configuration software, it seems implausible to do it for C/C++ as anything other than a gimmick. Even a simple program may need to link against a library or two to pull in some APIs unless absolutely every single function you use is in libc and it gets linked by default. If you want to target multiple platforms it gets worse because you may need things like per-platform defines set by the build system or per-platform .c/.h selection. Once you approach that level of complexity you should strongly consider just using existing build ecosystems like cmake - not because they're pleasant, but because the people maintaining them have identified a lot of the sharp edges that you're going to run into.
The performance of builds with this seems like it could also scale badly once your app gets big enough, since it doesn't seem to do incremental builds or avoid re-building individual .c files if a source file and its dependencies (headers) haven't changed.
Seems handy if you just want to fiddle around with a .c and .h file pair without writing a makefile, though!
The performance of builds with this seems like it could also scale badly once your app gets big enough, since it doesn't seem to do incremental builds or avoid re-building individual .c files if a source file and its dependencies (headers) haven't changed.
Seems handy if you just want to fiddle around with a .c and .h file pair without writing a makefile, though!
> Seems handy if you just want to fiddle around with a .c and .h file pair without writing a makefile, though!
Doesn't make already do this, with built-in targets?
Eg, with hello.c, hello.h, no Makefile:
Doesn't make already do this, with built-in targets?
Eg, with hello.c, hello.h, no Makefile:
make hello
there's a whole lot of default rules in GNU make: https://www.gnu.org/software/make/manual/html_node/Implicit-... but not sure they are portable
I imagine that using Zig to compile C++ projects is probably a better idea?
Can you actually write Zig build scripts to build C++ projects? If that’s possible then that might be a good option.
Yes, here is an example for a non-trivial mixed C/C++ project:
https://github.com/floooh/sokol-tools/blob/master/build.zig
Compared to cmake this means giving up IDE support like Xcode or Visual Studio though, it's really "just" a build system, not a meta-build system which can generate IDE project files.
The Zig build system also has some minor warts that don't matter much for a pure Zig project but do matter for C/C++ projects, mainly that build settings of dependencies (like header search paths or linker input) mostly don't automatically 'travel' upwards to the root, instead each dependency and root project needs to be configured on its own.
There are plenty of upsides too though, like much simplified cross-compilation, all in all it looks very promising.
https://github.com/floooh/sokol-tools/blob/master/build.zig
Compared to cmake this means giving up IDE support like Xcode or Visual Studio though, it's really "just" a build system, not a meta-build system which can generate IDE project files.
The Zig build system also has some minor warts that don't matter much for a pure Zig project but do matter for C/C++ projects, mainly that build settings of dependencies (like header search paths or linker input) mostly don't automatically 'travel' upwards to the root, instead each dependency and root project needs to be configured on its own.
There are plenty of upsides too though, like much simplified cross-compilation, all in all it looks very promising.
Can it also generate compile_commands.json automatically? That is the only thing you need for IDE/editor support nowadays.
AFAIK no, sounds like a good feature to have though.
Judging from https://zig.news/kristoff/make-zig-your-c-c-build-system-28g... , this should be possible. Considering Zig's features, I wonder if even the automated build logic in this Qb system couldn't be ported into a Zig build script completely.
I admit the C/C++ build setup has deterred me from prototyping at times. I used to solve this by keeping a sandbox project where I replace code in main().
> I admit the C/C++ build setup has deterred me from prototyping at times.
I don't understand how that can be a barrier to entry. There are plenty of free IDEs which already create ready-to-go projects, and build systems such as cmake make it quite trivial to setup a C++ project from scratch. Case in point:
I don't understand how that can be a barrier to entry. There are plenty of free IDEs which already create ready-to-go projects, and build systems such as cmake make it quite trivial to setup a C++ project from scratch. Case in point:
project(foo LANGUAGES CXX)
add_executable(foo main.cpp)
I got lazy and installed https://github.com/ryanmjacobs/c . (Other than that, I intend to learn Zig including its C++ build capabilities.)
Similiar story for me with rust and js. The cost for the first setup with everything dialed in just right (proper linker, comp flags...) is often deterring for the prototyping stage.
Maybe that's a case for boilerplates? Idk, though I really liked the idea of create-react-app and co.
Maybe that's a case for boilerplates? Idk, though I really liked the idea of create-react-app and co.
I once made something similar for my project as an excercise - it parsed the source code for #includes and figured out automatically what needs to be compiled. It was OKayish for my self-contained project which only used the standard library but it wouldn't work if you had to add third-party dependencies. Today I'd rather just use CMake.
The discussion here reminded me about a blog post I made back in 2011, concluding with meta-meta-build systems.
Unfortunately, Mike Acton didn't maintain the AltDevBlogADay blog due to technical (build system?) issues here a copy (warning, a wall of text!):
++++++++++++++
Erwin Coumans, 2011:
"This is a story about my search for a hassle-free cross-platform open source (meta-/meta-meta-) build system.
For our open source Bullet physics engine, I’ve been distributing the source code in a way that should make it as easy as possible to build out-of-the box. This means that for all supported platforms, the user (a developer who downloaded the Bullet SDK) should be able to download and unzip the zipfile or tarball on his machine, and get started as soon as possible. It should be hassle-free for the user but also for me, so I rather don’t manual updating too many files for each release.
Aside from all the different platforms, we need to also support various compilers, compiler settings and integrated developer environments (IDEs). Common platforms are Windows, Mac OSX, iPhone, Linux as well as PlayStation 3 and XBox 360. Some common compilers are Visual Studio compiler, GCC, SN Systems and LLVM.
Here are a couple of build systems and modifications that I tried or considered to try:
Visual Studio project files. Shipping manual generated visual studio files works hassle-free for the user, but isn’t hassle-free for myself especially if you want to support all versions from MSVC 6 up to MSVC 2010. Obviously it is also not cross-platform so it needs to be combined with another build system. For example autotools.
autotools: for most unix flavors, autotools does a great job. It is often already installed and is fairly hassle-free, as long as the user doesn’t run into version compatibilities of automake, autoconf or libtool. It is a bit of a hassle for me, because I’m not too familiar with autotools and can only provide most basic support. A bigger issue is that autotools doesn’t support IDEs such as Xcode.
Xcode, the environment for Mac OSX and iPhone, requires its own project files. Shipping manual generated project files is a burden, when dealing with versions and platform settings.
jam and msvcgen: jam is a fast build system and it is small enough to ship the entire jam source code together with my own project (it is just a few hundred kilobytes with no external dependencies). The msvcgen extension could autogenerate standalone Visual Studio project files from MSVC 6 onwards, so that was a nice extra. For several years I’ve been shipping the Bullet SDK with autotools + jam and msvcgen autogenerated Visual Studio projects for MSVC 6, 7 and 2005. No proper support for XCode, until I noticed cmake.
cmake: this is almost a dream meta build system but unfortunately it cannot properly generate distributable build systems. For example the generated Visual Studio project files only work on the local machine. I’ve been hacking the cmake source code so that it does generate distributable project files, and submitted a few patches back to cmake. Unfortunately, the main cmake developers don’t want to support distributable project file generation, and I felt they were very reluctant to discuss further changes to support distributable project files. For example, I need the option to avoid creating copy rules that depend on cmake being installed. Furthermore any environment variables should not be expanded in absolute paths but stay environment variables for shipping project files (for example DXSDK_DIR, ATISTREAMSDKROOT or NVSDKCOMPUTE_ROOT for some recent OpenCL and DirectCompute accelerated optional modules). This means that I need to keep on using and updating a forked version of cmake to generate and distribute MSVC projectfiles. As a side note: the wxWidgets project even created their own build system, just because of this cmake shortcoming. Another shortcoming of cmake is lack of multiple compiler support, so generating projects for both PlayStation PPU and SPU is not possible. After bringing up my build system issue on the PS3 Devnet forums, someone mentioned premake.
premake is a very promising meta build system and I keep on thinking of trying it out. I’ve heard people who create PS3 and XBox 360 project files, and it supports most other platforms. Premake can be optionally extended using Lua, but it is not a requirement for Lua to be installed, unlike the Python requirement for scons. Unfortunately, I’ve heard that premake has some issues with dependency tracking, has less features that cmake and it is not widely supported yet. This means that integrating it into another project might become a hassle.
This leaves me to either wait until premake matures and becomes more widely used, or writing my own meta-meta build system that can generate both cmake, premake and other project files."
++++++++++++++++
Not a whole lot changed in those 11 years. Bullet focus moved to robotics and reinforment learning and Python/PyBullet and with that: setuptools as one more build system, so you can trivially install using: pip install pybullet
Unfortunately, Mike Acton didn't maintain the AltDevBlogADay blog due to technical (build system?) issues here a copy (warning, a wall of text!):
++++++++++++++
Erwin Coumans, 2011:
"This is a story about my search for a hassle-free cross-platform open source (meta-/meta-meta-) build system.
For our open source Bullet physics engine, I’ve been distributing the source code in a way that should make it as easy as possible to build out-of-the box. This means that for all supported platforms, the user (a developer who downloaded the Bullet SDK) should be able to download and unzip the zipfile or tarball on his machine, and get started as soon as possible. It should be hassle-free for the user but also for me, so I rather don’t manual updating too many files for each release.
Aside from all the different platforms, we need to also support various compilers, compiler settings and integrated developer environments (IDEs). Common platforms are Windows, Mac OSX, iPhone, Linux as well as PlayStation 3 and XBox 360. Some common compilers are Visual Studio compiler, GCC, SN Systems and LLVM.
Here are a couple of build systems and modifications that I tried or considered to try:
Visual Studio project files. Shipping manual generated visual studio files works hassle-free for the user, but isn’t hassle-free for myself especially if you want to support all versions from MSVC 6 up to MSVC 2010. Obviously it is also not cross-platform so it needs to be combined with another build system. For example autotools.
autotools: for most unix flavors, autotools does a great job. It is often already installed and is fairly hassle-free, as long as the user doesn’t run into version compatibilities of automake, autoconf or libtool. It is a bit of a hassle for me, because I’m not too familiar with autotools and can only provide most basic support. A bigger issue is that autotools doesn’t support IDEs such as Xcode.
Xcode, the environment for Mac OSX and iPhone, requires its own project files. Shipping manual generated project files is a burden, when dealing with versions and platform settings.
jam and msvcgen: jam is a fast build system and it is small enough to ship the entire jam source code together with my own project (it is just a few hundred kilobytes with no external dependencies). The msvcgen extension could autogenerate standalone Visual Studio project files from MSVC 6 onwards, so that was a nice extra. For several years I’ve been shipping the Bullet SDK with autotools + jam and msvcgen autogenerated Visual Studio projects for MSVC 6, 7 and 2005. No proper support for XCode, until I noticed cmake.
cmake: this is almost a dream meta build system but unfortunately it cannot properly generate distributable build systems. For example the generated Visual Studio project files only work on the local machine. I’ve been hacking the cmake source code so that it does generate distributable project files, and submitted a few patches back to cmake. Unfortunately, the main cmake developers don’t want to support distributable project file generation, and I felt they were very reluctant to discuss further changes to support distributable project files. For example, I need the option to avoid creating copy rules that depend on cmake being installed. Furthermore any environment variables should not be expanded in absolute paths but stay environment variables for shipping project files (for example DXSDK_DIR, ATISTREAMSDKROOT or NVSDKCOMPUTE_ROOT for some recent OpenCL and DirectCompute accelerated optional modules). This means that I need to keep on using and updating a forked version of cmake to generate and distribute MSVC projectfiles. As a side note: the wxWidgets project even created their own build system, just because of this cmake shortcoming. Another shortcoming of cmake is lack of multiple compiler support, so generating projects for both PlayStation PPU and SPU is not possible. After bringing up my build system issue on the PS3 Devnet forums, someone mentioned premake.
premake is a very promising meta build system and I keep on thinking of trying it out. I’ve heard people who create PS3 and XBox 360 project files, and it supports most other platforms. Premake can be optionally extended using Lua, but it is not a requirement for Lua to be installed, unlike the Python requirement for scons. Unfortunately, I’ve heard that premake has some issues with dependency tracking, has less features that cmake and it is not widely supported yet. This means that integrating it into another project might become a hassle.
This leaves me to either wait until premake matures and becomes more widely used, or writing my own meta-meta build system that can generate both cmake, premake and other project files."
++++++++++++++++
Not a whole lot changed in those 11 years. Bullet focus moved to robotics and reinforment learning and Python/PyBullet and with that: setuptools as one more build system, so you can trivially install using: pip install pybullet
> cmake: this is almost a dream meta build system but unfortunately it cannot properly generate distributable build systems. For example the generated Visual Studio project files only work on the local machine.
can Visual Studio project files even be portable ? before CMake, I remember having to go edit variables in a FUCKING GUI to indicate to Visual Studio where to find OpenCV, Boost, etc. For organizations it isn't an issue because everyone has the same FS tree and the libraries in the same place so the VS files can have hardcoded paths, but that just does not work for stuff where someone may have opencv in c:\opencv\3.4 and someone else in d:\libs\cv\opencv\ or c:\program files (x86)\opencv\...
can Visual Studio project files even be portable ? before CMake, I remember having to go edit variables in a FUCKING GUI to indicate to Visual Studio where to find OpenCV, Boost, etc. For organizations it isn't an issue because everyone has the same FS tree and the libraries in the same place so the VS files can have hardcoded paths, but that just does not work for stuff where someone may have opencv in c:\opencv\3.4 and someone else in d:\libs\cv\opencv\ or c:\program files (x86)\opencv\...
MSVC project files can be portable, if you are careful about using relative paths, and use environment variables for CUDA, PYTHON_PATH, OpenCV etc (if needed).
I remember this article, thanks for the memory :)
I would disagree that not much have changed though I agree that I would prefer if we were positioned even better. For example the dominant use of CMake has made using other projects as a dependency easier. there are more package managers than there used to be.
I would disagree that not much have changed though I agree that I would prefer if we were positioned even better. For example the dominant use of CMake has made using other projects as a dependency easier. there are more package managers than there used to be.
Agreed, I'm glad cmake became more widely used.
For Python I prefer using setuptools without further dependency to cmake, to make it easier for installation.
Also, I was at Google, which used blaze/bazel mostly.
Curious how the C++ world looks like in 10 years from now, or maybe we moved to Rust?
For Python I prefer using setuptools without further dependency to cmake, to make it easier for installation.
Also, I was at Google, which used blaze/bazel mostly.
Curious how the C++ world looks like in 10 years from now, or maybe we moved to Rust?
[ FD: CMake developer, but this is just my personal opinion. ]
I find Bazel, Buck, and other similar tools from other FAANG-like companies to mostly target monorepos because they expect to control everything from the toolchain up. Dependencies get vendored and use of things from outside the repo is difficult and actively discouraged. It solves its problems, but that's not the realm I work in (or, frankly, care a lot about as these projects have enough zeroes backing their development anyways).
IME, these kinds of things doesn't really happen in FOSS codebases organically. I also don't see Rust taking over much of the C or C++ world in any kind of replacement way. Rust will become more popular and as projects die over time, they'll just be more likely to have been in C or C++ than Rust. I have no doubts that Rust will have this time in its lifecycle too. But the languages will never truly die.
The best thing that could happen to C and C++ ecosystems is for projects to stop caring about and relying on build flags, `-D`-directed configuration, and flag soup-based dependency management (`pkg-config`). An in-language way to do conditional compilation based on concrete queries (e.g., ask the compiler about itself, the target, some dependency info) would do wonders here to replace the inscrutable black box that is the preprocessor and its magic names. It would to enable projects to just say "I use X" and have that mean something other than work for build systems to discover, extract, and then communicate to the compiler about what that actually means through each one's local command line flag dialect.
But I don't expect that to ever see the light of day before C++32 and C40 nevermind being something one can rely on until after 2050 or so.
I find Bazel, Buck, and other similar tools from other FAANG-like companies to mostly target monorepos because they expect to control everything from the toolchain up. Dependencies get vendored and use of things from outside the repo is difficult and actively discouraged. It solves its problems, but that's not the realm I work in (or, frankly, care a lot about as these projects have enough zeroes backing their development anyways).
IME, these kinds of things doesn't really happen in FOSS codebases organically. I also don't see Rust taking over much of the C or C++ world in any kind of replacement way. Rust will become more popular and as projects die over time, they'll just be more likely to have been in C or C++ than Rust. I have no doubts that Rust will have this time in its lifecycle too. But the languages will never truly die.
The best thing that could happen to C and C++ ecosystems is for projects to stop caring about and relying on build flags, `-D`-directed configuration, and flag soup-based dependency management (`pkg-config`). An in-language way to do conditional compilation based on concrete queries (e.g., ask the compiler about itself, the target, some dependency info) would do wonders here to replace the inscrutable black box that is the preprocessor and its magic names. It would to enable projects to just say "I use X" and have that mean something other than work for build systems to discover, extract, and then communicate to the compiler about what that actually means through each one's local command line flag dialect.
But I don't expect that to ever see the light of day before C++32 and C40 nevermind being something one can rely on until after 2050 or so.
> An in-language way to do conditional compilation...
Nim has some of this. But of course, it is not C.
Nim has some of this. But of course, it is not C.
This exactly matches my own research from a few years ago.
I also landed on Premake, which did work well enough for me at the time and remains the leading contender.
That said, I've since been keeping a keen eye on the Ziglang project as they seem to be the most likely (only?!) real candidate for a minimal, cross target build environment.
I also landed on Premake, which did work well enough for me at the time and remains the leading contender.
That said, I've since been keeping a keen eye on the Ziglang project as they seem to be the most likely (only?!) real candidate for a minimal, cross target build environment.
We can also try https://github.com/xmake-io/xmake
go code to compile c/c++ code? What is the reason for this complexity?
What does this project solve that existing package managers do not? (eg compared to vcpkg, spack or hunter)
After a week of fighting with vcpkg, i really doubt that this project is worth anyone's time.
After a week of fighting with vcpkg, i really doubt that this project is worth anyone's time.
Go is surprisingly good in this role. Super easy cross compilation, good platform support, and easy distribution means that it's very easy to get the tool where you need it. For a lot of file munging, Go is serviceable.
To my understanding, the problem is how to easily and consistently build several versions of software stacks(libraries/executables) with different compilers(primarily MSVC/icc/gcc/clang) on windows/linux/osx for windows/linux/osx/android.
I do not see how go (or any other tool) is more or less suited to solving this problem and given the variety of existing tools (and their failure to do it), makes me suspect that this project is merely an alpha stage effort in solving this problem and mostly an exercise in go.
I do not see how go (or any other tool) is more or less suited to solving this problem and given the variety of existing tools (and their failure to do it), makes me suspect that this project is merely an alpha stage effort in solving this problem and mostly an exercise in go.
[deleted]
For example, "exe", "dll" and "lib" are types of projects, not of individual files: no apparent way to have multiple and heterogeneous end products (e.g dynamic libraries + static libraries + test executables + something not C/C++ like Doxygen) or to define variants like compiling the same program as a statically linked big executable or as a thin executable and a bundle of dynamic libraries.
Lack of documentation is another serious problem. For example, how do I pass exotic GCC flags and how can I ensure that qb uses GCC instead of MSVC?
"Zero configuration" in serious systems means self-configuration, e.g. automatically finding and inspecting available compilers, to provide added value over completely manual configuration; denying configuration, or at least not thinking of it because no toy example has needed it so far, isn't very productive for the end user.