Show HN: Using C++23 to get proper crash logs in C++ programs(github.com)
github.com
Show HN: Using C++23 to get proper crash logs in C++ programs
https://github.com/TylerGlaiel/Crashlogs
94 comments
Don't forget `WIN32_EXTRA_LEAN`! Still no idea what that does/did.
For those curious about WIN32_LEAN_AND_MEAN - it reduces compile time by not auto-including a number of windows headers: https://devblogs.microsoft.com/oldnewthing/20091130-00/?p=15...
For those curious about WIN32_LEAN_AND_MEAN - it reduces compile time by not auto-including a number of windows headers: https://devblogs.microsoft.com/oldnewthing/20091130-00/?p=15...
Do you mean VC_EXTRALEAN? Or is WIN32_EXTRA_LEAN also a thing?
First time I hear of VC_EXTRALEAN, always used WIN32_EXTRA_LEAN.
The bad news is that WIN32_EXTRA_LEAN has no effect, you have to define VC_EXTRALEAN if you want to exclude extra stuff from the MFC and VC headers.
The good news is also that it has no effect. :)
You can verify this for yourself by grepping the Visual Studio headers for VC_EXTRALEAN and WIN32_EXTRA_LEAN.
The good news is also that it has no effect. :)
You can verify this for yourself by grepping the Visual Studio headers for VC_EXTRALEAN and WIN32_EXTRA_LEAN.
But it did in 2005?
> VC_EXTRALEAN defines WIN32_LEAN_AND_MEAN and a number of NOservice definitions, such as NOCOMM and NOSOUND.
> https://gamedev.net/forums/topic/367942-win32_lean_and_mean-...
> VC_EXTRALEAN defines WIN32_LEAN_AND_MEAN and a number of NOservice definitions, such as NOCOMM and NOSOUND.
> https://gamedev.net/forums/topic/367942-win32_lean_and_mean-...
WIN32_EXTRA_LEAN != WIN32_LEAN_AND_MEAN
As always, Raymond Chen wrote about it! https://devblogs.microsoft.com/oldnewthing/20091130-00/?p=15...
Its also my favorite btw.
Its also my favorite btw.
We've had to use this at my work a couple times. I forget the exact reasoning, but IIRC if you're using including parts of the Win32 API, you get some things that would stomp on C or C++ names, which is bad. Macros like that one prevent loading things you don't want.
One example was min and max - Win32 includes those which messes with trying to use std::min and std::max.
One example was min and max - Win32 includes those which messes with trying to use std::min and std::max.
There's NOMINMAX, NOGDI, etc. for that.
not only min, max, other "fun" ones to debug are "near", "far", "small"
I think we had to use that when importing FDI.h (the CAB-extracting part of winapi) to avoid it bringing more legacy cruft.
I've always been a fan of _CRT_SECURE_NO_WARNINGS, personally.
Probably because you're the same generation as the dude at MS, whereas you want the kids at Github to get off your lawn?
[deleted]
If you don't care about exotica like async or signal safety, and just need to see the callstack from arbitray points, this can do the job without C++23:
https://github.com/Ardour/ardour/blob/master/libs/pbd/stackt...
(2 different implementations, one for POSIX-y systems with the execinfo.h header, and one for Windows)
The demange() function is elsewhere.
https://github.com/Ardour/ardour/blob/master/libs/pbd/stackt...
(2 different implementations, one for POSIX-y systems with the execinfo.h header, and one for Windows)
The demange() function is elsewhere.
Ime the execinfo.h backtraces are unfortunately not that useful in practice, due to being unable to resolve symbol names of static functions. But FWIW, you can use it for async signals with the _fd variant.
There are parts of C++ standard library that no one should ever use.
The examples are: regex, iostreams, locale...
My main concern - this can also become such a dead weight.
The examples are: regex, iostreams, locale...
My main concern - this can also become such a dead weight.
What's wrong with std::regex? Seems to work fine for me.
And iostreams - they're not great. Bad programming UI, poor performance, etc. Issues abound. But should you never use them? What do you use instead? *printf methods have lots of issues too. And so does depending on Boost::format. And so does writing your own Logger/wrapping code (which is what everyone does AFAICT).
locale is bad though
And iostreams - they're not great. Bad programming UI, poor performance, etc. Issues abound. But should you never use them? What do you use instead? *printf methods have lots of issues too. And so does depending on Boost::format. And so does writing your own Logger/wrapping code (which is what everyone does AFAICT).
locale is bad though
Well, `std::regex` is literally orders of magnitude slower than other common regex libraries found in JS, Python, Perl, C, etc. It allocates a ton, is poorly implemented, and can never be fixed due to ABI constraints. The entire <regex> subsystem is a mess and should have never been standardized as is.
While that is true it can still be a good thing to use on a non-perofrmance critical path instead of adding a dependency on an external library.
Maybe it shouldn't be there and some better thing should be there, but given it exists ... as long as one is aware of alternatives using it is fine.
Maybe it shouldn't be there and some better thing should be there, but given it exists ... as long as one is aware of alternatives using it is fine.
There's "not fast but usable enough" of course, but I would never ship std::regex code on any user facing software. Forget realtime, std::regex fails to be interactive in examples where other libraries resolve quickly.
the last time I tried it (it was years ago) std::regex was taking a measurable number of milliseconds to evaluate which is kind of a very long time even outside of performance critical paths.
There's nothing wrong with the std::regex design as it is the standard.
std::regex only sucks because the developers of gcc and clang never bothered to optimize it. (Too much work and they have other stuff to worry about.)
std::regex only sucks because the developers of gcc and clang never bothered to optimize it. (Too much work and they have other stuff to worry about.)
"can't be fixed without breaking ABI" sounds plausible for C++.
There's generally not all that much stdc++ specific optimisation stuff in clang. There might be parts of regex that are worth implementing as compiler intrinsics, that seems to be the existing pattern for making bits much faster.
The really heavy lifting you want for regex is to partially evaluate and split them. They're a separate language unto themselves and benefit from being optimised as such. There's nowhere ideal in the clang/llvm pipeline to do that though.
There's generally not all that much stdc++ specific optimisation stuff in clang. There might be parts of regex that are worth implementing as compiler intrinsics, that seems to be the existing pattern for making bits much faster.
The really heavy lifting you want for regex is to partially evaluate and split them. They're a separate language unto themselves and benefit from being optimised as such. There's nowhere ideal in the clang/llvm pipeline to do that though.
C++ regexes are literally just copy-pasted ECMAScript regexes. They could have just used an existing regex library, but C++ compiler developers presumably don't want to support an extra dependency.
That's the only real reason why std::regex is slow.
That's the only real reason why std::regex is slow.
std::regex also depends on locale, which is reason enough to avoid it, regardless of performance.
> What do you use instead [of std::iostream]?
This is what I want to know. Having come from C to C++, iostreams were a big improvement over the "strings" and print functions of C. I even extended a base iostream class to have a "teebuf" logger, that could output to multiple streams and had the standard logging levels.
It's been a while since I last had mastery of C++, but I'd like to hear what is as portable and better than iostreams.
This is what I want to know. Having come from C to C++, iostreams were a big improvement over the "strings" and print functions of C. I even extended a base iostream class to have a "teebuf" logger, that could output to multiple streams and had the standard logging levels.
It's been a while since I last had mastery of C++, but I'd like to hear what is as portable and better than iostreams.
Strings in C++ are nice, especially now that we have std::string_view, but <iostream> is one of the worst pieces of the C++ standard library.
- <iostream> makes localization more difficult, compared to printf (localizing <iostream> code is beyond awful)
- <iostream> makes thread safety more difficult, compared to printf (it is safe to printf/fprintf from multiple threads, simultaneously, without any extra work)
- The <iostream> operator overloading syntax is bad (my sense is that the operator overloading abuse in <iostream> was a contributing factor for why Java doesn't allow operator overloading)
- Streams in <iostream> are stateful, and it's easy to accidentally leave them in the wrong state (radix, padding, field width, etc)
- Performance of <iostream>, out of the box, is mediocre (to get decent performance, you need to change some defaults)
The main advantage of <iostream> was that it provided type safety, but IMO that advantage has long since been irrelevant. You get type safety with std::printf, with most compilers, assuming you enable -Wformat on GCC or similar options in other compilers.
The only remaining advantage of <iostream> is that you can overload operator<<. I don't think that's much of an advantage, especially weighed against the numerous disadvantages.
Using std::printf is better and more portable. Libfmt is also better and more portable, and it is now part of the standard library as std::format.
https://www.moria.us/articles/iostream-is-hopelessly-broken/
- <iostream> makes localization more difficult, compared to printf (localizing <iostream> code is beyond awful)
- <iostream> makes thread safety more difficult, compared to printf (it is safe to printf/fprintf from multiple threads, simultaneously, without any extra work)
- The <iostream> operator overloading syntax is bad (my sense is that the operator overloading abuse in <iostream> was a contributing factor for why Java doesn't allow operator overloading)
- Streams in <iostream> are stateful, and it's easy to accidentally leave them in the wrong state (radix, padding, field width, etc)
- Performance of <iostream>, out of the box, is mediocre (to get decent performance, you need to change some defaults)
The main advantage of <iostream> was that it provided type safety, but IMO that advantage has long since been irrelevant. You get type safety with std::printf, with most compilers, assuming you enable -Wformat on GCC or similar options in other compilers.
The only remaining advantage of <iostream> is that you can overload operator<<. I don't think that's much of an advantage, especially weighed against the numerous disadvantages.
Using std::printf is better and more portable. Libfmt is also better and more portable, and it is now part of the standard library as std::format.
https://www.moria.us/articles/iostream-is-hopelessly-broken/
>The <iostream> operator overloading syntax is bad
In an obnoxious way. The first code someone will see of a new language is often 'hello world'. In C++, 'hello world' is an advertisement for the fact that operator overloading exists.
In an obnoxious way. The first code someone will see of a new language is often 'hello world'. In C++, 'hello world' is an advertisement for the fact that operator overloading exists.
I still don't get the hate for <<
it's the coolest part
it's the coolest part
libfmt is definitely recommended.
About 6-7 years ago back when my employer was running code compiled with gcc-4.4.7 and running it on Linux 2.6.32 boxes even though it was pretty old even back then, it took me a lot of convincing the company to give libfmt a try.
It was such a boost to developer happiness. People were literally overjoyed, writing to me on Slack how much of a pleasure string formatting has become.
About 6-7 years ago back when my employer was running code compiled with gcc-4.4.7 and running it on Linux 2.6.32 boxes even though it was pretty old even back then, it took me a lot of convincing the company to give libfmt a try.
It was such a boost to developer happiness. People were literally overjoyed, writing to me on Slack how much of a pleasure string formatting has become.
> I'd like to hear what is as portable and better than iostreams
Have a look at fmtlib. The interface is more like printf, but type safe and format strings are parsed at compile time. I believe it’s what std::format is based on, which could also be an option for you depending on how recent your compiler/language version is.
Have a look at fmtlib. The interface is more like printf, but type safe and format strings are parsed at compile time. I believe it’s what std::format is based on, which could also be an option for you depending on how recent your compiler/language version is.
> The interface is more like printf
See, I don't believe that's an improvement. Having used printf in C, I was relieved to be able to "redirect" whatever to a stream, and not care about whether it should be "%d" or "%02f" or even if it was a struct/class.
On top of this, treating files as streams, strings as streams, or even extending streams to make a tee-stream[0] all seem clunkier to me with a printf like system.
Maybe fmt fixes these problems, I don't know. But I feel a lot of people don't like iostreams because they have some form of Stockholm syndrome with printf.
[0] - https://wordaligned.org/articles/cpp-streambufs#tee-streams
See, I don't believe that's an improvement. Having used printf in C, I was relieved to be able to "redirect" whatever to a stream, and not care about whether it should be "%d" or "%02f" or even if it was a struct/class.
On top of this, treating files as streams, strings as streams, or even extending streams to make a tee-stream[0] all seem clunkier to me with a printf like system.
Maybe fmt fixes these problems, I don't know. But I feel a lot of people don't like iostreams because they have some form of Stockholm syndrome with printf.
[0] - https://wordaligned.org/articles/cpp-streambufs#tee-streams
> Maybe fmt fixes these problems, I don't know.
Yeah, it looks like you did a lot of guesswork in that comment, and a lot of those guesses were inaccurate. Not really trying to be hostile here, but you did acknowledge that you were unfamiliar with std::format.
The part that fmtlib / std::format has, which is printf-like, is the idea of having a format string and arguments, rather than having a bunch of separate, piecemeal strings.
The std::print interface can work equally well with FILE or std::ofstream, or whatever you want. This is C++, and so you can just use a templated output iterator—or one of the overloads that creates one automatically.
There are a lot of problems with <iostream>. I think it’s telling that lots of languages have copied printf, but nobody (or almost nobody) thought <iostream> was good enough to copy. There are just too many serious design flaws with <iostream>. It would be one thing if <iostream> were just annoying to use, but it poses problems for localization, thread-safety, accidental misuse through its statefulness, and its operator overloading syntax is bad.
Yeah, it looks like you did a lot of guesswork in that comment, and a lot of those guesses were inaccurate. Not really trying to be hostile here, but you did acknowledge that you were unfamiliar with std::format.
The part that fmtlib / std::format has, which is printf-like, is the idea of having a format string and arguments, rather than having a bunch of separate, piecemeal strings.
// Old printf code, works ok for most people
std::printf("failed to clone %s from %s", target, src);
// <iostream>
std::cout << "failed to clone " << target << " from " << src;
// New std::format / fmtlib
std::print("failed to clone {} from {}", target, src);
The std::print interface can work equally well with FILE or std::ofstream, or whatever you want. This is C++, and so you can just use a templated output iterator—or one of the overloads that creates one automatically.
There are a lot of problems with <iostream>. I think it’s telling that lots of languages have copied printf, but nobody (or almost nobody) thought <iostream> was good enough to copy. There are just too many serious design flaws with <iostream>. It would be one thing if <iostream> were just annoying to use, but it poses problems for localization, thread-safety, accidental misuse through its statefulness, and its operator overloading syntax is bad.
> It would be one thing if <iostream> were just annoying to use, but it poses problems for localization, thread-safety, accidental misuse through its statefulness, and its operator overloading syntax is bad.
It's Bjarne (Stroustrup)'s pet feature. You'll notice that many expert practitioners of C++ write std::println("Hello, world"); or similar for a canonical C++ 23 Hello World, but Bjarne pointedly still writes it with std::cout and the iostream operator overloads.
Now that Bjarne is back at Columbia teaching, there's a risk he'll infect impressionable young people with this nonsense, COMSW4995 was not an intro class and I suspect Bjarne's jetsetting makes it impractical to teach such a class, so this risk is small but not zero.
It's Bjarne (Stroustrup)'s pet feature. You'll notice that many expert practitioners of C++ write std::println("Hello, world"); or similar for a canonical C++ 23 Hello World, but Bjarne pointedly still writes it with std::cout and the iostream operator overloads.
Now that Bjarne is back at Columbia teaching, there's a risk he'll infect impressionable young people with this nonsense, COMSW4995 was not an intro class and I suspect Bjarne's jetsetting makes it impractical to teach such a class, so this risk is small but not zero.
fmtlib knows the types of all the arguments, and those types are checked against the format string at compile time. So type safety ("%d" vs "%s" etc) is not really an issue.
FWIW, I have waaaay more experience with iostreams than printf. I once wrote a replacement for most parts of std::ostream purely because I had a logging system that was loosely based on log4cxx (which is based on iostreams) and I needed much faster formatting.
Right now I'm in the process of switching to a different logging system that uses fmtlib. I can say that I really, really do not miss the extreme verbosity of iostreams at all. The statefulness I could take or leave, although on balance I'd say it's usually more of a negative. Thankfully I don't have to give up type safety or extensibility.
As for treating files/strings/whatever as streams--my impression of fmtlib is that it's concerned primarily with formatting, which frankly I'm fine with. It can write to an in-memory buffer or to a FILE* or to a std::ostream, which covers pretty much all of my needs. Given the performance of iostreams (lots of virtual method calls) I never really saw the point of using it as a more generalized 'streaming' interface, even knowing that it's possible.
FWIW, I have waaaay more experience with iostreams than printf. I once wrote a replacement for most parts of std::ostream purely because I had a logging system that was loosely based on log4cxx (which is based on iostreams) and I needed much faster formatting.
Right now I'm in the process of switching to a different logging system that uses fmtlib. I can say that I really, really do not miss the extreme verbosity of iostreams at all. The statefulness I could take or leave, although on balance I'd say it's usually more of a negative. Thankfully I don't have to give up type safety or extensibility.
As for treating files/strings/whatever as streams--my impression of fmtlib is that it's concerned primarily with formatting, which frankly I'm fine with. It can write to an in-memory buffer or to a FILE* or to a std::ostream, which covers pretty much all of my needs. Given the performance of iostreams (lots of virtual method calls) I never really saw the point of using it as a more generalized 'streaming' interface, even knowing that it's possible.
Considering the origins of "stockholm syndrome" this may not be exactly the message you intended.
However, printf and its ilk get it right; presentation is a property of the context in which the entity is to be presented, not of the entity itself. You can kvetch about the markup syntax, or the type safety issues the C implementation has, but the architecture is fundamentally correct.
However, printf and its ilk get it right; presentation is a property of the context in which the entity is to be presented, not of the entity itself. You can kvetch about the markup syntax, or the type safety issues the C implementation has, but the architecture is fundamentally correct.
It’s impossible to implement std::regex efficiently due to internal but exposed character handling requirements. To the sibling comments about iostream, thank vzeverovich and everyone that worked on fmt and turning it into std::format (edit: accidentally wrote std::fmt initially), which is vastly superior and not dependent on mutable global state.
1. Historically, std::regex was offered in GCC before it was actually fully implemented. Much hilarity ensued...
2. Some existing implementations have efficiency issues, e.g. performing many allocations.
3. It is claimed (e.g. by Titus Winters) that the ABI of std::regex is problematic, and without breaking it, the implementations cannot be good enough
See these points and others at:
https://www.reddit.com/r/cpp/comments/e16s1m/what_is_wrong_w...
2. Some existing implementations have efficiency issues, e.g. performing many allocations.
3. It is claimed (e.g. by Titus Winters) that the ABI of std::regex is problematic, and without breaking it, the implementations cannot be good enough
See these points and others at:
https://www.reddit.com/r/cpp/comments/e16s1m/what_is_wrong_w...
> What's wrong with std::regex?
In my experience, stdlibc++ <regex> is VERY slow, especially on debug builds. We are using g_regex instead, which in turn uses pcre2.
> And iostreams
<iostream> achieves too little with too much code. We instead use:
I also like that fmt has helpers for pointers (fmt::ptr), enums (fmt::underlying) and arrays (fmt::join). It's both easy on the eyes and feels consistent.
[1]: https://github.com/emilk/loguru
In my experience, stdlibc++ <regex> is VERY slow, especially on debug builds. We are using g_regex instead, which in turn uses pcre2.
> And iostreams
<iostream> achieves too little with too much code. We instead use:
std::cout << fmt::format(...);
I also like that fmt has helpers for pointers (fmt::ptr), enums (fmt::underlying) and arrays (fmt::join). It's both easy on the eyes and feels consistent.
[1]: https://github.com/emilk/loguru
IMHO, the printf-style API is superior to the stream based API, because it can be adapted into a logging API that defers string formatting or delegates it to another thread. Overloading by type can be accomplished with printf-style API as well. Cluttering business code with string formatting can be a liability in code that needs to be performant/low latency, which is one of the primary reasons people use C++.
iostreams are not great but are nothing compared to the awkwardness and lack of extensibility of printf style format strings.
iostream is replaced by format.
Can you expand on the problems with regex?
The implementations are bad and implementers are refusing to fix their own bad implementations so as to not break their own ABI, but that has nothing to do with C++ the standard.
From what I’ve heard, std::regex is notoriously inefficient (lots of memory allocations, no allocator support). But I’ve never used it myself.
oh yeah C++ regex is stupidly inefficient, like "python is faster" inefficient. I tried to use it for text replacements and pretty much immediately abandoned it
Apparently typical implementations of std::regex are inefficient like "'popen("perl..")' is faster" is inefficient!
I thing boost::regex is significantly faster although not particularly fast
I thing boost::regex is significantly faster although not particularly fast
[deleted]
The code:
//a decent amount of this was copied/modified from backward.cpp (https://github.com/bombela/backward-cpp)
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.Both have "MIT License" and he explicitly acknowledge the source. He probably forgot that he had to add an additional line with "Copyright 2013 Google Inc." in his own license file.
This is only an issue if the original author chooses to enforce. You don't know - this code may have been given special permission to omit the notice by the original author. It isn't up to random joe to find copyrights they think have been violated.
Reliable in-process crash reporting is exceptionally difficult.
The code must be fully async-safe, which means you cannot use <stacktrace>. You also cannot acquire mutexes, use any of the standard allocators, etc etc etc.
The code must be fully async-safe, which means you cannot use <stacktrace>. You also cannot acquire mutexes, use any of the standard allocators, etc etc etc.
The best way I've found is - patching LLVM's libunwind to make it fully async-signal safe, and sending the stack trace to another thread for symbolization. This is implemented in ClickHouse.
1. Can you link to that?
2. Have these changes been offered as patch for libunwind or boost::stacktrace?
2. Have these changes been offered as patch for libunwind or boost::stacktrace?
https://github.com/ClickHouse/libunwind/
There were multiple steps:
1. Avoid using malloc/free inside libunwind.
2. Avoid using FDECache that required a mutex.
3. Avoid using dl_iterate_phdr (a mutex inside libc).
4. Protection from dereferencing wrong pointers due to incorrect unwind tables.
Most of the changes were integrated to libunwind, but not everything. Example: https://bugs.llvm.org/show_bug.cgi?id=48186
There were multiple steps:
1. Avoid using malloc/free inside libunwind.
2. Avoid using FDECache that required a mutex.
3. Avoid using dl_iterate_phdr (a mutex inside libc).
4. Protection from dereferencing wrong pointers due to incorrect unwind tables.
Most of the changes were integrated to libunwind, but not everything. Example: https://bugs.llvm.org/show_bug.cgi?id=48186
What is async-unsafe in using `<stacktrace>`?
As for not using standard allocators - not a problem, just have a fixed area set aside as a buffer for crash reporting. Yes, it might not fit an extremely long report, but it's not that much of an issue.
As for not using standard allocators - not a problem, just have a fixed area set aside as a buffer for crash reporting. Yes, it might not fit an extremely long report, but it's not that much of an issue.
It’s not guaranteed to be async-safe. From the C++ proposal (P0881R7):
> Note about signal safety: this proposal does not attempt to provide a signal-safe solution for capturing and decoding stacktraces. Such functionality currently is not implementable on some of the popular platforms.
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p08...
[edit] Replying here, because HN is doing its occasional obnoxious rate-limiting of replies:
Signal-safe and async-safe are effectively the same thing, and “async-safe” absolutely isn’t the same thing as “thread-safe”.
A code path that acquires a mutex can be thread-safe; that’s absolutely not async-safe.
If boost implemented a fully async-safe stack unwinder, complete with DWARF expression support, Apple compact unwind encoding support, and all the other features required across platforms, then good for them — but that’s not what <stacktrace> is guaranteed to provide, and such a thing is still not sufficient to implement anything but the most barebones portion of a real crash reporter.
> Note about signal safety: this proposal does not attempt to provide a signal-safe solution for capturing and decoding stacktraces. Such functionality currently is not implementable on some of the popular platforms.
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p08...
[edit] Replying here, because HN is doing its occasional obnoxious rate-limiting of replies:
Signal-safe and async-safe are effectively the same thing, and “async-safe” absolutely isn’t the same thing as “thread-safe”.
A code path that acquires a mutex can be thread-safe; that’s absolutely not async-safe.
If boost implemented a fully async-safe stack unwinder, complete with DWARF expression support, Apple compact unwind encoding support, and all the other features required across platforms, then good for them — but that’s not what <stacktrace> is guaranteed to provide, and such a thing is still not sufficient to implement anything but the most barebones portion of a real crash reporter.
1. signal-safe and async-safe/thread-safe is not quite the same thing, but fair enough.
2. The boost::stacktrace library (on which the standardization was mostly based IIANM) has a `safe_dump_to()` function for these cases.
See here: https://github.com/boostorg/stacktrace/blob/develop/include/...
2. The boost::stacktrace library (on which the standardization was mostly based IIANM) has a `safe_dump_to()` function for these cases.
See here: https://github.com/boostorg/stacktrace/blob/develop/include/...
What's the benefit of in-process crash reporting compared to just using something like crashpad/breakpad?
To the extent that in-process crash reporting is even possible... seems the most common class of crashes would be entirely unrecoverable.
To the extent that in-process crash reporting is even possible... seems the most common class of crashes would be entirely unrecoverable.
Ease of integration, because having literally anything is typically better than nothing. Honestly Crashpad isn't fun to integrate unless you use a fork like backtrace's (which adds CMake support), which I think doesn't help. I don't know of any alternatives.
A version of Crashpad or something like it with a single turnkey server for database dumps, a one-line "defaults are good enough" integration, would be a real great thing to see.
A version of Crashpad or something like it with a single turnkey server for database dumps, a one-line "defaults are good enough" integration, would be a real great thing to see.
I found Sentry's crash reporting (which uses crashpad) simple enough to configure into an existing CMake build within an afternoon.
Building Sentry/crashpad from source in a few lines of CMake: https://github.com/ArmageddonGames/ZQuestClassic/commit/3471...
And a few lines in the main function: https://github.com/ArmageddonGames/ZQuestClassic/commit/3471...
Building Sentry/crashpad from source in a few lines of CMake: https://github.com/ArmageddonGames/ZQuestClassic/commit/3471...
And a few lines in the main function: https://github.com/ArmageddonGames/ZQuestClassic/commit/3471...
Neat, I didn't know Sentry also had a good fork, I haven't tried it! But in contrast, here's an in-process fault library that I whipped up (from forking Phusion Passenger) about 10 years ago that I still reach for sometimes, which is surprisingly robust to most of the original complaints about async safety, but still not perfect: https://github.com/thoughtpolice/libfault
You add one C file and 6 lines of code in `main()`, and you can do this in pretty much any programming language with a tiny extra bit of glue. It takes 3 minutes to do this in any C/C++ codebase of mine. It is build system agnostic and works immediately, with zero outside deps. It's something, and that's better than nothing, in practice. So people reach for that. I reach for it. And not just because I wrote it.
I want to be clear: Crashpad is 10000x better than mine in every way, except this one way. And I really wish it wasn't. To add onto this, I really don't like CMake for example, so this problem isn't just a "well I like my thing." I want something that will also work in my Java programs, or Rust programs, for instance! Sometimes they crash too. I don't need to add any dependencies except like 2 or 3 C function calls, which almost every langauge supports with a native FFI out of the box. The friction is extremely low.
I'm reminded of something Yann Collet once said about the design of zstd, and getting people to adopt new compression technology. If you make a compressor and it's better than an alternative in one or more dimensions, but worse in another (size, decompressor speed), then friction is actually significantly increased by that one failure. But if you make it better in every dimension -- so it gives an equal ratio and compression and decompression are always better than alternatives -- the friction is eliminated and people will just reach for it. Even though you only did worse in one spot, people find ways to make it matter. It really makes people think twice. But if it's always better, in every way, then using and reaching for it is just instinctive -- it replaces the old thing entirely.
So that's what I really wish we had here. I think that's what you would need to see a lot better crash handling and reporting become more widely used. There needs to be a version of Crashpad, or any robust out of process crash collector, that you can just drop into any language and any build system with a little C glue (or Rust! Sure! Whatever!) in 5 minutes and it should have a crash database server and crash handler process which should instantly work for most uses.
You add one C file and 6 lines of code in `main()`, and you can do this in pretty much any programming language with a tiny extra bit of glue. It takes 3 minutes to do this in any C/C++ codebase of mine. It is build system agnostic and works immediately, with zero outside deps. It's something, and that's better than nothing, in practice. So people reach for that. I reach for it. And not just because I wrote it.
I want to be clear: Crashpad is 10000x better than mine in every way, except this one way. And I really wish it wasn't. To add onto this, I really don't like CMake for example, so this problem isn't just a "well I like my thing." I want something that will also work in my Java programs, or Rust programs, for instance! Sometimes they crash too. I don't need to add any dependencies except like 2 or 3 C function calls, which almost every langauge supports with a native FFI out of the box. The friction is extremely low.
I'm reminded of something Yann Collet once said about the design of zstd, and getting people to adopt new compression technology. If you make a compressor and it's better than an alternative in one or more dimensions, but worse in another (size, decompressor speed), then friction is actually significantly increased by that one failure. But if you make it better in every dimension -- so it gives an equal ratio and compression and decompression are always better than alternatives -- the friction is eliminated and people will just reach for it. Even though you only did worse in one spot, people find ways to make it matter. It really makes people think twice. But if it's always better, in every way, then using and reaching for it is just instinctive -- it replaces the old thing entirely.
So that's what I really wish we had here. I think that's what you would need to see a lot better crash handling and reporting become more widely used. There needs to be a version of Crashpad, or any robust out of process crash collector, that you can just drop into any language and any build system with a little C glue (or Rust! Sure! Whatever!) in 5 minutes and it should have a crash database server and crash handler process which should instantly work for most uses.
Thanks for sharing, I'm sure that will come in handy for me some day!
This all feels like a failure of our modern OSes - why must the application layer know how to report on when it crashes? It seems like functionality that the OS should provide! Instead, we're stuck reaching for these random extensions solving the same problem in the same way everywhere - or, if you're lucky, this gets provided by the language framework for "free" to application developers (but not the language developers).
This all feels like a failure of our modern OSes - why must the application layer know how to report on when it crashes? It seems like functionality that the OS should provide! Instead, we're stuck reaching for these random extensions solving the same problem in the same way everywhere - or, if you're lucky, this gets provided by the language framework for "free" to application developers (but not the language developers).
Actually, that's a really a good point. It's also especially weird considering we get lots of debugging support through the OS stack on every system. They have APIs, libraries, tools, etc. But actual crash control, a kind of thing where you want black box recordings after the fact, isn't really a thing anywhere unless you roll it yourself. It's a big shame.
I guess it is easier to print out some interesting process variables, compared to trying to save and then make sense of dumps etc.
[deleted]
It has value in embedded code where you can stop the world to handle or log fault conditions.
A question:
Would it be possible to pass the stacktrace of the current thread to another, so that the stacktrace would be traceable across threadpools or worker threads?
I am not sure if i understand the question correctly. But once collected, stack traces are just regular object that can be passed around thread as other object. It's possible that some implementation have references to some stack addresses (like for example the address of a function parameter), in which case you would need to serialize the stack trace before storing them/ moving then another thread.
> once collected, stack traces are just regular object that can be passed around thread as other object.
> It's possible that some implementation have references to some stack addresses (like for example the address of a function parameter), in which case you would need to serialize the stack trace before storing them/ moving then another thread.
So which of these two mutually exclusive options is it? As I understand it, that was the question.
> It's possible that some implementation have references to some stack addresses (like for example the address of a function parameter), in which case you would need to serialize the stack trace before storing them/ moving then another thread.
So which of these two mutually exclusive options is it? As I understand it, that was the question.
> So which of these two mutually exclusive options is it? As I understand it, that was the question.
Well the stack_entry/stack_trace object can be moved around between thread, as in the object itself is copyable and movable. However, the handle_type is implementation defined, so it might be the case that extracting the information out of the object only works on the producing thread.
Well the stack_entry/stack_trace object can be moved around between thread, as in the object itself is copyable and movable. However, the handle_type is implementation defined, so it might be the case that extracting the information out of the object only works on the producing thread.
As far as I understand, basic_stack trace is just a container of stacktrace_entries, which are Regular types. So the default distinct-objects type safety rules apply.
You should be able to, for example, collect the stacktace on one thread, transport it to another and print it.
You should be able to, for example, collect the stacktace on one thread, transport it to another and print it.
When I debug multithreaded programs, the stacktrace of a breakpoint usually ends somewhere in a worker thread.
What I want is that the worker thread's stacktrace part is replaced by the one who put the work into it.
Kinda like the program wasn't multithreaded at all.
Ah. I guess you can capture the stack trace at task creation point, then stitch together a new stack trace by replacing the generic common prefix of your worker thread trace with the task creation one.
But you can't use the basic_stacktrace container itself as it is immutable and not constructibe from a range, so you have to roll your own. You should be able to use the stacktrace_entries though.
Most importantly, I expect that capturing a stacktrace is quite expensive, so you might not be able to do it at task creation time, and it is too late to do it later. Maybe you want this only in debug mode.
Note I haven't actually tried any if this, it is just guesswork.
But you can't use the basic_stacktrace container itself as it is immutable and not constructibe from a range, so you have to roll your own. You should be able to use the stacktrace_entries though.
Most importantly, I expect that capturing a stacktrace is quite expensive, so you might not be able to do it at task creation time, and it is too late to do it later. Maybe you want this only in debug mode.
Note I haven't actually tried any if this, it is just guesswork.
Exactly this.
I didn't try this, and I suppose that some low level pointer rewriting would be necessary to do this.
I'm not sure if it's expensive though, maybe you can replace the pointers without resolving the stacktrace.
The problem is that to get the stacktrace of the task creation you have to traverse the stack at that point in time. You can't really do it later. And stack traversal using DWARF unwind info, for example, is neither cheap nor simple. You might have better luck if you compile with frame pointer though.
I think what you looking for is "task tracing" not really stack tracing. The relationship between the task (like where was a task added in the thread pool) are not reflected in the stacktrace the way you want them.
To address those, you need to have special handshake between the debugger and your task api. You can also instrument the "add_task" function call to log every time a task is added to you queue and do some some offline stack stiching.
if you actually wanted to you could probably wrap thread to pass the stacktrace of the spawning thread into the worker thread whenever you spawn a thread and then output that upon a crash as well. the library seems pretty simple and flexible.
Yet another library feature that will be used by nobody... While having a standard library is good, system programming presents tricky aspects like async-safety that are not adequately addressed. Therefore, considering the challenges involved, I believe it would be better to utilize existing libraries like crashpad to handle such scenarios.
I'm afraid you are right. Without async-signal safety, the library is not terribly useful.
I may be wrong, but as I recall it is good form to chain exception filter calls by making note of the return from `SetUnhandledExceptionFilter`. For example, if I want to use `<stacktrace>` and do copy-on-write using memory protection in the same program.
Maybe I am doing something wrong but the example code does not seem to work on VS 2022 (no output file after a div by zero or stack overflow crash).
This does not even remotely look to be signal safe to me?
The signal handler just saves the trace and then wakes up another thread which does the reporting. The signaled thread uses only <stacktrace> and mutex/cv, all signal-safe as far as I can tell.
The crashing thread might hold a lock in the memory allocator - which could either self deadlock when saving the stack trace (which certainly seems to do memory allocation and thus isn't a-signal safe), or could deadlock with the crash reporting thread which definitely allocates memory all over.
I also quite doubt that std::mutex, and even more so std::condition_variable are guaranteed to be signal safe.
I also quite doubt that std::mutex, and even more so std::condition_variable are guaranteed to be signal safe.
> I also quite doubt that std::mutex, and even more so std::condition_variable are guaranteed to be signal safe.
They are not. Use sem_t for signaling.
edit: also even if they where, the signal handler might wait forever for a mutex owned by the blocked thread.
They are not. Use sem_t for signaling.
edit: also even if they where, the signal handler might wait forever for a mutex owned by the blocked thread.
It was more a rethorical doubting because I did not want to look the standard up on my phone....
stack_trace allows you to specify a custom allocator, which would protect against a lock held in the allocator (never ran into this in the real world though).
You're right about mutex & cv in the general case though
You're right about mutex & cv in the general case though
That assumes the specified allocator actually controls all allocations - somewhat doubtful across all platforms as things like dl_iterate_phdr() IIRC allocate memory (and take locks). And there's a lot more to writing signal safe code than not calling malloc. Unless an interface documents to be signal safe you're take better of assuming it is not.
FWIW I've run into malloc self dreadlocks due to rare signals plenty of time :(. In production workloads.
FWIW I've run into malloc self dreadlocks due to rare signals plenty of time :(. In production workloads.
Why is it that even though Github or what ever has cutsie unicorns (or whatever it is) as error messages it feels fake and contrived while this define just feels like some random dude at MS naming it before going off to write Solitaire?