Undefined Behavior in C and C++(lumagraph.ie)
lumagraph.ie
Undefined Behavior in C and C++
https://lumagraph.ie/undefined-behavior
85 comments
Right. To be clear, the purpose of the article is not 'zounds, C and C++ have footguns!' but 'C and C++ have footguns – yeah, this is not exactly breaking news – but here is a hopefully helpful summary of where they are, why they exist, and what you can do to avoid them'.
If you are already satisfied you know how to avoid them, and you don't need any more help with that, then you are not the target audience, and should by all means ignore the article.
If you are already satisfied you know how to avoid them, and you don't need any more help with that, then you are not the target audience, and should by all means ignore the article.
But tyfighter has some reason. People take such articles, and use them to beat up anyone writing C++, arguing that they are stupid to use such an undependable tool.
So, yes, people who know what they're doing can ignore such articles, as a first-order effect. But there are second-order effects from such articles, and while they don't change anything, they are rather unpleasant. Hence tyfighter's anger - he gets tired of being on the receiving end of the fallout from such articles.
So, yes, people who know what they're doing can ignore such articles, as a first-order effect. But there are second-order effects from such articles, and while they don't change anything, they are rather unpleasant. Hence tyfighter's anger - he gets tired of being on the receiving end of the fallout from such articles.
I do actually sympathize with that! I tried to keep a level tone, and maximize the ratio of useful information to flame-war ammunition, but that ratio unfortunately has an upper bound well short of infinity.
You haven't changed but compilers have changed, unfortunately. Unless you stick on -O0 or -fno-strict-aliasing all the time, the chance is that your UB-ridden code can break in the future with more powerful compilers exploiting more UBs. So that's why you have to care now if you didn't so far. (Or you can argue that optimizations should be turned off, which is indeed another valid, though uncommon, answer preferred by djb for example.)
Actually, I have changed, and I've changed corporate C/C++ MANY times to satisfy compiler upgrades. It is always just shitty code. It's never something insidious. Bugs happen. It probably wasn't your intention, but you've made UB sound pretty awesome.
If you are mainly dealing with shitty code, which frequently contains an UB, then most UBs you've encountered should have been from shitty code. That doesn't mean all UBs indicate shitty code.
> This is yet another article that goes on and on about nonsensical situations that are just shitty code.
> Who cares?
The short flippant answer is: because everyone writes shitty code at some point. It generally doesn't get committed or released, but during development, shitty buggy code with Undefined Behavior happens.
Here's a concrete example of some code that I actually wrote (simplified greatly so it could be a small illustrative example): https://godbolt.org/z/xzehrWE57
Knowing about UB is a useful way to describe what's going on in this code example, and why the compiler is doing what it's doing. If you see your code behaving in "impossible" ways, knowing about UB can give you some hints about where to look.
> Who cares?
The short flippant answer is: because everyone writes shitty code at some point. It generally doesn't get committed or released, but during development, shitty buggy code with Undefined Behavior happens.
Here's a concrete example of some code that I actually wrote (simplified greatly so it could be a small illustrative example): https://godbolt.org/z/xzehrWE57
Knowing about UB is a useful way to describe what's going on in this code example, and why the compiler is doing what it's doing. If you see your code behaving in "impossible" ways, knowing about UB can give you some hints about where to look.
As an engineer, my job isn't to write code, it's to deliver systems that do specific things. That means that I need to understand the defined behavior of the code I put into the system. Undefined behavior anywhere means you lack defined behavior everywhere in C/C++.
You can't work around this by writing more code or eliminate undefined behavior with tools like linters and tests. Your one and only option is to write perfect code that only has defined behavior. The number of people that can accomplish this in practice rounds to zero.
So yeah, how can you not care about UB? It's the semantic elephant in the room. Every conversation has to include it, implicitly or not.
You can't work around this by writing more code or eliminate undefined behavior with tools like linters and tests. Your one and only option is to write perfect code that only has defined behavior. The number of people that can accomplish this in practice rounds to zero.
So yeah, how can you not care about UB? It's the semantic elephant in the room. Every conversation has to include it, implicitly or not.
What universe are you working in where you think ANY of that is actually true? In the land of reality where I live and work (I work in hardware), I'm not constructing philosophical prose about well-defined systems. This is another bad faith argument where undefined behavior is made out to be some house of cards. I hate to break it to you, but every computer and all it's software you've ever used is a monument to the glory of undefined behavior, because people just didn't worry about it.
I'm exceedingly well-aware of how prevalent UB is and how "rarely" it actually turns into an issue in practice. The problem is that you have no way of knowing when or if a particular instance of UB will be dangerous. Even if you somehow know the impact today, that can change without warning in the future.
There's a wealth of studies on this subject, like this one [0] documenting cases where undefined behavior leads to miscompilations or examples like [1] where undefined behavior leads to security vulnerabilities. There's a quote from that second link that's deeply applicable here:
> This blog post provides an exploit technique demonstrating that treating these bugs as universally innocuous often leads to faulty evaluations of their relevance to security.
[0] http://dx.doi.org/10.1145/2517349.2522728
[1] https://googleprojectzero.blogspot.com/2023/01/exploiting-nu...
There's a wealth of studies on this subject, like this one [0] documenting cases where undefined behavior leads to miscompilations or examples like [1] where undefined behavior leads to security vulnerabilities. There's a quote from that second link that's deeply applicable here:
> This blog post provides an exploit technique demonstrating that treating these bugs as universally innocuous often leads to faulty evaluations of their relevance to security.
[0] http://dx.doi.org/10.1145/2517349.2522728
[1] https://googleprojectzero.blogspot.com/2023/01/exploiting-nu...
Aside from the contrived examples in the paper, the rest are bugs. The kernel exploit was a just a lack of a NULL check; another bug. Bugs are going to happen, and they're going to have unpredictable consequences. What does that have to do with the language and undefined behavior? These are all just more evidence of needing to know what you're doing if you're going to write code at this level, but really not because the vast majority of people aren't writing code where bugs in the form of crashes or security exploits will have serious consequences or can't be fixed.
I'm not sure what you're going for by trying to call the examples I linked bugs. Yes...?
The issue is that you can't solve these at the code level. The kernel vuln could have been solved by a null check only because the kernel build system explicitly tells the compiler not to omit null checks as a fix for earlier exploits [0] caused by the language allowing the compiler to omit null checks.
I don't think it's reasonable to brush these off as things that only affect "serious" code. For one, someone needs to write that important code and history has repeatedly demonstrated that even the best programmers write UB occasionally. Secondly, "important code" is pretty much the biggest remaining niche for large scale C development, and C++ to a lesser extent. Very few people are using Ada/SPARK for safety critical development, for example. Compilers have also become significantly more aggressive at optimizing against UB and security significantly more important, which means this problem is far worse than it was 30 years ago.
[0] https://lwn.net/Articles/342420/
The issue is that you can't solve these at the code level. The kernel vuln could have been solved by a null check only because the kernel build system explicitly tells the compiler not to omit null checks as a fix for earlier exploits [0] caused by the language allowing the compiler to omit null checks.
I don't think it's reasonable to brush these off as things that only affect "serious" code. For one, someone needs to write that important code and history has repeatedly demonstrated that even the best programmers write UB occasionally. Secondly, "important code" is pretty much the biggest remaining niche for large scale C development, and C++ to a lesser extent. Very few people are using Ada/SPARK for safety critical development, for example. Compilers have also become significantly more aggressive at optimizing against UB and security significantly more important, which means this problem is far worse than it was 30 years ago.
[0] https://lwn.net/Articles/342420/
UB is far from the only source of systems not doing the desired thing - writing code that ends up at UB is as wrong as writing code that was written with an incorrect understanding of the invoked behavior.
Sure, the neat trick of a+1<a not working is perhaps undesirable, but, even if signed addition was defined to wrap, in most contexts an "a+1" subtracting four billion is not gonna be the specific thing you want it to do in your system.
Alternatively, signed overflow could be defined to return exactly 31415, which would be very concrete defined behavior, but barely if at all more useful compared to it being UB.
Sure, the neat trick of a+1<a not working is perhaps undesirable, but, even if signed addition was defined to wrap, in most contexts an "a+1" subtracting four billion is not gonna be the specific thing you want it to do in your system.
Alternatively, signed overflow could be defined to return exactly 31415, which would be very concrete defined behavior, but barely if at all more useful compared to it being UB.
I hope I didn't imply that UB was the only source of bugs. It obviously isn't. It's just the only source of bugs that has the side effect of undefining the semantics of all your other code.
Just for fun let's take your example and say signed overflow returns integer pi. That now means the compiler has to implement your (hypothetical) next line checking if the result is 31415 rather than omitting it under the assumption that it's unreachable because it would imply UB. All of that code suddenly has defined behavior, even if it's silly.
Just for fun let's take your example and say signed overflow returns integer pi. That now means the compiler has to implement your (hypothetical) next line checking if the result is 31415 rather than omitting it under the assumption that it's unreachable because it would imply UB. All of that code suddenly has defined behavior, even if it's silly.
But what does it get you that it's a "defined but completely unusable value" versus "undefined"? Indexing an array by it, adding it to some previously-meaningful value, or doing anything else with it, is still gonna all do practically arbitrary things.
I suppose in some cases it can lead to bugs being harder to exploit, but it's still a bug and still wrong and still should be fixed. Being defined is not a get out of exploitability free card.
(ok I do have one case where "defined but completely arbitrary" is actually meaningful over "undefined" with no reasonable alternative in C - for a floating-point x, "x==(int)x" for checking if x exactly fits in an int - e.g. gcc on aarch64 or x86+AVX (requiring -fno-trapping-math for whatever reason) optimizes that to "x==floor(x)" as an fp-to-integer cast is undefined on overflowing result)
I suppose in some cases it can lead to bugs being harder to exploit, but it's still a bug and still wrong and still should be fixed. Being defined is not a get out of exploitability free card.
(ok I do have one case where "defined but completely arbitrary" is actually meaningful over "undefined" with no reasonable alternative in C - for a floating-point x, "x==(int)x" for checking if x exactly fits in an int - e.g. gcc on aarch64 or x86+AVX (requiring -fno-trapping-math for whatever reason) optimizes that to "x==floor(x)" as an fp-to-integer cast is undefined on overflowing result)
It means you could know what the code will do, that's it. Even that's useful though. It means you can write complete formal models of the language and apply them against your code. The current situation is that you can only build partial formal models, and the assumptions those models rely on evaporate in the presence of UB. It's a really shitty way to do proofs.
Not knowing what the code will do also means that most of the safety critical code in your life is verified through a checkbox that essentially says "I promise there's no undefined behavior". For example, here's what MISRA says about undefined behavior:
Not knowing what the code will do also means that most of the safety critical code in your life is verified through a checkbox that essentially says "I promise there's no undefined behavior". For example, here's what MISRA says about undefined behavior:
Rule 1.3: There shall be no occurrence of undefined or critical unspecified behaviour
Analysis: Undecidable, System
You can absolutely write formal models with the presence of UB - encountering UB is just a call to do_anything(), and the scenarios in which UB happens is itself well-defined. Determining whether any UB can happen is as "undecidable" as determining whether the program follows a given specification - undecidable in the general case, but likely decidable for most specific cases.
Time travel may feel a little funky as you end up not being able to ensure anything leading up to UB happened, but that might not matter much - even if you have "shut_down_engines(); UB();" and are afraid of engines not ever getting shut down, the UB could equivalently also just run start_engines_back_up(), or even without UB some later code sees your off-by-four-billion number and thinks it really needs to (though yes you could have some truly-supposed-to-be-irreversible actions).
I'm pretty sure engineers expected to follow "there shall be no occurrence of UB" are also expected to follow "there shall be no occurrence of behavior we didn't ask you to write" in general - in a car/plane/train integer overflow is likely gonna result in some pretty undesirable behavior regardless of whether that's because the compiler messed with it or because now all your calculations are off by four billion. (and sometimes the compiler can even optimize based on UB to some more desirable code, e.g. "x-y<0" to "x<y" for signed integers, or expanding the range of lengths a loop works on by promoting the index variable)
And you do have UB sanitizers (and perhaps it'd be neat to have compilers have an option to define as much as is reasonable for absolutely critical software that for whatever reason was written in C).
And you cannot even meaningfully have an equivalent to sanitizers on defined operations - if an operation is explicitly defined, people may rely on it, and therefore it is unacceptable to ever warn on it! (ok rust does do a funky thing of making integer overflow trap on debug builds, and be defined to wrap on release ones, but to me this does not seem like a reasonable approach to have on many things)
Time travel may feel a little funky as you end up not being able to ensure anything leading up to UB happened, but that might not matter much - even if you have "shut_down_engines(); UB();" and are afraid of engines not ever getting shut down, the UB could equivalently also just run start_engines_back_up(), or even without UB some later code sees your off-by-four-billion number and thinks it really needs to (though yes you could have some truly-supposed-to-be-irreversible actions).
I'm pretty sure engineers expected to follow "there shall be no occurrence of UB" are also expected to follow "there shall be no occurrence of behavior we didn't ask you to write" in general - in a car/plane/train integer overflow is likely gonna result in some pretty undesirable behavior regardless of whether that's because the compiler messed with it or because now all your calculations are off by four billion. (and sometimes the compiler can even optimize based on UB to some more desirable code, e.g. "x-y<0" to "x<y" for signed integers, or expanding the range of lengths a loop works on by promoting the index variable)
And you do have UB sanitizers (and perhaps it'd be neat to have compilers have an option to define as much as is reasonable for absolutely critical software that for whatever reason was written in C).
And you cannot even meaningfully have an equivalent to sanitizers on defined operations - if an operation is explicitly defined, people may rely on it, and therefore it is unacceptable to ever warn on it! (ok rust does do a funky thing of making integer overflow trap on debug builds, and be defined to wrap on release ones, but to me this does not seem like a reasonable approach to have on many things)
The scenarios in which UB can happen aren't actually well defined by the standards. They're just the negative space outside the constraints. I'll grant that most of the useful scenarios are listed though.
Time travel and inconsistency also prevent the "do_anything()" model from working. There is no consistent behavior in the presence of UB, and the program is not even guaranteed to be translated correctly leading up to that point.
As for running sanitizers on defined operations, all you would need to do is add a new kind of behavior alongside implementation defined, unspecified, and UB with defined behavior that it's explicitly illegal to rely on. You could also treat unspecified in this way, though I'd need to think how dangerous that is.
Speaking of sanitizers, most certified compilers don't actually support them. I've unsuccessfully tried to convince a couple vendors that they're important and even gave them an appropriate bare metal runtime to use if only they'd do the work of calling it. No luck.
Time travel and inconsistency also prevent the "do_anything()" model from working. There is no consistent behavior in the presence of UB, and the program is not even guaranteed to be translated correctly leading up to that point.
As for running sanitizers on defined operations, all you would need to do is add a new kind of behavior alongside implementation defined, unspecified, and UB with defined behavior that it's explicitly illegal to rely on. You could also treat unspecified in this way, though I'd need to think how dangerous that is.
Speaking of sanitizers, most certified compilers don't actually support them. I've unsuccessfully tried to convince a couple vendors that they're important and even gave them an appropriate bare metal runtime to use if only they'd do the work of calling it. No luck.
What happened up to "do_anything()" cannot matter - if you don't like interpreting it as actual time travel, you could alternatively interpret it as the UB rearranging the atoms of the universe to look like some different past happened - no time travel, but result is the same. (done literally you might encounter some issues with physics, but in most practical scenarios reversing some operation after it has happened is plenty simple; and in cases where it's not a C compiler most likely couldn't even have a way to optimize it out, as arbitrary code may include "exit()" at which point removing the invocation is wrong)
"defined behavior that is explicitly illegal to rely on" is a nice oxymoron.
What your certified compilers do or don't support is all a question of self-inflicted problems. (I happen to believe "certified" compilers are primarily a waste of time - with humans writing code/specifications, miscompilations are gonna be an extremely insignificant source of problems, and basically none if you do any amount of testing)
"defined behavior that is explicitly illegal to rely on" is a nice oxymoron.
What your certified compilers do or don't support is all a question of self-inflicted problems. (I happen to believe "certified" compilers are primarily a waste of time - with humans writing code/specifications, miscompilations are gonna be an extremely insignificant source of problems, and basically none if you do any amount of testing)
Again, you can't usefully encode "do_anything()" into a formal model. As an aside, that definition would also break the fundamental abstractions of the standard in amazingly deep ways. Regardless, my point in this particular comment thread is that eliminating undefined behavior is useful, not that I have some grudge against incompleteness.
The standards already have defined behavior that it's explicitly illegal to rely on, so I'm not sure why it's an oxymoron. Strictly conforming programs are prohibited from relying on implementation-defined behavior. You could start dealing with the issue of UB by a 3 word modification of the rules in 4-3 (N3096), though any actual attempt would have to be much more surgical to avoid undoing a decade of compiler optimizations. This isn't an easy issue and I've never pretended otherwise.
Can't say I disagree about certified compilers (though it's extremely hard to detect miscompilations via testing). Regardless, they exist and regulators/certification authorities effectively require them. Since we all have to trust the code they produce with our lives, we may as well not ignore them.
The standards already have defined behavior that it's explicitly illegal to rely on, so I'm not sure why it's an oxymoron. Strictly conforming programs are prohibited from relying on implementation-defined behavior. You could start dealing with the issue of UB by a 3 word modification of the rules in 4-3 (N3096), though any actual attempt would have to be much more surgical to avoid undoing a decade of compiler optimizations. This isn't an easy issue and I've never pretended otherwise.
Can't say I disagree about certified compilers (though it's extremely hard to detect miscompilations via testing). Regardless, they exist and regulators/certification authorities effectively require them. Since we all have to trust the code they produce with our lives, we may as well not ignore them.
Some attempts to come up with a case where gcc or clang optimize in a way not easily describable as a specific "do_anything()":
- printf (or any other external call) before UB - both gcc & clang keep the printf.
- write to atomic before UB - easy to reverse by writing the old value, the interim value needn't ever be visible.
- write to atomic/volatile, spinlock, UB - cannot be optimized out as the loop may be infinite (even in C++ as atomic & volatile are exceptions to "no infinite loops allowed")
- write to volatile before UB - both gcc and clang keep the write.
- read from volatile before UB - gcc keeps the read, but clang removes it. This is the closest I've got, but it's quite far from something you'd actually encounter (and could be easily countered by expecting volatile accesses to potentially exit(), at which point removing them is incorrect)
Now, granted, C doesn't guarantee that all UB time travel must be of the easily-reversed kind, but, seemingly, basically nothing would be lost if it were.
- printf (or any other external call) before UB - both gcc & clang keep the printf.
- write to atomic before UB - easy to reverse by writing the old value, the interim value needn't ever be visible.
- write to atomic/volatile, spinlock, UB - cannot be optimized out as the loop may be infinite (even in C++ as atomic & volatile are exceptions to "no infinite loops allowed")
- write to volatile before UB - both gcc and clang keep the write.
- read from volatile before UB - gcc keeps the read, but clang removes it. This is the closest I've got, but it's quite far from something you'd actually encounter (and could be easily countered by expecting volatile accesses to potentially exit(), at which point removing them is incorrect)
Now, granted, C doesn't guarantee that all UB time travel must be of the easily-reversed kind, but, seemingly, basically nothing would be lost if it were.
Because behavior does eventually get defined somewhere. Just because it's not defined in the C standard it does not mean you can't reason about it.
No, if it was defined somewhere, it'd have a consistent behavior and it wouldn't "time-travel" the way UB can. The word for this in the standards is unspecified behavior. Undefined behavior doesn't need to have any requirements. Different parts of the toolchain and runtime environment (or even different compiler passes) may assume different behaviors for the construct. Even different calls to the same function with the same arguments may produce different behaviors.
Let's walk through a simple example to make this clear. Let's assume you have a macro function foo() that triggers some trivial UB, perhaps integer overflow. Let's also say that this macro function is called the same way in two different translation units. Because there are no requirements on UB by definition, there's no guarantee that those calls will do the same thing, even on the same runtime, using the same compiler, with the same flags. Even the same line of code calling the same arguments may see different things every time, because again there are no required behaviors.
Even code that does not itself trigger UB, but is on an execution path with UB does not have a defined behavior and will commonly be omitted by optimizing compilers like GCC. This has resulted in Linux vulnerabilities where null pointer checks were omitted from the actual binary because other code was "proven" by the compiler to dereference the pointer first.
Let's walk through a simple example to make this clear. Let's assume you have a macro function foo() that triggers some trivial UB, perhaps integer overflow. Let's also say that this macro function is called the same way in two different translation units. Because there are no requirements on UB by definition, there's no guarantee that those calls will do the same thing, even on the same runtime, using the same compiler, with the same flags. Even the same line of code calling the same arguments may see different things every time, because again there are no required behaviors.
Even code that does not itself trigger UB, but is on an execution path with UB does not have a defined behavior and will commonly be omitted by optimizing compilers like GCC. This has resulted in Linux vulnerabilities where null pointer checks were omitted from the actual binary because other code was "proven" by the compiler to dereference the pointer first.
>Because there are no requirements on UB by definition, there's no guarantee that those calls will do the same thing, even on the same runtime, using the same compiler, with the same flags.
Reread my comment. You are talking about behavior not defined by the C standard which I addressed in that comment. Compilers are deterministic. Reproducible builds are a thing.
Reread my comment. You are talking about behavior not defined by the C standard which I addressed in that comment. Compilers are deterministic. Reproducible builds are a thing.
Reproducibility is an entirely unrelated issue. The same compiler can produce different assembly for the same code depending on the surrounding context, or any number of other reasons. A reproducible build just means that you'll get the same binary each time you build it. Furthermore, the same generated assembly can produce different results each time it's run, as data races do. In that case, the only "definition" comes down to the essentially unknowable physical state of the system.
>Reproducibility is an entirely unrelated issue.
No, reproducibility is about having a defined output for a given source code and toolchain.
No, reproducibility is about having a defined output for a given source code and toolchain.
I wrote up a quick example demonstrating UB compiling to two different implementations:
https://godbolt.org/z/nd7GrP44s
Ignore how silly the actual code is and notice that the -O0 assembly checks the pointers before dereferencing them while the -O2 assembly does not. Same compiler, same translation unit, different assembly. Calling each with null pointers will behave differently too. Run this with whatever reproducible toolchain you want. Reproducible builds are not about making undefined behavior deterministic, they're a separate and largely unrelated topic.
Ignore how silly the actual code is and notice that the -O0 assembly checks the pointers before dereferencing them while the -O2 assembly does not. Same compiler, same translation unit, different assembly. Calling each with null pointers will behave differently too. Run this with whatever reproducible toolchain you want. Reproducible builds are not about making undefined behavior deterministic, they're a separate and largely unrelated topic.
In order to make this example you showed me you were successfully able to reason about the output of the compiler despite using UB. You understood how things were defined differently for different optimization levels.
Being able to reason about a particular instance of a particular compiler on a specific undefined construct does not imply UB is defined generally.
I never said it is defined generally. I said that it eventually gets defined as in it may come down to the source code of a specific version of the compiler that defines the behavior.
Yeah, no. Yes, in theory undefined behavior can destroy your entire program. In practice? Not so much.
I do not care about bogeymen that exist in theory. I don't even care about bogeymen that affect your code. I only care about bogeymen that actually affect my code.
I do not care about bogeymen that exist in theory. I don't even care about bogeymen that affect your code. I only care about bogeymen that actually affect my code.
As a user, I do care when people who declare that UB is not a problem because "you just have to write good code" still end up repeatedly shipping apps and libraries with vulnerabilities in them. Which with C and C++ specifically happens all the time, and much more often than in languages with significantly less UB. The proof is in the pudding.
>"Undefined behavior anywhere means you lack defined behavior everywhere in C/C++."
Well, stop programming then. Undefined behavior is everywhere. Your hardware, CPU microcode, any software written in any language etc. etc.
>"As an engineer"
Your statements suggest otherwise.
Well, stop programming then. Undefined behavior is everywhere. Your hardware, CPU microcode, any software written in any language etc. etc.
>"As an engineer"
Your statements suggest otherwise.
Overall I agree with you. But the people writing the C++ standard library have to care.
A core part of the problem of UB in C and C++, is that it is gratuitously over applied.
Mercifully the article calls out the BS argument of "old hardware" justifying UB. It is simply a false argument. The overwhelming majority of UB in C and C++ should be either implementation defined or unspecified behaviour. Security vulnerabilities due to overflow or null dereferences being UB should never have been possible because there are no platforms in which those operations are not defined (some trap, some wrap, some go to infinity), but that is all under the banner of implementation defined behavior. Labelling these things as UB is _solely_ to allow performance optimizations in narrow cases, at the cost of safety in all cases.
In committee meetings I've been in recently the new refrain I'm hearing/reading that has replaced "we need to support various hardware" is an even more stupid argument: if we make it so that these aren't UB then people will rely on the common behavior and write code that is incorrect on platforms that behave differently. e.g. instead of software that is always wrong on one platform, you make software that is semi-randomly wrong on all platforms (because whether or not a compiler removes UB in one case is dependent on compiler version, flags, inlining, etc and if any of those change then suddenly the same code you had yesterday has a security bug when shipped).
Mercifully the article calls out the BS argument of "old hardware" justifying UB. It is simply a false argument. The overwhelming majority of UB in C and C++ should be either implementation defined or unspecified behaviour. Security vulnerabilities due to overflow or null dereferences being UB should never have been possible because there are no platforms in which those operations are not defined (some trap, some wrap, some go to infinity), but that is all under the banner of implementation defined behavior. Labelling these things as UB is _solely_ to allow performance optimizations in narrow cases, at the cost of safety in all cases.
In committee meetings I've been in recently the new refrain I'm hearing/reading that has replaced "we need to support various hardware" is an even more stupid argument: if we make it so that these aren't UB then people will rely on the common behavior and write code that is incorrect on platforms that behave differently. e.g. instead of software that is always wrong on one platform, you make software that is semi-randomly wrong on all platforms (because whether or not a compiler removes UB in one case is dependent on compiler version, flags, inlining, etc and if any of those change then suddenly the same code you had yesterday has a security bug when shipped).
Ub is a bug. We can define what happens, but your code is still wrong if it gets there. Leaving it undefined mean the optimizer can make useful optimizations sith no harm as your code should be useful anyway.
Not always. [fs.race.behavior] makes it undefined behavior to use the C++ filesystem library in a way that introduces a race on the filesystem, including with _other processes_:
https://eel.is/c++draft/fs.race.behavior
I'm not sure how it is possible for a program to avoid this.
https://eel.is/c++draft/fs.race.behavior
I'm not sure how it is possible for a program to avoid this.
Many enough UBs are just a mismatch between the specification and programmer intents. Strict aliasing is a good example: why should aliasing be only allowed through `union` and otherwise UB? Only because it's easier to analyze and optimize. The specification could have instead defined any pair of explicitly aliasing types should be considered aliased, but then the possible optimization will be severely limited (for example, aliasing in one translation unit can inhibit an optimization in other units).
In what sense? C and C++ aren’t memory safe, so the specification has to say something about what happens if you’re dereferencing an invalid pointer (random value, out of bounds, frees pointer, etc).
That’s what UB exists for: there’s no behaviour we can actually define for some operations.
That’s what UB exists for: there’s no behaviour we can actually define for some operations.
My favourite description of undefined behaviour. The poster is corrected later on in the thread about whether the specific operation discussed would invoke undefined behaviour, but the description of what happens when undefined behaviour occurs is gold:
https://groups.google.com/g/comp.lang.c/c/ZE2B2UorTtM/m/1ROv...
Joona I Palaste, 2001-01-19, comp.lang.c
https://groups.google.com/g/comp.lang.c/c/ZE2B2UorTtM/m/1ROv...
Joona I Palaste, 2001-01-19, comp.lang.c
This isn't about the post-increment operator, this is about the order
of evaluation of the operands.
Since you're modifying the value of i twice without a sequence point
in between, either of the two results are exactly as much "expected".
Also, equally "expected" behaviour includes incrementing every
variable in the array, flipping all the bits in every variable in the
array, converting all instances of the text string "/usr" in memory
to "fsck", changing the colours of your screen to purple, calling the
police on your modem line and telling them you're being attacked by
a one-eyed Martian wielding a herring while singing "Hi ho, it's off
to work we go", and even weirder stuff.
So... what it all boils to... when writing your compiler, just flip
a coin and use the one of the two behaviours you listed that
corresponds with the coin's face.And yet the standard explicitly states that undefined behaviour can behave in some documented manner characteristic of the environment. As a simple question of quality of implementation, we should surely be able to demand that nothing confusing happens.
I don't disagree, and I think the quote above follows that idea. Undefined behaviour means that anything _could_ happen, but compiler writers should ensure something sensible happens in those cases. At least, that's what I took from it.
At this point we should just bite the bullet and make it not just defined, but defined in a way that results in safe code, even if that code is slower (e.g. for overflow, panic). We have computers that are many orders of magnitude faster than anything that was around back in the days C++, much less C, was originally designed. And most code that runs on them is not performance critical, so we could absolutely turn on null checks, overflow checks, bounds checks etc most everywhere and things would still be fine - but with less (and more visible, thus easier to find) bugs. This whole mentality that if you are writing in C++, you must let the compiler squeeze every last bit of perf out of your code, is both dangerous and unneeded.
Not the first discussion of this topic, by any means. In this case, I've tried to boil it down to the essential points a practical programmer needs to know, but the article still ended up longer than I initially aimed for.
One hopefully constructive comment… I didn’t find this a motivating example as intended:
I’m sure there’s a better example of why UB helps optimization, but this one didn’t work for me.
int foo_or_bar(int which) {
// Assumes you don't mind both functions being called
int x = foo();
int y = bar();
return *(&x + which);
}
I’m sure there’s a better example of why UB helps optimization, but this one didn’t work for me.
> Well, how about the rule is that if I take the address of x, it can’t be put in a register?
The issue is that y might end up in a register, and you didn't take the address of y.
The issue is that y might end up in a register, and you didn't take the address of y.
Ohhh. I misunderstood the example. That would not only depend on a lack of register usage, but depend on where the variables are stored on the stack, which I don’t think anyone could reasonably demand even in 1976.
So I still feel there must be a more reasonable and therefore motivating example of optimizations one would want that are enabled by surprising you with UB. (Which I think was the idea behind this example.)
So I still feel there must be a more reasonable and therefore motivating example of optimizations one would want that are enabled by surprising you with UB. (Which I think was the idea behind this example.)
> depend on where the variables are stored on the stack, which I don’t think anyone could reasonably demand even in 1976
You'd be surprised. A typical compiler of that era would be single-pass, and allocating variables on the stack in order in which they were declared was not uncommon. Don't forget, we're talking about a language that actually had a "register" keyword solely to tell the compiler to enregister the variable!
You'd be surprised. A typical compiler of that era would be single-pass, and allocating variables on the stack in order in which they were declared was not uncommon. Don't forget, we're talking about a language that actually had a "register" keyword solely to tell the compiler to enregister the variable!
Well, now that you mention it, that’s true, I was there. :) But I just want to go back to 1992, not 1976.
Here's another interesting post if you want to delve further into an example of undefined behavior created by gcc optimization: https://thephd.dev/c-undefined-behavior-and-the-sledgehammer....
Also, this quote comes to mind: "C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off": https://www.stroustrup.com/quotes.html
Also, this quote comes to mind: "C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off": https://www.stroustrup.com/quotes.html
Here's some more on gcc optimization from 2010, discusses how GCC optimization started eliminating null pointer checks in Linux kernel code necessiting compiling at a lower optimization level (towards the bottom). On the 'good' side, it also mentions tight loops can speed up 30-50% if the compiler can ignore signed integer overflow. Also has 'best practices' list:
https://blog.regehr.org/archives/213
https://blog.regehr.org/archives/213
[deleted]
[deleted]
In the bit where he shows
void error(const char* msg);
int successor(int a) {
if (a + 1 < a) error("Integer overflow!");
return a + 1;
}
It's optimized out on both gcc and clang on -O1 and above. -O3 is presumably just what the author defaults to for enabling optimizations (I also write -O3 everywhere by default).
I recommend reading the resources under https://en.cppreference.com/w/c/language/behavior#External_l... (–> External links).
I don't get it.
How can UB on double-free, use-after-free, dangling pointers, etc lead to optimizations?
How can UB on double-free, use-after-free, dangling pointers, etc lead to optimizations?
Making double-free an UB makes `free` more efficient because there are less checks to make. Combined with use-after-free as an UB, that deallocated memory can be immediately reused for the next allocation without any repercussion. And making dangling pointer an UB makes most pointer analysis much more doable.
Once again, I want to plead. At least have a Warning option to annotate any time undefined behavior is encountered by a compiler. The goal should be to promote optimizations to written code and improve code quality. Not just the result of one particular compiler.
Hi! I'm a former compiler engineer who specialized on undefined behaviour.
Would you like warnings on:
On the flip side, I can offer -fsanitize=undefined which will catch when you do many things that have UB at runtime. It does not change the ABI which means that there are some bugs it can't catch, but deploying it is easier since you do not need to recompile all your libraries with it (like your C++ standard library and C library, in particular). You can use this to help you build unit tests that send intentionally overflowing values into your functions and show that they do not overflow. It turns untestable problem (since you cannot check for UB after it happens) into a problem you can write deterministic tests for.
Would you like warnings on:
* int f(int x, int y) { return x + y; }
* int get_x_coord(Point *p) { return p->x; }
* void compute_and_cache(const char *key) { *get_cache_bucket_for(key) = compute_value_for(key); }
On the flip side, I can offer -fsanitize=undefined which will catch when you do many things that have UB at runtime. It does not change the ABI which means that there are some bugs it can't catch, but deploying it is easier since you do not need to recompile all your libraries with it (like your C++ standard library and C library, in particular). You can use this to help you build unit tests that send intentionally overflowing values into your functions and show that they do not overflow. It turns untestable problem (since you cannot check for UB after it happens) into a problem you can write deterministic tests for.
The problem isn’t overflow, the problem is the backwards logic of the compiler assuming there will never be any execution that leads to overflow, so the code that overflows just vanishes completely.
In other words,
In other words,
bool overflowed = (x+1)<x;
How about if the x+1 was from an inlined function? a macro? perhaps even just a variable defined 10 lines up?
In a 7-character sequence it may seem pretty obvious that it's probably intended to mean something, but when joining together multiple independent things doing redundant operations it's much less trivial.
In a 7-character sequence it may seem pretty obvious that it's probably intended to mean something, but when joining together multiple independent things doing redundant operations it's much less trivial.
int game_logic(int prev_score, int bonus) {
if (bonus < 0) { // check for bad argument
return -1;
}
int new_score = prev_score + bonus;
// ...
if (new_score < prev_score) { // sanity check
abort();
}
return new_score;
}
// in the above the abort path can (and is) already optimized out, but an invocation of game_logic(x,1) becomes exactly x+1<x bool overflowed = (x+1)<x;
Nevertheless, I don't know what to do about the suggestion "it may or may not do what you want on any given architecture, but it shouldn’t just be assumed false." The compiler needs to know what it can and can't do. The common advice of "just do what the CPU does" doesn't work, we need to know what to do when cross-compiling, when constant folding, and we need to know which instructions are valid to select. If I selected PADDSW for this add (a saturating addition operation) but then later when you use x+1 I select a non-saturating addition, would you be happy with that? Probably not. The compiler needs actual rules to follow.
I don't know how to apply the suggestion about warnings when we end up deleting dead code. The compiler deletes dead code all the time, consider a case like "vector<t> v; v.push_back(a); v.push_back(b);", each push_back begins with an if-statement on whether reallocation is required, and that becomes constant with inlining. Tracking "this code became dead because", well, because which situations exactly?
Cross-compiling is irrelevant, because if the behavior is processor-specific then the cross-compiler knows the behavior. Constant folding shouldn’t happen in this case because x is not constant. If you choose a saturating add consistently on this processor target, and document it, that’s fine.
Deleting dead code because the code demonstrably wouldn’t do anything (that is, it has defined behavior that is not observable) makes sense and is of course hugely useful. Deleting code that isn’t dead, it just doesn’t have a universally defined behavior, is the issue.
Deleting dead code because the code demonstrably wouldn’t do anything (that is, it has defined behavior that is not observable) makes sense and is of course hugely useful. Deleting code that isn’t dead, it just doesn’t have a universally defined behavior, is the issue.
> Deleting code that isn’t dead, it just doesn’t have a universally defined behavior, is the issue.
Can I delete "if (x & 3 == 16)" without a warning? There is no 'x' which makes that expression true, so I can safely fold it to false without a warning?
Can I delete "if (x + 1 < x)" without a warning? There is no signed 'x' which makes that expression true, so I can safely fold it to false without a warning?
How about this:
Or this:
The problem with UB is that you can use it to set up impossible situations, like create an 'x' where x & 3 == 16 is true or a variable whose address was never taken being modified through a pointer, and so on. If you account for UB then "code that doesn't have a universally defined behaviour" becomes all code.
Ideally I think the first two examples should have warnings, though not because we delete the code, and the last two shouldn't? The warning should be because it's a tautology so the human likely didn't mean to write that (for instance if the human wrote it indirectly through macros, then we shouldn't warn on it).
Can I delete "if (x & 3 == 16)" without a warning? There is no 'x' which makes that expression true, so I can safely fold it to false without a warning?
Can I delete "if (x + 1 < x)" without a warning? There is no signed 'x' which makes that expression true, so I can safely fold it to false without a warning?
How about this:
int x = 7;
call_function_outside_this_file();
if (x != 7) { /* dead */ }
Or this:
void f(int *x, float *y) {
*x = 1;
*y = 2;
if (*x != 1) { /* dead */ }
The problem with UB is that you can use it to set up impossible situations, like create an 'x' where x & 3 == 16 is true or a variable whose address was never taken being modified through a pointer, and so on. If you account for UB then "code that doesn't have a universally defined behaviour" becomes all code.
Ideally I think the first two examples should have warnings, though not because we delete the code, and the last two shouldn't? The warning should be because it's a tautology so the human likely didn't mean to write that (for instance if the human wrote it indirectly through macros, then we shouldn't warn on it).
You’re on to something with that last example. The idea that those two pointers can’t alias is one place C has diverged from my understanding. Of course they can alias. Which is why I wouldn’t naturally write that code, I’d write:
I’m obviously just a holdover from the 90s, but it does seem we’ve leaned too far into hidden assumptions that the compiler thinks I share, rather than doing what the code says, or a simplification of what the code says.
void f(int *px, float *py) {
int x = *px;
x = 1;
*y = 2;
if (x != 1) { /* dead */ }
*px = x;
I’m obviously just a holdover from the 90s, but it does seem we’ve leaned too far into hidden assumptions that the compiler thinks I share, rather than doing what the code says, or a simplification of what the code says.
> If I put in a dereference, I expect a dereference to happen. Not dereferencing the pointer when I wrote a dereference operator seems like going too far.
Surely not? I mean, you probably didn't intend to include unevaluated contexts like "sizeof(ptr)" where putting in a memory access is forbidden, but I think nearly-all programmers fully expect the compiler to delete the dead store in "ptr = a; ptr = b;" or "ptr = x; free(ptr);" and would get annoyed if it didn't. Especially if we can't just take a scalar computation in a loop, move the memory access to register, then store it to memory only once when we're done.
I once did a cleanup of undefined behaviour dereferencing NULL pointers (-fsanitize=null) and I got a lot of pushback from people complaining about "&ptr" where the ptr is NULL, because the compiler doesn't emit any assembly for that, so their code is just fine as is.
The rule for memory is that all memory you've stored to has an effective-type -- same as the static types but for addresses at runtime -- and a pointer has to point to an object with the effective-type matching the pointer's static type. Further details aside (uninitialized pointers, pointers to data you just freed, freshly malloc'd memory which has no effective type yet, unions) when you think of it in this model, the fact you can't have an int and float* pointing to the memory feels natural.
Surely not? I mean, you probably didn't intend to include unevaluated contexts like "sizeof(ptr)" where putting in a memory access is forbidden, but I think nearly-all programmers fully expect the compiler to delete the dead store in "ptr = a; ptr = b;" or "ptr = x; free(ptr);" and would get annoyed if it didn't. Especially if we can't just take a scalar computation in a loop, move the memory access to register, then store it to memory only once when we're done.
I once did a cleanup of undefined behaviour dereferencing NULL pointers (-fsanitize=null) and I got a lot of pushback from people complaining about "&ptr" where the ptr is NULL, because the compiler doesn't emit any assembly for that, so their code is just fine as is.
The rule for memory is that all memory you've stored to has an effective-type -- same as the static types but for addresses at runtime -- and a pointer has to point to an object with the effective-type matching the pointer's static type. Further details aside (uninitialized pointers, pointers to data you just freed, freshly malloc'd memory which has no effective type yet, unions) when you think of it in this model, the fact you can't have an int and float* pointing to the memory feels natural.
If code is dead or unreached, and therefore deleted / no instructions emitted; clearly THAT is of a level a warning should cover, if not an error.
Dead code happens all the time. Adjusting the example in my comment you're replying to:
vector<t> v;
v.push_back(a);
v.push_back(b);
v.push_back(c);
v.push_back(d);
template <typename T> struct vector<t> {
size_t len = 0;
size_t storage_len = 0;
T *storage = nullptr;
void push_back(T t) {
if (len + 1 > storage_len) {
storage_len = storage_len ? storage_len * 2 : 1;
storage = realloc(storage, storage_len);
}
storage[len] = t;
++len;
}
};
vector<t> v;
v.push_back(a);
v.push_back(b);
v.push_back(c);
v.push_back(d);
len = 0;
storage_len = 0;
storage = nullptr;
if (len + 1 > storage_len) {
storage_len = storage_len ? storage_len * 2 : 1;
storage = realloc(storage, storage_len);
}
storage[len] = a;
++len;
if (len + 1 > storage_len) {
storage_len = storage_len ? storage_len * 2 : 1;
storage = realloc(storage, storage_len);
}
storage[len] = b;
++len;
if (len + 1 > storage_len) {
storage_len = storage_len ? storage_len * 2 : 1;
storage = realloc(storage, storage_len);
}
storage[len] = c;
++len;
if (len + 1 > storage_len) {
storage_len = storage_len ? storage_len * 2 : 1;
storage = realloc(storage, storage_len);
}
storage[len] = d;
++len;
len = 0;
storage_len = 0;
storage = nullptr;
if (0 + 1 > 0) {
storage_len = 1;
storage = realloc(storage, 1);
}
storage[0] = a;
len = 1;
if (1 + 1 > 1) {
storage_len = 2;
storage = realloc(storage, 2);
}
storage[1] = b;
len = 2;
if (2 + 1 > 2) {
storage_len = 4;
storage = realloc(storage, 4);
}
storage[2] = c;
len = 3;
if (3 + 1 > 4) {
storage_len = 8;
storage = realloc(storage, 8);
}
storage[3] = d;
len = 4;
As a hypothetical, lets assume there's a compiler that has a deep transformation around alloc/reallocs to optimize them when they're only fed 'static' input sizes.
// Source by nlewycky
template <typename T> struct vector<t> {
size_t len = 0;
size_t storage_len = 0;
T *storage = nullptr;
void push_back(T t) {
if (len + 1 > storage_len) {
// FIXME: what if storage_len > (size_t_max >> 1) ??
// ? Define maximum expected growth unit? 1K? 4K? 64K?
// Assignment from trinary op visually confusing, add ( )
storage_len = ( storage_len ? storage_len * 2 : 1 );
storage = realloc(storage, storage_len);
}
storage[len] = t;
++len;
}
};
//
vector<t> v;
v.push_back(a);
v.push_back(b);
v.push_back(c);
v.push_back(d);
// Pseudo code
template <typename T> vector<T> v = {
// final state at end of code section
size_t len = 4;
size_t storage_len = 8;
T *storage = realloc(NULL, storage_len);
};
// Compiled and included, forgot template function syntax.
void template <typename T> vector<T>.push_back(T t);
(v[0], v[1], v[2], v[3]) = (a, b, c, d);
No, because the code as it was written is still evaluated and executed. Said execution happened to become static at the time the program was compiled and 3 out of the 4 times it was executed an effect and binary output was generated.
In all the cases that the hypothetical -Wubelim would cover the written code would be evaluated by the compiler and 'as it's undefined it can't ever be true, silent elimination'. That's the case where the human that wrote the code and the compiler that's interpreting a specification to claim that code can't ever do anything. Rather than transliterate the code as it was written to the machine instructions the programmer probably expected to see were this native machine assembly they'd written, the compiler behaved differently, silently.
In all the cases that the hypothetical -Wubelim would cover the written code would be evaluated by the compiler and 'as it's undefined it can't ever be true, silent elimination'. That's the case where the human that wrote the code and the compiler that's interpreting a specification to claim that code can't ever do anything. Rather than transliterate the code as it was written to the machine instructions the programmer probably expected to see were this native machine assembly they'd written, the compiler behaved differently, silently.
I think the -fwrapv switch in GCC allows this to work properly. I don't actually know about this specific case, but -fwrapv makes signed and unsigned arithmetic to wrap around so that only the appropriate number of low bits are kept.
(I commonly use -fwrapv when writing programs in C.)
(I commonly use -fwrapv when writing programs in C.)
[deleted]
Different levels of warnings might be useful.
-Wub # Warn _anytime_ there is detected potential undefined behavior, irrespective of if there is an associated optimization.
-Wubelim # Warn any time code is eliminated as a result of undefined behavior / assumptions.
-Wub... # Any other classes of UB optimizations that change the program as (incorrectly) written.
Again, the goal is to provide feedback that improves the program and possibly educates / reminds the programmer about how their meanings might be misunderstood.
-Wub # Warn _anytime_ there is detected potential undefined behavior, irrespective of if there is an associated optimization.
-Wubelim # Warn any time code is eliminated as a result of undefined behavior / assumptions.
-Wub... # Any other classes of UB optimizations that change the program as (incorrectly) written.
Again, the goal is to provide feedback that improves the program and possibly educates / reminds the programmer about how their meanings might be misunderstood.
> -Wub # Warn _anytime_ there is detected potential undefined behavior, irrespective of if there is an associated optimization.
Can I ask you, have you tried any existing tools? Coverity static analysis, Klocwork, PVS-Studio, clang static analysis, tis-interpreter, Frama-C? What did you think of those? If not, why not (how important is the problem to you)?
My understanding of these tools is that they start by marking every spot potential UB could happen -- every add is potentially overflowing, every pointer dereference is potentially null or freed or whatnot, and then they use solvers to prove that the UB does not occur, and print out the rest. The benefit they have is that they can examine more than one file at a time (the compiler may only look at one .c file at a time) and they have permission to take much longer than compiling.
> -Wubelim # Warn any time code is eliminated as a result of undefined behavior / assumptions.
This doesn't happen, the compiler doesn't detect your UB and use that to delete your code. Consider this:
Another way to think about it is that the assumptions we make about your program being free of UB are completely indistinguishable from all the rest of the correct and working code. "int x = 4;" should declare a new variable, named x, with an int's worth of memory, initialized to the value 4. That is precisely as true to the compiler as any UB-performing, code. When you write "p->xcoord" you are telling the compiler that 'p' is a valid pointer to an object of its type at this moment, and it believes you. Trust the programmer, and all that.
> -Wub... # Any other classes of UB optimizations that change the program as (incorrectly) written.
"UB optimizations" isn't a thing. It just isn't. The optimizations never change the program, at least, not unless the compiler is buggy. The compiler's job is to find some assembly which meets the specification we call the program. With the optimizer enabled, we spend more time so that we can select assembly that minimizes a cost model we have for the execution time on the underlying machine (or sometimes file size).
> Again, the goal is to provide feedback that improves the program and possibly educates / reminds the programmer about how their meanings might be misunderstood.
FWIW we agree on the goal.
The model for warnings in clang at least has been to look at the code as it is typed, and focus on errors that programmers make. We have all kinds of complex rules for warnings, like "if (3 < 4)" issues a warning (-Wtautological-compare) but "if (MAX_THREADS < MAX_CORES)" with #define MAX_THREADS 3 and #define MAX_CORES 4 doesn't. We've put a ton of effort into getting this sort of thing right, and that includes warnings that code will always produce UB when run, even if it was expanded through macros or templates. It's not an exhaustive system, the warnings work was guided by actual bugs we've encountered in real systems.
There might be another way to do this. The C++ constexpr feature has the compiler evaluate some functions at compile time and detect any UB they encounter as they run. The clang implementation of this can also handle working with values that are not known at compile time, and working with dynamic allocations. One could try to run every function with the constexpr evaluator and see whether it does a better job at producing good warnings, then remove the redundant warnings (made by pattern matching on the AST) and see if the result is fast enough to use as part of regular compilation.
Can I ask you, have you tried any existing tools? Coverity static analysis, Klocwork, PVS-Studio, clang static analysis, tis-interpreter, Frama-C? What did you think of those? If not, why not (how important is the problem to you)?
My understanding of these tools is that they start by marking every spot potential UB could happen -- every add is potentially overflowing, every pointer dereference is potentially null or freed or whatnot, and then they use solvers to prove that the UB does not occur, and print out the rest. The benefit they have is that they can examine more than one file at a time (the compiler may only look at one .c file at a time) and they have permission to take much longer than compiling.
> -Wubelim # Warn any time code is eliminated as a result of undefined behavior / assumptions.
This doesn't happen, the compiler doesn't detect your UB and use that to delete your code. Consider this:
int x = 4;
int y;
int *p = &y - sizeof(int);
*p = 7;
printf("%d", x);
Another way to think about it is that the assumptions we make about your program being free of UB are completely indistinguishable from all the rest of the correct and working code. "int x = 4;" should declare a new variable, named x, with an int's worth of memory, initialized to the value 4. That is precisely as true to the compiler as any UB-performing, code. When you write "p->xcoord" you are telling the compiler that 'p' is a valid pointer to an object of its type at this moment, and it believes you. Trust the programmer, and all that.
> -Wub... # Any other classes of UB optimizations that change the program as (incorrectly) written.
"UB optimizations" isn't a thing. It just isn't. The optimizations never change the program, at least, not unless the compiler is buggy. The compiler's job is to find some assembly which meets the specification we call the program. With the optimizer enabled, we spend more time so that we can select assembly that minimizes a cost model we have for the execution time on the underlying machine (or sometimes file size).
> Again, the goal is to provide feedback that improves the program and possibly educates / reminds the programmer about how their meanings might be misunderstood.
FWIW we agree on the goal.
The model for warnings in clang at least has been to look at the code as it is typed, and focus on errors that programmers make. We have all kinds of complex rules for warnings, like "if (3 < 4)" issues a warning (-Wtautological-compare) but "if (MAX_THREADS < MAX_CORES)" with #define MAX_THREADS 3 and #define MAX_CORES 4 doesn't. We've put a ton of effort into getting this sort of thing right, and that includes warnings that code will always produce UB when run, even if it was expanded through macros or templates. It's not an exhaustive system, the warnings work was guided by actual bugs we've encountered in real systems.
There might be another way to do this. The C++ constexpr feature has the compiler evaluate some functions at compile time and detect any UB they encounter as they run. The clang implementation of this can also handle working with values that are not known at compile time, and working with dynamic allocations. One could try to run every function with the constexpr evaluator and see whether it does a better job at producing good warnings, then remove the redundant warnings (made by pattern matching on the AST) and see if the result is fast enough to use as part of regular compilation.
>int *p = &y - sizeof(int);
Funny how arguing about UB shows that even compiler writers can miss when they are juggling with lots of sharp objects :) There is no way you meant to write integer pointer value - 4 if your intent was 'next to it'. And double the fun is that some future hardware may do sizeof(int) == 1 and have it running 'just fine' :)
As gamedev engine person doing bugs all the time I wish we had some middle ground but a lot of smart people do not want to find it. I guess it is cheaper that way when we just build on the existing C code and hope for the best.
Funny how arguing about UB shows that even compiler writers can miss when they are juggling with lots of sharp objects :) There is no way you meant to write integer pointer value - 4 if your intent was 'next to it'. And double the fun is that some future hardware may do sizeof(int) == 1 and have it running 'just fine' :)
As gamedev engine person doing bugs all the time I wish we had some middle ground but a lot of smart people do not want to find it. I guess it is cheaper that way when we just build on the existing C code and hope for the best.
I've forgotten if 'address of' ( & ) is signed or unsigned. Worse, in a quick search online I can't seem to find anyplace that mentions what the return type of address of is, just a bunch of basic working with pointers pages.
The pointer math line should still be legal, even if it would be 'unsafe' in some popular other languages. Useful for determining where to write to memory mapped files or IO. Not so useful in the above case.
The warnings/errors I'd expect would be: *p = 7; // out of known bounds write
The pointer math line should still be legal, even if it would be 'unsafe' in some popular other languages. Useful for determining where to write to memory mapped files or IO. Not so useful in the above case.
The warnings/errors I'd expect would be: *p = 7; // out of known bounds write
> I've forgotten if 'address of' ( & ) is signed or unsigned.
Neither, pointers are their own types which have no sign, only integral types come in signed and unsigned. There exist intptr_t (signed) and uintptr_t (unsigned) which are integral types you can losslessly cast a pointer to. Also the difference of two pointers is a ptrdiff_t which is a signed integral type.
> The warnings/errors I'd expect would be: *p = 7; // out of known bounds write
In that regard I picked a bad example, this code always executes undefined behaviour, so this could indeed be detected by a compiler warning at the cost of doing a simulated execution of the code. Most real problems come about where the code is UB only for certain runtime inputs.
The reason I chose that example was to make a point about how the compiler doesn't realize it's doing anything to "change" the program, and isn't going out of its way to optimize based on undefined behavior. It assumes that a local variable's value can't change with it being assigned to, so how could it know whether to issue the warning?
Another way to put it is:
> The pointer math line should still be legal, even if it would be 'unsafe' in some popular other languages.
I'm not sure what you mean by "should", you might be suggesting a change to C or you might be stating what you think the code currently does. Right now in C, the very creation of an invalid pointer is UB whether you use the thing or not. I'm told this is because of very old CPU designs that had distinct pointer and integer registers and loading an invalid address into the pointer register would trap.
Neither, pointers are their own types which have no sign, only integral types come in signed and unsigned. There exist intptr_t (signed) and uintptr_t (unsigned) which are integral types you can losslessly cast a pointer to. Also the difference of two pointers is a ptrdiff_t which is a signed integral type.
> The warnings/errors I'd expect would be: *p = 7; // out of known bounds write
In that regard I picked a bad example, this code always executes undefined behaviour, so this could indeed be detected by a compiler warning at the cost of doing a simulated execution of the code. Most real problems come about where the code is UB only for certain runtime inputs.
The reason I chose that example was to make a point about how the compiler doesn't realize it's doing anything to "change" the program, and isn't going out of its way to optimize based on undefined behavior. It assumes that a local variable's value can't change with it being assigned to, so how could it know whether to issue the warning?
Another way to put it is:
void written_in_asm_in_another_file();
void test() {
int x = 4;
written_in_asm_in_another_file();
printf("%d", x);
}
> The pointer math line should still be legal, even if it would be 'unsafe' in some popular other languages.
I'm not sure what you mean by "should", you might be suggesting a change to C or you might be stating what you think the code currently does. Right now in C, the very creation of an invalid pointer is UB whether you use the thing or not. I'm told this is because of very old CPU designs that had distinct pointer and integer registers and loading an invalid address into the pointer register would trap.
I wouldn't like warnings for these things. I'd like them all to trap in a well-defined (but non-recoverable) way if UB actually gets triggered. And I'd like this to be the default behavior, even in release builds. Safety should never be opt-in.
I can offer you -fsanitize=undefined -fsanitize-trap=undefined, which you'd need to put in your configuration for release builds, presumably you have other flags in your build system (like -O2) already.
It's possible that a program terminating based on attacker influenced values could be used as a channel to leak confidential data to the attacker, so I'd suggest that developers decide whether to use this on a case-by-case basis. (Maybe it should default to on, but we'd need user education so people who are building sensitive systems know they need to turn it off.)
It's possible that a program terminating based on attacker influenced values could be used as a channel to leak confidential data to the attacker, so I'd suggest that developers decide whether to use this on a case-by-case basis. (Maybe it should default to on, but we'd need user education so people who are building sensitive systems know they need to turn it off.)
UB is not an event that happens. It’s an assumption baked into the design of the compiler.
For example, on 64-bit arch if you index arrays by an int, compiler can use CPU’s 64-bit addressing modes even when they don’t overflow when 32-bit int would. The compiler is taking advantage of it all the time. It wouldn’t make sense to warn about every array, but OTOH the compiler can’t know at compile time if your pointer arithmetic will ever overflow an int.
For example, on 64-bit arch if you index arrays by an int, compiler can use CPU’s 64-bit addressing modes even when they don’t overflow when 32-bit int would. The compiler is taking advantage of it all the time. It wouldn’t make sense to warn about every array, but OTOH the compiler can’t know at compile time if your pointer arithmetic will ever overflow an int.
Undefined behavior are runtime conditions, in the general case, not compile-time conditions.
Eliminating a statement by assuming UB will never happen is a compile-time condition.
I think the problem is more that it’s not as if there’s a single place in the compiler saying “aha! UB! let’s surprise the developer!”. It’s the effect of propagation through multiple optimization steps.
I think the problem is more that it’s not as if there’s a single place in the compiler saying “aha! UB! let’s surprise the developer!”. It’s the effect of propagation through multiple optimization steps.
Not quite, it’s the fact that if you have to assume the UB condition and make behavior defined for that condition, then you can’t apply certain optimizations, and/or you have to generate extra code to detect and handle the UB condition.
In any case, it would mean that existing programs that do not exhibit UB (but that contain expressions that could be UB when executed in the context of a different program) would suddenly compile to less efficient code. It’s not surprising that compiler vendors have little interest in agreeing to such changes to the C standard, which effectively would mean a performance regression for their compilers.
You can’t change the situation without either forfeiting some performance or changing existing programs.
This is how UB came about in the first place, in the first ANSI C version. Everything the compiler vendors couldn’t agree on to specify even just an implementation-defined behavior for, became UB.
In any case, it would mean that existing programs that do not exhibit UB (but that contain expressions that could be UB when executed in the context of a different program) would suddenly compile to less efficient code. It’s not surprising that compiler vendors have little interest in agreeing to such changes to the C standard, which effectively would mean a performance regression for their compilers.
You can’t change the situation without either forfeiting some performance or changing existing programs.
This is how UB came about in the first place, in the first ANSI C version. Everything the compiler vendors couldn’t agree on to specify even just an implementation-defined behavior for, became UB.
I am anger commenting, because I'm just sick of this, but this article still says nothing to convince me that any of this matters.