Integers in C quiz(acepace.net)
acepace.net
Integers in C quiz
https://www.acepace.net/integerQuiz/
243 comments
The trouble with explicit casting is if the code is refactored to change the underlying integer type, the explicit casts may silently truncate the integer, introducing bugs.
D follows the C integral promotion rules, with a couple crucial modifications:
1. No implicit conversions are done that throw away information - those will require an explicit cast. For example:
Continuing with the notion that casts should be avoided where practical is the cast expression has its own keyword. This makes casting greppable, so the code review can find them. C casts require a C parser with lookahead to find.
One other difference: D's integer types have fixed sizes. A char is 8 bits, a short is 16, an int is 32, and a long is 64. This is based on my experience that a vast amount of C programming time is spent trying to account for the implementation-defined sizes of the integer types. As a result, D code out of the box tends to be far more portable than C.
D also defines integer math as 2's complement arithmetic. All that 1's complement stuff belongs in the dustbin of history.
D follows the C integral promotion rules, with a couple crucial modifications:
1. No implicit conversions are done that throw away information - those will require an explicit cast. For example:
int i = 1999;
char c = i; // not allowed
char d = cast(char)i; // explicit cast
int i = 1999;
char c = i & 0xFF; // allowed
Continuing with the notion that casts should be avoided where practical is the cast expression has its own keyword. This makes casting greppable, so the code review can find them. C casts require a C parser with lookahead to find.
One other difference: D's integer types have fixed sizes. A char is 8 bits, a short is 16, an int is 32, and a long is 64. This is based on my experience that a vast amount of C programming time is spent trying to account for the implementation-defined sizes of the integer types. As a result, D code out of the box tends to be far more portable than C.
D also defines integer math as 2's complement arithmetic. All that 1's complement stuff belongs in the dustbin of history.
Yeah for sure, having more expressive casts and differentiating between "upcasts" and "downcasts" is definitely better. My point that even "lossy" casts are better than C's weird arcane implicit promotion rules by quite a margin IMO.
Rust's current handling of this issue is by no mean perfect, although it's been steadily improving and I definitely don't use `as` as much as I used to.
>D also defines integer math as 2's complement arithmetic.
I think modern C standards does so as well, but I think than signed overflow is still UB, so it's mostly about defining signed-to-unsigned conversions. There are flags on many compilers to tell them to assume wrapping arithmetic but obviously that's not standard...
Rust's current handling of this issue is by no mean perfect, although it's been steadily improving and I definitely don't use `as` as much as I used to.
>D also defines integer math as 2's complement arithmetic.
I think modern C standards does so as well, but I think than signed overflow is still UB, so it's mostly about defining signed-to-unsigned conversions. There are flags on many compilers to tell them to assume wrapping arithmetic but obviously that's not standard...
It's not perfect. People do complain that:
char a,b,c;
a = b + c; // error
a = cast(char)(b + c); // ok
Arguably Rust does better for this particular example because, for debug builds and optionally release builds, overflows trigger an error. If you explicitly want wrapping arithmetic in Rust you should use b.wrapping_add(c) in this case, otherwise you'll get a runtime error.
So for instance the following code generates an error (I add to add the function call indirection otherwise rustc would even refuse to compile the straightforward `200u8 + 100u8`):
So for instance the following code generates an error (I add to add the function call indirection otherwise rustc would even refuse to compile the straightforward `200u8 + 100u8`):
fn main() {
println!("{}", add(200, 100));
}
fn add(a: u8, b: u8) -> u8 {
a + b
}
// thread 'main' panicked at 'attempt to add with overflow', test.rs:6:5
// note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
I actually dislike this solution, because it relies on tests, while this could easily be a compile-time error.
Not sure I like that D promotes implicitly and then complains about the result of the addition not being assignable to a `char`.
But in theory, every integer could just have the maximum range of values encoded in its type, and for every arithmetic operation the compiler could check if the maximum range of the integer type is exceeded.
You could set the range explicitly:
There are crates that add ranged integers as types, but I haven’t found them satisfactory to work with.
Not sure I like that D promotes implicitly and then complains about the result of the addition not being assignable to a `char`.
But in theory, every integer could just have the maximum range of values encoded in its type, and for every arithmetic operation the compiler could check if the maximum range of the integer type is exceeded.
You could set the range explicitly:
fn mult_col(color:u8<0..31>, factor:u8<0..8>): u8<0..248> { color*factor }
There are crates that add ranged integers as types, but I haven’t found them satisfactory to work with.
> The drawback is that it incurs a performance hit and is therefore only enabled in debug builds by default.
D has core library functions to check for integer overflow. In my not-so-humble opinion, a targeted solution like this is better than a global switch to turn it on and off. Adding is deeply embedded into computation, and very very few are at any risk of overflowing.
D has core library functions to check for integer overflow. In my not-so-humble opinion, a targeted solution like this is better than a global switch to turn it on and off. Adding is deeply embedded into computation, and very very few are at any risk of overflowing.
Rust also has checked operations (i.e., checked_add/sub/mul/etc. that return None when there's an over/underflow). You use wrapped operations when that's what's supposed to happen, checked operations when that's what's supposed to happen, debug builds to assert check all operations, and release builds to skip checks anywhere they're not explicit.
> The trouble with explicit casting is if the code is refactored to change the underlying integer type, the explicit casts may silently truncate the integer, introducing bugs.
That depends on how the casting is provided. For the C-style casting, or Rust's `as` casting, yes that is a problem. However, another way casting could be provided is through conversion functions that are only infallible if information isn't lost. For example, let's say we have the functions `to_u16` and `to_i16`. For an `i8` the first function could return `Option<u16>`, the second `i16`, while for a `u8` they would return `u16` and `i16`. That way, any change to the types that could cause it to now silently truncate would instead cause a compiler error because of the type mismatch.
Rust almost gets there with its `Into` and `TryInto` traits which do provide that functionality, but trying to use them in an expression causes type inference to fail, which just makes them a pain in the ass to use.
That depends on how the casting is provided. For the C-style casting, or Rust's `as` casting, yes that is a problem. However, another way casting could be provided is through conversion functions that are only infallible if information isn't lost. For example, let's say we have the functions `to_u16` and `to_i16`. For an `i8` the first function could return `Option<u16>`, the second `i16`, while for a `u8` they would return `u16` and `i16`. That way, any change to the types that could cause it to now silently truncate would instead cause a compiler error because of the type mismatch.
Rust almost gets there with its `Into` and `TryInto` traits which do provide that functionality, but trying to use them in an expression causes type inference to fail, which just makes them a pain in the ass to use.
You can use From and TryFrom, like u32::try_from(5i32). https://doc.rust-lang.org/std/convert/trait.From.html , https://doc.rust-lang.org/std/convert/trait.TryFrom.html
And, notably in this context, Rust's traits are auto-implemented in a chain, so if From<Foo> for Bar, then Into<Bar> for Foo, and thus TryFrom<Foo> for Bar, and thus in turn TryInto<Bar> for Foo.
This means that if first_thing used to have a non-overlapping value space so that converting it to other_thing might fail, so you wrote
This means that if first_thing used to have a non-overlapping value space so that converting it to other_thing might fail, so you wrote
other_thing = first_thing.try_into().blahblahblah;
Yeah, having two different cast operations can help here. But I like the simpler approach.
I never really clicked with D, but I always like reading your discussions on these details.
D doesn't have any one big feature. It's an accumulation of little things, like a nice desk vs one made out of plywood. :-) People who've used it for a while discover that they're more productive and their code has fewer bugs.
Sort of like putting a nut on a grade 8 bolt rather than a hardware store bolt.
People don't always agree on what's better:
https://digitalmars.com/d/archives/digitalmars/D/Movement_ag...
and sometimes the rationale for things isn't obvious at all, and even counter-intuitive.
Sort of like putting a nut on a grade 8 bolt rather than a hardware store bolt.
People don't always agree on what's better:
https://digitalmars.com/d/archives/digitalmars/D/Movement_ag...
and sometimes the rationale for things isn't obvious at all, and even counter-intuitive.
In the examples from the quiz the most tricky promotions IMO are that unsigned shorts are promoted to signed int instead of unsigned int, as signed and unsigned have very different behaviors.
Back before the 1989 C Standard, there was a huge debate about whether to do "value preserving" versus "sign preserving" integer promotions. About half the C compilers did the former, half the latter. Compromise wasn't possible. Value preserving won.
And hence unsigned shorts are promoted to signed ints.
And hence unsigned shorts are promoted to signed ints.
The most recent C standards also explicitly define integer math as 2’s complement. Admittedly, much later than D.
The last 1's complement machine I encountered was, never. Not in nearly 50 years of programming. I think those machines left for the gray havens before I was born.
C should also make char unsigned (D does). Optionally signed chars are an abomination.
C should also make char unsigned (D does). Optionally signed chars are an abomination.
I learned C in the late 1970s (although most of my programming back then was in FORTRAN, Assembly, and Pascal). Around the same time I was using a ones-complement system, a CDC 6000 series machine. This was an important machine at the time. The CDC 6600 was the first “super computer” and was three times faster that the preceding fastest computer.
The peculiar vagueness of the early C standards with respect to integers was probably due to the variety of hardware we used in the 1970s. For example:
The CDC 6000/7000s had 60-bit words, ones compliment arithmetic, 6 bit chars (but machine addresses were only of the 60 bit words).
The peripheral processors of the CDC machines (responsible for I/O) had 12-bit words as I recall.
IBMs very popular 360 mainframes supported several native hardware number formats: twos complement, unsigned, zoned decimal (one decimal digit per 8-bit byte), and packed decimal (two digits per byte).
I believe C was first developed on the DEC PDP-11. This was a very popular computer with 16 bit words and 8 bit bytes. It was twos complement, but mixed endianess (big endian for the 16 bit words making up a 32-bit long and little endian for the bytes in the 16-bit words).
Other DEC computers of the 1960s used 12 bit program counters and accumulators; I did some assembly language programming on the PDP-6 (it cost $300,000 back then and 24 were sold).
I seem to recall using a system with 18 bit registers and 9 bit bytes, but I can’t remember the details.
One system that I did assembly language programming on for a year was the TI 960. This machine was notable because it had two separate 64KB address spaces, one for data and one for instruction, known as a Harvard architecture instead of Von Neumann architecture. The advantage of the Harvard architecture was that a program couldn’t inadvertently modify the code it was running, a frequent and difficult to untangle occurrence when programming assembly. The Von Neumann architecture became dominant because it didn’t chop the available memory in two and partly because self modifying code was believed to be sometimes useful (yikes).
The peculiar vagueness of the early C standards with respect to integers was probably due to the variety of hardware we used in the 1970s. For example:
The CDC 6000/7000s had 60-bit words, ones compliment arithmetic, 6 bit chars (but machine addresses were only of the 60 bit words).
The peripheral processors of the CDC machines (responsible for I/O) had 12-bit words as I recall.
IBMs very popular 360 mainframes supported several native hardware number formats: twos complement, unsigned, zoned decimal (one decimal digit per 8-bit byte), and packed decimal (two digits per byte).
I believe C was first developed on the DEC PDP-11. This was a very popular computer with 16 bit words and 8 bit bytes. It was twos complement, but mixed endianess (big endian for the 16 bit words making up a 32-bit long and little endian for the bytes in the 16-bit words).
Other DEC computers of the 1960s used 12 bit program counters and accumulators; I did some assembly language programming on the PDP-6 (it cost $300,000 back then and 24 were sold).
I seem to recall using a system with 18 bit registers and 9 bit bytes, but I can’t remember the details.
One system that I did assembly language programming on for a year was the TI 960. This machine was notable because it had two separate 64KB address spaces, one for data and one for instruction, known as a Harvard architecture instead of Von Neumann architecture. The advantage of the Harvard architecture was that a program couldn’t inadvertently modify the code it was running, a frequent and difficult to untangle occurrence when programming assembly. The Von Neumann architecture became dominant because it didn’t chop the available memory in two and partly because self modifying code was believed to be sometimes useful (yikes).
>The last 1's complement machine I encountered was, never. Not in nearly 50 years of programming. I think those machines left for the gray havens before I was born.
Unisys OS 2200 uses one's complement[1] and Unisys MCP uses signed magnitude[2]. Both are still around.
[1] - https://public.support.unisys.com/2200/docs/CP19.0/78310422-... (page 108) - "UCS C represents an integer in 36-bit ones complement form (or 72-bit ones complement form, if the long long type attribute is specified)."
[2] - https://public.support.unisys.com/aseries/docs/ClearPath-MCP... (page 304) - "ClearPath MCP C uses a signed-magnitude representation for integers instead of two’s-complement representation. Furthermore, ClearPath MCP C integers use only 40 of the 48 bits in the word: a separate sign bit and the low order 39 bits for the absolute value."
Unisys OS 2200 uses one's complement[1] and Unisys MCP uses signed magnitude[2]. Both are still around.
[1] - https://public.support.unisys.com/2200/docs/CP19.0/78310422-... (page 108) - "UCS C represents an integer in 36-bit ones complement form (or 72-bit ones complement form, if the long long type attribute is specified)."
[2] - https://public.support.unisys.com/aseries/docs/ClearPath-MCP... (page 304) - "ClearPath MCP C uses a signed-magnitude representation for integers instead of two’s-complement representation. Furthermore, ClearPath MCP C integers use only 40 of the 48 bits in the word: a separate sign bit and the low order 39 bits for the absolute value."
IMO everyone gets it wrong. In a CPU instruction set, signed and unsigned are operations, not values, so why not extend that to low level languages? An int32 is neither signed nor unsigned. When you compare them, you choose signed or unsigned comparison.
Rust gets points in my book for being more explicit and avoiding arbitrary promotion rules. Plus the standard lib has specific functions for things like add with overflow which reflect what the CPU flags are doing.
Rust gets points in my book for being more explicit and avoiding arbitrary promotion rules. Plus the standard lib has specific functions for things like add with overflow which reflect what the CPU flags are doing.
> When you compare them, you choose signed or unsigned comparison.
True. I've done a lot of assembler programming. Getting the right jxx instruction after a compare was a prolific source of bugs. I've had a lot fewer bugs with signed types :-)
> Plus the standard lib has specific functions for things like add with overflow which reflect what the CPU flags are doing.
D does, too:
https://dlang.org/phobos/core_checkedint.html
along with integer types that automatically check for overflow:
https://dlang.org/phobos/std_checkedint.html
True. I've done a lot of assembler programming. Getting the right jxx instruction after a compare was a prolific source of bugs. I've had a lot fewer bugs with signed types :-)
> Plus the standard lib has specific functions for things like add with overflow which reflect what the CPU flags are doing.
D does, too:
https://dlang.org/phobos/core_checkedint.html
along with integer types that automatically check for overflow:
https://dlang.org/phobos/std_checkedint.html
Nice. I haven't really looked into D before but maybe I'll spend some time with it soon.
So same would have to work for signed vs unsigned addition, subtraction, bit shifts, powers, etc., right?
Yes.
Still, I would rather not have Rust's 'as' cast. I should like to see all, or at least most uses of 'as' deprecated in a later Rust edition.
Rust's 'as' will silently throw away data to achieve what you asked. Sometimes you wanted that, sometimes you didn't realise, and requiring into() and try_into() instead helps fix that.
For example suppose I have a variable named beans, I forgot what type it is, but it's got the count of beans in it, there should definitely be a non-negative value because we were counting beans, and it shouldn't be more than a few thousand, so we're going to put that in a u16 variable named enough. let enough = beans as u16;
This works just fine, even though beans is i64 (ie a signed 64-bit integer). Until one day beans is -1 due to an arithmetic error elsewhere, and now enough is 65535, which is pretty surprising. It's not undefined but it is surprising.
If we instead write let enough = beans.into(); we get a compiler error, the compiler can't see any way to turn i64 into u16 without risk of loss, so this won't work. We can write beans.try_into().unwrap() and get a panic if actually beans was out of range, or we can actually write the try_into() handler properly if we realise, seeing it can fail, what we're actually dealing with here.
Rust's 'as' will silently throw away data to achieve what you asked. Sometimes you wanted that, sometimes you didn't realise, and requiring into() and try_into() instead helps fix that.
For example suppose I have a variable named beans, I forgot what type it is, but it's got the count of beans in it, there should definitely be a non-negative value because we were counting beans, and it shouldn't be more than a few thousand, so we're going to put that in a u16 variable named enough. let enough = beans as u16;
This works just fine, even though beans is i64 (ie a signed 64-bit integer). Until one day beans is -1 due to an arithmetic error elsewhere, and now enough is 65535, which is pretty surprising. It's not undefined but it is surprising.
If we instead write let enough = beans.into(); we get a compiler error, the compiler can't see any way to turn i64 into u16 without risk of loss, so this won't work. We can write beans.try_into().unwrap() and get a panic if actually beans was out of range, or we can actually write the try_into() handler properly if we realise, seeing it can fail, what we're actually dealing with here.
If I can get away with using .into() over as, I will. However, there are so many cases where you can't, and .try_into().unwrap() is just way too unwieldy. In particular, there's no way for me to go "I know I'm only ever going to run on 64-bit machines where sizeof(usize) == sizeof(u64), so usize should implement From<u64>" (and even more annoying, I might be carefully making sure this works on 32-bit and 64-bit machines and get screwed because Rust thinks it could be portable to a 16-bit usize despite the fact I'm allocating hundreds of MB of memory).
And of course there are times I do want to bitcast negative values to large positive unsigned values or vice versa, without error. So while I do understand why maybe you shouldn't use as, at the end of the day, it just ends up being easier to use it than not use it.
And of course there are times I do want to bitcast negative values to large positive unsigned values or vice versa, without error. So while I do understand why maybe you shouldn't use as, at the end of the day, it just ends up being easier to use it than not use it.
> try_into().unwrap() is just way too unwieldy.
I always carry with me a tiu() alias method for exactly that reason (in a TryIntoUnwrap trait implemented for every pair of types which implements TryInto).
I always carry with me a tiu() alias method for exactly that reason (in a TryIntoUnwrap trait implemented for every pair of types which implements TryInto).
You can also do try_from()? if you impl From<TryIntError> on your local error type.
I feel you. I'd really like a way for "as" to be transformed into "try_into().unwrap()", say in a later edition. Or even just "into()" which would throw a nice compile-time error - This is the Rust Way.
A flip side is that some safe conversions also produce compiler errors. On a 64-bit system, usize and u64 are the same width, but they are not convertible: neither is Into the other, so you cannot lean on the compiler in the way you describe.
You might use try_into(), but then you risk panicking on a 32-bit system, instead of getting a compile-time error.
You might use try_into(), but then you risk panicking on a 32-bit system, instead of getting a compile-time error.
These examples do look scary. However, in my 16 years of experience of mostly writing In C, I have never found a single chance that I need to actually use the expressions that I got the answers wrong.
Never mix unsigned and signed integer comparison unless you know exactly what you are doing.
And never do arithmetic on boundaries definitions like INT_MAX, they are boundaries , why you need to compute a derived value on boundaries?
If you do not need arithmetic behavior, do not use signed integer, use unsigned. Because computer does not understand sign, so if you do not need a sign, do not use it.
You do not need to be a language committee member to write C , you just need to understand the reason behind its design.
Never mix unsigned and signed integer comparison unless you know exactly what you are doing.
And never do arithmetic on boundaries definitions like INT_MAX, they are boundaries , why you need to compute a derived value on boundaries?
If you do not need arithmetic behavior, do not use signed integer, use unsigned. Because computer does not understand sign, so if you do not need a sign, do not use it.
You do not need to be a language committee member to write C , you just need to understand the reason behind its design.
The problem is that for unsigned one of the boundaries is 0, which is an extremely common number - there isn't a couple months where I don't find a bug due to a size-1 somewhere
Subtraction would presumably fall under the arithmetic behavior that OP was talking about.
Maybe. But subtraction is also perfectly well defined for unsigned ints in C.
If you wanted to be extremely careful, you might pass unsigned ints into a function by copy, and cast it as a signed int to perform arithmetic, then do a boundary check before returning a result to be assigned back to an unsigned integer.
The unsignedness of the integer lets readers of the code know that it's a constant of some kind, and that comparisons should be performed with care. Normally, when I've worked with C (fairly limited), unsigned numbers normally get used as flags for bitwise comparisons. Sometimes your flags may have a partial ordering defined on them (so a sense of flag A being 'smaller' than flag B), so you might need some arithmetic to do those checks you need to be explicitly careful with the boundary checks though.
The unsignedness of the integer lets readers of the code know that it's a constant of some kind, and that comparisons should be performed with care. Normally, when I've worked with C (fairly limited), unsigned numbers normally get used as flags for bitwise comparisons. Sometimes your flags may have a partial ordering defined on them (so a sense of flag A being 'smaller' than flag B), so you might need some arithmetic to do those checks you need to be explicitly careful with the boundary checks though.
> And never do arithmetic on boundaries definitions like INT_MAX, they are boundaries
I'd argue something stronger: if you care about boundaries like INT_MAX, you should never be comparing them using your regular comparison tools. I.e., even though there are correct ways to compute whether x + 1 will overflow, don't bother trying to do that and instead always use __builtin_add_overflow since you can't fuck it up. Trying to do these sorts of edge checks are incredibly hard and it has lead to numerous security vulnerabilities due to checks being optimized out. The builtins do exactly what they say and you don't have to worry about UB blowing your foot off.
I'd argue something stronger: if you care about boundaries like INT_MAX, you should never be comparing them using your regular comparison tools. I.e., even though there are correct ways to compute whether x + 1 will overflow, don't bother trying to do that and instead always use __builtin_add_overflow since you can't fuck it up. Trying to do these sorts of edge checks are incredibly hard and it has lead to numerous security vulnerabilities due to checks being optimized out. The builtins do exactly what they say and you don't have to worry about UB blowing your foot off.
> If you do not need arithmetic behavior, do not use signed integer, use unsigned. Because computer does not understand sign, so if you do not need a sign, do not use it.
That's interesting. In my career (embedded development) I've learned to do the opposite. Always use signed unless you have a reason not to. Even if a value can't naturally be negative, use signed. Use unsigned only if you need the extra bit, or if you're doing bitwise operations.
> Because computer does not understand sign
Computers understand signs just fine, we're long past the days of the 6502 N-flag being a glorified bit 7 check. All CPUs have signed instructions.
That's interesting. In my career (embedded development) I've learned to do the opposite. Always use signed unless you have a reason not to. Even if a value can't naturally be negative, use signed. Use unsigned only if you need the extra bit, or if you're doing bitwise operations.
> Because computer does not understand sign
Computers understand signs just fine, we're long past the days of the 6502 N-flag being a glorified bit 7 check. All CPUs have signed instructions.
> That's interesting. In my career (embedded development) I've learned to do the opposite. Always use signed unless you have a reason not to. Even if a value can't naturally be negative, use signed. Use unsigned only if you need the extra bit, or if you're doing bitwise operations.
In this case, since you make sure that the extra size gained by unsigned is not important to you all, then you can also go with signed by default. Basically it is the tradeoff between 1. robust in majority of the use cases 2. capability to do signed arithmetic 3. additional positive integer range. As long as you make a consistent selection and be mindful when you are in the danger zone, it can be handled.
In this case, since you make sure that the extra size gained by unsigned is not important to you all, then you can also go with signed by default. Basically it is the tradeoff between 1. robust in majority of the use cases 2. capability to do signed arithmetic 3. additional positive integer range. As long as you make a consistent selection and be mindful when you are in the danger zone, it can be handled.
> Always use signed unless you have a reason not to. Even if a value can't naturally be negative, use signed.
Can you elaborate on what benefits this approach has? I would feel that, especially when a number cannot be negative, unsigned integers seem like a proper representation of the data?
Can you elaborate on what benefits this approach has? I would feel that, especially when a number cannot be negative, unsigned integers seem like a proper representation of the data?
A big problem is that C doesn't make you stick to the representation.
If you accidentally subtract and go into 'negative' numbers, using an unsigned in C will just happily give you the wrong answer.
Now in something like Haskell, it's much more robust advice to make invalid states unrepresentable. But C's compiler and runtime don't typically help you, when you are trying to break your representation.
If you accidentally subtract and go into 'negative' numbers, using an unsigned in C will just happily give you the wrong answer.
Now in something like Haskell, it's much more robust advice to make invalid states unrepresentable. But C's compiler and runtime don't typically help you, when you are trying to break your representation.
A couple of things I can think of off the top of my head: Since the compiler knows that certain things are UB using signed offers better (often significantly better) optimization opportunities then using unsigned. Plus, you get to use UBSan to detect bad code.
Here is an example: https://wesmckinney.com/blog/avoid-unsigned-integers/
It's not always obvious that you're mixing signed and unsigned, though.
I think the biggest source of these kinds of bugs is because size_t is unsigned but ptrdiff_t is signed. So the moment you mix pointer subtraction with sizeof, footguns abound.
I think the biggest source of these kinds of bugs is because size_t is unsigned but ptrdiff_t is signed. So the moment you mix pointer subtraction with sizeof, footguns abound.
Is it your experience that there are many projects where you can reliably say some variable will never need arithmetic behavior?
I don't have 16 years of experience writing C, but
> If you do not need arithmetic behavior, do not use signed integer, use unsigned
does seem in a roundabout way to match the advice that I've gotten from other folks -- usually stick to signed (because you never know if somebody is going to want to use an integer in, say, a downwards loop). Your comment just seems to highlight the less usual case, where you can be sure that nobody will ever need that arithmetic behavior... maybe it depends on the type of applications, though.
I don't have 16 years of experience writing C, but
> If you do not need arithmetic behavior, do not use signed integer, use unsigned
does seem in a roundabout way to match the advice that I've gotten from other folks -- usually stick to signed (because you never know if somebody is going to want to use an integer in, say, a downwards loop). Your comment just seems to highlight the less usual case, where you can be sure that nobody will ever need that arithmetic behavior... maybe it depends on the type of applications, though.
It always depends on the area of focus. You can also consistently choose the default int type to signed while being fully aware of you lose half range but gain signed integer arithmetic. For me I am mostly doing system and networking so I tends to to unsigned. The key is to choose a consistent default while aware of that you are in a non-intuitive zone when that default type puts you in.
For me using uint8_t and family makes things simpler (working predominately with packets)
It is a container for X bits and thats it, its easy to understand overflow and corner case behaviors.
I rarely find good reason to use signed ints, so only when I need them i worry about sign handling.
> because you never know if somebody is going to want to use an integer in, say, a downwards loop
Why risk a possible bug, for a hypothetical case of someone wanting signed var?
If they really want to use signed vars they can handle it themselves.
It is a container for X bits and thats it, its easy to understand overflow and corner case behaviors.
I rarely find good reason to use signed ints, so only when I need them i worry about sign handling.
> because you never know if somebody is going to want to use an integer in, say, a downwards loop
Why risk a possible bug, for a hypothetical case of someone wanting signed var?
If they really want to use signed vars they can handle it themselves.
Yeah. My context was doing linear algebra stuff. It is apparently a known mistake to say "well we aren't going to have to describe negative positions for matrix elements, so why not make the integer unsigned?" but there are lots of cases where you want a negative offset for whatever reason. ("I want to stop right before the diagonal").
I can definitely see why you might not ever need a sign in networking stuff.
I can definitely see why you might not ever need a sign in networking stuff.
The compiler will also warn about these cases where you better be explicit to avoid undesired behavior.
Found a disappointing bug in the test:
The left operand of > has type unsigned short, before promotion. On C implementations for today's popular, mainstream machines, short is narrower than int; therefore, whether it is signed or unsigned it goes to int.
In that common case, we are doing a 1 > -1 comparison in the int type.
However, unsigned short may be exactly as wide as int, in which case it cannot promote to int, because its values do not fit into that type. It promotes to unsigned int in that case. Both sides will go to unsigned int{*}, and so we are comparing 1 > UINT_MAX which is 0.
Maybe the author should name this "GCC integer quiz for 32 bit x86", and drop the harder choices like "implementation-defined".
---
{*} It's more nuanced here. If we have a signed and unsigned integer operand of the same rank, it could go either way. If the unsigned type has a limited range so that all its vallues are representable in the signed type, then the unsigned type goes to signed. My remark represents only the predominant situation whereby signed and unsigned types of the same rank have overlapping ranges that don't fit into each other: the unsigned version of a type not only lacks negatives, but has extra positives.
(unsigned short)1 > -1
The left operand of > has type unsigned short, before promotion. On C implementations for today's popular, mainstream machines, short is narrower than int; therefore, whether it is signed or unsigned it goes to int.
In that common case, we are doing a 1 > -1 comparison in the int type.
However, unsigned short may be exactly as wide as int, in which case it cannot promote to int, because its values do not fit into that type. It promotes to unsigned int in that case. Both sides will go to unsigned int{*}, and so we are comparing 1 > UINT_MAX which is 0.
Maybe the author should name this "GCC integer quiz for 32 bit x86", and drop the harder choices like "implementation-defined".
---
{*} It's more nuanced here. If we have a signed and unsigned integer operand of the same rank, it could go either way. If the unsigned type has a limited range so that all its vallues are representable in the signed type, then the unsigned type goes to signed. My remark represents only the predominant situation whereby signed and unsigned types of the same rank have overlapping ranges that don't fit into each other: the unsigned version of a type not only lacks negatives, but has extra positives.
The site says:
> All other things being equal, assume GCC/LLVM x86/x64 implementation-defined behaviors.
> All other things being equal, assume GCC/LLVM x86/x64 implementation-defined behaviors.
that's still wrong, the integer data model is defined by the target, which is not specified in the quiz, not solely by the compiler, ex. clang uses 32-bit longs when compiling for windows, and 64-bit longs when compiling for linux:
* windows - https://godbolt.org/z/WWWzarEsK
* linux - https://godbolt.org/z/57WK45o6e
same for gcc on linux/windows (mingw)
* windows - https://godbolt.org/z/WWWzarEsK
* linux - https://godbolt.org/z/57WK45o6e
same for gcc on linux/windows (mingw)
Discussed in this thread:
https://news.ycombinator.com/item?id=32712578
https://news.ycombinator.com/item?id=32712578
I thought I did well until the bit-shifting questions. Who in their right mind designs a language where shifting the bits of a u16 silently converts into an i32? Doubly so since that fact alone directly causes UB — keeping the u16 would have been perfectly fine.
(Edit, since some respondents seem to miss this, explanations about efficiency or ISAs might justify promoting to u32 (though even that's debatable), but not i32. A design that auto-promotes an unsigned type, where every operation is nicely defined and total, into a signed type, where you run into all kinds of undefined behavior on overflow, is simply crazy.)
(Edit, since some respondents seem to miss this, explanations about efficiency or ISAs might justify promoting to u32 (though even that's debatable), but not i32. A design that auto-promotes an unsigned type, where every operation is nicely defined and total, into a signed type, where you run into all kinds of undefined behavior on overflow, is simply crazy.)
Because everything smaller than an int is usually promoted to int. int is the 'word' in C that things are calculated in. Even character constants are ints, all enum constants are ints, the default type was int when default types were still a thing.
And that makes writing a correct and portable C futile. Yes, defined-size types are a thing and I almost exclusively use them, but since the size of `int` itself is unknown and integer promotion depends on that size, the meaning of the code using only defined-size types can still vary across platforms. intN_t etc. are only typedefs and do not form their own type hierarchy, which in my opinion is a huge mistake.
Yes. The model C has is that the CPU has an ALU that that operates as (word,word)->word with int being the smallest word size. This explains many of C's conversion rules: to operate on a single integer, it first has to be promoted to a word size; to operate on two integers, they first have to be converted to a common word size, etc.
But why not turn u16 into u32? Why switch it to being signed on promotion?
ANSI describes why in their rationale: https://www.lysator.liu.se/c/rat/c2.html#3-2-1 .
The unsigned preserving rules greatly increase the number of situations where unsigned int confronts signed int to yield a questionably signed result, whereas the value preserving rules minimize such confrontations. Thus, the value preserving rules were considered to be safer for the novice, or unwary, programmer. After much discussion, the Committee decided in favor of value preserving rules, despite the fact that the UNIX C compilers had evolved in the direction of unsigned preserving.Believe it or not, this makes the behavior more like what you'd expect in many cases. For example:
If it promoted to unsigned int, then it would also clear bits 32-63.
uint8_t x = 4;
extern volatile uint64_t *reg;
*reg &= ~x;
If it promoted to unsigned int, then it would also clear bits 32-63.
extern volatile uint64_t *reg;
*reg &= ~x;
extern volatile uint64_t *reg;
uint64_t tmp = *reg;
tmp &= ~x;
*reg = temp;
I don’t find that very convincing. It’s simply ambiguous, I might want this:
*reg &= ~(uint64_t)x;
*reg &=(uint64_t)(uint8_t)~x;
That second line can be written:
*reg &= (uint8_t)~x;
*reg &= ~x & 0xFF;I'm sure the answer is going to be along the lines of "because PCC did that and they standardized it" :-/
Here's a fun standardization problem I came across recently (nothing to do with C): http://mywiki.wooledge.org/BashFAQ/105
Here's a fun standardization problem I came across recently (nothing to do with C): http://mywiki.wooledge.org/BashFAQ/105
If you have a 32-bit architecture, you don't necessarily have 8-bit and 16-bit hardware operations outside of memory operations (load/store).
Now, I don't find this reasoning persuasive--it's not that hard to emulate an 8-bit or 16-bit operation--and judging from the history of post-C languages, most other language designers are equally unmoved by this reasoning, but I can see someone in their right mind designing a language that acts like this. Especially if the first architecture they're developing on is precisely such on architecture (PDP11 doesn't have byte-sized add/sub/mul/div).
Now, I don't find this reasoning persuasive--it's not that hard to emulate an 8-bit or 16-bit operation--and judging from the history of post-C languages, most other language designers are equally unmoved by this reasoning, but I can see someone in their right mind designing a language that acts like this. Especially if the first architecture they're developing on is precisely such on architecture (PDP11 doesn't have byte-sized add/sub/mul/div).
I think the prevailing philosophy for C (and later C++) was that code should map closely to the hardware and not expand to complicated "microcode". A left shift in Cshould just be a left shift in the underlying ISA, within reason. That's where most of the undefined behaviours come from. Having to add masking and other operations to emulate a 16bit shift on a 32bit architecture feels un-C-like, for better or worse.
IMO the real issue is not so much the fact that all shifts of any type < int is treated as if it were an int, it's that the language doesn't force you to acknowledge that in the code. If you got a compilation error when trying to shift a short and had to explicitly promote to int in order to make it through, at the very least it can't lead to an oversight from a careless programmer.
C is trying to be clever but only goes half way, resulting in the worst of both worlds IMO.
IMO the real issue is not so much the fact that all shifts of any type < int is treated as if it were an int, it's that the language doesn't force you to acknowledge that in the code. If you got a compilation error when trying to shift a short and had to explicitly promote to int in order to make it through, at the very least it can't lead to an oversight from a careless programmer.
C is trying to be clever but only goes half way, resulting in the worst of both worlds IMO.
The ISA might force someone to extend a value to 32 bits (debatable, but let’s go with it). It never forces you to treat an unsigned int as signed. It also doesn’t require inserting UB into the process.
I agree, the whole signed vs. unsigned generally feels like an afterthought in C (probably because, to a certain extent, it was). `char`'s sign being implementation-defined is a pretty wild design choice that wasted many hours of my life while porting code between ARM and x86.
UBs are not required, but you need them if you want C to behave as a macro-assembler as well as allowing for aggressive optimizations. For instance `a << b` if b is greater than a's width is genuinely UB if you write portable code, different CPUs will do different things in this situation. Defining the behaviour means that the compiler would have to insert additional opcodes to make the behaviour identical on all platforms.
You may argue that it's still better than having UB but that's just not C's design philosophy, for better or worse.
UBs are not required, but you need them if you want C to behave as a macro-assembler as well as allowing for aggressive optimizations. For instance `a << b` if b is greater than a's width is genuinely UB if you write portable code, different CPUs will do different things in this situation. Defining the behaviour means that the compiler would have to insert additional opcodes to make the behaviour identical on all platforms.
You may argue that it's still better than having UB but that's just not C's design philosophy, for better or worse.
That’s true, I’d add something more. Your reasoning about hardware differences would only justify implementation-defined behavior, not undefined behavior. The distinction is important here: undefined behavior is when the compiler can make surprising optimizations in other parts of the code assuming something doesn’t happen.
I imagine the reason oversized shifts are UB is because some 40-year-old computer hardware trapped on oversized shifts, as traps are always UB in C.
> For instance `a << b` if b is greater than a's width is genuinely UB if you write portable code, different CPUs will do different things in this situation. Defining the behaviour means that the compiler would have to insert additional opcodes to make the behaviour identical on all platforms.
Your seem to be mixing up implementation defined behavior and undefined behavior. It would have been perfectly reasonable to make this choice if signed integer ovwrflow were implementation-defined, but it is unfortunately not - it is undefined behavior instead. This means that a program containing this instruction is not valid C and may "legally" have any effect whatsoever.
Your seem to be mixing up implementation defined behavior and undefined behavior. It would have been perfectly reasonable to make this choice if signed integer ovwrflow were implementation-defined, but it is unfortunately not - it is undefined behavior instead. This means that a program containing this instruction is not valid C and may "legally" have any effect whatsoever.
It's worse than that. `char`'s signedness being implementation defined is one thing, but then having the standard library provide a function called `getchar()` that returns not a `char` but an `unsigned char` cast to an `int` is diabolical.
> Who in their right mind designs a language where shifting the bits of a u16 silently converts into an i32?
Yeah, hence why I asked this question years ago: https://stackoverflow.com/questions/39964651/is-masking-befo...
Yeah, hence why I asked this question years ago: https://stackoverflow.com/questions/39964651/is-masking-befo...
Because sub-integer types are for storage, not computation.
Yes, it'd be better if you had to explicitly cast to int or unsigned to perform arithmetic, but that ship has sailed.
Yes, it'd be better if you had to explicitly cast to int or unsigned to perform arithmetic, but that ship has sailed.
Beware that this is very much not a "C Integer quiz", but a "ILP32/LP64 Iinteger quiz". As sono as you move not to some weird exotic architecture, but simply to 64bit Windows(!!) some quiz answers will not hold.
For a website meant to educate programmers on C language gotchas, this is a pretty lackluster effort.
Even the initial disclaimer, "assume x86/x86-64 GCC/CLang", is wrong, as the compiler does not have anything to do with integer widths.
For a website meant to educate programmers on C language gotchas, this is a pretty lackluster effort.
Even the initial disclaimer, "assume x86/x86-64 GCC/CLang", is wrong, as the compiler does not have anything to do with integer widths.
My impression is this wasn't muxh to educate programmers on C language gotchas but to remind you just how messy and fragile this language is (so you can stay far away).
One more reason not to give the impression that you can assume that long is wider than int which is wider then short!
I noticed this when I got to some of the x86 vs. x86-64 questions.
Occasionally I'll tutor C and C++ to college students, and I swear that I spend half of my time in such cases correcting erroneous (or absurdly incomplete) info provided by instructors, Stack Exchange articles, etc. It's especially challenging when I get, "but I read it on HN?!"
Occasionally I'll tutor C and C++ to college students, and I swear that I spend half of my time in such cases correcting erroneous (or absurdly incomplete) info provided by instructors, Stack Exchange articles, etc. It's especially challenging when I get, "but I read it on HN?!"
The quiz is wrong because they assume that the length of short is less than int in some questions. A short can be the same length as an int. It just can't be longer.
[deleted]
Another interesting thing about C integers that the quiz doesn't cover is that remainder is not modulus. For example, in Python:
>>: 2 % -5
-3
2 % -5 == 2
long mod(long x, long y) {
if (y == -1) return 0;
return x - y * (x / y - (x % y && (x ^ y) < 0));
}Mathematically speaking, the mod is usually taken to be positive. Any reasonable definition would involve taking the quotient of Z over the congruence relation, so I can see both -3 and 2 being reasonable conventions to represent partitions.
OTOH, the largest "wat" of C-like languages is the following:
Python here is once again correct:
OTOH, the largest "wat" of C-like languages is the following:
> -8 % 5
-3
Python here is once again correct:
>>> -8 % 5
2Mathematically speaking if you ask for the modulus of -5 you may as well get a negative number, and I think people may validly have their own interpretations of what's "correct" at this point.
I didn't say it's wrong.
You may as well use {white, blue, black, red, green} to represent congruences mod 5, mathematically speaking it's not wrong, as long as they respect the axioms of a field.
What's "wat" (in the same sense as the js "wat" talk) is that the answer changes if the argument becomes negative. By all reasonable definitions of modular arithmetic, -8 and 7 are in the same class mod 5. Why is then:
You may as well use {white, blue, black, red, green} to represent congruences mod 5, mathematically speaking it's not wrong, as long as they respect the axioms of a field.
What's "wat" (in the same sense as the js "wat" talk) is that the answer changes if the argument becomes negative. By all reasonable definitions of modular arithmetic, -8 and 7 are in the same class mod 5. Why is then:
> (-8 % 5) == (7 % 5)
false
> What's "wat" (in the same sense as the js "wat" talk) is that the answer changes if the argument becomes negative.
It's the same in Python though? you just prefer the way it behaves (though to be fair it's generally more useful and less troublesome): Python's remainder follows the sign of the divisor (because it uses floored division), while C's follows the sign of the dividend (because it uses truncated division).
Strictly speaking, neither is an euclidian modulo.
It's the same in Python though? you just prefer the way it behaves (though to be fair it's generally more useful and less troublesome): Python's remainder follows the sign of the divisor (because it uses floored division), while C's follows the sign of the dividend (because it uses truncated division).
Strictly speaking, neither is an euclidian modulo.
It's not symmetric. I believe most people with strong mathematical background (like Guido:)) expect lambda x: x % m to be some "computer approximation" to the standard mapping from Z to Z/mZ, while not having deep expectations about lambda m: x % m.
The really useful fact of C's remainder operator is that:
x == x / y * y + x % y, provided nothing overflowed. This is usually what you want when doing accurate calculations with integer division.
The modulo operation together with floor division also has this property. And unlike C's operators, it also has the useful property that (x + y) % y == x % y.
That's an inherent property of both the modulus interpretation and the remainder interpretation.
Haskell does it right. Instead of having operators like %, it gives the programmer choice to choose either the modulus or the remainder. Using GP's example:
Haskell does it right. Instead of having operators like %, it gives the programmer choice to choose either the modulus or the remainder. Using GP's example:
ghci> 2 `div` (-5)
-1
ghci> 2 `mod` (-5)
-3
ghci> 2 `quot` (-5)
0
ghci> 2 `rem` (-5)
2
One of my favorite tables: https://en.m.wikipedia.org/wiki/Modulo_operation#In_programm...
Fortran supports both of these, with `mod` as the C-like truncated modulo and `modulo` as the floored modulo. Having both is convenient, but you do get errors from people who don't realize the difference
Common Lisp as well, Rem and Mod
Surprised I got all the questions right because I tend to code around those limits, sticking to defining always safe types for the problem at hand, in all the cases I'm not sure about ranges, possibile overflows and so forth. What I mean is that if you have to remember the rules in a piece of code you are writing, it is better to rewrite the code with new types that are obviously safe.
Admittedly my C is (very) rusty, but I was surprised at how few of the answers I knew, so your comment made me feel even worse. But then I saw your nickname. Yeah. :) (redis rocks btw)
Completely agree with you, and not just in C. Using esoteric features of any language is equivalent to putting landmines in front of your teammates - bad idea.
Completely agree with you, and not just in C. Using esoteric features of any language is equivalent to putting landmines in front of your teammates - bad idea.
> Using esoteric features of any language is equivalent to putting landmines in front of your teammates
Also, outside of production code, using esoteric features is a good way to get familiar with every corner of that language - which is useful so that you can diffuse those landmines when they are accidentally created. i.e know them to avoid them.
Also, outside of production code, using esoteric features is a good way to get familiar with every corner of that language - which is useful so that you can diffuse those landmines when they are accidentally created. i.e know them to avoid them.
The first rule of C good programming "Treat warnings as errors". Compile with all warnings enabled, selectively ignore only warnings you know for sure are not important for program semantics.
Simple MISRA C with all warnings on is already close to language with strict type checking. C compilers give you a choice, use it. If you are programming for money, good static analyzer makes it possible to write safety critical code.
Simple MISRA C with all warnings on is already close to language with strict type checking. C compilers give you a choice, use it. If you are programming for money, good static analyzer makes it possible to write safety critical code.
I do agree about compiler warnings, but MISRA C imposes a lot of unnecessary rules, a lot of unhelpful rules on how you write expressions, and tries to also act like C's type system works a different way than it does. In practice I have found it to actually create bugs. Read the MISRA rules and appendices on their effective type model and the C standard, they have gaps where MISRA actually forces you to write code that looks correct and doesn't work with C's model. I strongly recommend against using MISRA, even in automotive or aviation code (although it may unfortunately be a requirement on any such project).
Worth noting Dennis own words,
"Although the first edition of K&R described most of the rules that brought C's type structure to its present form, many programs written in the older, more relaxed style persisted, and so did compilers that tolerated it. To encourage people to pay more attention to the official language rules, to detect legal but suspicious constructions, and to help find interface mismatches undetectable with simple mechanisms for separate compilation, Steve Johnson adapted his pcc compiler to produce lint [Johnson 79b], which scanned a set of files and remarked on dubious constructions."
Dennis M. Ritchie -- https://www.bell-labs.com/usr/dmr/www/chist.html
Unfortunately too many think they know better than the language authors themselves.
"Although the first edition of K&R described most of the rules that brought C's type structure to its present form, many programs written in the older, more relaxed style persisted, and so did compilers that tolerated it. To encourage people to pay more attention to the official language rules, to detect legal but suspicious constructions, and to help find interface mismatches undetectable with simple mechanisms for separate compilation, Steve Johnson adapted his pcc compiler to produce lint [Johnson 79b], which scanned a set of files and remarked on dubious constructions."
Dennis M. Ritchie -- https://www.bell-labs.com/usr/dmr/www/chist.html
Unfortunately too many think they know better than the language authors themselves.
What is the history of "undefined behavior" [in C compilers and the standard] in general? I suppose originally it was supposed to guide compiler engineers, but we all know that backfired, as many compilers try to exploit undefined behavior to optimize code, but that can be problematic in security sensitive code (e.g. if uninitialized memory is optimized away) - there have been discussions between security engineers, kernel developers and GCC hackers about how to implement/interpret the standard.
Would it be possible to have a standard, where undefined behavior is just a compile error? What would we lose - apart from legacy compatibility?
Would it be possible to have a standard, where undefined behavior is just a compile error? What would we lose - apart from legacy compatibility?
C sucked on 8 and 16 bit home computers, to be fair, all high level systems programming languages had their own set of issues regarding optimal code generation, thus Assembly was the name of the name for ultimate performance.
UB started as means to not kick out computer architectures that would otherwise not be able to be targeted by fully compliant ISO C compilers.
Given that C prefers to be a kind of portable macro assembler than care about security, it was only a matter of time until those escape hatches started to be taken advantage for optimizations.
Same applies to other languages, however since their communities tend to prefer security before ultimate performance, some optimization paths are not considered as that would hurt their safety goals.
In what concerns C, C++ and Objective-C, dropping UB optimizations would mean going back to the 1990's in terms of code quality.
UB started as means to not kick out computer architectures that would otherwise not be able to be targeted by fully compliant ISO C compilers.
Given that C prefers to be a kind of portable macro assembler than care about security, it was only a matter of time until those escape hatches started to be taken advantage for optimizations.
Same applies to other languages, however since their communities tend to prefer security before ultimate performance, some optimization paths are not considered as that would hurt their safety goals.
In what concerns C, C++ and Objective-C, dropping UB optimizations would mean going back to the 1990's in terms of code quality.
> Would it be possible to have a standard, where undefined behavior is just a compile error?
No [if you're aiming for something in the same vein as C]. Undefined behavior is ultimately an inherently dynamic property--certain values could make a statement execute undefined behavior, and consequently, virtually every statement could potentially cause undefined behavior. Note that this remains true even in languages like Rust: Rust has loads of undefined behavior, but you do have to wrap code in unsafe blocks to potentially cause undefined behavior.
> What would we lose - apart from legacy compatibility?
In particular, it is clear at this point that if you want to permit converting integers to pointers, you will either have to live with undefined behavior (via pointer provenance) or forgo basically all optimization whatsoever.
No [if you're aiming for something in the same vein as C]. Undefined behavior is ultimately an inherently dynamic property--certain values could make a statement execute undefined behavior, and consequently, virtually every statement could potentially cause undefined behavior. Note that this remains true even in languages like Rust: Rust has loads of undefined behavior, but you do have to wrap code in unsafe blocks to potentially cause undefined behavior.
> What would we lose - apart from legacy compatibility?
In particular, it is clear at this point that if you want to permit converting integers to pointers, you will either have to live with undefined behavior (via pointer provenance) or forgo basically all optimization whatsoever.
Would
int foo(int x, int y) { return x+y; }
I think what OP is saying is that it should compile and not have any UB, because signed integer overflow should be well-defined.
(This is the case in e.g. C#.)
(This is the case in e.g. C#.)
Presumably, that function would be a compiler error like this.
Unless your program can prove, that the function is only ever called with arguments that don't hit UB.
Unless your program can prove, that the function is only ever called with arguments that don't hit UB.
Unfortunately some of this stuff just can't be detected statically. So while warnings are an excellent starting point, I recommend also building and testing with UBSan+ASan enabled.
> What does the expression SCHAR_MAX == CHAR_MAX evaluate to?
> Sorry about that -- I didn't give you enough information to answer this one. The signedness of the char type is implementation-defined
...why not have an "implementation-defined" answer button then, because that's what people should know (instead of knowing all ABIs) and what the question is about anyway?
> If these operators were right-associative, the expression [x - 1 + 1] would be defined for all values of x.
That's just wrong, no? If + and - were right-associative, it would be parsed as x - (1 + 1), which is decidedly not "defined for all values of x".
> Sorry about that -- I didn't give you enough information to answer this one. The signedness of the char type is implementation-defined
...why not have an "implementation-defined" answer button then, because that's what people should know (instead of knowing all ABIs) and what the question is about anyway?
> If these operators were right-associative, the expression [x - 1 + 1] would be defined for all values of x.
That's just wrong, no? If + and - were right-associative, it would be parsed as x - (1 + 1), which is decidedly not "defined for all values of x".
I raised an eyebrow on that as well, in some questions you have implentation defined for an answer so I assumed the author just wrongly assumed undefined covers that
The answer for question about SCHAR_MAX == CHAR_MAX is just wrong there. Of course it is "undefined", or actually, it is configurable with "-funsigned-char" flag (for GCC/Clang/ICC).
https://godbolt.org/z/PTEoW91v5
https://godbolt.org/z/PTEoW91v5
Other questions might also be "implementation defined", thus a caveat to assume GCC/LLVM implementations on x86/amd64 at the start.
I.e. C standard enforces undefined very sparingly iirc, and most of the corner cases are implementation defined.
I may also be misremembering things: it's been 20 years since I've carefully read C99 (draft, really) for fun :))
I.e. C standard enforces undefined very sparingly iirc, and most of the corner cases are implementation defined.
I may also be misremembering things: it's been 20 years since I've carefully read C99 (draft, really) for fun :))
Even the compiler assumption isn't enough. There is also an implicit assumption in the answers that x86-64 is using LP64 when some platforms will use LLP64 (Windows) or ILP64 (Cray).
The answer of the final question (INT_MIN % -1) is wrong in a way that's somewhat dangerous.
If you read the text of C99 carefully, yes, it's implied that INT_MIN % -1 should be well-defined to be 0. However, the % operator is usually implemented in hardware as part of the same instruction that does division, which means that on hardware where INT_MIN / -1 traps (thereby causing undefined behavior), INT_MIN % -1 will also trap. The wording was changed in C11 (and C++11) to make INT_MIN % -1 explicitly undefined behavior, and given the reasoning for why the wording was changed, users should expect that it retains its undefined behavior even in C89 and C99 modes, even on 20-year-old compilers that predate C11.
If you read the text of C99 carefully, yes, it's implied that INT_MIN % -1 should be well-defined to be 0. However, the % operator is usually implemented in hardware as part of the same instruction that does division, which means that on hardware where INT_MIN / -1 traps (thereby causing undefined behavior), INT_MIN % -1 will also trap. The wording was changed in C11 (and C++11) to make INT_MIN % -1 explicitly undefined behavior, and given the reasoning for why the wording was changed, users should expect that it retains its undefined behavior even in C89 and C99 modes, even on 20-year-old compilers that predate C11.
>The wording was changed in C11
And here's an exact quote from the C11 standard:
>If the quotient a/b is representable, the expression (a/b)*b + a%b shall equal a; otherwise, the behavior of both a/b and a%b is undefined.
http://port70.net/~nsz/c/c11/n1570.html#6.5.5p6
And here's an exact quote from the C11 standard:
>If the quotient a/b is representable, the expression (a/b)*b + a%b shall equal a; otherwise, the behavior of both a/b and a%b is undefined.
http://port70.net/~nsz/c/c11/n1570.html#6.5.5p6
Didn’t notice I got some wrong because the green color made me think I got them right.
Yeah that's a UX bug in the quiz
That’s to keep the obscurity vibe
Third question is wrong. It's implementation defined, because unsigned short might be the same rank as unsigned int in some implementations, in which case it remains unsigned when promotion occurs.
They may have the same width, but not the same rank.
From C99 §6.3.1.1:
> — The rank of long long int shall be greater than the rank of long int, which shall be greater than the rank of int, which shall be greater than the rank of short int, […]
> — The rank of any unsigned integer type shall equal the rank of the corresponding signed integer type, if any.
From C99 §6.3.1.1:
> — The rank of long long int shall be greater than the rank of long int, which shall be greater than the rank of int, which shall be greater than the rank of short int, […]
> — The rank of any unsigned integer type shall equal the rank of the corresponding signed integer type, if any.
That's true, but it doesn't matter to the point being made.
According to the standard[1], if short and int have the same size (even if not the same rank) both numbers are converted to unsigned int (that is, the unsigned integer type corresponding to int) because int can't represent all values of unsigned short.
The usual arithmetic conversions never "promote" an unsigned to signed if doing so would change the value.
[1] http://port70.net/~nsz/c/c99/n1256.html#6.3.1.8
According to the standard[1], if short and int have the same size (even if not the same rank) both numbers are converted to unsigned int (that is, the unsigned integer type corresponding to int) because int can't represent all values of unsigned short.
The usual arithmetic conversions never "promote" an unsigned to signed if doing so would change the value.
[1] http://port70.net/~nsz/c/c99/n1256.html#6.3.1.8
[deleted]
He does say on the first page: All other things being equal, assume GCC/LLVM x86/x64 implementation-defined behaviors. Are there any normal, modern machines where short and int are the same size? I think the last machine where that was true was the 8086.
That's still a bad question: the behavior is implementation-defined according to the standard, so having "implementation-defined" as a wrong option is ambiguous and confusing.
The question would be fine (given that note) if "impementation-defined" was not an option, like for example the question about "SCHAR_MAX == CHAR_MAX".
The question would be fine (given that note) if "impementation-defined" was not an option, like for example the question about "SCHAR_MAX == CHAR_MAX".
On x86 systems like 8086 short and int were the same size. And that appears to be a footnote they've added after being called out for being wrong. The question gives "implementation defined" as an option and in other questions seems to specify the ABI, and in some assumes it without saying again. Very inconsistent, they should fix their quiz really.
The last x86 processor I know of to have these sizes is 80286.
The last x86 processor I know of to have these sizes is 80286.
Avr microcontrollers still have 16bit ints, probably 8051 and pic too, but I don't use those. Lots of people do though. TI dsp uses 48 bit long, so don't count on int and long being the same either.
> I think the last machine where that was true was the 8086.
The last machine with a word size of 16 bits? The 286 was as well. There were others like the WDC 65816 (the Apple IIgs and SNES CPU).
It just so happens that there simply far fewer 16 bit CPUs then there were 8 or 32 (or “32/16” like the 68k). Also 8 bit CPUs are simply a poor fit for C by their nature and the assumptions C makes. But the numerous ones still relevant today will use 16 bit ints.
The use of Real or V86 mode on the x86 went on for many years after the demise of the 8086. I think it is a somewhat of a joke at this point that they’re teaching Turbo C in some developing countries.
The last machine with a word size of 16 bits? The 286 was as well. There were others like the WDC 65816 (the Apple IIgs and SNES CPU).
It just so happens that there simply far fewer 16 bit CPUs then there were 8 or 32 (or “32/16” like the 68k). Also 8 bit CPUs are simply a poor fit for C by their nature and the assumptions C makes. But the numerous ones still relevant today will use 16 bit ints.
The use of Real or V86 mode on the x86 went on for many years after the demise of the 8086. I think it is a somewhat of a joke at this point that they’re teaching Turbo C in some developing countries.
Define "machine". All kinds of microcontrollers are programmed with C.
Neat quiz. Reminds me that the absolute value of INT_MIN in C (and many other languages) is undefined, but will generally still return a negative value. This is a "gotcha" that a lot of people are unaware of.
> abs(-2147483648) = -2147483648
> abs(-2147483648) = -2147483648
A consequence of most, if not all, CPUs today using two's complement integers.
I think one's complement is more sensible since it doesn't have this problem, but it loses out because it requires a more complex ISA and implementation.
I think one's complement is more sensible since it doesn't have this problem, but it loses out because it requires a more complex ISA and implementation.
It's annoying that negation, ABS, and division can overflow with two's complement. But how I look at it: lots of operations can already overflow, just a fact of signed integers, and you need to guard against that overflow in portable code already. It doesn't seem to be fundamentally worse that those extra operations can overflow.
Ones' complement (correct spelling) has negative zero, which I would argue is a far worse problem.
Why in the fuck is it “ones' complement”, but “two's complement”?
According to Wikipedia:
> The name "ones' complement" (note this is possessive of the plural "ones", not of a singular "one") refers to the fact that such an inverted value, if added to the original, would always produce an 'all ones' number
> The name "ones' complement" (note this is possessive of the plural "ones", not of a singular "one") refers to the fact that such an inverted value, if added to the original, would always produce an 'all ones' number
And two's complement refers to what?
The radix complement in base two. They are closely related but not identical - the two's complement representation of a negative number is the ones' complement representation + 1 (ignoring the final carry). In ones' complment there are two representations for 0 (all 0s or all 1s), but every number has a positive and negative representation. In two's complement, there is a single 0, but there is also negative number that doesn't have a corresponding positive representation (INT_MIN).
In fact, for every numerical base there is an equivalent. For example, in base 3 you can represent a number in twos' complement notation (each trit X is replaced with 2-X, so -3 is represented as 212 on three trits, with +3 being 010) is or in three's complement by adding 1 (212+1 = 220). For base 10, you can do nines' complement (-3 on 3 digits is 996) or ten's complement (996+1 = 997).
In fact, for every numerical base there is an equivalent. For example, in base 3 you can represent a number in twos' complement notation (each trit X is replaced with 2-X, so -3 is represented as 212 on three trits, with +3 being 010) is or in three's complement by adding 1 (212+1 = 220). For base 10, you can do nines' complement (-3 on 3 digits is 996) or ten's complement (996+1 = 997).
[deleted]
It is undefined since it involves integer overflow
I only have cursory experience with C, mostly by programming some Arduino stuff, so I can only add this: good god this is a minefield.
Almost all of these examples would be sloppy programming in real code. The “int”, “short”, “long”, “long long” and “char” types are generic labels which shouldn’t be used where size and signedness matters. For a guarantee of size and signedness one should use the “[u]intXX_t” types. For sizes and pointer arithmetic - size_t. For subtracting pointers - ptrdiff_t. You simply wouldn’t have most of these issues if you stick to the correct types.
Unfortunately, even if you’re using uint16_t (which, remember, the C standard does not guarantee exists) on a platform with (say) 32-bit integers, you’re still actually computing in signed int due to promotion.
The true types (int, unsigned etc) are correct for local variables, since they describe registers. The sized aliases are correct for struct fields and for arrays in memory.
You should prefer signed types for computing (if not storing) sizes and for pointer arithmetic, since they are more forgiving with underflow.
You should prefer signed types for computing (if not storing) sizes and for pointer arithmetic, since they are more forgiving with underflow.
Absolutely. int is a "code smell" for us, except in some well-defined cases (storing file descriptors for example). If someone is using int to iterate a loop, then the code is more usually wrong than right, they should be using size_t.
I don't think a for loop using an int is bad or even "more wrong than right". If anything int is much better than using size_t.
Using an integer, in the 1000s of for loops I've written, none get even remotely close to the billions - it is optimizing for a 1 in a million case, and if I know something can run into the billions of iterations I'm going to pay more attention to anyway. I've seen 0 occurrences of bugs relating to this kind of overflow.
Using a size_t, it is effectively an unsigned integer that risks underflowing which can easily cause bugs like infinite loops if decrementing or other bugs if doing any index arithmetic. I've seen many occurrences of these kind of bugs.
Using an integer, in the 1000s of for loops I've written, none get even remotely close to the billions - it is optimizing for a 1 in a million case, and if I know something can run into the billions of iterations I'm going to pay more attention to anyway. I've seen 0 occurrences of bugs relating to this kind of overflow.
Using a size_t, it is effectively an unsigned integer that risks underflowing which can easily cause bugs like infinite loops if decrementing or other bugs if doing any index arithmetic. I've seen many occurrences of these kind of bugs.
On Linux/x86-64 int is 31 bits, so you've probably introduced "1000s" of security bugs where the attacker only needs the persistence to add 2 billion items to a network input, local file or similar, to generate a negative pointer of their choosing.
Any such code submitted to one of our projects would be rejected or fixed to use the proper type.
Any such code submitted to one of our projects would be rejected or fixed to use the proper type.
Please don't make this adversarial.
I've not introduced a security bug in every for loop I've written. What I've written shouldn't be controversial, just take a look at Googles style guide:
"We use int very often, for integers we know are not going to be too big, e.g., loop counters. Use plain old int for such things. You should assume that an int is at least 32 bits, but don't assume that it has more than 32 bits. If you need a 64-bit integer type, use int64_t or uint64_t.
For integers we know can be "big", use int64_t.
You should not use the unsigned integer types such as uint32_t, unless there is a valid reason such as representing a bit pattern rather than a number, or you need defined overflow modulo 2^N. In particular, do not use unsigned types to say a number will never be negative. Instead, use assertions for this.
If your code is a container that returns a size, be sure to use a type that will accommodate any possible usage of your container. When in doubt, use a larger type rather than a smaller type.
Use care when converting integer types. Integer conversions and promotions can cause undefined behavior, leading to security bugs and other problems."
I've not introduced a security bug in every for loop I've written. What I've written shouldn't be controversial, just take a look at Googles style guide:
"We use int very often, for integers we know are not going to be too big, e.g., loop counters. Use plain old int for such things. You should assume that an int is at least 32 bits, but don't assume that it has more than 32 bits. If you need a 64-bit integer type, use int64_t or uint64_t.
For integers we know can be "big", use int64_t.
You should not use the unsigned integer types such as uint32_t, unless there is a valid reason such as representing a bit pattern rather than a number, or you need defined overflow modulo 2^N. In particular, do not use unsigned types to say a number will never be negative. Instead, use assertions for this.
If your code is a container that returns a size, be sure to use a type that will accommodate any possible usage of your container. When in doubt, use a larger type rather than a smaller type.
Use care when converting integer types. Integer conversions and promotions can cause undefined behavior, leading to security bugs and other problems."
I've been a professional C programmer for nearly 40 years and didn't do especially well (I got about 2/3rds right). In my view it doesn't really matter. Enable maximum warnings and fix everything the compiler warns about. I also use Coverity once in a while, and every test is run with and without valgrind. If you do that you'll be fine.
> In my view it doesn't really matter.
One of these days, the compiler will do something surprising to one of your expressions involving signed integer overflow, like converting x < x + 1 to true. Or it'll delete a whole loop because it noticed that your code is guaranteed to trigger an out-of-bounds array read, e.g.: https://devblogs.microsoft.com/oldnewthing/?p=633
> If you do that you'll be fine.
I would not trust code written based on the methodology you described. However, if you also add -fsanitize=undefined,address (UBSan and ASan) and pass those tests, then I would trust your code.
One of these days, the compiler will do something surprising to one of your expressions involving signed integer overflow, like converting x < x + 1 to true. Or it'll delete a whole loop because it noticed that your code is guaranteed to trigger an out-of-bounds array read, e.g.: https://devblogs.microsoft.com/oldnewthing/?p=633
> If you do that you'll be fine.
I would not trust code written based on the methodology you described. However, if you also add -fsanitize=undefined,address (UBSan and ASan) and pass those tests, then I would trust your code.
Yes we do use the sanitization options too.
Yeah, in practice these sort of gotchas ought to trigger warnings. And for a C/C++ developer, warnings should never be allowed to persist in a code base.
I thought some deductive reasoning and thinking about how things work at the byte level would save me. Computer said no.
When you write C programs, you are coding on the C abstract machine as specified by the language standard. If your code triggers signed integer overflow, then the C abstract machine says that it's undefined behavior, and the compiler and runtime can make your code behave however it wishes. It doesn't matter that the underlying concrete machine (POSIX, x86 ISA, etc.) has wraparound signed overflow, because your code has to interact with both the abstract machine and the concrete machine at the same time. See: https://www.youtube.com/watch?v=ZAji7PkXaKY
> When you write C programs, you are coding on the C abstract machine as specified by the language standard.
Are you really, though? I would argue that it's a matter of perspective and/or semantics.
The Linux kernel is built with -fwrapv and with -fno-strict-aliasing, and uses idioms that depend on it directly. We can surmise from that that the kernel must be:
1. Exhibiting undefined behavior (according to a literal interpretation of the standard)
OR:
2. Not written in C.
Either way, it's quite reasonable to wonder just how much practical applicability your statement really has in any given situation -- since you didn't have any caveats. It's not as if the kernel is some esoteric, obscure case; it's arguably the single most important C codebase in the world. Plus there are plenty of other big C codebases that take the same approach besides Linux.
Lots of compiler people seem to take the same hard line on the issue -- "the C abstract machine" and whatnot. It always surprises me, because it seems to presuppose that the only thing that matters is what the ISO standard says. The actual experience of people working on large C codebases doesn't seem to even get acknowledged. Nor does the fact that the committee and the people that work on compilers have significant overlap.
I'm not claiming that "low-level C hackers are right and the compiler people are wrong". I'm merely pointing out that there is a vast cultural chasm that just doesn't seem to be acknowledged.
Are you really, though? I would argue that it's a matter of perspective and/or semantics.
The Linux kernel is built with -fwrapv and with -fno-strict-aliasing, and uses idioms that depend on it directly. We can surmise from that that the kernel must be:
1. Exhibiting undefined behavior (according to a literal interpretation of the standard)
OR:
2. Not written in C.
Either way, it's quite reasonable to wonder just how much practical applicability your statement really has in any given situation -- since you didn't have any caveats. It's not as if the kernel is some esoteric, obscure case; it's arguably the single most important C codebase in the world. Plus there are plenty of other big C codebases that take the same approach besides Linux.
Lots of compiler people seem to take the same hard line on the issue -- "the C abstract machine" and whatnot. It always surprises me, because it seems to presuppose that the only thing that matters is what the ISO standard says. The actual experience of people working on large C codebases doesn't seem to even get acknowledged. Nor does the fact that the committee and the people that work on compilers have significant overlap.
I'm not claiming that "low-level C hackers are right and the compiler people are wrong". I'm merely pointing out that there is a vast cultural chasm that just doesn't seem to be acknowledged.
Doesn't list my favorite footgun.
x and y are unsigned short. The expression `x*y` has defined behavior for...
a) all values of x and y.
b) some values of x and y.
c) no values of x and y.
x and y are unsigned short. The expression `x*y` has defined behavior for...
a) all values of x and y.
b) some values of x and y.
c) no values of x and y.
If short = 16 bits and int = 16 bits, then x and y will be promoted to unsigned int. Unsigned multiplication has wraparound behavior, so x*y will be defined for all values.
If short = 16 bits and int = 32 bits (or heck even 17 bits), then x and y will be promoted to signed int. Signed multiplication overflow is undefined behavior, so x*y will be undefined for some values when x*y is too large. In particular, 0xFFFF * 0xFFFF = 0xFFFE_0001, which is larger than INT_MAX = 0x7FFF_FFFF.
If short = 16 bits and int = 64 bits (or even 33 bits), then x and y will be promoted to signed int. The range of x*y will always fit int, so no overflow occurs, and the expression is defined for all input values.
Isn't C fun?
If short = 16 bits and int = 32 bits (or heck even 17 bits), then x and y will be promoted to signed int. Signed multiplication overflow is undefined behavior, so x*y will be undefined for some values when x*y is too large. In particular, 0xFFFF * 0xFFFF = 0xFFFE_0001, which is larger than INT_MAX = 0x7FFF_FFFF.
If short = 16 bits and int = 64 bits (or even 33 bits), then x and y will be promoted to signed int. The range of x*y will always fit int, so no overflow occurs, and the expression is defined for all input values.
Isn't C fun?
(B), assuming int is 32-bit and short is 16-bit. The multiplication promotes both operands and result to signed int, right? So if both x and y are uint16_max, the result overflows signed int and is UB, I think.
But if int were larger (eg 64 bit) and short remained 16 bits, there’s no overflow and the answer is (A). I think.
But if int were larger (eg 64 bit) and short remained 16 bits, there’s no overflow and the answer is (A). I think.
There was definitely a question that covered the auto-promotion to int, maybe with slightly different types.
I am gonna say A but I guess I dont the foot gun here
The shorts are not promoted to ints (if you even declared the result as that type) until after the multiplication. The result will first be put into a short and then promoted to int. It's basically asking for an overflow error. Given that 256^2 is 65536 you don't have to be multiplying large numbers before hitting that overflow.
All operands are always promoted to int (or unsigned int, long, unsigned long, etc.) before any operation. The footgun in this example is that even though you're using unsigned integers and would expect guaranteed wrapping semantics, the promotion to signed int makes it UB for USHORT_MAX*USHORT_MAX.
I don't follow, assuming short is half the size of int it would only overflow if most significant bits of both values are 1. Where did that 256^2 come from? It wouldn't overflow if a and b were 256
I missed that the result would be promoted to signed int, which only most significant bits are set
I missed that the result would be promoted to signed int, which only most significant bits are set
[deleted]
Unsigned short has defined overflow, though.
[deleted]
I have used C quite a bit, and I’m amazed at how many I got wrong. Nice example of why C is hated by so many I suppose.
Personally, I love C but I don’t use it in anything serious. Probably for the best given I apparently cannot remember how C ints work.
Personally, I love C but I don’t use it in anything serious. Probably for the best given I apparently cannot remember how C ints work.
It's important to notice the disclaimer that this is applies to x86/x86-64 GCC like platforms, in particular int is assumed to be 32 bits.
As antirez said as well, I don't pride myself in understanding the intricacies of integer arithmetic and promotion well. I try to write clear code by writing around the less commonly understood rules. Nevertheless I wanted to test myself. There are two questions that surprised me somewhat.
Apparently you can't left-shift a negative value even by a zero amount, as in (-1 << 0).
And is it true that the value of "-1L > 1U" is platform dependent? I had assumed that 1U would be promoted to 1L in this expression, even on x86 where unsigned int and long have the same number of bits. According to the following document, long int has a "higher rank" than unsigned int. https://wiki.sei.cmu.edu/confluence/display/c/INT02-C.+Under... . (Edit: according to rules 4 and 5 of "Usual arithmetic conversions", it's not only the rank but also about "the values that can be represented")
As antirez said as well, I don't pride myself in understanding the intricacies of integer arithmetic and promotion well. I try to write clear code by writing around the less commonly understood rules. Nevertheless I wanted to test myself. There are two questions that surprised me somewhat.
Apparently you can't left-shift a negative value even by a zero amount, as in (-1 << 0).
And is it true that the value of "-1L > 1U" is platform dependent? I had assumed that 1U would be promoted to 1L in this expression, even on x86 where unsigned int and long have the same number of bits. According to the following document, long int has a "higher rank" than unsigned int. https://wiki.sei.cmu.edu/confluence/display/c/INT02-C.+Under... . (Edit: according to rules 4 and 5 of "Usual arithmetic conversions", it's not only the rank but also about "the values that can be represented")
One thing to note is that long int is the same size as int on x64 Windows, at least in the MSVC ABI. clang also conforms to this.
This is relevant to the question asking if -1L > 1U.
This is relevant to the question asking if -1L > 1U.
same with ARM-EABI (32bit cortex-M MCUs). Int and long int are both 32 bits.
I think that is expected (it’s what happens for x86 as well) what’s surprising about the parent is that long is apparently 32 bit on windows x64. Usually long should be 64 bit on a machine with 64 bit words
Undefined behavior is probably the worst way we can imagine to define constraints usable by optimizers. Sad that major languages and implementations went this way.
Yeah, I think a lot of C's `undefined behaviour` semantics around assignments to ints should be reconsidered and changed to `unspecified` or `implementation defined` behaviour, and compilers can just do "whatever the hardware does". If that includes traps on one arch, fine, let it trap.
I think `undefined behaviour` still has its place in C - dereferencing a freed pointer comes to mind as an obvious example - but I think a good proportion of the really unintuitive UB conditions could be made saner without sacrificing portability or optimisation opportunities.
I think `undefined behaviour` still has its place in C - dereferencing a freed pointer comes to mind as an obvious example - but I think a good proportion of the really unintuitive UB conditions could be made saner without sacrificing portability or optimisation opportunities.
If you extend "unspecified" to allow traps, reading a bogus pointer can also be unspecified, only writes undefined in the current sense.
I don't think so. With "unspecified" and "implementation-defiend" behaviours, the implementation has to pick a behaviour and be consistent with it. The difference is whether they have to document that behaviour or not.
If the hardware traps on bogus pointers, then reading a bogus pointer may trap. But if you read a recently-freed pointer, it may still be valid according to the hardware (e.g. will have valid PTEs into the processes address space) so won't trap. Therefore you won't be able to guarantee any particular behaviour on an invalid read, so I don't think you'd be able to get away with "unspecified" or "implementation defined" behaviour on most hardware.
If the hardware traps on bogus pointers, then reading a bogus pointer may trap. But if you read a recently-freed pointer, it may still be valid according to the hardware (e.g. will have valid PTEs into the processes address space) so won't trap. Therefore you won't be able to guarantee any particular behaviour on an invalid read, so I don't think you'd be able to get away with "unspecified" or "implementation defined" behaviour on most hardware.
AFAIR reading uninitialized int, for example, is "unspecified" (any value could be there). If we consider adding "implementation defined with possible trap" for overflow, we might as well add "unspecified with possible trap" for reading an invalid pointer (any value could be there, or it could trap, but no nasal demons).
Using unitialized values is UB. Going back to naive pointers is a lost cause because compilers have started to do crazy optims (not even currently allowed by the standards...) like origin analysis. You can't steal that toy from the people implementing optims.
"Implementation defined" is worse in a lot of ways. Now the compiler has way less authority to tell you to stop! And we still have the problem of the application probably not doing what you want.
Current compilers don't "tell you to stop". They silently transform your program into complete garbage without even causality restraints. The sanitizers can help but they are merely dynamic when the dangerous tranformations are static in the first place.
It is true that the language has failed to provide tools that help developers prevent UB and that this is a very bad thing for the ecosystem. It can change, though.
In practice, compilers aren't actually adversarial. A lot of the discussion around UB is catastrophizing and talks about how the compiler will order you pizza or delete your disk. Some problems are real and I know some people whose graduate school work was very specifically on the problems that this causes for security-related checks but compilers really really do not transform your program into "complete garbage." They transform your program into a program with a bug, which is a true statement about that program.
I'm reminded of the apocryphal story about being asked how a computer knows how to do the right thing if given the wrong inputs. This feels similar.
In practice, compilers aren't actually adversarial. A lot of the discussion around UB is catastrophizing and talks about how the compiler will order you pizza or delete your disk. Some problems are real and I know some people whose graduate school work was very specifically on the problems that this causes for security-related checks but compilers really really do not transform your program into "complete garbage." They transform your program into a program with a bug, which is a true statement about that program.
I'm reminded of the apocryphal story about being asked how a computer knows how to do the right thing if given the wrong inputs. This feels similar.
Not apocryphal:
> On two occasions I have been asked, — "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.
-- Charles Babbage
https://en.wikiquote.org/wiki/Charles_Babbage#Passages_from_...
> On two occasions I have been asked, — "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.
-- Charles Babbage
https://en.wikiquote.org/wiki/Charles_Babbage#Passages_from_...
I do agree that the story around UB in C and C++ sucks. But UB doesn't exist because the compiler engineers want to be able to stick in optimizations. Once they are in the spec it makes sense to follow the rules but most UB comes from a desire for portability and to not privilege one platform over another.
And further, defining a lot of UB won't actually improve things. Imagine we define signed integer overflowing behavior. Hooray. Now your program just has a different bug. If you've accidentally got signed integer overflow in your application then "did some weird things because the compiler assumed it would never overflow" is going to cause exactly the same amount of havoc as "integer overflowed and now your algorithm is almost certainly wrong."
And further, defining a lot of UB won't actually improve things. Imagine we define signed integer overflowing behavior. Hooray. Now your program just has a different bug. If you've accidentally got signed integer overflow in your application then "did some weird things because the compiler assumed it would never overflow" is going to cause exactly the same amount of havoc as "integer overflowed and now your algorithm is almost certainly wrong."
> And further, defining a lot of UB won't actually improve things. Imagine we define signed integer overflowing behavior. Hooray. Now your program just has a different bug.
This is exactly what JF Bastien argues in his hourlong talk: https://youtu.be/JhUxIVf1qok?t=2284
So yeah, defining signed integer overflow isn't going to fix bugs in the vast majority of existing programs.
That being said, enforcing signed overflow wraparound at least makes debugging easier because it's reproducible. This is how it is in Java land - int overflow wraps around, and integer type widths are fixed, so if you trigger an overflow while testing then it is reliably reproducible across all conforming Java compiler and VM versions and platforms.
This is exactly what JF Bastien argues in his hourlong talk: https://youtu.be/JhUxIVf1qok?t=2284
So yeah, defining signed integer overflow isn't going to fix bugs in the vast majority of existing programs.
That being said, enforcing signed overflow wraparound at least makes debugging easier because it's reproducible. This is how it is in Java land - int overflow wraps around, and integer type widths are fixed, so if you trigger an overflow while testing then it is reliably reproducible across all conforming Java compiler and VM versions and platforms.
It also makes tools less able to loudly yell at you for having it in your application. Yes, you can turn on optional warnings if you want but if you've got one or two intended uses for this behavior then now you need suppressions and all sorts of mess.
I have an issue with this one:
Assume x has type int . Is the expression x<<32 ...
Defined for all values of x
Defines for some values of x
Defined for no values of x
(chose second answer) Wrong answer
Shifting (in either direction) by an amount equalling or exceeding the bitwidth of the promoted operand is an error in C99.
So according to Wikipedia: https://en.wikipedia.org/wiki/C_data_types
int signed signed int Basic signed integer type. Capable of containing at least the [−32,767, +32,767] range.[3][a]
So a minimum of 16 bits is used for int, but no maximum is specified. Thus, if my C compiler on my 64-bit architecture uses 64 bits for int, this is perfectly allowed by the specification and my answer is correct.
Defined for all values of x
Defines for some values of x
Defined for no values of x
(chose second answer) Wrong answer
Shifting (in either direction) by an amount equalling or exceeding the bitwidth of the promoted operand is an error in C99.
So according to Wikipedia: https://en.wikipedia.org/wiki/C_data_types
int signed signed int Basic signed integer type. Capable of containing at least the [−32,767, +32,767] range.[3][a]
So a minimum of 16 bits is used for int, but no maximum is specified. Thus, if my C compiler on my 64-bit architecture uses 64 bits for int, this is perfectly allowed by the specification and my answer is correct.
The top of the page says:
> All other things being equal, assume GCC/LLVM x86/x64 implementation-defined behaviors.
However, you're right to point out that the quiz would be better if you can't make any more assumptions than guaranteed by the basic language standard.
> All other things being equal, assume GCC/LLVM x86/x64 implementation-defined behaviors.
However, you're right to point out that the quiz would be better if you can't make any more assumptions than guaranteed by the basic language standard.
One of my friends likes to scold me about using a programming language (Python) that doesn't enforce type declarations. I gently remind him that his language (C) has eight different types of integers.
Heh, it has way more than 8.
For char, you have 3: signed char, unsigned char and char. It's not specified if char without keyword is signed or unsigned.
You have integer types such as size_t, ssize_t and ptrdiff_t. They may, under the hood, match one of the other standard int types, however this differs per platform, so you can't e.g. just easily print size_t using the standard printf formatters, you really have to treat is as its own type. Also wchar_t and such of course.
Then you have all the integers in stdint.h and inttypes.h. Same here applies as for size_t. At least you know how many bits you get from several of them, unlike from something like "long".
Then your compiler may also provide additional types such as __int128 and __uint128_t.
For char, you have 3: signed char, unsigned char and char. It's not specified if char without keyword is signed or unsigned.
You have integer types such as size_t, ssize_t and ptrdiff_t. They may, under the hood, match one of the other standard int types, however this differs per platform, so you can't e.g. just easily print size_t using the standard printf formatters, you really have to treat is as its own type. Also wchar_t and such of course.
Then you have all the integers in stdint.h and inttypes.h. Same here applies as for size_t. At least you know how many bits you get from several of them, unlike from something like "long".
Then your compiler may also provide additional types such as __int128 and __uint128_t.
> so you can't e.g. just easily print size_t using the standard printf formatters
This has been fixed in C99. For size_t, it's "%zu", for ptrdiff_t it's "%td", for ssize_t it's "&zd" and for wchar_t, it's "&lc".
This has been fixed in C99. For size_t, it's "%zu", for ptrdiff_t it's "%td", for ssize_t it's "&zd" and for wchar_t, it's "&lc".
The quiz is a little broken. CHAR_MAX is implementation defined but for some reason that question doesn’t have that option, even though other questions do.
Just a note for the author: The Green background on wrong answers was entirely confusing. Thought I was a C god at first.
Haven't written C in a long time. What a nightmare this is. Got most of them predictably wrong.
This tool I recently made can help: https://www.nayuki.io/page/summary-of-c-cpp-integer-rules#ar...
For example, you can see what (signed long) > (unsigned int) would convert to, under different environment bit width assumptions.
Also, there's a discussion on Reddit: https://www.reddit.com/r/cpp/comments/x4x01f/cc_arithmetic_c...
For example, you can see what (signed long) > (unsigned int) would convert to, under different environment bit width assumptions.
Also, there's a discussion on Reddit: https://www.reddit.com/r/cpp/comments/x4x01f/cc_arithmetic_c...
I taught first-semester C for several years, and still got a couple of these wrong - although, to be honest, the way we taught the class at my alma mater we steered well clear of these situations. We told students that C perform type promotions and automatic conversions, but either had them use a uniform type to begin with - int typically - or told them to convert explicitly and avoid
Still, the most important suggestion I would make here is: Always compile with warnings enabled, in particular:
* -Wstrict-overflow=1 and maybe even -Wstrict-overflow=3
* -Wsign-compare
* -Wsign-conversion
* -Wfloat-conversion
* -Wconversion
* -Wshift-negative-value
(or just `-Wall -Wextra`)
And maybe also:
* -fsanitize=signed-integer-overflow
* -fsanitize=float-cast-overflow
(These are GCC flags, should work for clang as well.)
Still, the most important suggestion I would make here is: Always compile with warnings enabled, in particular:
* -Wstrict-overflow=1 and maybe even -Wstrict-overflow=3
* -Wsign-compare
* -Wsign-conversion
* -Wfloat-conversion
* -Wconversion
* -Wshift-negative-value
(or just `-Wall -Wextra`)
And maybe also:
* -fsanitize=signed-integer-overflow
* -fsanitize=float-cast-overflow
(These are GCC flags, should work for clang as well.)
Exactly why I don't use C anymore. Other languages like Go don't allow this stuff.
https://go.dev/play/p/D2J6Y9ol41W
https://go.dev/play/p/D2J6Y9ol41W
Another fun one:
struct { long unsigned a : 31; } t = { 1 };
Correct answer to most of these questions: "I don't know, and I don't care, because it would never even cross my mind to write code in which most of these questions would come up. If forced to review or debug code written by someone else who knew the answers and leveraged that knowledge, I would either complain about it loudly or rewrite it quietly."
C, like Rome, is a wilderness of tigers. Why ask for trouble?
C, like Rome, is a wilderness of tigers. Why ask for trouble?
It's very worth pointing out and in fact advocating for the compiler options -fwrapv and -ftrapv
Which both make signed integers take the expected two's complement behavior, or traps signed integer overflow, respectively.
N.B. left shifting into the sign bit is still undefined behavior... On x86-64 gcc and clang seem to perform the shift as if the number is interpreted unsigned, then shifted, then interpreted signed.
Which both make signed integers take the expected two's complement behavior, or traps signed integer overflow, respectively.
N.B. left shifting into the sign bit is still undefined behavior... On x86-64 gcc and clang seem to perform the shift as if the number is interpreted unsigned, then shifted, then interpreted signed.
[deleted]
In the end, it's very simple and intuitive.
1- If you need arithmetic, use signed; if you need a bitmask, use signed. C is not assembly, and bit shift is no multiplication nor division.
2- Make sure you stay within bound, as in: don't even think you can approach the boundaries. C is not assembly, and overflow flag does not exist.
1- If you need arithmetic, use signed; if you need a bitmask, use signed. C is not assembly, and bit shift is no multiplication nor division.
2- Make sure you stay within bound, as in: don't even think you can approach the boundaries. C is not assembly, and overflow flag does not exist.
I know this is a tangential comment, but the fact that the site author went to the effort to add a css reset, and then doesn't go ahead and add in any kind of margins is pretty bizarre.
The fact that the text butts up to the left of the page with no margin is pretty incredible.
The fact that the text butts up to the left of the page with no margin is pretty incredible.
It's a bit frustrating that/I don't know why "implementation defined" and "assume x86" are both present. For most of these there is a fairly specific x86 implementation detail. For example, short char vs char size.
-Wall -Werror
#include <stdint.h>
and don't use primitive types, and you will avoid many of these issues.
#include <stdint.h>
and don't use primitive types, and you will avoid many of these issues.
The correct answer for many of these is "don't do that". :-)
That is the correct answer to the questions in this quiz!
Most professionals don't worry about these effects, instead choosing to cast and use parenthesis directly. Those instruct the compiler instead of relying upon warnings from the compiler.
Most professionals don't worry about these effects, instead choosing to cast and use parenthesis directly. Those instruct the compiler instead of relying upon warnings from the compiler.
[deleted]
In my opinion the three original sins of C are: nullptr, null-terminated strings, and implicit conversions.
Abolish those three concepts and C is a significantly improved language!
Abolish those three concepts and C is a significantly improved language!
> nullptr, null-terminated strings
Any system that lacks them will have to hack them back in at some point, in one form or another.
Option<types> are not an option for C, and sentinels like NULL work just fine.
Any system that lacks them will have to hack them back in at some point, in one form or another.
Option<types> are not an option for C, and sentinels like NULL work just fine.
> Any system that lacks them will have to hack them back in at some point, in one form or another.
Only because the C FFI is the lingua franca of operating systems.
> Option<types> are not an option for C
Disagree. C could absolutely support SumTypes / TaggedUnions.
> sentinels like NULL work just fine.
Spans/Slices are objectively superior. Null terminated strings aren't the biggest source of pain in the universe, but they're problematic.
I'm not arguing that C _should_ add these things today. I'm saying that C made bad choices 50 years ago and we're paying the price to this day. This isn't pointing fingers. It'd be upsetting if the industry hadn't made progress and didn't learn better ways to do things!
Only because the C FFI is the lingua franca of operating systems.
> Option<types> are not an option for C
Disagree. C could absolutely support SumTypes / TaggedUnions.
> sentinels like NULL work just fine.
Spans/Slices are objectively superior. Null terminated strings aren't the biggest source of pain in the universe, but they're problematic.
I'm not arguing that C _should_ add these things today. I'm saying that C made bad choices 50 years ago and we're paying the price to this day. This isn't pointing fingers. It'd be upsetting if the industry hadn't made progress and didn't learn better ways to do things!
> Only because the C FFI is the lingua franca of operating systems.
No, because they are genuinely useful. How should you best implement Option<type> for pointers? Right, using NULL as a sentinel. And representing sum types explicitly in the type system (instead of ad-hoc using sentinels) would add a huge amount of complexity.
I still like the simplicity of zero-terminated strings. They are self-contained strings that can be passed around as a simple char-pointer. No other format can give you this simplicity. They are still an acceptable default for string literals. And whenever you have a non-trivial usecase, just use the best string representation for that. You don't need to use zero-terminated strings everywhere.
Use a macro like this for a dumb length-delineated string
No, because they are genuinely useful. How should you best implement Option<type> for pointers? Right, using NULL as a sentinel. And representing sum types explicitly in the type system (instead of ad-hoc using sentinels) would add a huge amount of complexity.
I still like the simplicity of zero-terminated strings. They are self-contained strings that can be passed around as a simple char-pointer. No other format can give you this simplicity. They are still an acceptable default for string literals. And whenever you have a non-trivial usecase, just use the best string representation for that. You don't need to use zero-terminated strings everywhere.
Use a macro like this for a dumb length-delineated string
struct MyString { const char *ptr; int length; };
#define S(stringlit) (struct MyString) { "" slit, sizeof slit - 1 };
no bounds checkings, enums are hardly better than #define, pointer decay....
> pointer decay
This is an issue? Works fine for me.
This is an issue? Works fine for me.
It does wonders for security researchers jobs, because the idea of typing &arr[0] instead was asking too much.
Why _can_ you write &arr[0] in the first place? There is at least some logic behind it (which probably breaks down with multidimensional arrays, a feature that probably wasn't there from the start). In C, I suspect that at least in the beginning, there were never arrays on the type and expression level - only pointers.
What security problems are exactly caused by pointer decay?
A similar thing also exists with calls through function pointers, and I do not think you can find many programmers who are willing to write (*foo)(x, y, z) when using function pointers (at least not when there are many, like when using e.g. OpenGL). Or worse, (&foo)(x, y, z) with normal functions.
What security problems are exactly caused by pointer decay?
A similar thing also exists with calls through function pointers, and I do not think you can find many programmers who are willing to write (*foo)(x, y, z) when using function pointers (at least not when there are many, like when using e.g. OpenGL). Or worse, (&foo)(x, y, z) with normal functions.
The usual stuff that array lengths on function declarations are only guidelines, given that they are plain pointers, and when one calculates the array length not taking that into account, somewhere out there there is an happy hacker.
I'm a heavy use of C, and most of what I got wrong were around integer promotion.
I'm glad I run clang with -Weverything and use ASan and UBSan.
I'm glad I run clang with -Weverything and use ASan and UBSan.
Tangentially related, https://cppquiz.org/, for C++.
It would have been good if the feedback wasn't green (and for bonus points; not the same) for wrong answers.
[deleted]
Got all these correct. I was expecting something more convoluted or exotic undefined behavior.
I felt pretty smart getting that first question, then proceeded to fall down the stairs.
Watch out: The box becomes green no matter if you get it right or wrong.
On Windows, `long` and `int` are the same size, even when building for x64.
Seems like a good example of why interview quizzes aren't a panacea.
At this point, I'm Jon Snow. I'll just... anyway...
Wait so does an unsigned int promote to signed int for shift?
typically you shall avoid shift on signed integers, especially NEVER left shift on signed char|short|int|whatever.
I limit shift strictly to unsigned integer numbers.
I limit shift strictly to unsigned integer numbers.
Following this rule strictly can be tricky.
On a weird but legal platform where int is 24 bits wide, the expression (uint16_t)0xFFFF << 15 will cause undefined behavior.
My workaround for this is to force promotion to unsigned int: (0U + x) << 15. https://stackoverflow.com/questions/39964651/is-masking-befo...
uint16_t x = (...);
uint16_t y = x << 15;
On a weird but legal platform where int is 24 bits wide, the expression (uint16_t)0xFFFF << 15 will cause undefined behavior.
My workaround for this is to force promotion to unsigned int: (0U + x) << 15. https://stackoverflow.com/questions/39964651/is-masking-befo...
I really hate C.
I made a similar test for floating point: https://beza1e1.tuxen.de/no_real_numbers.html
Since most languages use the same IEEE-754, you can hate them all.
Since most languages use the same IEEE-754, you can hate them all.
Unbiased, equal hate, fully in line with today's political correctness? Fine, I really hate programming languages.
This quiz only reminded me how little I know about C (thankfully those case are corner-cases of incompetent programming so other and I can ignore their existence).
Good grief, thanks Quiz for reminding me to stay away from that mess of language.
Good grief, thanks Quiz for reminding me to stay away from that mess of language.
Rust not having a default "int" type and forcing you to explicitly cast everything is such an improvement. Truly a poster child for "less is more". Yeah it makes the code more verbose, but at least I don't have to worry about "lmao you have a signed int overflow in there you absolute nincompoop, this is going to break with -O3 when GCC 16 releases 8 years from now, but only on ARM32 when compiled in Thumb mode!"