Essential C [pdf](cslibrary.stanford.edu)
cslibrary.stanford.edu
Essential C [pdf]
http://cslibrary.stanford.edu/101/EssentialC.pdf
77 comments
> "Could you tell the linux kernel devs to stop […]"
As you seem to care about this, have you made this argument on LKML yourself? What Linux kernel related work are you active on that drives your position?
As you seem to care about this, have you made this argument on LKML yourself? What Linux kernel related work are you active on that drives your position?
Bummer, that train is gone. There recently was a kernel community decision to explicitly move to C11.
Its nicely documented on LWN https://lwn.net/SubscriberLink/885941/01fdc39df2ecc25f/
It was also discussed here on HN https://news.ycombinator.com/item?id=30459634
Its nicely documented on LWN https://lwn.net/SubscriberLink/885941/01fdc39df2ecc25f/
It was also discussed here on HN https://news.ycombinator.com/item?id=30459634
Standard C is not suitable for writing kernels anymore due to all the ridiculous hostile interpretions of the standard (mainly typed based aliasing rules).
I mean you can't even write an allocator in standard C.
I mean you can't even write an allocator in standard C.
Both sides are true IMO. The kernel isn't capable of being written totally in standards compliant C, and it'd probably be better off if that fact wasn't used as carte blanche to write non standards compliant code where it doesn't provide a tangible benefit.
> I mean you can't even write an allocator in standard C.
I might be missing something, but I don't believe this is true? The type aliasing rules have an explicit carve-out for `char *` aliases of other types, which is intended to solve this exact issue.
Similarly, C99 and C11 both allow aliasing through a union of pointers without violating the strict aliasing rule.
I might be missing something, but I don't believe this is true? The type aliasing rules have an explicit carve-out for `char *` aliases of other types, which is intended to solve this exact issue.
Similarly, C99 and C11 both allow aliasing through a union of pointers without violating the strict aliasing rule.
You can't implement `malloc()` in C because the pointers it returns are defined to be pointers to different objects, and if you can see the implementation of `malloc` then you know it's returning pointers to the same object (the page it gets from mmap/sbrk). There's no functionality in C to actually do what it says except for the function itself.
My understanding is that this is covered by the "effective type" language in both C99 and C11[1].
malloc itself is specified to return a void pointer, but the effective type established during assignment circumvents what would otherwise be UB via aliasing[2].
Edit: Specifically, the reasoning is:
1. Allocated objects have no declared type;
2. 6.5.6: Objects with no declared type have the type of their accessing lvalue
3. 6.5.7: Access of a stored value is valid for lvalue expressions that are type compatible with the effective type.
So there's no UB here. `malloc` itself doesn't need to know about the effective type produced by the lvalue, and C (at least C99 onwards) respects that type for strict aliasing purposes.
[1]: https://port70.net/%7Ensz/c/c11/n1570.html#6.5p6
[2]: https://port70.net/%7Ensz/c/c11/n1570.html#6.5p7
malloc itself is specified to return a void pointer, but the effective type established during assignment circumvents what would otherwise be UB via aliasing[2].
Edit: Specifically, the reasoning is:
1. Allocated objects have no declared type;
2. 6.5.6: Objects with no declared type have the type of their accessing lvalue
3. 6.5.7: Access of a stored value is valid for lvalue expressions that are type compatible with the effective type.
So there's no UB here. `malloc` itself doesn't need to know about the effective type produced by the lvalue, and C (at least C99 onwards) respects that type for strict aliasing purposes.
[1]: https://port70.net/%7Ensz/c/c11/n1570.html#6.5p6
[2]: https://port70.net/%7Ensz/c/c11/n1570.html#6.5p7
The issue in C99 is:
> The lifetime of an allocated object extends from the allocation until the deallocation. Each such allocation shall yield a pointer to an object disjoint from any other object.
If the implementation of malloc is visible, then it can be inlined etc., and then you have to deal with the effects of this obviously not being true.
> The lifetime of an allocated object extends from the allocation until the deallocation. Each such allocation shall yield a pointer to an object disjoint from any other object.
If the implementation of malloc is visible, then it can be inlined etc., and then you have to deal with the effects of this obviously not being true.
It would have been better, if the writer had not used the word “object”.
Here's an outline of a toy version:
#include <stdlib.h> /* for size_t */
static char data_storage[10000000];
static char *nextp = data_storage;
void *malloc(size_t size)
{
void *p = nextp;
nextp += size;
return p;
}
void free(void *p){}This does not "yield a pointer to an object disjoint from any other object". So it only works if the implementation is not visible to callers and they can pretend it's compliant.
Huh? My version never returns a pointer to space that was already allocated because it never deallocates anything. So the objects are ALWAYS disjoint from every other object. Or are you concerned that they are aliased with the static 'data_storage' char array?
More context around your quote:
> The lifetime of an allocated object extends from the allocation until the deallocation. Each such allocation shall yield a pointer to an object disjoint from any other object.
https://port70.net/%7Ensz/c/c11/n1570.html#7.22.3
In context, I take that to mean that all allocated objects must be disjoint from one another throughout their lifetime. It wouldn't make sense otherwise.
More context around your quote:
> The lifetime of an allocated object extends from the allocation until the deallocation. Each such allocation shall yield a pointer to an object disjoint from any other object.
https://port70.net/%7Ensz/c/c11/n1570.html#7.22.3
In context, I take that to mean that all allocated objects must be disjoint from one another throughout their lifetime. It wouldn't make sense otherwise.
> So the objects are ALWAYS disjoint from every other object. Or are you concerned that they are aliased with the static 'data_storage' char array?
Yes, plus if you call it twice it returns two pointers to the same object, because there's no way in C to create another "object". Of course, as long as the callers don't know this it's fine, but it would be a problem if eg you compile your custom malloc with ASAN/Valgrind and don't tell it that it's a malloc.
I think C++ partially addresses this with "placement new" but not sure how far.
Yes, plus if you call it twice it returns two pointers to the same object, because there's no way in C to create another "object". Of course, as long as the callers don't know this it's fine, but it would be a problem if eg you compile your custom malloc with ASAN/Valgrind and don't tell it that it's a malloc.
I think C++ partially addresses this with "placement new" but not sure how far.
> if you call it twice it returns two pointers to the same object
How are you defining an object? This draft version of the C11 spec defines 'object' as a "region of data storage in the execution environment, the contents of which can represent values". https://port70.net/%7Ensz/c/c11/n1570.html#3.15
How are you defining an object? This draft version of the C11 spec defines 'object' as a "region of data storage in the execution environment, the contents of which can represent values". https://port70.net/%7Ensz/c/c11/n1570.html#3.15
The drafting is not the best. The key point is that objects have bounds (even though it doesn't explicitly say it), and pointers point to a base object, and it's UB if those pointers go outside the bounds of that object.
malloc is defined to return a pointer to a new "base object". But your code doesn't do that; you can see by reading it that it returns a pointer to `data_storage`. That means the UB conditions for using that pointer don't match the spec.
You could say it's supposed to magically work if the function is named `malloc`, but my understanding of all C implementations is they don't do that.
malloc is defined to return a pointer to a new "base object". But your code doesn't do that; you can see by reading it that it returns a pointer to `data_storage`. That means the UB conditions for using that pointer don't match the spec.
You could say it's supposed to magically work if the function is named `malloc`, but my understanding of all C implementations is they don't do that.
You never could implement malloc() in C anyway, because the language lacks the ability to manipulate the MMU.
C runs on platforms such as microcontrollers that don't have MMU's. Malloc is available on those platforms.
The point isn't that malloc is available, rather that it is impossible to write in ISO C, other than mapping a global memory segment.
It's strict aliasing that's the issue, not the mapping of memory to a address space. You run into the same issues with a global segment.
Both are a problem, because they can only be implemented outside ISO C semantics.
That is why I tend to always write ISO C instead of C.
That is why I tend to always write ISO C instead of C.
One's not a blocker to a malloc implementation since you could otherwise just cut up a static char buffer. Coordinating with the OS is a nice to have. Strict aliasing is hard blocker. Hence why that's the issue under discussion.
And I'm not sure I understand why you're using the inability to describe certain actions as the reason why you use ISO C.
And I'm not sure I understand why you're using the inability to describe certain actions as the reason why you use ISO C.
Implementations of C have more functionality than C itself - like inline assembly or syscalls or machine-specific intrinsics, so it can do more. ISO C only has what's written in the standard.
(A syscall is an example of "something you can only do because the implementation isn't visible to the caller" - it can violate aliasing that way.)
Also, my argument isn't about type aliasing, it's about UB on out of bounds pointers. Could be some other aliasing issues though.
(A syscall is an example of "something you can only do because the implementation isn't visible to the caller" - it can violate aliasing that way.)
Also, my argument isn't about type aliasing, it's about UB on out of bounds pointers. Could be some other aliasing issues though.
Then in this case the portability offered by C is false. Could be an equally valid approach to place the required behavior into separate modules written with an assembler.
It was never true, "portable" C programs are full of #ifdef spaghetti.
well, alternative compilers can excel in not providing broken optimizations based on UB. -Oboring is a thing in the kernel. chibicc e.g
I have often wondered if a C89 standard with no UB would be the definitive go to language for systems. I mean you COULD define things if you chose to, right?
Some kinds of UB can easily be handled that way. GCC offers a flag to guarantee wrapping on signed arithmetic overflow, for instance. [0] It wouldn't be easy to nicely handle every instance of UB though. The current state-of-the-art for that kind of thing would be something like Valgrind, an extremely 'intrusive' runtime system.
This is because of the particulars of the C language. For instance, the way it handles arrays and pointer arithmetic make it difficult to detect every instance of out-of-bounds access.
[0] -fwrapv, see https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html
This is because of the particulars of the C language. For instance, the way it handles arrays and pointer arithmetic make it difficult to detect every instance of out-of-bounds access.
[0] -fwrapv, see https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html
I think we're talking about different things, but maybe not.
Using out of bounds references in arrays and pointer arithmetic are unsafe but they can be defined. The definition of "array accesses using indexes that are outside the range of 0 - the declared array size will access memory in the data segment. The compiler will treat that access as if the addressed memory had the type specified for the array." The behavior is defined, it is unsafe, but it is defined. Not like "the compiler may or may not choose to optimize out loads and stores, or re-order them." which especially on embedded systems creates bugs where the things like "read the status register THEN read the data register (which clears the status register when read)" those get re-ordered and suddenly your loop never exits because your status bit is never set. That kind of UB needs to die in a fire IMHO.
Using out of bounds references in arrays and pointer arithmetic are unsafe but they can be defined. The definition of "array accesses using indexes that are outside the range of 0 - the declared array size will access memory in the data segment. The compiler will treat that access as if the addressed memory had the type specified for the array." The behavior is defined, it is unsafe, but it is defined. Not like "the compiler may or may not choose to optimize out loads and stores, or re-order them." which especially on embedded systems creates bugs where the things like "read the status register THEN read the data register (which clears the status register when read)" those get re-ordered and suddenly your loop never exits because your status bit is never set. That kind of UB needs to die in a fire IMHO.
> which especially on embedded systems creates bugs where the things like "read the status register THEN read the data register (which clears the status register when read)" those get re-ordered and suddenly your loop never exits because your status bit is never set.
Why are the embedded systems programmers not using `volatile`, which exists for this exact reason?
Why are the embedded systems programmers not using `volatile`, which exists for this exact reason?
"volatile" is not a synchronization primitive and doesn't forbid reordering amongst access to different memory locations.
All it does is forbid reordering or removing accesses to a particular memory location.
Historically, many compilers implemented it as a hard memory barrier, but that isn't how the standard defines it.
All it does is forbid reordering or removing accesses to a particular memory location.
Historically, many compilers implemented it as a hard memory barrier, but that isn't how the standard defines it.
Volatile operations are considered a side effect and as such may not be reordered with other volatile operations. How is that not literally sufficient for the operations that GGP described?
I spoke perhaps a little strongly. But that said:
The compiler is free to reorder accesses to multiple volatile variables if they happen prior to the same sequence point. So (roughly) expressions involving two volatile variables do not have those accesses sequenced.
But you are right that the usage described earlier is OK, as long as both variables are marked volatile, and the accesses straddle a sequence point.
The more common mistake with volatiles is to use the for multithreading primitives.
The compiler is free to reorder accesses to multiple volatile variables if they happen prior to the same sequence point. So (roughly) expressions involving two volatile variables do not have those accesses sequenced.
But you are right that the usage described earlier is OK, as long as both variables are marked volatile, and the accesses straddle a sequence point.
The more common mistake with volatiles is to use the for multithreading primitives.
It's downright perverse to insist that you should be able to assume all accesses are volatile reads/ stores just because that was momentarily convenient.
C lacks the intrinsics you'd actually want for this (explicit load/ store of various machine sizes) but it does provide the "volatile" storage qualifier which is what you should be using to do what you apparently wanted in C today. It makes no sense to demand everybody else writes some sort of "not-volatile" qualifier in front of every variable to tell the compiler that actually this is just a variable and it's OK to optimise.
C lacks the intrinsics you'd actually want for this (explicit load/ store of various machine sizes) but it does provide the "volatile" storage qualifier which is what you should be using to do what you apparently wanted in C today. It makes no sense to demand everybody else writes some sort of "not-volatile" qualifier in front of every variable to tell the compiler that actually this is just a variable and it's OK to optimise.
> "[...] The compiler will treat that access as if the addressed memory had the type specified for the array." The behavior is defined, it is unsafe, but it is defined.
I believe performance (compiler optimisation) is the reason the language isn't defined that way. Permitting the compiler to assume that the runtime error will never arise, opens the door to all sorts of optimisations. (At least, that's the idea.)
C permits the 'union trick' to (roughly speaking) access the bit-pattern of a value as another type, which is to say an escape-hatch is offered in the language.
Similarly the strict aliasing rule is surprising to people who are new to C, but the C standard committee seem to be committed to keeping it, presumably for performance reasons.
> Not like "the compiler may or may not choose to optimize out loads and stores, or re-order them." which especially on embedded systems creates bugs
Right, but it's defined that way to enable compiler optimizations, not to spite the programmer. As others have mentioned, C has features like volatile specifically to address this kind of thing. If the C standard required memory fences to be inserted everywhere, performance would be ruined.
> That kind of UB needs to die in a fire IMHO.
C cannot easily be made into a safe language, and I think the committee is doing the right thing in declining to try to make C into something it isn't. On the plus side, there are plenty of other languages around, many of them with compelling advantages over C. Ada, Rust, and Zig, for instance.
I believe performance (compiler optimisation) is the reason the language isn't defined that way. Permitting the compiler to assume that the runtime error will never arise, opens the door to all sorts of optimisations. (At least, that's the idea.)
C permits the 'union trick' to (roughly speaking) access the bit-pattern of a value as another type, which is to say an escape-hatch is offered in the language.
Similarly the strict aliasing rule is surprising to people who are new to C, but the C standard committee seem to be committed to keeping it, presumably for performance reasons.
> Not like "the compiler may or may not choose to optimize out loads and stores, or re-order them." which especially on embedded systems creates bugs
Right, but it's defined that way to enable compiler optimizations, not to spite the programmer. As others have mentioned, C has features like volatile specifically to address this kind of thing. If the C standard required memory fences to be inserted everywhere, performance would be ruined.
> That kind of UB needs to die in a fire IMHO.
C cannot easily be made into a safe language, and I think the committee is doing the right thing in declining to try to make C into something it isn't. On the plus side, there are plenty of other languages around, many of them with compelling advantages over C. Ada, Rust, and Zig, for instance.
WG14 could perfectly add proper string and vector types, even if library based structs, they didn't had any qualms to add complex numbers in C99.
Then again, just use C++ alongside std::vector, std::array and std::string with FORTIFY turned on (or equivalent) and be done with it.
Then again, just use C++ alongside std::vector, std::array and std::string with FORTIFY turned on (or equivalent) and be done with it.
Out of bounds is only defined for length + 1, for anything else anything goes.
That doesn't sound right. C (and C++) permit you to do pointer arithmetic to derive a pointer value pointing to one element beyond the final element of an array. This special treatment doesn't extend to dereferencing though - if you deference that pointer, that's UB.
If you do pointer arithmetic to derive a pointer value pointing 2 or more elements beyond the final element of an array, that's undefined behaviour, even if you never dereference that pointer.
If you do pointer arithmetic to derive a pointer value pointing 2 or more elements beyond the final element of an array, that's undefined behaviour, even if you never dereference that pointer.
Sure, I just didn't want to spend too much time explaining all details, my failure.
How do you eliminate ub without memory safety?!
For example, as mentioned in another comment, by making all read and write `volatile`, that way, dangling pointer and out of bounds are "defined" to be memory corruption or crash, and not the compiler optimizing the code in a way that the programmer did not enticipate.
Wouldn't that bring a performance penalty though?
Yes.
But the current trade-off (performance always wins) means that kernels and other embedded-style programs cannot rely on the compiler doing the "reasonable" thing for UB because it's explicitly allowed to do whatever it wants (which is generally, try for better performance).
For kernel-style work, slightly lower performance but predictable/defined behavior for some of what is currently UB, makes life much simpler.
But the current trade-off (performance always wins) means that kernels and other embedded-style programs cannot rely on the compiler doing the "reasonable" thing for UB because it's explicitly allowed to do whatever it wants (which is generally, try for better performance).
For kernel-style work, slightly lower performance but predictable/defined behavior for some of what is currently UB, makes life much simpler.
You don’t need to do what youre saying for that; Rust provides a way to get both performance and safety
How is _Generic toxic?
It has some annoying misfeatures. Closely related types aren't treated as the same and you can't put multiple related types in a _Generic association list. You're forced to resort to explicit casts or call the type specific function directly which subverts the intended magic of using _Generic in the first place.
Some of this was fixed by DR 481 [http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2396.htm#dr...], can you give some specific cases/examples at issue?
it's ugly and doesn't feel like a first class citizen of the language
This is a vague and emotional argument to a technical question.
> char ASCII character -- at least 8 bits. Pronounced "car".
Never heard it pronounced car!
Never heard it pronounced car!
Really? I've always pronounced it car (kar) because it's short for character which is pronounced karakter in the first place.
Everybody seems to say ree-verb as short for reverberation. I don't say "reeverberation", I say something more like "ruhverberation" — which I don't think is an odd pronunciation — hence I say "ruhverb", but people look at me funny when I do.
Likewise I say char as "car".
Likewise I say char as "car".
Right, but what sound does ‘a’ have for you? In my region, car, carrot, and care are 3 different vowel sounds (with ‘car’ being totally different from the others) and ‘character’ sounds like ‘carrot’ and not at all like ‘car’.
If "carrot" and "care" don't rhyme with each other, what other words do they rhyme with?
Again, it's regional. I have lived in places (Washington state, I think, as well as the South) where to my ear people said 'carrot' like 'care'. But where I'm from (northeast) carrot sounds like calculus, car sounds like calm, and care sounds like mare. Care is closer to cake ('longer'), carrot is 'short' like 'at'. Car is like with a little short 'o' mixed in, like in Spanish or the posh version of 'amen'.
I wish I knew the formal phonetic symbols of these, but I don't. But disambiguating these is just what they're for.
I wish I knew the formal phonetic symbols of these, but I don't. But disambiguating these is just what they're for.
I live in Washington state but hail from Illinois, I’ve never heard anyone say carrot like car. It’s always been care-at
Well, carrot rhymes with parrot :-) But also, hat or cat.
Car rhymes with are, or charge.
And care with pear, pair, hair, etc.
But I'm Australian, so I talk funny.
Car rhymes with are, or charge.
And care with pear, pair, hair, etc.
But I'm Australian, so I talk funny.
I'm from NJ and that all aligns with my pronunciations
I used to live in Jersey City, maybe that explains it?
Maybe! I was actually considering saying I was from NY just to not make anyone think of a "Joisey" accent. I grew up in the suburbs which means a close tie to NYC despite the distance. And Jersey City being right across the river from NY certainly takes influence from there.
The three pronunciations I've heard used are "kar", "char" (like "charge"), and "care".
Thank you! Most useful C book I ever got.
This one seems to be probably most in-depth free book for C programming in practice. https://beej.us/guide/bgc/
I learned C from that document with devc++ & mingw on windows :)
This is legitimately nice. Thanks for sharing it.
I have gone over half of it, and I will start recommending it left and right.
I have gone over half of it, and I will start recommending it left and right.
Can someone explain to me why in Section 4: Functions they compute Twice by multiplying the number by three then subtracting the number once? Is it just to showcase more math, or is there a detail I'm missing?
> /* Computes double of a number. Works by tripling the number, and then subtracting to get back to double. /
> /* Computes double of a number. Works by tripling the number, and then subtracting to get back to double. /
static int Twice(int num) {
int result = num * 3;
result = result - num;
return(result);**A great book!
[deleted]
For instance in the whole network stack, some "agents of the planned obsolescence" did not use constant expressions in switch/case statements or initializers. Also tell those who are using that toxic c11 "Generic" keyword to use explicit code instead. I think, the main issue for C now is the standard body trying to obsolete stable in time C code and forcing upon devs/integrators the usage of the only few compilers which are able to follow fast enough their tantrums (latest gcc/latest clang). Results: all attempts to provide "working" alternative C compilers is beyond reasonable. This is becoming so accute, it is more likely to be a worldwide scam than anything else.
Yep, open source software is not enough anymore, it needs to be lean, simple but able to do reasonably "the job", and ofc very stable in time hence C89 only with benign bits of c99/c11, or "standard assembly".