Clang Format Tanks Performance(travisdowns.github.io)
travisdowns.github.io
Clang Format Tanks Performance
https://travisdowns.github.io/blog/2019/11/19/toupper.html
155 comments
I'm going to take the contrarian view here, and say it's not clickbait.
I admit that I'm not a C++ expert, but naively I would never have expected clang-format to have any effect on my code. If you stopped and suggested header re-ordering to me my first thought would have been "I guess it must not do that. Maybe in practice header re-ordering doesn't actually matter with reasonable code?".
The title here doesn't describe the contents of the article, but it describes a very important consequence of it that C++ programmers should be aware of. It gives a specific example of the consequence. I'm ok with that.
I admit that I'm not a C++ expert, but naively I would never have expected clang-format to have any effect on my code. If you stopped and suggested header re-ordering to me my first thought would have been "I guess it must not do that. Maybe in practice header re-ordering doesn't actually matter with reasonable code?".
The title here doesn't describe the contents of the article, but it describes a very important consequence of it that C++ programmers should be aware of. It gives a specific example of the consequence. I'm ok with that.
I agree with most of that, although as the author I think it's still clickbait in the sense that:
1) clang-format is not really at fault here at all - the same effect could have happened by adding a new header, reordering the headers manually, switching to a system with a different libc version or a different transitive header include tree, etc.
2) The title doesn't tell you wtf is up. You have to get at least half way through the post to figure that out. It's in the vein of "This one food could wreck your health" posts.
That said, other aspects of clickbait are advertising or monetization to profit from the clicks. Except for the crypto mining bot I load into your browser, and which subsequently writes itself to your MBR or UEFI partition, I have no ads, affiliate links, patreon-alikes or other monetization vectors.
1) clang-format is not really at fault here at all - the same effect could have happened by adding a new header, reordering the headers manually, switching to a system with a different libc version or a different transitive header include tree, etc.
2) The title doesn't tell you wtf is up. You have to get at least half way through the post to figure that out. It's in the vein of "This one food could wreck your health" posts.
That said, other aspects of clickbait are advertising or monetization to profit from the clicks. Except for the crypto mining bot I load into your browser, and which subsequently writes itself to your MBR or UEFI partition, I have no ads, affiliate links, patreon-alikes or other monetization vectors.
Of course clang format is at fault; reordering includes, even standard or system includes, will almost certainly effect how a complex C/C++ program compiles. That's just how it is, with the preprocessor and such.
This is a _bad_ clang-format bug.
This is a _bad_ clang-format bug.
You can disable include reordering in clang-format. I don't think this is a bug, because I see the default here as a matter of policy decision.
Given that reordering includes can have consequences, reordering includes blindly and casually like the OP is a form of Russian roulette: an order that works should be preserved until it stops working.
The article describes false complexity caused by looking atr the issue in the wrong way: a "mysterious" performance problem, to be diagnosed, rather than the obvious consequence of a broken workflow that includes an extremely hazardous operation.
In a voluntary header reordering experiment, all kinds of breakage could be detected with a trivial before and after comparison of compiled code.
The article describes false complexity caused by looking atr the issue in the wrong way: a "mysterious" performance problem, to be diagnosed, rather than the obvious consequence of a broken workflow that includes an extremely hazardous operation.
In a voluntary header reordering experiment, all kinds of breakage could be detected with a trivial before and after comparison of compiled code.
Yes, but reordering the headers is usually safe. Having order-dependent behavior is a serious bug and should generally be fixed by the header authors. When you add a header do you usually carefully consider the interaction between the new header and all the exist ones? No, you just add it to the end of the existing list or at its sorted position? Yeah, me too.
I agree that a formatting tool changing the behavior or performance is a serious issue, but I don't think it's a bug in clang-format in that I suspect it is deliberate design choice, and users can opt out if they want. Maybe the default could be changed or more visible warnings added to the documentation, but the real fix here needs to be in glibc.
I agree that a formatting tool changing the behavior or performance is a serious issue, but I don't think it's a bug in clang-format in that I suspect it is deliberate design choice, and users can opt out if they want. Maybe the default could be changed or more visible warnings added to the documentation, but the real fix here needs to be in glibc.
No, I don’t normally think about header order, but if I’m unlucky enough to be getting some weird behavior based on header order, I think I’d be able to debug it before landing my change.
I’d be pissed if the issue were introduced by a formatting pass.
I’d be pissed if the issue were introduced by a formatting pass.
Headers really shouldn’t be affecting each other. Header order shouldn’t matter. The fact that it does is arguably a bad design flaw in C/C++, although the C modules system pushed by Apple fixed this.
Tooling developers don’t get to say “it shouldn’t matter”. It does matter, so the tools should account for this.
clang-format does account for this: if you break your includes into groups with blank lines, clang-format will sort within but not across the groups. That is, clang-format offers a crude but simple API to the programmer to express which headers have order dependency, and which don't.
Given that headers can have order dependency, it is sadly part of a C++ programmer's responsibility to know which do, just as it is their responsibility to know which functions have side-effects, etc. If they are using clang-format, it is then their responsibility to communicate this knowledge to it.
Given that headers can have order dependency, it is sadly part of a C++ programmer's responsibility to know which do, just as it is their responsibility to know which functions have side-effects, etc. If they are using clang-format, it is then their responsibility to communicate this knowledge to it.
That reordering is simply a feature of clang-format. Some coding standards require sorting includes alphabetically, possibly to avoid dupes and to make all include usage identical between source files.
In terms of C/C++ headers, that's like requiring all scopes have their expressions alphanumerically sorted.
C/C++ headers are literal code-injection mechanisms; they aren't modifying a runtime environment by importing symbols, they inject code into the compilation unit.
C/C++ headers are literal code-injection mechanisms; they aren't modifying a runtime environment by importing symbols, they inject code into the compilation unit.
> Some coding standards require sorting includes alphabetically
Those are some pretty dumb coding standards if that's true.
(I mean, yeah, the original fault is with the C preprocessor and the fact that header order matters, but since we're stuck with that, having those kinds of coding standards is pretty darn silly!)
Those are some pretty dumb coding standards if that's true.
(I mean, yeah, the original fault is with the C preprocessor and the fact that header order matters, but since we're stuck with that, having those kinds of coding standards is pretty darn silly!)
Well given that the order of the headers matter, wouldn't it make sense to have the coding standard specify one true include order ? That's the only way to have a deterministic result.
That ties namespacing to performance, which just sounds horrible to deal with.
Of course, the bug is horrible. But given that those type of bugs exist, having a fixed ordering of include statements seem like the least painful way to go.
Indeed, and one reason I dislike sorting header includes alphanumerically is that it makes it easy for include-order-dependencies between them to creep in, as they're always used in the same relative order.
At least you’re forthcoming.
Clickbait, but _good_ clickbait :) I do so love good deep technical blog posts that dig into why something works a certain way, and the title is a legit statement based on the initial cause. Thanks for writing this!
Just to make sure I'm understanding this correctly: the performance issue you found actually amounts to include-order dependent behavior in GCC and glibc; clang doesn't really have anything to do with anything here except that you used its code formatter to sort your include statements.
If that's the case, the clickbait-y headline's bordering on useless, since the performance differences here have nothing to do with clang itself-- you'd see the same results if you had ordered your includes the same way by hand.
If that's the case, the clickbait-y headline's bordering on useless, since the performance differences here have nothing to do with clang itself-- you'd see the same results if you had ordered your includes the same way by hand.
I'm talking about clang-format, a formatting tool for C and C++ code, which has nothing to do with compiling your code with clang. You can use it with any compiler.
That said, the title is still a lie in that clang-format is not really at a fault here at all: clang-format triggered the issue on my codebase, by sorting header files, and this in turn triggered a libc performance issue.
So as I initially say it, a commit which purely clang-format 'd my codebase caused a massive regression - but the full story is doesn't implicate clang-format at all, really.
That said, the title is still a lie in that clang-format is not really at a fault here at all: clang-format triggered the issue on my codebase, by sorting header files, and this in turn triggered a libc performance issue.
So as I initially say it, a commit which purely clang-format 'd my codebase caused a massive regression - but the full story is doesn't implicate clang-format at all, really.
The title would be a /little/ clearer with a dash between Clang and format.
Good point, I put a hyphen a few places but not everywhere nor in the title. It seems hyphen is definitely the standard though.
Fixed now [1], although the HN title will remain hyphen-free free I guess.
---
[1] https://github.com/travisdowns/travisdowns.github.io/commit/...
Fixed now [1], although the HN title will remain hyphen-free free I guess.
---
[1] https://github.com/travisdowns/travisdowns.github.io/commit/...
Or "header inclusion order may affect performance." If someone points to this as a reason not to use clang-format...
clang-format reordering includes is a terrible behaviour. It would break many C/C++ projects.
It's optional, although it's enabled by default in the LLVM and Chromium styles.
That's a bizarre optional feature to have; it's tantamount to having a "blindly reorder logic" function that sorts your source alphanumerically.
Not really too bizarre. Header files should include or declare everything they need; they should not introduce include-order dependencies. Listing your includes in lexicographic order is a good way to enforce header completeness.
It is by convention that C/C++ headers rely on preprocessor state to determine what blocks to reveal, macros to use, et al.
It is quite common to have an auto-generated configuration header, for instance; or a precompiled header; or optional headers that, when present, mutate the behaviour of other headers.
Every time you run a configure script for a C project there's a good chance you're interacting with code in this way.
It is quite common to have an auto-generated configuration header, for instance; or a precompiled header; or optional headers that, when present, mutate the behaviour of other headers.
Every time you run a configure script for a C project there's a good chance you're interacting with code in this way.
clang-format still allows some ordering when the SortIncludes feature is enabled. Search for IncludeBlocks in https://clang.llvm.org/docs/ClangFormatStyleOptions.html
Yeah, and if you want to use clang-format include reordering in such a project, you should read the docs for clang-format's "IncludeCategories". It allows you to separate your includes into blocks by providing regexes and associated priorities. Alphabetic sorting is done only within a block.
It's not like clang-format was written by idiots who never used the language before. Maybe read up on what the tool actually does before you get all mad at them?
It's not like clang-format was written by idiots who never used the language before. Maybe read up on what the tool actually does before you get all mad at them?
I'm not mad?
I think it's a harmful anti-feature that's worthy of criticism.
I think it's a harmful anti-feature that's worthy of criticism.
One other ordering I've encountered which helps remove errors is to list the headers in reverse generic order. Meaning that you go from the most specific header files to the least specific. So <algorithm> and <windows.h> would be getting included last.
This helps ensure that your specific project level header files have all the necessary forward declares and includes to be a fully functioning and complete header.
This helps ensure that your specific project level header files have all the necessary forward declares and includes to be a fully functioning and complete header.
I also sort my includes this way. The trouble is, determining how generic an include (inclusion?) is is a fuzzy problem. Sometimes I get stuck trying to figure out which header is more abstract, and at times I've run into side effects like this even with reverse generic sorting.
Yeah, this header sorting is only meant to be a tool to help craft header files which are independent. If that's not your project structure then you can use whatever sorting you want.
In the end though you'll have to make a decision on how to sort the projects and which one is more or less specific than another. Kind of like deciding to just sort by pointer address if all else is equal.
In the end though you'll have to make a decision on how to sort the projects and which one is more or less specific than another. Kind of like deciding to just sort by pointer address if all else is equal.
No, the headline pretty clearly states "Clang Format". I've used clang-format on MSVC codebases, and imagine there are some who use it with GCC (say, for instance, the author). It's still the best C++ formatter, as far as I know, and is compiler-independent.
It's basically the gorilla in the space, and it's good.
There is no gcc equivalent. In any case there no 1:1 relationship between formatters anyways: you might compiler you codebase with N compilers, but you'll only have one formatter unless you are insane. So my codebases are not "gcc" or "clang" codebases, but basically "linux" codebases - but the formatter is always clang-format.
There is no gcc equivalent. In any case there no 1:1 relationship between formatters anyways: you might compiler you codebase with N compilers, but you'll only have one formatter unless you are insane. So my codebases are not "gcc" or "clang" codebases, but basically "linux" codebases - but the formatter is always clang-format.
It would actually be interesting to see if this phenomenon happens with clang, which has its own implementation of the standard library. And perhaps with musl libc or some such.
Clang has its own libc++ implementation, but I think clang still uses the system libc and associated headers?
Yes, i think so too. So a different <algorithm>, but the same <ctype.h>.
The crucial __NO_CTYPE define is in <bits/os_defines.h>, which is part of the GCC paraphernalia. I don't know if clang's stdlib would also have to end up defining it.
The crucial __NO_CTYPE define is in <bits/os_defines.h>, which is part of the GCC paraphernalia. I don't know if clang's stdlib would also have to end up defining it.
I clicked because of the title and laughed when I read the reason, don't sweat about anyone accusing you of clickbait
clang format can tank performance” would, IMO, not decrease the number of people who click the link, yet remove the clickbait.
Could be, but I feel it would equally dishonest?
This really has nothing to do with clang-format: I only mention that once (twice with an update: where clang format actually helps because it reorders stdlib.h after ctype.h) - it's all about header include order.
This really has nothing to do with clang-format: I only mention that once (twice with an update: where clang format actually helps because it reorders stdlib.h after ctype.h) - it's all about header include order.
Maybe someone should make a tool that randomly reorders your header includes until optimal performance is found.
I think clang-format is definitely relevant; the take-away here is that no formatter should change the order of includes.
I was wondering about this only yesterday - tidying up some C++ and C code. I don't remember any standard that says it's OK to re-order headers, and clearly it can matter a lot.
I was wondering about this only yesterday - tidying up some C++ and C code. I don't remember any standard that says it's OK to re-order headers, and clearly it can matter a lot.
That wouldn't be my take away.
My takeway would be anything that changes in semantics or performance due to header includes should be considered a bug by the (possibly joint) writers of the involved headers.
clang-format is an innocent victim here. Maybe I should make that clearer.
My takeway would be anything that changes in semantics or performance due to header includes should be considered a bug by the (possibly joint) writers of the involved headers.
clang-format is an innocent victim here. Maybe I should make that clearer.
Clang isn't the root cause, but as a (relatively) new tool in an existing ecosystem it needs to work with the existing ecosystem and not some imaginary idealistic world.
Changing semantics due to header order is considered a feature in C/C++. If you aren't willing to waste hours working through esoteric header issues, you just aren't tough enough.
Why does every title have to be a mystery? Why not tell people what it contains?
In this case it's an excerpt of a larger post that I was writing while I ran into this.
I don't think there is a good single-sentence title that would encapsulate the whole thing without being an already well-known fact, like say "Header include order matters".
Also, I wrote this one as a narrative, not revealing the conclusion at the start - basically a reconstructed version of how I encountered it myself. I don't write all my posts like that and it's not for everyone: but once I've decided on that, it makes no sense to reveal the conclusion in the title, I think?
The HN headline could be changed though if it places a priority on information-full titles.
I don't think there is a good single-sentence title that would encapsulate the whole thing without being an already well-known fact, like say "Header include order matters".
Also, I wrote this one as a narrative, not revealing the conclusion at the start - basically a reconstructed version of how I encountered it myself. I don't write all my posts like that and it's not for everyone: but once I've decided on that, it makes no sense to reveal the conclusion in the title, I think?
The HN headline could be changed though if it places a priority on information-full titles.
It places priority on not-clickbait, above most things. You could have a title that isn't super descriptive, asks a question, maintains an air of mystery, whatever. But a title that deliberately elevates some near-irrelevant detail for attention is purposefully deceptive and, well, less than good (by HN's rules, that is).
Eh, sure - but you are arguing for non-clickbait titles, while I was replying to the OP who complained about not revealing more about the conclusion in the title. It seems we agree there that some mystery is allowed: the point of contention is on "click-baitiness".
I don't have a similar principled defense of clickbait titles, either in general or mine specifically! I just thought it was a funny one and didn't expend more effort on coming up with a non-clickait one.
That's an independent issue from revealing more info in the title.
I don't have a similar principled defense of clickbait titles, either in general or mine specifically! I just thought it was a funny one and didn't expend more effort on coming up with a non-clickait one.
That's an independent issue from revealing more info in the title.
Oh, I don't really have any argument about what titles you should put on your own writing. It's your writing, if you feel its narrative purpose is served by an intricate web of title feints, line noise, whatnot, more power to you.
I'm just explaining the HN title rules, you can see them here (do a find title):
https://news.ycombinator.com/newsguidelines.html
There's nothing in there about titles having to be 'information-full'. Lots of perfectly HN-good titles aren't.
I'm just explaining the HN title rules, you can see them here (do a find title):
https://news.ycombinator.com/newsguidelines.html
There's nothing in there about titles having to be 'information-full'. Lots of perfectly HN-good titles aren't.
Got it, I didn't understand you were talking about HN titles, rather than the blog post title.
I didn't submit it to HN so I can't comment on the title choice, other than say it at least reflects the literal title of the page which I guess would be the default when submitting.
I tend to outsource my comments to HN, since my blog is a github pages static site and introducing comments without tracking and ads is a pain, but I definitely don't choose my post titles to satisfy HN rules.
I didn't submit it to HN so I can't comment on the title choice, other than say it at least reflects the literal title of the page which I guess would be the default when submitting.
I tend to outsource my comments to HN, since my blog is a github pages static site and introducing comments without tracking and ads is a pain, but I definitely don't choose my post titles to satisfy HN rules.
Right, technically, fixing the title in this case would have been on the poster. Then there'd probably be a long subthread about why the title was changed so drastically. Title-lawyering is a professional metasport around here, as has been amply demonstrated.
Grabs popcorn :).
Pretty much the HN version of nerd sniping.
https://xkcd.com/356/
https://xkcd.com/356/
So that you can't just respond to the title.
> Are my standard algorithms letting me down? Does std::transform have some fatal flaw? Not really. Well, not at all.
This part lost me because I can't see how you can use std::transform (which is in <algorithm>) without hitting this flaw. Unless you edit the headers? I guess you mean the algorithm isn't flawed.
I also didn't understand the title because I don't use clang-format and I skimmed past the single mention of it (my fault).
Regardless, interesting post.
This part lost me because I can't see how you can use std::transform (which is in <algorithm>) without hitting this flaw. Unless you edit the headers? I guess you mean the algorithm isn't flawed.
I also didn't understand the title because I don't use clang-format and I skimmed past the single mention of it (my fault).
Regardless, interesting post.
The title was basically clickbait.
It reflects how I encountered the issue, but the investigation doesn't have much of that, because I was starting from a point where I suddenly had a slow and fast algorithm (the actual scenario was more complicated than shown).
The investigation doesn't have much to do with clang-format because I'm just trying to see why a raw loop is faster, and ultimately has nothing to do with "raw" vs "std" at all, really. Only when I understood that includes and include order matter was I able to map it back to a header order swap made by clang-format.
> This part lost me because I can't see how you can use std::transform (which is in <algorithm>) without hitting this flaw.
No, two ways.
You can get the fast performance with std::transform if you happen to include <ctype.h> before <algorithm>. The order matters.
Similarly, you can get the slow performance with a raw loop if you happen to include <algorithm> before <ctype.h> in the file for the raw loop.
In fact, I'd say that the raw loop and std::transform will have the same performance almost all the time. If they are in the same file, they will have the same performance. If they are in a C++ file, you are almost certainly including some C++ header that triggers the issue. Only in special cases, like a coding convention that separates C and C++ headers with whitespace with C first (clang-format doesn't sort headers separated by whitespace) are you likely to come into it "natively".
Even if you do, it's absolutely no fault of std::transform or <algorithm> - it's a weird effect of interaction between C, C++ and OS headers, nothing inherent to the C++ algorithms.
It reflects how I encountered the issue, but the investigation doesn't have much of that, because I was starting from a point where I suddenly had a slow and fast algorithm (the actual scenario was more complicated than shown).
The investigation doesn't have much to do with clang-format because I'm just trying to see why a raw loop is faster, and ultimately has nothing to do with "raw" vs "std" at all, really. Only when I understood that includes and include order matter was I able to map it back to a header order swap made by clang-format.
> This part lost me because I can't see how you can use std::transform (which is in <algorithm>) without hitting this flaw.
No, two ways.
You can get the fast performance with std::transform if you happen to include <ctype.h> before <algorithm>. The order matters.
Similarly, you can get the slow performance with a raw loop if you happen to include <algorithm> before <ctype.h> in the file for the raw loop.
In fact, I'd say that the raw loop and std::transform will have the same performance almost all the time. If they are in the same file, they will have the same performance. If they are in a C++ file, you are almost certainly including some C++ header that triggers the issue. Only in special cases, like a coding convention that separates C and C++ headers with whitespace with C first (clang-format doesn't sort headers separated by whitespace) are you likely to come into it "natively".
Even if you do, it's absolutely no fault of std::transform or <algorithm> - it's a weird effect of interaction between C, C++ and OS headers, nothing inherent to the C++ algorithms.
I bet the performance would look different in a real program anyways. The random input data probably results in more branch predict misses than would occur in normal text, and the tight loop of the benchmark basically ensures the lookup table is always in cache.
Definitely, but I think it's unlikely that it will close the gap much between the fast and slow std::toupper() compilations (my own toupper is mostly just in there as an interesting reference point).
In addition to more instructions and a function call, the slow version has many more memory dereferences, so if everything is very cold it is likely to suffer more misses.
In addition to more instructions and a function call, the slow version has many more memory dereferences, so if everything is very cold it is likely to suffer more misses.
A disagree with your conclusion that the slowness doesn't matter. If it doesn't matter to anybody then why does the optimized version even exist?
IMO this is a library bug. <algorithm> shouldn't be limping toupper or anything else.
IMO this is a library bug. <algorithm> shouldn't be limping toupper or anything else.
The conclusion near the end of the blog post?
That's not really because performance doesn't matter, but because toupper() shouldn't really matter in most modern C and C++ programs: it just can't support Unicode. So either you are using a different way of supporting strings entirely (if you need any kind of Unicode or multibyte support), or this is some internal ASCII-only text processing in which case you are better off using your own methods (e.g., the lookup table I mention at the end) since they'll provide an easy speedup and you aren't accidentally introducing locale-dependent code where you don't want it.
That's not really because performance doesn't matter, but because toupper() shouldn't really matter in most modern C and C++ programs: it just can't support Unicode. So either you are using a different way of supporting strings entirely (if you need any kind of Unicode or multibyte support), or this is some internal ASCII-only text processing in which case you are better off using your own methods (e.g., the lookup table I mention at the end) since they'll provide an easy speedup and you aren't accidentally introducing locale-dependent code where you don't want it.
[deleted]
I read the title and thought the article was about (clang-format's performance) on a (program simulating military tanks)... Apparently you meant "tanks" as a verb...
Yes, although a quick search reveals that Oxford considers this a US usage so it was probably not the ideal choice for a title.
The "three observations that aren’t really part of the story" were really interesting and doesn't seem to be fully resolved. Will you be discussing them more in the future post?
Maybe.
This is really a three part answer, since there were three quite-different effects I glossed over.
The first one is interesting, but it is also well-covered elsewhere, e.g., [1], [2] and [3]. That said, I have a ton of stuff to say on branch prediction, so I probably will at some point. Those posts are slow going, however. It may never emerge.
For the second and third points, I am partly blocked by the fact the effect disappeared or is only intermittently reproducible.
I am more interested in the third uninteresting effect than the second. The second thing could be a lot of things, since it shows up when running a python script with results piped to disk. What I've seen in the past is that the "driver" program (driver.py) undergoes some kind of phase change, e.g., suddenly causing a bunch of context switches or allocating/freeing a munch of memory which affects the program under test (despite CPU pinning, since there are shared resources). They are phantoms (but still interesting!).
The third one is very interesting to me, because it involves a CPU-bound loop with only L1 hits. It goes to the core of the Skylake uarch and the kind of fine-grained uarch details I'm interested in. However, it disappeared as I was tracking it down. I was getting a mix of 2.07 cycles and 1.57 cycles (slow and fast) running the benchmark repeatedly, trying differnet perf counters and doing the needful to track it down and then suddenly I only got 2.07. A new plot showed "blue" only at ~2.07. So I don't know. I have seen similar things though, and I've tracked down an similar "mode shift" and will post on it soon. Stay tuned.
---
[1] https://lemire.me/blog/2019/11/12/unrolling-your-loops-can-i...
[2] https://lemire.me/blog/2019/10/16/benchmarking-is-hard-proce...
[3] https://lemire.me/blog/2019/10/15/mispredicted-branches-can-...
This is really a three part answer, since there were three quite-different effects I glossed over.
The first one is interesting, but it is also well-covered elsewhere, e.g., [1], [2] and [3]. That said, I have a ton of stuff to say on branch prediction, so I probably will at some point. Those posts are slow going, however. It may never emerge.
For the second and third points, I am partly blocked by the fact the effect disappeared or is only intermittently reproducible.
I am more interested in the third uninteresting effect than the second. The second thing could be a lot of things, since it shows up when running a python script with results piped to disk. What I've seen in the past is that the "driver" program (driver.py) undergoes some kind of phase change, e.g., suddenly causing a bunch of context switches or allocating/freeing a munch of memory which affects the program under test (despite CPU pinning, since there are shared resources). They are phantoms (but still interesting!).
The third one is very interesting to me, because it involves a CPU-bound loop with only L1 hits. It goes to the core of the Skylake uarch and the kind of fine-grained uarch details I'm interested in. However, it disappeared as I was tracking it down. I was getting a mix of 2.07 cycles and 1.57 cycles (slow and fast) running the benchmark repeatedly, trying differnet perf counters and doing the needful to track it down and then suddenly I only got 2.07. A new plot showed "blue" only at ~2.07. So I don't know. I have seen similar things though, and I've tracked down an similar "mode shift" and will post on it soon. Stay tuned.
---
[1] https://lemire.me/blog/2019/11/12/unrolling-your-loops-can-i...
[2] https://lemire.me/blog/2019/10/16/benchmarking-is-hard-proce...
[3] https://lemire.me/blog/2019/10/15/mispredicted-branches-can-...
[deleted]
I may eat downvotes but here's an honest opinion nonetheless: this article shows almost exactly why I wouldn't touch C++ with a ten-foot pole still, after running away screaming from it 11-12 years ago.
Not saying it's an unique problem with C++'s tooling. I'm just saying that it has been my experience that one is much more likely to stumble upon such horrors while working with C++ as opposed to at least 7 other languages I worked with.
I'll take the slower Rust and Go and Elixir, thank you very much.
Don't take this as trash-talking, though. I simply want to get things done, not fight my environment.
Not saying it's an unique problem with C++'s tooling. I'm just saying that it has been my experience that one is much more likely to stumble upon such horrors while working with C++ as opposed to at least 7 other languages I worked with.
I'll take the slower Rust and Go and Elixir, thank you very much.
Don't take this as trash-talking, though. I simply want to get things done, not fight my environment.
> Don't take this as trash-talking, though. I simply want to get things done, not fight my environment.
I think people don't understand just how badly C++ is broken in reality (vs conceptually) until their paycheck is tied to a real, years-old, multi-developer C++ project. It is not trash talking, it is pointing out truths.
It goes well beyond "any long-lived project has its issues." I think you and I (and many others) know that it is much larger than this, it is a fundamental issue with the language, the standards body surrounding it, and the mess of toolings.
This persists today because C++ is often used to bridge a performance gap between scripted languages and C among younger programmers and academics. (Admit it: we all like that sweet sweet dopamine hit when we take our Python to C++ and the syntax is almost the same.) But that slight performance boost and that "hey this looks like Python but it's really C++" feeling are soon demolished by the accretion of unresolved issues which have enumerated ad-nauseum on HN, including this article.
I've been working with C++ since 1993 (I got a big Borland C++ Compiler as an intern plus 12 VHS tapes of Bjarne himself explaining the language)... my experience with C++ continues right up to today as a contractor having to deal with Arm mbedOS (even Keil struggles to manage hypersensitivity, GNU is a trainwreck for embedded C++) and Tensorflow/OpenCV... and C++ has yet to shake off its flaws. In fact, I think it has added to them exponentially.
I think people don't understand just how badly C++ is broken in reality (vs conceptually) until their paycheck is tied to a real, years-old, multi-developer C++ project. It is not trash talking, it is pointing out truths.
It goes well beyond "any long-lived project has its issues." I think you and I (and many others) know that it is much larger than this, it is a fundamental issue with the language, the standards body surrounding it, and the mess of toolings.
This persists today because C++ is often used to bridge a performance gap between scripted languages and C among younger programmers and academics. (Admit it: we all like that sweet sweet dopamine hit when we take our Python to C++ and the syntax is almost the same.) But that slight performance boost and that "hey this looks like Python but it's really C++" feeling are soon demolished by the accretion of unresolved issues which have enumerated ad-nauseum on HN, including this article.
I've been working with C++ since 1993 (I got a big Borland C++ Compiler as an intern plus 12 VHS tapes of Bjarne himself explaining the language)... my experience with C++ continues right up to today as a contractor having to deal with Arm mbedOS (even Keil struggles to manage hypersensitivity, GNU is a trainwreck for embedded C++) and Tensorflow/OpenCV... and C++ has yet to shake off its flaws. In fact, I think it has added to them exponentially.
Yep, I am in full agreement with your points. Especially this:
> It goes well beyond "any long-lived project has its issues."
I heard this mantra as a justification for bad coding practices likely 500 times.
I appreciate that C++ still drives a lot of our virtual world but that's more a proof of the tenacity of those programs' creators than the virtues of C++ itself. IMO it's time tools like `ripgrep`, `dua` / `ds-cli` (both calculate directory contents' sizes) to become the norm: Rust tools, multicore friendly, insanely fast, and with no quirks and weirdly interacting CLI switches.
> It goes well beyond "any long-lived project has its issues."
I heard this mantra as a justification for bad coding practices likely 500 times.
I appreciate that C++ still drives a lot of our virtual world but that's more a proof of the tenacity of those programs' creators than the virtues of C++ itself. IMO it's time tools like `ripgrep`, `dua` / `ds-cli` (both calculate directory contents' sizes) to become the norm: Rust tools, multicore friendly, insanely fast, and with no quirks and weirdly interacting CLI switches.
If you want to just get things done then this sort of complexity wouldn't impact you in the slightest using C++. You could just write code without needing to know this information. This is something for people really needing to squeeze performance out of systems.
If you are looking for performance though, other languages have similar but different pitfalls and bits of esoteric knowledge that you need. Constructs that you should use or avoid, the layout of data in memory, when to run or prevent running of a GC, etc.
If you are looking for performance though, other languages have similar but different pitfalls and bits of esoteric knowledge that you need. Constructs that you should use or avoid, the layout of data in memory, when to run or prevent running of a GC, etc.
True. I am simply looking for a compiled not-far-away-from-C++ levels of performance language, plus one with the least amount of pitfalls / gotchas / WTFs to avoid. So far Rust seems to hit a sweet spot, although its style (procedural) and syntax leave much to be desired.
Achieving “not far away from C++ levels of performance” with anything but a procedural-style language is very difficult, rare, and usually comes with severe trade-offs, unfortunately.
For example: Haskell has been trying to bring the performance of C to a functional language for almost 30 years now. While Haskell is a great language with impressive performance for its category, I don’t think anyone believes it renders C/C++ obsolete for scenarios where predictably high performance is crucial.
This is an interesting topic to me, because while I try to find ways to write high-performance code as functionally as possible, I can’t seem to escape the relationship where more procedural-styled code usually yields consistently/predictably higher performance.
Functional (and other paradigms) can sometimes match C’s performance -- but in non-toy scenarios, the “sometimes” clause here compounds its probabilities to ultimately become “virtually never”, as the scope and complexity of a real project grows.
As a result, performance-critical projects written in languages without predictable performance characteristics often evolve into a situation later in development where 90% of your development time is spent poking at 'black boxes' (the compiler optimizer and garbage collector), hoping (sometimes futilely) that you can prod it into spitting out the procedural machine code patterns you already knew you needed anyway — if you’re lucky. And what makes this even worse is that this situation usually arises late enough into development that it's very costly (if not impossible) to backtrack and rewrite everything in a procedural language.
Of course, writing procedurally ends up trading off readability and robustness too human error, vs better performance. I too wish there was a better way to achieve both — I just haven’t found it yet.
For example: Haskell has been trying to bring the performance of C to a functional language for almost 30 years now. While Haskell is a great language with impressive performance for its category, I don’t think anyone believes it renders C/C++ obsolete for scenarios where predictably high performance is crucial.
This is an interesting topic to me, because while I try to find ways to write high-performance code as functionally as possible, I can’t seem to escape the relationship where more procedural-styled code usually yields consistently/predictably higher performance.
Functional (and other paradigms) can sometimes match C’s performance -- but in non-toy scenarios, the “sometimes” clause here compounds its probabilities to ultimately become “virtually never”, as the scope and complexity of a real project grows.
As a result, performance-critical projects written in languages without predictable performance characteristics often evolve into a situation later in development where 90% of your development time is spent poking at 'black boxes' (the compiler optimizer and garbage collector), hoping (sometimes futilely) that you can prod it into spitting out the procedural machine code patterns you already knew you needed anyway — if you’re lucky. And what makes this even worse is that this situation usually arises late enough into development that it's very costly (if not impossible) to backtrack and rewrite everything in a procedural language.
Of course, writing procedurally ends up trading off readability and robustness too human error, vs better performance. I too wish there was a better way to achieve both — I just haven’t found it yet.
Insightful. I will be treading a very similar path soon.
So, isn't there a way to emulate FP's `map` mechanics for example, in Rust, without losing performance?
EDIT: To reflect your edits, this is why I want to learn Rust. I am as happy as one can be with Elixir due to Erlang's transparent super-robust and predictable parallel performance; this thing simply does not lag! However, I'd like to be prepared and be able to utilise Rust on the performance hot-spots.
So, isn't there a way to emulate FP's `map` mechanics for example, in Rust, without losing performance?
EDIT: To reflect your edits, this is why I want to learn Rust. I am as happy as one can be with Elixir due to Erlang's transparent super-robust and predictable parallel performance; this thing simply does not lag! However, I'd like to be prepared and be able to utilise Rust on the performance hot-spots.
Yes, both Rust and Zig are two of the most promising languages in this field IMO. Rust's generics is probably the best example of the 'best of both worlds' (high level abstraction, with predictably high performance), but it's also not without tradeoffs[1].
Rust is more mature, well-known, and akin to C++ in its philosophy (big language with powerful features and abstraction capabilities), while Zig[2] is less well-known but IMO a beautifully designed language akin to C in its emphasis on simplicity, while remaining modern in design (e.g. unlike Google Go, which is very simple but intentionally repeats many of C's mistakes[3]).
[1] Such powerful generics usually come with unavoidable explosions in compile time (and sometimes mental complexity for the coder). This is because while it may look like you're simply calling a library function and passing in a reference to your callback function, what it's really doing most of the time (to achieve good performance) is recompiling the entire library from scratch (or nearly so) with your function inlined. Also, the performance characteristics of any given block of code won’t be as obvious and predictable at-a-glance as it would be in a simpler language.
[2] https://ziglang.org/
[3] https://www.lucidchart.com/techblog/2015/08/31/the-worst-mis...
Rust is more mature, well-known, and akin to C++ in its philosophy (big language with powerful features and abstraction capabilities), while Zig[2] is less well-known but IMO a beautifully designed language akin to C in its emphasis on simplicity, while remaining modern in design (e.g. unlike Google Go, which is very simple but intentionally repeats many of C's mistakes[3]).
[1] Such powerful generics usually come with unavoidable explosions in compile time (and sometimes mental complexity for the coder). This is because while it may look like you're simply calling a library function and passing in a reference to your callback function, what it's really doing most of the time (to achieve good performance) is recompiling the entire library from scratch (or nearly so) with your function inlined. Also, the performance characteristics of any given block of code won’t be as obvious and predictable at-a-glance as it would be in a simpler language.
[2] https://ziglang.org/
[3] https://www.lucidchart.com/techblog/2015/08/31/the-worst-mis...
The major problem with functional programming is that many idioms requires a garbage collector. For example having pure operations on binary trees by just replacing dirty nodes leaving all old references intact wouldn't be possible in rust.
Regarding binary trees: You can always go back to using old-fashioned arrays and indices. A few simple contiguous arrays of nodes (e.g. a key array and value array) is often all you need, where nodes simply refer to each other via int indices into these arrays. You may be surprised that this often yields some of the best-performing data structures, sometimes better than those using raw pointers and non-contiguous heap allocations.
And, this works in Rust just as it does in C. But at this point, you lose out on the benefits of Rust's static type system and borrow-checker:
1. The compiler will no longer be able to correctness-check the validity of these 'int' style references to other memory.
2. If something does go wrong and you read the array with a bad integer index, Rust will just 'panic' and crash the application. Unlike C, there will be no risk of memory errors or related security vulnerabilities. But on the other hand, each read being bounds-checked will make such Rust code slightly slower than is possible with C/C++. (Though I think you can use 'unsafe' blocks if you want to hyper-optimize akin to C.)
But regarding GC langauges, I generally agree; they're almost always more trouble than they're worth in any performance-sensitive context. You often end up using approaches like this to optimize around the GC (e.g. int indexes into pre-allocated arrays) which ultimately means you're coding C-style in a GC language anyway, which defeats the whole point of GC's productivity-enhancing benefit -- at that point, why not just go all the way and use C or Zig or Rust etc.?
And, this works in Rust just as it does in C. But at this point, you lose out on the benefits of Rust's static type system and borrow-checker:
1. The compiler will no longer be able to correctness-check the validity of these 'int' style references to other memory.
2. If something does go wrong and you read the array with a bad integer index, Rust will just 'panic' and crash the application. Unlike C, there will be no risk of memory errors or related security vulnerabilities. But on the other hand, each read being bounds-checked will make such Rust code slightly slower than is possible with C/C++. (Though I think you can use 'unsafe' blocks if you want to hyper-optimize akin to C.)
But regarding GC langauges, I generally agree; they're almost always more trouble than they're worth in any performance-sensitive context. You often end up using approaches like this to optimize around the GC (e.g. int indexes into pre-allocated arrays) which ultimately means you're coding C-style in a GC language anyway, which defeats the whole point of GC's productivity-enhancing benefit -- at that point, why not just go all the way and use C or Zig or Rust etc.?
It isn't entirely impossible. Erlang's processes (actors, not actual OS processes; they are like mini-green threads / fibers) very often just throw away all their data after they are done with their work, and that data is often entirely stored entirely in the stack, without any GC needed -- even if the language is GC-ed. Also, all GC is per actor so there's no stop-the-world GC pauses.
But the scenario you outline, yep, it does seem impossible with a borrow checker.
And don't get me wrong. I am gradually learning Rust and I love it. But I can see myself being a little grumpy with the procedural style and often quirky syntax. Oh well, can't have it all, right?
But the scenario you outline, yep, it does seem impossible with a borrow checker.
And don't get me wrong. I am gradually learning Rust and I love it. But I can see myself being a little grumpy with the procedural style and often quirky syntax. Oh well, can't have it all, right?
[deleted]
While Go and Elixir might indeed be slower, Rust is designed to generate optimal code and offer zero-cost abstractions like C and C++ and I would expect it to do so in this case.
That's what I keep hearing but here on HN periodically do pop up some benchmarks that show Rust being stomped.
But oh well, most benchmarks aren't pure or non-opinionated so there's that as well.
So far I like Rust quite a lot but -- as I said in another sibling comment -- its procedural style and syntax aren't stellar, in my eyes at least.
But oh well, most benchmarks aren't pure or non-opinionated so there's that as well.
So far I like Rust quite a lot but -- as I said in another sibling comment -- its procedural style and syntax aren't stellar, in my eyes at least.
Well, generally you can, approximately, given C/C++ code, produce Rust code that should give equivalent LLVM bitcode to the one produced by clang on the C/C++ code, so it should not be possible for Rust to routinely lose benchmarks to clang with a significant margin given enough effort in benchmark construction.
Only if you use unsafe Rust, otherwise you'd have to prove that safe Rust semantics are enough to describe the same operations used to squeeze out that performance in C/C++.
It's more complicated than that. Compilers love restrictions, because that means they can optimize more aggressively. And there are features that Rust doesn't have that C++ does have that can make Rust faster; for example, because Rust doesn't have move constructors, moving is an incredibly cheap operation, whereas it can execute arbitrary code (including throwing an exception!) in C++.
There are also... I don't know how to frame this, social issues? Rust's guarantees often let you do some very aggressive things that you could write in C++, but wouldn't, because it's not maintainable. For example, if you must use reference counting, but don't need threads, Rust has a split between Rc<T> and Arc<T>. You can afford to not use the atomic version in Rust code because if you ever refactor later, you know the compiler will catch this for you, whereas you're just asking for latent, hard to track down bugs if you tried this in C++.
This goes both ways; there is some stuff that C++ is better at than Rust too. It's more complex than "unsafe Rust lets you do anything C++ can do directly and so if it's 100% unsafe in Rust it will be the same speed and if it's safe Rust it will be slower."
There are also... I don't know how to frame this, social issues? Rust's guarantees often let you do some very aggressive things that you could write in C++, but wouldn't, because it's not maintainable. For example, if you must use reference counting, but don't need threads, Rust has a split between Rc<T> and Arc<T>. You can afford to not use the atomic version in Rust code because if you ever refactor later, you know the compiler will catch this for you, whereas you're just asking for latent, hard to track down bugs if you tried this in C++.
This goes both ways; there is some stuff that C++ is better at than Rust too. It's more complex than "unsafe Rust lets you do anything C++ can do directly and so if it's 100% unsafe in Rust it will be the same speed and if it's safe Rust it will be slower."
That's what I am falling in love with as I am learning Rust: both sensical and compiler-helping restrictions.
Rust can use less ways to do the same thing, I keep hearing from people, and I am hopeful it can be tackled in the future with clever deprecations or simply de facto libraries / ways of doing things.
Thanks for your hard work, it's much appreciated.
Rust can use less ways to do the same thing, I keep hearing from people, and I am hopeful it can be tackled in the future with clever deprecations or simply de facto libraries / ways of doing things.
Thanks for your hard work, it's much appreciated.
since simd intrinsics got introduced there shouldn't be any reason for rust to get 'stomped'
Sorry, I don't mean to be demeaning. I am aware Rust's creators make huge strides and I am absolutely on their side. I already prefer Rust over a lot of languages, C and C++ included.
Just having some healthy skepticism, apologies if it reads like anything else.
Just having some healthy skepticism, apologies if it reads like anything else.
I’d agree with you, given the chance to do so in a codebase that has vanilla dependencies. If you work in robotics or do a lot of parallel computing for example, the libraries you use are almost always based on C++, so there’s not an easy way to code against them. Even if there is such a solution, a gap between the languages can lead to non-trivial behaviors and that in itself is more costly to fix than to write in the first class language C++ anyway.
Rust is slower than C++?
I am undecided on it since I am not in the C++ ecosystem for a long time now, and I am just a beginner in Rust. I get the message there are ways to write suboptimal (in terms of performance) code in Rust as well but I also keep hearing there are much less ways to shoot oneself in the foot, which works very well for me.
Truthfully, even if Rust can be 2x-5x slower than C++ somewhere, I'll still choose it as my static-and-strongly-typed / compiled-to-machine-code language regardless. Least WTFs per minute is my most important metric.
But I am open to see any and all benchmarks which show both strengths and weaknesses of Rust. So far I sadly haven't seen many of them (but I've been mightily impressed by `actix` in the TechEmpower benchmarks!).
Truthfully, even if Rust can be 2x-5x slower than C++ somewhere, I'll still choose it as my static-and-strongly-typed / compiled-to-machine-code language regardless. Least WTFs per minute is my most important metric.
But I am open to see any and all benchmarks which show both strengths and weaknesses of Rust. So far I sadly haven't seen many of them (but I've been mightily impressed by `actix` in the TechEmpower benchmarks!).
Yes, at best 20% slower but can be far more slower if you hit worst case scenario.
Where did you pull those numbers?
Looking at https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
While the worst case is 40% slower (where even C perfored abysmally), most of the time the Rust is within a 10% margin of error, and in quite a few cases it's faster.
Looking at https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
While the worst case is 40% slower (where even C perfored abysmally), most of the time the Rust is within a 10% margin of error, and in quite a few cases it's faster.
Strange benchmark, comparing llvm(rust) to gcc(c++ code)
As nozzz said, this is using clang.
So I should correct my statement, c++ is in average 30% faster as gcc is in average 20% faster than llvm.
"at best 20%" slower is a clearly non factual statement.
Probably the one scenario where it would be far more slower than that would be a heavily AVX-512 intrinsics program because Rust doesn't have those yet (I think), but then that isn't really either language anymore. its intrinsics.
Probably the one scenario where it would be far more slower than that would be a heavily AVX-512 intrinsics program because Rust doesn't have those yet (I think), but then that isn't really either language anymore. its intrinsics.
Re-read as 30% slower in average and then it's a factual statement.
Why is Mozilla Firefox written in Rust, then?
The last time I looked at browser benchmarks was in 2016 when Google released benchmarks to brag about its performance over Firefox and Edge. It appears Webkit has released JetStream2 to benchmark JavaScript, WebAssembly and web workers, among other things. But I can't find data showing Mozilla Firefox is ~20% slower than Chrome/Edge today.
The last time I looked at browser benchmarks was in 2016 when Google released benchmarks to brag about its performance over Firefox and Edge. It appears Webkit has released JetStream2 to benchmark JavaScript, WebAssembly and web workers, among other things. But I can't find data showing Mozilla Firefox is ~20% slower than Chrome/Edge today.
Using Firefox to evaluate Rust performance is probably not a good idea since only parts of it are written in Rust today.
But it's the first time I hear someone say that Rust is supposedly fundamentally slower than C++. I don't believe this to be a statement that can be substantiated, seeing that some of the fastest programs out there (ripgrep and hyper to name just two) are written in Rust.
But it's the first time I hear someone say that Rust is supposedly fundamentally slower than C++. I don't believe this to be a statement that can be substantiated, seeing that some of the fastest programs out there (ripgrep and hyper to name just two) are written in Rust.
Facts:
On average rust is 10% slower on the banchmarkgame:
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
But they use llvm instead of gcc for c++, and gcc is in average 20% faster than clang. Therefore, rust is 30% slower, at least on heavily optimised programs.
And hyper is not the fastest http server, but the actix one is one of the fastest yes.
But they use llvm instead of gcc for c++, and gcc is in average 20% faster than clang. Therefore, rust is 30% slower, at least on heavily optimised programs.
And hyper is not the fastest http server, but the actix one is one of the fastest yes.
> Facts: On average rust is 10% slower on the banchmarkgame
That does not show it's 10% slower on average; it shows that C++ is 0.9917938291x as fast as Rust. They both have some where they win by 30%, most are closer, Rust is just faster on average.
> But they use llvm instead of gcc for c++,
C++ g++ g++ (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008
?
> gcc is in average 20% faster than clang.
Extremely citation needed.
That does not show it's 10% slower on average; it shows that C++ is 0.9917938291x as fast as Rust. They both have some where they win by 30%, most are closer, Rust is just faster on average.
> But they use llvm instead of gcc for c++,
C++ g++ g++ (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008
?
> gcc is in average 20% faster than clang.
Extremely citation needed.
> faster on average
What kind-of time measurements? What kind-of average?
This chart shows some time measurements:
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
:and the geometric mean is faster for the C++ programs, but the median seems faster for the rust programs.
Statistical artifacts?
What kind-of time measurements? What kind-of average?
This chart shows some time measurements:
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
:and the geometric mean is faster for the C++ programs, but the median seems faster for the rust programs.
Statistical artifacts?
> What kind-of time measurements? What kind-of average?
I interpreted the parent to mean a simple average, so I did exactly that: calculated the percentage for each, and averaged them. In the colloquial term, "mean".
I interpreted the parent to mean a simple average, so I did exactly that: calculated the percentage for each, and averaged them. In the colloquial term, "mean".
typo ? "the colloquial term" is "average", the specific term is "mean".
Geometric mean is the implicit default measurement for benchmarking
> On average rust is 10% slower on the banchmarkgame
If that claim was true …
> gcc is in average 20% faster than clang
and if that claim was true …
then we might guess the rust programs average could be 10% faster than the C++ programs average, if both used the same compiler backend
— because those rust programs were compiled with llvm but those c++ programs were compiled with gcc.
If that claim was true …
> gcc is in average 20% faster than clang
and if that claim was true …
then we might guess the rust programs average could be 10% faster than the C++ programs average, if both used the same compiler backend
— because those rust programs were compiled with llvm but those c++ programs were compiled with gcc.
IMO the jury is still out. I am pretty sure that C++ veterans that can cleverly utilise any and all performance tricks (minus resorting to an assembly language) and can thus beat any other language.
But I -- like yourself -- am much more interested in the applicability of a language in everyday tools, plus in getting the most performance with the least amount of dark magic involved.
BTW, `ripgrep` definitely made me work better and faster.
But I -- like yourself -- am much more interested in the applicability of a language in everyday tools, plus in getting the most performance with the least amount of dark magic involved.
BTW, `ripgrep` definitely made me work better and faster.
Firefox is far more than 20% slower than Chrome today but is not strictly a direct cause of using rust.
https://www.phoronix.com/scan.php?page=article&item=firefox-...
Isn't the correct way to write this loop actually:
See https://godbolt.org/z/xHYKft
Disclaimer: I haven't actually run this function. That would be too much like work.
void toupper_cmov(char* buf, size_t size) {
for (size_t i = 0; i < size; i++) {
char c = buf[i];
buf[i] = c - 32 * (c >= 'a' && c <= 'z');
}
}
See https://godbolt.org/z/xHYKft
Disclaimer: I haven't actually run this function. That would be too much like work.
Why not use -march=native? When I look at it, I hope the compiler knows what it's doing wrt cache and whatnot.
https://godbolt.org/z/zRQRkr
https://godbolt.org/z/zRQRkr
> Why not use -march=native?
The conditional jump remains even with -march=native.
https://godbolt.org/z/2p7uAB
The conditional jump remains even with -march=native.
https://godbolt.org/z/2p7uAB
Yep just noticed it.. wanted to delete the comment but was to slow apparently
You also get the slow version if you include <cctype> instead of <ctype.h>.
https://godbolt.org/z/cbeGc4
https://godbolt.org/z/cbeGc4
Yes, I saw that too and mention it:
> As a corollary, if you include <cctype> rather than <ctype.h> (the C++ version of the C header which puts functions in the std:: namespace) you also get the slow behavior because <cctype> ultimately includes <bits/os_defines.h>.
> As a corollary, if you include <cctype> rather than <ctype.h> (the C++ version of the C header which puts functions in the std:: namespace) you also get the slow behavior because <cctype> ultimately includes <bits/os_defines.h>.
Oh, so you do!
Please give us more than one pixel of color in the legend of your graph.
This is both technically in-depth while so hilarious. Love this post!
Can the std::transform version be written without a lambda?
std::transform(buf, buf + size, buf, toupper);Yes, it can!
In fact I had that at some point and clobbered it with a git reset (the perils of developing on two machines and once and using git a sync mechanism).
Fixed [1] and credited.
---
[1] https://github.com/travisdowns/travisdowns.github.io/commit/...
In fact I had that at some point and clobbered it with a git reset (the perils of developing on two machines and once and using git a sync mechanism).
Fixed [1] and credited.
---
[1] https://github.com/travisdowns/travisdowns.github.io/commit/...
This could actually change performance though as the lambda version gives you a unique templace instantiation of std::transform while the non-lambda version requires std::transform to be inlined into the caller (or cloned to an argument-specific version) before the compiler can make any optimizations based on its knowledge of toupper.
Good point. I would usually expect std::transform to be inlined, since it is quite simple - but it may not be and it's a very good point in general regarding the optimization probability for function pointers versus lambdas.
Ultimately, I ended up reverting the change, so using a lambda based on the advice the taking the address of a standard-library function is verboten, since they may introduce overloads at any time [1].
---
[1] https://twitter.com/foonathan/status/1197051249822195712
Ultimately, I ended up reverting the change, so using a lambda based on the advice the taking the address of a standard-library function is verboten, since they may introduce overloads at any time [1].
---
[1] https://twitter.com/foonathan/status/1197051249822195712
Yes and no. Technically that results in UB with many inputs with platforms that use a signed char type.
Messy (esp. standard library) header files like in this case is why C/C++ isn't really fun today, for me anyway.
> I was including stdlib.h just to get the definition for size_t.
size_t is also defined in stddef.h, and including that likely won't cause the same issues.
size_t is also defined in stddef.h, and including that likely won't cause the same issues.
A fascinating read. Is there a sane way to fix this in a future version of glibc?
I mean if you wanted the inline body to be visible to C++ you just have to slightly reorganize the ctype.h header so those definitions are available to C++ code (i.e., visible when __cplusplus is defined), because right now the bodies are getting caught the same #ifdef net that macro definitions are (but the body is not a macro).
That said, any exposure of implementation details, including inline bodies like this, is risky and constraining for future changes, since you bake in part of the implementation and also commit to supporting that implementation as far as you backwards compatibility strategy requires.
So it is entirely possible that the non-inlining is intentional, but then the part about include ordering mattering doesn't make sense, and if you are willing to commit for C programs, you are probably willing to commit in C++ too. You'd have to really spelunk the history of this files or ask the people responsible to be sure.
It seems more likely that that conditional compilation block is intended to avoid implementing standard library functions as macros, which is probably barred by the C++ standard (e.g., so you can use the name `std::tolower`) - and the inline bodies just got caught in the crossfire.
That said, any exposure of implementation details, including inline bodies like this, is risky and constraining for future changes, since you bake in part of the implementation and also commit to supporting that implementation as far as you backwards compatibility strategy requires.
So it is entirely possible that the non-inlining is intentional, but then the part about include ordering mattering doesn't make sense, and if you are willing to commit for C programs, you are probably willing to commit in C++ too. You'd have to really spelunk the history of this files or ask the people responsible to be sure.
It seems more likely that that conditional compilation block is intended to avoid implementing standard library functions as macros, which is probably barred by the C++ standard (e.g., so you can use the name `std::tolower`) - and the inline bodies just got caught in the crossfire.
Yeesh, I can't read assembly. Is there somewhere I learn what all movsxd, lea, cmp, ja, and friends do in plain english?
Follow the links in the article to the godbolt compiler explorer. The disassembly there has a short description of each instruction when you hover over it, and links to the full reference.
This online reference is helpful: https://www.felixcloutier.com/x86/index.html
For something more formal, the official Intel programming manuals describe what each instruction does in painstaking detail.
For something more formal, the official Intel programming manuals describe what each instruction does in painstaking detail.
Focusing on x86-64 (don't waste your time starting with the 32-bit variant when learning, let alone x87 FPU instructions, there's plenty of up-to-date materials for contemporary architectures nowadays):
- Chapter 10, Assembly Language (https://www3.nd.edu/~dthain/compilerbook/compilerbook.pdf#ch...) of http://compilerbook.org/
- intro_x86-64: Introduction to x86_64 assembly - https://gitlab.com/mcmfb/intro_x86-64
- Introduction to 64 Bit Assembly Language Programming for Linux and OS X - Ray Seyfarth - http://rayseyfarth.com/asm/
- Introduction to Computer Organization with x86-64 Assembly Language & GNU/Linux - Robert G. Plantz - http://bob.cs.sonoma.edu/IntroCompOrg-x64/book.html
- Modern X86 Assembly Language Programming - 2018; Daniel Kusswurm - Covers x86 64-bit, AVX, AVX2, and AVX-512 - https://github.com/Apress/modern-x86-assembly-language-progr...
- Understanding Assembly Language - a.k.a. Reverse Engineering for Beginners; https://yurichev.com/blog/UAL/ - https://beginners.re/
- x86-64 Assembly Language Programming with Ubuntu - Ed Jorgensen - http://www.egr.unlv.edu/~ed/x86.html
- Assembly Programming and Computer Architecture for Software Engineers (APCASE) - https://github.com/brianrhall/Assembly - Videos: https://www.youtube.com/channel/UCr0svQEez3UQvlj6-5EYS6w
Or, if you prefer talks:
- Just enough Assembly for Compiler Explorer - Anders Schau Knatten - NDC TechTown 2019 - https://www.youtube.com/watch?v=soeFwz0cOqU
- Modern x64 Assembly - https://www.youtube.com/playlist?list=PLKK11Ligqitg9MOX3-0tF...
- Bluff your way in x64 assembler - ACCU 2017; Roger Orr - https://www.youtube.com/watch?v=RI7VL-g6J7g
- Enough x86 Assembly to Be Dangerous - CppCon 2017; Charles Bailey - https://www.youtube.com/watch?v=IfUPkUAEwrk
More:
- Arm / AArch64: https://github.com/MattPD/cpplinks/blob/master/assembly.arm....
- RISC-V: https://github.com/MattPD/cpplinks/blob/master/assembly.risc...
- x86: https://github.com/MattPD/cpplinks/blob/master/assembly.x86....
- Chapter 10, Assembly Language (https://www3.nd.edu/~dthain/compilerbook/compilerbook.pdf#ch...) of http://compilerbook.org/
- intro_x86-64: Introduction to x86_64 assembly - https://gitlab.com/mcmfb/intro_x86-64
- Introduction to 64 Bit Assembly Language Programming for Linux and OS X - Ray Seyfarth - http://rayseyfarth.com/asm/
- Introduction to Computer Organization with x86-64 Assembly Language & GNU/Linux - Robert G. Plantz - http://bob.cs.sonoma.edu/IntroCompOrg-x64/book.html
- Modern X86 Assembly Language Programming - 2018; Daniel Kusswurm - Covers x86 64-bit, AVX, AVX2, and AVX-512 - https://github.com/Apress/modern-x86-assembly-language-progr...
- Understanding Assembly Language - a.k.a. Reverse Engineering for Beginners; https://yurichev.com/blog/UAL/ - https://beginners.re/
- x86-64 Assembly Language Programming with Ubuntu - Ed Jorgensen - http://www.egr.unlv.edu/~ed/x86.html
- Assembly Programming and Computer Architecture for Software Engineers (APCASE) - https://github.com/brianrhall/Assembly - Videos: https://www.youtube.com/channel/UCr0svQEez3UQvlj6-5EYS6w
Or, if you prefer talks:
- Just enough Assembly for Compiler Explorer - Anders Schau Knatten - NDC TechTown 2019 - https://www.youtube.com/watch?v=soeFwz0cOqU
- Modern x64 Assembly - https://www.youtube.com/playlist?list=PLKK11Ligqitg9MOX3-0tF...
- Bluff your way in x64 assembler - ACCU 2017; Roger Orr - https://www.youtube.com/watch?v=RI7VL-g6J7g
- Enough x86 Assembly to Be Dangerous - CppCon 2017; Charles Bailey - https://www.youtube.com/watch?v=IfUPkUAEwrk
More:
- Arm / AArch64: https://github.com/MattPD/cpplinks/blob/master/assembly.arm....
- RISC-V: https://github.com/MattPD/cpplinks/blob/master/assembly.risc...
- x86: https://github.com/MattPD/cpplinks/blob/master/assembly.x86....
Order of headers should not matter in any sane language. This just proves that C++ is not a sane language. Yet Bjarne Stroustrup keeps getting engineering awards! If I were king of programming languages, I would forbid any new lines of C++ to be brought into this world, and install mandatory programmes for rewriting legacy C++ code in sane languages.
But the root cause here lies with GCC or rather glibc and not the language? C++ is but a tool, just like every other language.
It likes with C++ not having a proper module system enforcing properties like module import order being irrelevant, and the glibc developers apparently failing to produce modules having such properties even if the language doesn't enforce it.
cctype is such garbage. not real functions, takes/returns int, not contexpr, and now this trash. bleh
[deleted]
Link time inlining would solve this.
In a sense, yes - specifically if libc could be link-time inlined.
I don't think many people are proposing that for libc though because it would shut down the possibility to patch libc for anything that has been link-timed inline, and more generally expose the internal implementation of libc in a way that will make evolving it very difficult.
That is, link-time inlining will see a lot more use within an application before it will ever apply to system libraries.
I don't think many people are proposing that for libc though because it would shut down the possibility to patch libc for anything that has been link-timed inline, and more generally expose the internal implementation of libc in a way that will make evolving it very difficult.
That is, link-time inlining will see a lot more use within an application before it will ever apply to system libraries.
Applying LTO to libc would be no worse than statistically linking it. I don't like static linking, but on Linux, we're stuck with supporting it.
Isn't glibc almost always dynamically linked on linux? I thought half the point of musl was that it supported static linking.
Almost everyone does dynamically link Glibc. You should dynamically link glibc too. But if you really want, you can statically link glibc.
As for static linking being a competitive advantage of musl: I don't know where you got that idea. Musl is less featureful than glibc, not more. In particular, I'm not going to use it until it supports dlclose like every other full-featured computing platform.
$ dpkg -S '*/libc.a'
libc6-dev:amd64: /usr/lib/x86_64-linux-gnu/libc.a
As for static linking being a competitive advantage of musl: I don't know where you got that idea. Musl is less featureful than glibc, not more. In particular, I'm not going to use it until it supports dlclose like every other full-featured computing platform.
> As for static linking being a competitive advantage of musl: I don't know where you got that idea.
That's how I've been told to build fully statically linked rust binaries, I can't say I've dug into why glibc isn't used for those.
That's how I've been told to build fully statically linked rust binaries, I can't say I've dug into why glibc isn't used for those.
The primary reason is that a statically linked glibc can't use NSS (Name Service Switch) modules from a different glibc version, so if you statically link glibc you can't reliably resolve names with getaddrinfo. You can do DNS with other libraries, and many applications do anyway because they want asynchronous DNS and NSS uses static DNS, but you can't really do anything other than DNS or static hosts files.
musl, on the other hand, never supports NSS, whether static or dynamic, and exclusively supports DNS. You can't resolve things like mDNS names, or even resolve localhost if on a system that doesn't hardcode it in /etc/hosts.
So either way, static linking and name resolution don't mix well, and musl and non-DNS name resolution don't mix well.
musl, on the other hand, never supports NSS, whether static or dynamic, and exclusively supports DNS. You can't resolve things like mDNS names, or even resolve localhost if on a system that doesn't hardcode it in /etc/hosts.
So either way, static linking and name resolution don't mix well, and musl and non-DNS name resolution don't mix well.
The usual reason is that if you want a binary that can be used "everywhere", you can't just build on your machine without more thought if you use the glibc because chances are you're going to be using symbols that aren't available in older versions of the glibc, making your binary incompatible with many (older) distros. musl is an easy way out of that. Another is to build against an old glibc.
The right way to solve this problem would be to teach the linker to link against the set of glibc symbols that existed as of a particular version number or date. Then you'd just pass a flag (along with a preprocessor macro, so the headers would know to define the right version of the linker symbols) specifying this version target when you built your program. For want of this relatively simple toolchain feature, we have people statically linking entire C libraries. It's a shame.
You can do a moral version of the same thing using linker scripts, but nobody bothers with linker scripts.
You can do a moral version of the same thing using linker scripts, but nobody bothers with linker scripts.
The macOS SDK does this better. It is indeed a shame that the glibc doesn't care.
True, static linking would be troublesome but what about at load time? Dynamic link time optimization could be very useful.
It's 2019 and people are still using languages where order of imports matters...
I'll own up to misleading-and-possibly-clickbait title, I just gave up trying to think of anything better without revealing the conclusion.