Catastrophic Cancellation (2020)(twitter.com)
twitter.com
Catastrophic Cancellation (2020)
https://twitter.com/raymondh/status/1275924648132149249
31 comments
If you add a really large number, then your relative error will decrease!
If you add a function which increments a dummy value 4000 times, your test coverage will increase!
Test coverage is a necessary but not sufficient condition for catching bugs in a line of code.
Am I the only one who thinks the concept of relative error is not meaningful in this context?
It gives a disproportionate meaning to 0 without real physical consideration, eg:
- 0.1⁰C ± 0.1 (wow 100% relative error) - 273.25K ± 0.1 (meh 0.04% relative error)
It gives a disproportionate meaning to 0 without real physical consideration, eg:
- 0.1⁰C ± 0.1 (wow 100% relative error) - 273.25K ± 0.1 (meh 0.04% relative error)
The Celsius temperature scale is an interval scale [0] which means that it is possible to calculate differences but not ratios. The Kelvin temperature scale is a ratio scale [0] (it has an "absolute zero") that allows to do that.
Beside that if there are uncertainties involved, one should do proper propagation of uncertainty anyways. [1]
[0] https://en.wikipedia.org/wiki/Level_of_measurement
[1] https://en.wikipedia.org/wiki/Propagation_of_uncertainty
Beside that if there are uncertainties involved, one should do proper propagation of uncertainty anyways. [1]
[0] https://en.wikipedia.org/wiki/Level_of_measurement
[1] https://en.wikipedia.org/wiki/Propagation_of_uncertainty
This is talking about the error in the difference between two values with the same units though. For temperature, it wouldn’t matter if you’re using C, K, or F for your starting values, the % error of the difference would be the same (I think).
Yes.
What would make a difference is to change to https://en.wikipedia.org/wiki/Thermodynamic_beta
(Essentially the same difference as miles per gallons vs litres per 100 km.)
What would make a difference is to change to https://en.wikipedia.org/wiki/Thermodynamic_beta
(Essentially the same difference as miles per gallons vs litres per 100 km.)
Yes, because there is an arbitrary choice of origin which renders the relative error dependant on units. If you're measuring a length, for instance, or an interval of time, the relative error is independent of which units you choose. If you're measuring e.g. a distance to some point, then again you have an arbitrary choice of origin.
I am sometimes joking with friends by suggesting: Let's meet at 12 o'clock ± 5%.
Does Python's Decimal type handle this correctly?
Yes it does and should be used when precision is important.
You could also use Python's fractions.
choeger(1)
I don't understand how rearranging operations, like suggested later in the thread, would avoid the large relative error in "how much older is the earth than the oceans" given that our estimates for the age of the earth and oceans are only so precise?
This is a strange tweet. It assumes people are familiar with classical cancellation error, but not familiar with error analysis. Which in my experience... people either understand both, or are ignorant of both concepts.
The general point is that "cancellation error" happens more than just in floating-point operations, but also in "classic scientific sig-fig error analysis".
---------
The tweet should either be dumbed down to discuss cancellation error in floating-point arithmetic, or elevated up and assume people know about sig-fig analysis. It sits at a weird point in the "assumed knowledge" curve.
---------
For people unfamiliar with cancellation error, try the two following statements in Python3 (which defaults to double-precision... aka 53-bits of mantissa).
9007199254740992.0 == 2^53. So it is impossible for a double-precision number to accurately represent +/- 1.0 at 2^53. (Note that +/- 2.0 will work out just fine).
Play around with 9007199254740992.0 +/- 1.0, or 2.0, and other values for about 15 minutes, and you'll probably learn everything you need to know about cancellation error from that playtime alone.
Double-precision numbers are composed of 52-explicit bits + 1 implicit bit + 1 sign bit + 11-bit exponent bits (yes, 65-bits total. The implicit bit "doesn't count", but makes 0.0 and subnormal numbers harder to deal with)
The general point is that "cancellation error" happens more than just in floating-point operations, but also in "classic scientific sig-fig error analysis".
---------
The tweet should either be dumbed down to discuss cancellation error in floating-point arithmetic, or elevated up and assume people know about sig-fig analysis. It sits at a weird point in the "assumed knowledge" curve.
---------
For people unfamiliar with cancellation error, try the two following statements in Python3 (which defaults to double-precision... aka 53-bits of mantissa).
poor_ordering = 9007199254740992.0 + 1.0 + 1.0 + 1.0 + 1.0 - 9007199254740992.0
good_ordering = 9007199254740992.0 - 9007199254740992.0 + 1.0 + 1.0 + 1.0 + 1.0
9007199254740992.0 == 2^53. So it is impossible for a double-precision number to accurately represent +/- 1.0 at 2^53. (Note that +/- 2.0 will work out just fine).
Play around with 9007199254740992.0 +/- 1.0, or 2.0, and other values for about 15 minutes, and you'll probably learn everything you need to know about cancellation error from that playtime alone.
Double-precision numbers are composed of 52-explicit bits + 1 implicit bit + 1 sign bit + 11-bit exponent bits (yes, 65-bits total. The implicit bit "doesn't count", but makes 0.0 and subnormal numbers harder to deal with)
It wouldn’t and cannot, because there’s only one operation in that calculation, so there’s nothing to rearrange.
That example only shows that relative errors can explode even when doing simple calculations.
Another thing is that, in computers, calculations often are imprecise.
Because of catastrophic cancellation and similar issues, that means that the computer result of a calculation can be quite different from the mathematical result.
To make matters worse, in real life, we often don’t know the exact values of things we measure, so even if our calculations are mathematically perfect, the outcome of a calculation by computer can be quite different from the real result.
So, if you do a computer calculation, say to compute how strong a bridge has to be, you really, really need to know now close the computed value, at worst, is to the mathematically exact result.
That’s what numerical analysis is about. For a given calculation, it might say such things as
“if the input is between 100 and 200, to get a result with n decimal digits of precision, you’ll have to compute all intermediate results with 4 × n digits.”
or
”but if you rearrange the computation like this, you only need to use 2 × n digits for n digits of precision in your result”
That example only shows that relative errors can explode even when doing simple calculations.
Another thing is that, in computers, calculations often are imprecise.
Because of catastrophic cancellation and similar issues, that means that the computer result of a calculation can be quite different from the mathematical result.
To make matters worse, in real life, we often don’t know the exact values of things we measure, so even if our calculations are mathematically perfect, the outcome of a calculation by computer can be quite different from the real result.
So, if you do a computer calculation, say to compute how strong a bridge has to be, you really, really need to know now close the computed value, at worst, is to the mathematically exact result.
That’s what numerical analysis is about. For a given calculation, it might say such things as
“if the input is between 100 and 200, to get a result with n decimal digits of precision, you’ll have to compute all intermediate results with 4 × n digits.”
or
”but if you rearrange the computation like this, you only need to use 2 × n digits for n digits of precision in your result”
I think it's a bad example, but illustrates an important point he doesn't make explicit, that sometimes the variable you need to estimate is the DX itself, not just X1 and X2 to produce X2 - X1 = DX. With a sufficiently high amount of variance in your approximations of X1 and X2 their difference will tell you little to nothing about DX.
maybe he wants to implictly display correlations. if you substract directly you make the implicit assumptions the two even dates are uncorrelated.
Doesn't help for the "older question." That's just used as an intuitive example for how imprecision can arise from subtraction.
Rearranging helps when you can only store intermediate results with finite precision, but you can compute them to arbitrary precision.
Rearranging helps when you can only store intermediate results with finite precision, but you can compute them to arbitrary precision.
I see. I might quibble that here the imprecision isn't just from the subtraction, but because we originally just have very rough estimates for the dates (though 'mam2 brings up a very interesting point about correlation if estimates are based on each other and using them like that isn't the ideal way of answering the question).
It threw me off enough that I didn't get the original point of it re: floating point numbers until your comment - in our floating point formats, the imprecision is often an accidental effect of the limits of the format, vs a true unknown, and then magnifying the relative size of that arbitrary limitation is a problem.
It threw me off enough that I didn't get the original point of it re: floating point numbers until your comment - in our floating point formats, the imprecision is often an accidental effect of the limits of the format, vs a true unknown, and then magnifying the relative size of that arbitrary limitation is a problem.
Wether it's imprecise measurements or imprecise calculations, there are still things you can do if you're aware of the problem. When measurement is imprecise, it might be possible to improve accuracy by using a different measurement.
For geology, it's often easier to put things in the right order due to rock layers rather than to figure how long ago they were from present.
In this case, obviously, the Earth is older than the oceans, even if the estimated ages were even rougher and the error bars implied they could be in the opposite order.
For history, you may be able to figure out the relative order of events without knowing what year they were on our calendar.
For geology, it's often easier to put things in the right order due to rock layers rather than to figure how long ago they were from present.
In this case, obviously, the Earth is older than the oceans, even if the estimated ages were even rougher and the error bars implied they could be in the opposite order.
For history, you may be able to figure out the relative order of events without knowing what year they were on our calendar.
Threaded version: https://threadreaderapp.com/thread/1275924648132149249.html
These tweets are already threaded on twitter, why is this app even needed?
It tends to pollute the replies more often than not, with more people invoking it than actually replying.
It tends to pollute the replies more often than not, with more people invoking it than actually replying.
threadreaderapp.com opens quickly, works on a browser without Javascript, doesn't nag about installing an app. twitter.com on a browser fails those criteria (especially on mobile). So, win for threadreaderapp.com.
Twitter doesn't thread properly. It does weird things based on likes.
Depends on context. Coming into some tweet midway in a very long thread (scroll paginated/page range limited) makes going to the start very frustrating. That’s bad UX on Twitter’s part, but rolling up the thread is one easy workaround.
Edit: it’s also great for preserving content which may be deleted.
Edit: it’s also great for preserving content which may be deleted.