Data scientists need to learn about significant digits(lemire.me)
lemire.me
Data scientists need to learn about significant digits
https://lemire.me/blog/2019/01/29/data-scientists-need-to-learn-about-significant-digits/
69 comments
I'm pretty sure most of us DS know about significant digits and are usually calculating the maximum to enable future flexibility. For a single output, I can understand how you'd be upset we don't round. But for the 20+ column tables we usually build, I've found most will calculate the maximum scale allowed by the database and call it a day. The best of us certainly find the right formatting for the presentation.
The problem is that it's not just "presentation": being sloppy about significant digits (or precision generally) early in the computation leads to bad statistical reasoning much later in the problem.
If your variable is x +/- 0.05, then 1/(x +/- 0.05) != 1/x +/- 0.05. If you're not careful about this when doing computations, you'll end up with answers that aren't actually meaningful. The computational implementation of these equations is only concerned with machine epsilon, but each one of those 20 database columns has a real-world +/- delta which isn't being correctly considered.
If your variable is x +/- 0.05, then 1/(x +/- 0.05) != 1/x +/- 0.05. If you're not careful about this when doing computations, you'll end up with answers that aren't actually meaningful. The computational implementation of these equations is only concerned with machine epsilon, but each one of those 20 database columns has a real-world +/- delta which isn't being correctly considered.
But the error propagation is not transmitted by the significant number. x, y being written with the correct number of significant number will not lead to f(x, y) being written to the correct number of significant number.
Usually, the best approach is to propagate the uncertainty, for example by saving the uncertainty as another variable in the database and using it directly when the number is used. If you do that, there is no practical needs to lose time to format the numbers. Using significant numbers seems a "cheap trick" that risk to mislead you more often than help.
Usually, the best approach is to propagate the uncertainty, for example by saving the uncertainty as another variable in the database and using it directly when the number is used. If you do that, there is no practical needs to lose time to format the numbers. Using significant numbers seems a "cheap trick" that risk to mislead you more often than help.
> But the error propagation is not transmitted by the significant number. x, y being written with the correct number of significant number will not lead to f(x, y) being written to the correct number of significant number.
Significant figures are not a convention for making your deliverable pretty. They have semantic meaning. Don't think about dumb rules from high school chemistry, think about the actual problem. There are two entwined sources of uncertainty I am referring to:
1) measurement uncertainty, due to a lack of precision in the instrument (or the quantity itself, e.g. many financial computations are not meaningful if they involve fractional cents)
2) computational uncertainty, which is exclusively due to algebraic propagation of measurement uncertainty
Far too many data scientists don't care about the first category of uncertainty because they don't care about where the data came from. And they don't even realize the second category is a problem.
Let's look at a specific example. Somebody tells us that they measured the side of a square as 1.0m. Their tape measure only went down to centimeters, so the uncertainty is +/- 0.01m. What is the area of this square? Let's look at it two ways:
1) The smallest possible side length is 1.00m - 0.01m = 0.99m, so the smallest possible area is 0.98m^2. The largest possible side length is 1.01m, so the largest possible area is 1.02m^2. Thus the area is 1.00m^2 +/- 0.02m^2.
2) The side length is (1.00m +/- 0.01m). So the area is
(1.00m +/- 0.01m)(1.00m +/- 0.01m) = 1.00m^2 +/- 0.02m^2 +/- 0.0001m^2 ~ 1.00m^2 +/- 0.02m^2
So the uncertainty is not +/- 0.01, it is +/- 0.02. This can add up quite dramatically. In general if you have x +/- delta, then f(x +/- delta) is not going to be f(x) +/- delta or f(x) +/- f(delta). It needs to be handled carefully.
Significant figures are not a convention for making your deliverable pretty. They have semantic meaning. Don't think about dumb rules from high school chemistry, think about the actual problem. There are two entwined sources of uncertainty I am referring to:
1) measurement uncertainty, due to a lack of precision in the instrument (or the quantity itself, e.g. many financial computations are not meaningful if they involve fractional cents)
2) computational uncertainty, which is exclusively due to algebraic propagation of measurement uncertainty
Far too many data scientists don't care about the first category of uncertainty because they don't care about where the data came from. And they don't even realize the second category is a problem.
Let's look at a specific example. Somebody tells us that they measured the side of a square as 1.0m. Their tape measure only went down to centimeters, so the uncertainty is +/- 0.01m. What is the area of this square? Let's look at it two ways:
1) The smallest possible side length is 1.00m - 0.01m = 0.99m, so the smallest possible area is 0.98m^2. The largest possible side length is 1.01m, so the largest possible area is 1.02m^2. Thus the area is 1.00m^2 +/- 0.02m^2.
2) The side length is (1.00m +/- 0.01m). So the area is
(1.00m +/- 0.01m)(1.00m +/- 0.01m) = 1.00m^2 +/- 0.02m^2 +/- 0.0001m^2 ~ 1.00m^2 +/- 0.02m^2
So the uncertainty is not +/- 0.01, it is +/- 0.02. This can add up quite dramatically. In general if you have x +/- delta, then f(x +/- delta) is not going to be f(x) +/- delta or f(x) +/- f(delta). It needs to be handled carefully.
You're talking like sig figs is error propagation, but it isn't. That's the point. The correct solution is error propagation (with appropriate estimates of the errors of the inputs), not arbitrarily rounding numbers at each step, which is a terrible waste of time and likely to introduce its own errors compared to error propagation.
> You're talking like sig figs is error propagation, but it isn't.
No, that is the exact opposite of what I said! For starters, "uncertainty" and "error" are not the same thing here. I am saying significant figures in a measurement encapsulates an inherent measurement-specific uncertainty conveyed by significant figures, and that this uncertainty must be considered when doing calculations with that measurement. Just like the person I responded to, I don't think you've thought about why significant figures actually exist in the first place.
> The correct solution is error propagation (with appropriate estimates of the errors of the inputs), not arbitrarily rounding numbers at each step
Nowhere in my comment did I arbitrarily round anything. I thoughtfully propagated the uncertainty, which is why it was +/- 0.01m in the measurement and +/- 0.02m^2 in the calculation.
The whole point of my argument is that uncertainties in calculated quantities can be rigorously determined from the uncertainty of the inputs, and measurement inputs have uncertainty determined by the significant figures. On the other hand, ignoring significant figures in calculations means we're ignoring a potential source of uncertainty in downstream analysis. If you think significant figures is about "arbitrarily rounding something" then you are thoughtlessly applying high school chemistry rules. Please read this carefully:
If I measure something with a meterstick that is broken down into centimeters, that measurement has an inherent uncertainty of either +/- 1cm or +/- 0.5cm - which one you use is a problem-specific choice similar to p95 or p99 for statistical significance (if it was a physical meterstick I'd choose 1cm because human eyeballs aren't very good; if it was laser inference I'd choose 0.5cm).
So if I am a data scientist with a database of direct measurements from a meterstick, each one has an inherent uncertainty of +/- 0.01m that's implied by the data source even if it's not in the database. This is the entire point of representing the data as 12.03m, 1.00m, etc, instead of 1.234m. If you represented a measurement as 1.234m that would imply your meterstick could measure decimeters, but it probably can't. So 1.234m isn't merely against the rules, it's inaccurate.
If you take a measured side length of 1.00m and say the calculated area is 1.00m^2, then naively someone might think the uncertainty in the area is +/- 0.01m^2 based on thoughtlessly applying dumb high school chemistry rules. But that's not true, the uncertainty in the calculated area is in reality +/- 0.02m^2. The measurement can be presented without an explicit +/- because the significant figures acts as a "shorthand" and we don't need to do calculations to estimate the uncertainty. But the calculation must present a calculated uncertainty.
Programmers and data scientists are lazy about significant figures because they don't care where the data is coming from, to them it's all doubles in a database, and significant figures is just a matter of rounding things correctly at the end. The area-of-a-square argument proves that this is a mistake.
No, that is the exact opposite of what I said! For starters, "uncertainty" and "error" are not the same thing here. I am saying significant figures in a measurement encapsulates an inherent measurement-specific uncertainty conveyed by significant figures, and that this uncertainty must be considered when doing calculations with that measurement. Just like the person I responded to, I don't think you've thought about why significant figures actually exist in the first place.
> The correct solution is error propagation (with appropriate estimates of the errors of the inputs), not arbitrarily rounding numbers at each step
Nowhere in my comment did I arbitrarily round anything. I thoughtfully propagated the uncertainty, which is why it was +/- 0.01m in the measurement and +/- 0.02m^2 in the calculation.
The whole point of my argument is that uncertainties in calculated quantities can be rigorously determined from the uncertainty of the inputs, and measurement inputs have uncertainty determined by the significant figures. On the other hand, ignoring significant figures in calculations means we're ignoring a potential source of uncertainty in downstream analysis. If you think significant figures is about "arbitrarily rounding something" then you are thoughtlessly applying high school chemistry rules. Please read this carefully:
If I measure something with a meterstick that is broken down into centimeters, that measurement has an inherent uncertainty of either +/- 1cm or +/- 0.5cm - which one you use is a problem-specific choice similar to p95 or p99 for statistical significance (if it was a physical meterstick I'd choose 1cm because human eyeballs aren't very good; if it was laser inference I'd choose 0.5cm).
So if I am a data scientist with a database of direct measurements from a meterstick, each one has an inherent uncertainty of +/- 0.01m that's implied by the data source even if it's not in the database. This is the entire point of representing the data as 12.03m, 1.00m, etc, instead of 1.234m. If you represented a measurement as 1.234m that would imply your meterstick could measure decimeters, but it probably can't. So 1.234m isn't merely against the rules, it's inaccurate.
If you take a measured side length of 1.00m and say the calculated area is 1.00m^2, then naively someone might think the uncertainty in the area is +/- 0.01m^2 based on thoughtlessly applying dumb high school chemistry rules. But that's not true, the uncertainty in the calculated area is in reality +/- 0.02m^2. The measurement can be presented without an explicit +/- because the significant figures acts as a "shorthand" and we don't need to do calculations to estimate the uncertainty. But the calculation must present a calculated uncertainty.
Programmers and data scientists are lazy about significant figures because they don't care where the data is coming from, to them it's all doubles in a database, and significant figures is just a matter of rounding things correctly at the end. The area-of-a-square argument proves that this is a mistake.
I still don't get it.
You are explaining error propagation, but my point is that _if you are doing error propagation (as you should do if you want to do things properly), significant figures ARE just for making deliverable pretty_.
You are talking about measurement uncertainty. Measurement uncertainty is written x +- y, with y being the uncertainty.
If you don't do that and use significant digit instead, you lose the information and precision: 10.0 +- 0.1 is 10.0, 10.0 +- 0.2 is 10.0, 10.0 +- 0.3 is 10.0, ...
This is why the other person was talking about "arbitrarily rounding".
You should _never_ said "well, it's a measurement of 10.0 with a 0.2 precision, so I can write 10.0", you should _always_ write 10.0 +- 0.2 (in which case, you can also write 10 +- 0.2 or 10.000 +- 0.2, the significant digits have no impact on any future results). Writing 10.0 instead of 10.0 +- 0.2 is just a terrible practice that does not have much justification, 10 +- 0.2 is always a better way. (and my point is that the problem you have with the significant number disappear if you teach people to use a non-clumsy way)
(and, no, you should not do the distinction "it's a measurement, so it's written differently", because in practice, a lot of "measurements" are in fact already a transformation, and sometimes you cannot even know for sure yourself. For example, a temperature sensor will measure an electrical resistance (with a measurement uncertainty) and then convert it into a temperature, and according to you, it should not be written the same way, just for arbitrary reasons)
You are explaining error propagation, but my point is that _if you are doing error propagation (as you should do if you want to do things properly), significant figures ARE just for making deliverable pretty_.
You are talking about measurement uncertainty. Measurement uncertainty is written x +- y, with y being the uncertainty.
If you don't do that and use significant digit instead, you lose the information and precision: 10.0 +- 0.1 is 10.0, 10.0 +- 0.2 is 10.0, 10.0 +- 0.3 is 10.0, ...
This is why the other person was talking about "arbitrarily rounding".
You should _never_ said "well, it's a measurement of 10.0 with a 0.2 precision, so I can write 10.0", you should _always_ write 10.0 +- 0.2 (in which case, you can also write 10 +- 0.2 or 10.000 +- 0.2, the significant digits have no impact on any future results). Writing 10.0 instead of 10.0 +- 0.2 is just a terrible practice that does not have much justification, 10 +- 0.2 is always a better way. (and my point is that the problem you have with the significant number disappear if you teach people to use a non-clumsy way)
(and, no, you should not do the distinction "it's a measurement, so it's written differently", because in practice, a lot of "measurements" are in fact already a transformation, and sometimes you cannot even know for sure yourself. For example, a temperature sensor will measure an electrical resistance (with a measurement uncertainty) and then convert it into a temperature, and according to you, it should not be written the same way, just for arbitrary reasons)
I don't think you understand what a measurement is. There's a very good, very short book that explains in more detail what I am talking about, in the context of physics experiments: https://www.amazon.com/Practical-Guide-Data-Analysis/dp/0521...
I think your notions are just too basic. It's a bit like in school when the teacher says "you should write all your sentences as subject + verb + complement". It is good at school, to teach students the basics and to put boundaries of the studied regions (you don't want to have students using more complex notions by accident and having to cover everything in lesson one), but as soon as you begin to be a professional writer, you realise it is better to ignore this rule.
I know the notion of measurement that you try to explain, I've studied it when I was an undergraduate students. Since then, I have passed beyond this notion and use something better. It's not a matter of "you don't understand", it's rather a matter of "you understand too well and see the limits of this notion and that it's not useful for you anymore".
The book you share seems to confirm that: it is for undergraduates. Things get more complicated with real world practice, and the basic rules used to forge the understanding needs to be left behind. For undergraduate students, they are going to do basic lab experiment with a ruler and a chronometer, and the goal is just to practice, not to answer to a real unknown situation. In real life, no one needs to measure things as trivial as what they are measuring. When people do that, they realise that the distinction between calculated value and measured value is meaningless and not helpful at all.
Again, as I've said, you just use x +- y and you don't have any problem. What would be the problem of using x +- y that you will not have otherwise (knowing. of course, that you are educated enough to understand very complex notion and that therefore you totally understand and know things as trivial as significant digit already)?
I know the notion of measurement that you try to explain, I've studied it when I was an undergraduate students. Since then, I have passed beyond this notion and use something better. It's not a matter of "you don't understand", it's rather a matter of "you understand too well and see the limits of this notion and that it's not useful for you anymore".
The book you share seems to confirm that: it is for undergraduates. Things get more complicated with real world practice, and the basic rules used to forge the understanding needs to be left behind. For undergraduate students, they are going to do basic lab experiment with a ruler and a chronometer, and the goal is just to practice, not to answer to a real unknown situation. In real life, no one needs to measure things as trivial as what they are measuring. When people do that, they realise that the distinction between calculated value and measured value is meaningless and not helpful at all.
Again, as I've said, you just use x +- y and you don't have any problem. What would be the problem of using x +- y that you will not have otherwise (knowing. of course, that you are educated enough to understand very complex notion and that therefore you totally understand and know things as trivial as significant digit already)?
I agree with that.
It reminds me of the tomato. Some people don't know the tomato is a fruit. Some people know the tomato is a fruit and treat them like a fruit. Some people know the tomato is a fruit and still treat them like a vegetable because it is what makes sense.
In practice, I have never saw someone mistreating a badly written number in a way that had any impact. If they don't know themselves the concept of significant digit and believe the number is 1.234567890 precisely and not 1.234567889, what would be the wrong decision they will take that they would not have taken if the number was 1.234567889?
It starts to matter when you have 10% or ~100% uncertainty, in which case, writing 1.2 or 1 is still not enough to convey the meaning to someone who does not get significant numbers, because for them, 1 = 1.0000 anyway. In this case, you need to explicitly explain the limitation on decision due to the uncertainty.
In practice, splitting hairs on the significant digit convention is just missing the point: if you apply the convention to people who are not informed on the precision, they will make bad decision anyway even if technically the number has the correct significant number.
It reminds me of the tomato. Some people don't know the tomato is a fruit. Some people know the tomato is a fruit and treat them like a fruit. Some people know the tomato is a fruit and still treat them like a vegetable because it is what makes sense.
In practice, I have never saw someone mistreating a badly written number in a way that had any impact. If they don't know themselves the concept of significant digit and believe the number is 1.234567890 precisely and not 1.234567889, what would be the wrong decision they will take that they would not have taken if the number was 1.234567889?
It starts to matter when you have 10% or ~100% uncertainty, in which case, writing 1.2 or 1 is still not enough to convey the meaning to someone who does not get significant numbers, because for them, 1 = 1.0000 anyway. In this case, you need to explicitly explain the limitation on decision due to the uncertainty.
In practice, splitting hairs on the significant digit convention is just missing the point: if you apply the convention to people who are not informed on the precision, they will make bad decision anyway even if technically the number has the correct significant number.
I think a distinction should be made for measurements and predictions. If you are measuring something then you have a scale, and you probably have a sense of the margin of error. That is easy to translate into significant figures. When you make predictions, things are much harder. There is also the question of whether you should take the variance of the model into account, or its accuracy.
The other thing people get wrong is treating significant figures and decimal places as the same thing.
123.45 Kg and 7.32 Kg have the same decimal places, but the former has more significant figures, and implies greater precision.
123.45 Kg and 7.32 Kg have the same decimal places, but the former has more significant figures, and implies greater precision.
Always seemed odd to me you can have a ruler that measures millimeters and get more significant digits measuring something to 9mm vs 12mm. The measurements are equally precise but one has more significant digits. Write it in hex and now they're the same, 9mm and Cmm. How does this make any sense? If I measure 123mm and divide it by 1.23 seconds I get to keep 3 sig figs but if it was 99mm I'd be forced to drop one? At least writing everything in scientific notation sidesteps this.
Working in finance, significant digits are (of course) super important. You always want to assume a stakeholder doesn’t have a perfect understanding of the data definition, so provide less precision (even if you know the actual value to a cent) to avoid the stakeholder misusing the data given a slightly flawed understanding of some obscure edge case.
If the number stays at 32.56m for a period of time, you want them to consider it sideways movement rather than start worrying about the fact that it did technically go up by a few hundred dollars.
If the number stays at 32.56m for a period of time, you want them to consider it sideways movement rather than start worrying about the fact that it did technically go up by a few hundred dollars.
I have mixed feelings. On the one hand, yes, people working with data should be more thoughtful about how many digits they report. But I’m not a big fan of the proposed solution (report everything to a small fixed number of significant digits, like two).
Massive computer experiments are common and (often) cheap. If I’m comparing two classification models on a large dataset a difference between 97% and 96.8% accuracy might well be both statistically and practically significant.
An error I’ve seen professional statisticians make is asserting that averaging samples (or performing more complex statistical analyses like regression) justifies increasing the number of significant digits. For example, if I have a hundred observations measured with two significant digits of precision, the mean of those observations is asserted to be accurate to three significant digits, the standard deviation of the mean being 1/10th that of each observation. This is a natural conclusion if you treat the significant digits as accounting only for independent random errors, but in fact significant digits are also used to account for systematic errors in measuring equipment. A scale that measures down to 0.1g, for example, may be miscalibrated by 0.03g. This makes reporting the mean of 100 samples down to 0.01g obviously nonsensical.
In short, how many digits to report deserves more thought than either most data scientists or the author give it.
Massive computer experiments are common and (often) cheap. If I’m comparing two classification models on a large dataset a difference between 97% and 96.8% accuracy might well be both statistically and practically significant.
An error I’ve seen professional statisticians make is asserting that averaging samples (or performing more complex statistical analyses like regression) justifies increasing the number of significant digits. For example, if I have a hundred observations measured with two significant digits of precision, the mean of those observations is asserted to be accurate to three significant digits, the standard deviation of the mean being 1/10th that of each observation. This is a natural conclusion if you treat the significant digits as accounting only for independent random errors, but in fact significant digits are also used to account for systematic errors in measuring equipment. A scale that measures down to 0.1g, for example, may be miscalibrated by 0.03g. This makes reporting the mean of 100 samples down to 0.01g obviously nonsensical.
In short, how many digits to report deserves more thought than either most data scientists or the author give it.
> 97% and 96.8% accuracy might well be both statistically and practically significant.
I don't think author ever objected to this.
I don't think author ever objected to this.
> Which answer do you give? Whatever your software tells you (e.g., 87.14234%) or a number made of a small and fixed number of significant digits (e.g., 87%).
> The latter is the right answer in almost all instances.
and
> Most human beings are happy with a 1% error margin.
While he doesn't say it as a rule outright, the author repeatedly uses a "small and fixed" two significant digits. He later says this:
> You must choose the number of significant digits deliberately.
which I agree with, but is at odds with the "small and fixed" dictum with which he leads off the post.
> The latter is the right answer in almost all instances.
and
> Most human beings are happy with a 1% error margin.
While he doesn't say it as a rule outright, the author repeatedly uses a "small and fixed" two significant digits. He later says this:
> You must choose the number of significant digits deliberately.
which I agree with, but is at odds with the "small and fixed" dictum with which he leads off the post.
Fixed doesn't mean 2. If it is statstically and practically significant. It could be 3 or 4.
If you're adjusting the number of significant figures you report based on what is practically and statistically significant then it's no longer fixed.
It doesn't need to be same for all the task, just fixed for a particular task. The creators of benchmark or the community could come up with digits of significance that should be the standard, which is based on expected error range. For imagenet it could be 4, for ARC it could be 2.
Omg yes. My coworkers often specify parameters (the results of what we would have called "regression" or "fitting" but what is now called "machine learning") to like 14 decimal places when we only know them to 20%. It would have yielded a failing grade in my high school chemistry class.
They are like "Chesterton's digits" when they appear unexplained in parameter files. Do we really know this number or is it just a wild-assed guess?
They are like "Chesterton's digits" when they appear unexplained in parameter files. Do we really know this number or is it just a wild-assed guess?
> Do we really know this number or is it just a wild-assed guess?
Does it matter? You have 23 bits of mantissa to fill one way or the other, and if you don't know the lower, say, 12, one pattern of bits is as good as any other pattern.
Does it matter? You have 23 bits of mantissa to fill one way or the other, and if you don't know the lower, say, 12, one pattern of bits is as good as any other pattern.
Precision beyond what can be justified by the uncertainty of the original input is wrong, not just as-good-as.
In terms of actual numbers in the physical world, sure, but the usual kind of floating point numbers don't have a concept of precision, and you're getting all 23 bits whether you want them or not.
Humans know the difference between "1.0" and "1.0000" but they're the same to the computer.
Humans know the difference between "1.0" and "1.0000" but they're the same to the computer.
rounding, especially in decimal, isn't actually any better. If you want to specify precision, specify it explicitly.
The reason you get rid of those extra digits is because it will always be tempting to find meaning in them.
Great answer!
That would be fine if the number were accompanied by a confidence interval.
puts on grumpy old man hat
Yeah, back in my day you had to learn about significant digits because we still used slide rules and the calculators didn't have as many functions and we liked it!
removes grumpy old man hat
That's not the reason sigfigs are important.
Oh, I'm aware. On the way to getting a bachelor's in Chemistry I was drilled on the importance of sigfigs as a way of preventing data contamination by artificial precision (e.g. trying to weigh 16.0000000000 lbs with a bathroom scale). It just seemed funnier this way...
Do people really report 5 decimal places?
In my experience reporting models in industry, even 1 or 2 decimal places are irrelevant to the overall decision to be made, at least if the number is around "87%" as stated in the example.
Of course it's context dependent. If you're evaluating incremental improvements perhaps that warrants more careful precision.
After all, by definition each decimal place is 10x less important.
In my experience reporting models in industry, even 1 or 2 decimal places are irrelevant to the overall decision to be made, at least if the number is around "87%" as stated in the example.
Of course it's context dependent. If you're evaluating incremental improvements perhaps that warrants more careful precision.
After all, by definition each decimal place is 10x less important.
> After all, by definition each decimal place is 10x less important.
I’m not sure about that, since it depends on the error margin tolerance one has.
Regardless, my biggest gripe with most reports of quantitative information I stumble across is that, lacking mention of significant digits accuracy, it’s very easy (and I’ve seen this happen a lot) for people to attribute more accuracy than what is there to a value.
One example: AWS cloudwatch reports external replication delay in MySQL Aurora in milliseconds, which is impossible, since the value reported by MySQL is in seconds. Yet many times I’ve been involved in discussions with people who say “our system can tolerate at most 500ms” of replication delay for reads, because they see the value reported in that unit.
I’m not sure about that, since it depends on the error margin tolerance one has.
Regardless, my biggest gripe with most reports of quantitative information I stumble across is that, lacking mention of significant digits accuracy, it’s very easy (and I’ve seen this happen a lot) for people to attribute more accuracy than what is there to a value.
One example: AWS cloudwatch reports external replication delay in MySQL Aurora in milliseconds, which is impossible, since the value reported by MySQL is in seconds. Yet many times I’ve been involved in discussions with people who say “our system can tolerate at most 500ms” of replication delay for reads, because they see the value reported in that unit.
I have a 9 1/2 counts multimeter, and when I calibrate DACs I need all of those. The last two digits are a bit noisy but hey.
There is no rule of thumb, except that people should understand that I can give the illusion of precision to every measurement by capturing crap. Those 32 bits on your laptops onboard audio interface are the real banger, when even the least significant digits in 16 bit mode are filled with electromagnetic interference and noise from the power rails. Real, actual precision is way more hard to reach and you will know when you reach it, because you will have spent a lot of money on the way.
Whether you need it, or wield it correctly is a different story.
There is no rule of thumb, except that people should understand that I can give the illusion of precision to every measurement by capturing crap. Those 32 bits on your laptops onboard audio interface are the real banger, when even the least significant digits in 16 bit mode are filled with electromagnetic interference and noise from the power rails. Real, actual precision is way more hard to reach and you will know when you reach it, because you will have spent a lot of money on the way.
Whether you need it, or wield it correctly is a different story.
What sort of meter is this? I’m aware of a few 8 1/2 digit units but thought that was it.
Highly field dependent. In most of biology, two figures is often way above pipetting skills of most mortals (and robots). In physics in my meager understanding they are looking at close to 10 in some cases.
Agreed. Based on the title, I took the context of the article as generic data science/analytics in industry, not cutting edge bio or physics.
It'd be even more insightful if the significant figures discussion touched upon rounding that may be inherent to the calculations.
Take sin(x), C math would give a different approx value for sin(x+2*pi) even though it should be the same. Well, if only pi was known exactly...
Take sin(x), C math would give a different approx value for sin(x+2*pi) even though it should be the same. Well, if only pi was known exactly...
In addition to the uncertainty of the value, there's also the context of how it will be used. E.g. for some quick business decision that needs to be made, it may be fine to round to 1 sig fig, e.g. 71.244% -> 7/10 -> the majority.
[deleted]
My favorite example is GPS locations stored as and then displayed as floats or even doubles, giving a location precise to a handful of nanometers.
It's not uncommon to have 10m accuracy with consumer gear, which is 4 digits to the right of the decimal in degrees, and differential GPS can do around 3 orders of magnitude better, or around 7 digits. That's hitting the limits of float precision. By the time you start doing calculations and error propagation on the calculations, doubles are absolutely reasonable choices.
Are all location estimates that accurate? No. Should the precision of your location estimate be limited by your storage format? Of course not. That's why doubles are frequently used.
Are all location estimates that accurate? No. Should the precision of your location estimate be limited by your storage format? Of course not. That's why doubles are frequently used.
The point isn’t how you store them. The point is how you display them.
Relative to the quality of the instrumentation chain.
I've heard that some on this site do spend time measuring plate drift and correcting for atmospheric path distortion on long soak GPS base stations (that spend months recording GPS data for a fixed position to smooth out satellite and path error).
Your point stands though, there's no sense displaying drama digits that lack meaning in context.
I've heard that some on this site do spend time measuring plate drift and correcting for atmospheric path distortion on long soak GPS base stations (that spend months recording GPS data for a fixed position to smooth out satellite and path error).
Your point stands though, there's no sense displaying drama digits that lack meaning in context.
Software often ignores significant digits. It'd be buch nicer if math libraries had built-in significance propagation.
Data scientists have better ways of quantifying uncertainty. They can estimate statistics (functions of noisy estimates) by modeling the noise distribution, and report the credible- or confidence interval. And this is standard practice, so there is nothing to worry about.
I was taught about significant digits in Junior High School science classes. (in a public school in US, late 1960s.) Answers were marked wrong if they had too many digits that weren't significant. Is this no longer taught in school?
I was taught this in High School and I know that it is still taught. We learned it because the option for such (and many other) errors was a zero on the work and likely an F in the class overall.
Teachers have a lot less latitude nowadays; see [1] for an example that is shocking in the 1960s context.
1. https://www.cbc.ca/news/canada/edmonton/teacher-who-gave-zer...
Teachers have a lot less latitude nowadays; see [1] for an example that is shocking in the 1960s context.
1. https://www.cbc.ca/news/canada/edmonton/teacher-who-gave-zer...
This was definitely still taught in my public high school in the 2010s - if they stopped teaching this, it was very recently.
Granted, the teacher that tried to teach me this had no understanding of it themselves and would incorrectly mark answers wrong when they had the correct number of significant digits, so it was pretty botched in my case. Still was attempted in the curriculum though.
Granted, the teacher that tried to teach me this had no understanding of it themselves and would incorrectly mark answers wrong when they had the correct number of significant digits, so it was pretty botched in my case. Still was attempted in the curriculum though.
I think some of the problem is that you can get into computing through many pathways now, and you don’t have to come through some science and math stream from high school. Even when I was doing physics I always had to remind myself of the accuracy.
Significant figures have always been taught in chemistry courses. I remember students getting confused at notation like "100. g" representing 100 grams with 3 significant figures.
[deleted]
I don't remember it in junior high, which for me was the late 1970s. High school yes, definitely.
> Yes, you have 10,000 samples and properly classified 5,124 of them so your mathematical precision is 0.5124
That’s the point. That’s the correct usage of significant figures. Your inputs have significant figures ; it determines exactly the significant figures of your output. Significant figures have nothing to do with things like sampling variance.
What you’re looking for is not significant figures, it’s confidence intervals.
That’s the point. That’s the correct usage of significant figures. Your inputs have significant figures ; it determines exactly the significant figures of your output. Significant figures have nothing to do with things like sampling variance.
What you’re looking for is not significant figures, it’s confidence intervals.
No, they need to learn the importance of transparency. Chemists need to learn that too but I am not holding my breathe. They are too busy trying not to blow themselves up to care. Significant digits, like prediction intervals and confidence intervals are all rendered meaningless by misunderstanding from just one thing of a great many implicit assumptions upstream of the analysis.
Xkcd Coordinate Precision: https://xkcd.com/2170/
NASA: pi 15 digits
"For JPL's highest accuracy calculations, which are for interplanetary navigation, we use 3.141592653589793. Let's look at this a little more closely to understand why we don't use more decimal places."
"Let's go to the largest size there is: the known universe. The radius of the universe is about 46 billion light years. Now let me ask (and answer!) a different question: How many digits of pi would we need to calculate the circumference of a circle with a radius of 46 billion light years to an accuracy equal to the diameter of a hydrogen atom, the simplest atom? It turns out that 37 decimal places (38 digits, including the number 3 to the left of the decimal point) would be quite sufficient. "
https://www.jpl.nasa.gov/edu/news/2016/3/16/how-many-decimal...
NASA: pi 15 digits
"For JPL's highest accuracy calculations, which are for interplanetary navigation, we use 3.141592653589793. Let's look at this a little more closely to understand why we don't use more decimal places."
"Let's go to the largest size there is: the known universe. The radius of the universe is about 46 billion light years. Now let me ask (and answer!) a different question: How many digits of pi would we need to calculate the circumference of a circle with a radius of 46 billion light years to an accuracy equal to the diameter of a hydrogen atom, the simplest atom? It turns out that 37 decimal places (38 digits, including the number 3 to the left of the decimal point) would be quite sufficient. "
https://www.jpl.nasa.gov/edu/news/2016/3/16/how-many-decimal...
[deleted]
I hate how values are rounded for presentation and precision is lost.
If you have a material sample with measured length of 1cm ±0.5cm, and a measured mass of 3g±0.5g, then reporting 3.3333333333g/cm for mass per unit length is just lying. It's between 0.14 and 0.6, so "about 0.3g/cm" is not lying, whereas "0.333333333333g/cm" is.
What do you think about 0.333 ± 0.222 g/cm?
0.3 (-0.2+0.3) g/cm would be more correct. You've still got 3 significant figures when the inputs only had 1. Just don't lie, it's not that hard.
Interesting definition of a "lie". I don't think anything about that is inaccurate or untruthful. At best it could be called verbose, but yours is actually no shorter. Confidence intervals supersede significant figures in all respects, and now you've introduced a small but unnecessary bias that depends on the peculiarities of base 10.
This is the issue that is addressed by paying attention to "significant figures." Digits that are significant are shown. Those that aren't, aren't. It's appropriate to round where your precision ends.
business ppl ~always~ prefer that approach.
When we go to industry or "the real world", it's as if we are forced to unlearn all good practices
When we go to industry or "the real world", it's as if we are forced to unlearn all good practices
Everyone does. I grind my teeth at things like “blue whales can be up to 30 meters (98.43 feet) long” or “my dog weighs 20 lbs (9.072 kg)”. Stop that, please.
In my experience, non tech/engineer/math people dislikes more than 1 or 2 digits.
Obs: Don't wanna sound a douche here but I can't help but notice that they're talking about accuracy citing Wikipedia.
Obs: Don't wanna sound a douche here but I can't help but notice that they're talking about accuracy citing Wikipedia.