Pitfalls of comparing BigDecimals in Java(igorstechnoclub.com)
igorstechnoclub.com
Pitfalls of comparing BigDecimals in Java
https://igorstechnoclub.com/java-bigdecimal/
49 comments
Why would the BigDecimal implementation of 'equals' not use semantic (numeric, in this case) equality? Is 'equals' always meant to be memberwise (or perhaps makes performance guarantees), and if so, shouldn't there be another function/operator that is always meant to be semantic? The article doesn't acknowledge these questions.
> Why would the BigDecimal implementation of 'equals' not use semantic (numeric, in this case) equality?
A BigDecimal with different precision is not really semantically equivalent - it will lead to different results when you use it in calculations.
A BigDecimal with different precision is not really semantically equivalent - it will lead to different results when you use it in calculations.
Why is it not?
The blog post states “BigDecimal is a class in Java used to represent floating-point numbers with arbitrary precision”
Which means that the order should not semantically matter.
That said even if the order necessarily does matter (infinite precision be alas unavailable), does not justify equality of different representations of the same value being incorrect.
Any justification for == not comparing the represented value applies equally to compareTo (which someone else suggested?)
More over operations on this type, if they can give precise/arbitrary representation should not matter if the input values are represented differently. If I perform the same operations on the same values, and in intermediate values are all representable, then the output should always be the same value.
If the actual intent of this type is “equals is an invalid way to compare values” the equals should throw when the values are the same but the representation is different.
The blog post states “BigDecimal is a class in Java used to represent floating-point numbers with arbitrary precision”
Which means that the order should not semantically matter.
That said even if the order necessarily does matter (infinite precision be alas unavailable), does not justify equality of different representations of the same value being incorrect.
Any justification for == not comparing the represented value applies equally to compareTo (which someone else suggested?)
More over operations on this type, if they can give precise/arbitrary representation should not matter if the input values are represented differently. If I perform the same operations on the same values, and in intermediate values are all representable, then the output should always be the same value.
If the actual intent of this type is “equals is an invalid way to compare values” the equals should throw when the values are the same but the representation is different.
> The blog post states “BigDecimal is a class in Java used to represent floating-point numbers with arbitrary precision”
> Which means that the order should not semantically matter.
Arbitrary precision in the sense that you can set an arbitrary precision on your BigDecimal which then gets carried through. 1/3 = 0, 1.0/3=0.3, 1.00/3=0.33 etc.
> Any justification for == not comparing the represented value applies equally to compareTo (which someone else suggested?)
No it doesn't. equals (not the same as == in Java) is meant to mean equivalent in the program, liskov-substitutable for. compareTo is meant to compare ordering. It's ok to have values that are nonequal but compare the same for ordering purposes.
> If the actual intent of this type is “equals is an invalid way to compare values” the equals should throw when the values are the same but the representation is different.
equals is valid, it just compares something different from the mathematical value. This is the normal behaviour in programming (equals does not understand the domain, if your program has multiple representations for things that are equivalent in the domain then those representations will should still compare unequal unless they have completely equivalent behaviour in your program)
> Which means that the order should not semantically matter.
Arbitrary precision in the sense that you can set an arbitrary precision on your BigDecimal which then gets carried through. 1/3 = 0, 1.0/3=0.3, 1.00/3=0.33 etc.
> Any justification for == not comparing the represented value applies equally to compareTo (which someone else suggested?)
No it doesn't. equals (not the same as == in Java) is meant to mean equivalent in the program, liskov-substitutable for. compareTo is meant to compare ordering. It's ok to have values that are nonequal but compare the same for ordering purposes.
> If the actual intent of this type is “equals is an invalid way to compare values” the equals should throw when the values are the same but the representation is different.
equals is valid, it just compares something different from the mathematical value. This is the normal behaviour in programming (equals does not understand the domain, if your program has multiple representations for things that are equivalent in the domain then those representations will should still compare unequal unless they have completely equivalent behaviour in your program)
> The blog post states “BigDecimal is a class in Java used to represent floating-point numbers with arbitrary precision”
That doesn’t mean BigDecimal is used exclusively for that purpose… If you ask an engineer if 2.5 and 2.50 are the same, they will say no. It’s clear from the implementation that BigDecimal was meant for more than just representing arbitrary precision floating points, which is why there is a pitfall if that’s all you use it for.
Btw that’s the reason it’s got Decimal in the class name - it’s modeling base10 strings, not numbers. If the strings are different it shouldn’t be equal.
That doesn’t mean BigDecimal is used exclusively for that purpose… If you ask an engineer if 2.5 and 2.50 are the same, they will say no. It’s clear from the implementation that BigDecimal was meant for more than just representing arbitrary precision floating points, which is why there is a pitfall if that’s all you use it for.
Btw that’s the reason it’s got Decimal in the class name - it’s modeling base10 strings, not numbers. If the strings are different it shouldn’t be equal.
>If you ask an engineer if 2.5 and 2.50 are the same, they will say no
They will say it depends on the context. If you don't look into the implementation, the class name and maybe a short summary provides that context and clearly misleads people
Right, but in this case the short summary being posted is poor. In the context of a language like Java, floating point is synonymous with IEEE base-2 floats. BigDecimal is not used to store arbitrary precision floats, but rather fixed precision decimals. Otherwise it would be called BigFloat.
Also the scale parameter really isn’t an implementation detail. Virtually every non-trivial method defined in the class requires you to understand the scale parameter in order to get the result you want.
Also the scale parameter really isn’t an implementation detail. Virtually every non-trivial method defined in the class requires you to understand the scale parameter in order to get the result you want.
Oh really??? That would make sense, though it raises more questions with perhaps not so brief answers.
3 / 2 = 1, 3.0 / 2 = 1.5. So 3.equals(3.0) is false, as this "equals" is meant to have programming semantics rather than mathematical ones.
It looks like you're comparing integer arithmetic and float arithmetic. Are you saying BigDecimal can represent both??
That's a special case of decimal precision. 1/3 = 0, 1.0/3=0.3, 1.00/3=0.33 is probably a better example. (where by 1.0 I mean a BigDecimal with precision=2 - in fact the default is 40)
Oh, thanks, I think the phrase "programming semantics" threw me off, since I don't think of the precision behavior of ints and floats as "programming semantics", but rather "int and float semantics". In other words, it makes sense to me that integer 3/2 = 1 while float 3/2 = 1.5, because int can not represent 1.5, not because that's just how "programming" works.
So, extrapolating from that logic, since BigDecimal can represent 1.5, I would expect 3/2 = 1.5 no matter how 3 and 2 are internally represented. However, what I think I'm coming to understand, is that the precision of a BigDecimal is intentionally a semantic part of the value (duh, that's what you've been saying).
The only reason I can think of (as a math layman) for including the precision as a semantic part is that there are real world use cases for indicating precision and carrying it forward into results via rounding, like physical measurements. This is useful, but it also vexes me slightly because it means BigDecimal is not just for getting more precision than other numeric types, it is also for knowing/controlling the amount of precision, meaning there is no type dedicated only to the former. But I suppose the cases where you want more precision tend to also be the cases where you want to know/control the precision... It's just that this demonstrably doesn't occur to everybody using the API who might just be reaching for it because "I want more precision". I don't know how you would make it more obvious.
Then again, I am essentially guessing.
So, extrapolating from that logic, since BigDecimal can represent 1.5, I would expect 3/2 = 1.5 no matter how 3 and 2 are internally represented. However, what I think I'm coming to understand, is that the precision of a BigDecimal is intentionally a semantic part of the value (duh, that's what you've been saying).
The only reason I can think of (as a math layman) for including the precision as a semantic part is that there are real world use cases for indicating precision and carrying it forward into results via rounding, like physical measurements. This is useful, but it also vexes me slightly because it means BigDecimal is not just for getting more precision than other numeric types, it is also for knowing/controlling the amount of precision, meaning there is no type dedicated only to the former. But I suppose the cases where you want more precision tend to also be the cases where you want to know/control the precision... It's just that this demonstrably doesn't occur to everybody using the API who might just be reaching for it because "I want more precision". I don't know how you would make it more obvious.
Then again, I am essentially guessing.
> This is useful, but it also vexes me slightly because it means BigDecimal is not just for getting more precision than other numeric types, it is also for knowing/controlling the amount of precision, meaning there is no type dedicated only to the former. But I suppose the cases where you want more precision tend to also be the cases where you want to know/control the precision... It's just that this demonstrably doesn't occur to everybody using the API who might just be reaching for it because "I want more precision".
In practice someone who just "wants more precision" doesn't touch the precision and gets 40 significant figures of it, which will probably do what they want. Exact rationals have their own problems, and if you're not using exact rationals then however much precision you've got you'll always have rounding (as soon as you divide by anything other than 2 or 5).
In practice someone who just "wants more precision" doesn't touch the precision and gets 40 significant figures of it, which will probably do what they want. Exact rationals have their own problems, and if you're not using exact rationals then however much precision you've got you'll always have rounding (as soon as you divide by anything other than 2 or 5).
[deleted]
My suggestion would be to use compareTo where possible, and rely on equals only when you know for sure that it's implemented correctly
Which implies the type is not compatible with the built in collections? Or have they moved aware from equals?
>only when you know for sure that it's implemented correctly
This is pretty disappointing advice for a standard/bundled API :(
This is pretty disappointing advice for a standard/bundled API :(
"0.0 + 0.05" and "0.00 + 0.05" are not the same so "0.0" and "0.00" should probably not be considered to be the same.
What are the results of those two examples?
Maybe GP was thinking of the optional rounding modes? With plain old add(), you get a BigDecimal 0.05 with scale 2 in both cases and the two examples are equal according to the equals() method.
var a = new BigDecimal("0.0");
var b = new BigDecimal("0.05");
var result1 = a.add(b);
var c = new BigDecimal("0.00");
var result2 = c.add(b);
System.out.println(result1); // 0.05
System.out.println(result2); // 0.05
System.out.println(result1.equals(result2)); // true
System.out.println(result1.scale()); // 2
System.out.println(result2.scale()); // 2Yeah, my bad, I was thinking about rounding.
I'm surprised that a.add(b) promotes to an extra decimal place.
That's a touch ... odd.
I'm surprised that a.add(b) promotes to an extra decimal place.
That's a touch ... odd.
You could also think of it as declining to _lose_ a decimal place; a.add(b) ought to be the same as b.add(a), so there's no reason to prefer a's precision to b. You need to take the higher precision of the two for an exact result; I assume that's the rationale.
> The BigDecimal class gives its user complete control over rounding behavior. If no rounding mode is specified and the exact result cannot be represented, an exception is thrown ...
- https://docs.oracle.com/javase/8/docs/api/java/math/BigDecim...
> The BigDecimal class gives its user complete control over rounding behavior. If no rounding mode is specified and the exact result cannot be represented, an exception is thrown ...
- https://docs.oracle.com/javase/8/docs/api/java/math/BigDecim...
Thanks for the link.
Apparently, restricted behavior like I expected is reserved for if you use a MathContext.
> Instead, for the BigDecimal operations taking a MathContext parameter, if the MathContext has a nonzero precision, the set of possible representable values for the result is determined by the precision of the MathContext argument.
Very strange. I would have expected something that is going to be mostly used for currency to not do implicit things by default.
Apparently, restricted behavior like I expected is reserved for if you use a MathContext.
> Instead, for the BigDecimal operations taking a MathContext parameter, if the MathContext has a nonzero precision, the set of possible representable values for the result is determined by the precision of the MathContext argument.
Very strange. I would have expected something that is going to be mostly used for currency to not do implicit things by default.
https://docs.oracle.com/en/java/javase/21/docs/api/java.base... is probably a better link (though it's not that different)
> Why would the BigDecimal implementation of 'equals' not use semantic (numeric, in this case) equality?
https://stackoverflow.com/a/66180493:
“The general rule for equals is that two equal values should be substitutable for one another. That is, if performing a computation using one value gives some result, substituting an equals value into the same computation should give a result that equals the first result. This applies to objects that are values, such as String, Integer, BigDecimal, etc.
Now consider BigDecimal values 2.0 and 2.00. We know they are numerically equal, and that compareTo on them returns 0. But equals returns false. Why?
Here's an example where they are not substitutable:
https://stackoverflow.com/a/66180493:
“The general rule for equals is that two equal values should be substitutable for one another. That is, if performing a computation using one value gives some result, substituting an equals value into the same computation should give a result that equals the first result. This applies to objects that are values, such as String, Integer, BigDecimal, etc.
Now consider BigDecimal values 2.0 and 2.00. We know they are numerically equal, and that compareTo on them returns 0. But equals returns false. Why?
Here's an example where they are not substitutable:
var a = new BigDecimal("2.0");
var b = new BigDecimal("2.00");
var three = new BigDecimal(3);
a.divide(three, RoundingMode.HALF_UP)
==> 0.7
b.divide(three, RoundingMode.HALF_UP)
==> 0.67
https://docs.oracle.com/javase/8/docs/api/java/math/BigDecim...
> Note: care should be exercised if BigDecimal objects are used as keys in a SortedMap or elements in a SortedSet since BigDecimal's natural ordering is inconsistent with equals. See Comparable, SortedMap or SortedSet for more information.
https://docs.oracle.com/javase/8/docs/api/java/math/BigDecim...
> Compares this BigDecimal with the specified Object for equality. Unlike compareTo, this method considers two BigDecimal objects equal only if they are equal in value and scale (thus 2.0 is not equal to 2.00 when compared by this method).
equals remains object comparisons.
Consider the problem if `2.0` and `2.00` returned true for equals call and they were keys to a HashMap - you'd only be able to put one of them in there.
https://github.com/openjdk/jdk/blob/master/src/java.base/sha...
> Note: care should be exercised if BigDecimal objects are used as keys in a SortedMap or elements in a SortedSet since BigDecimal's natural ordering is inconsistent with equals. See Comparable, SortedMap or SortedSet for more information.
https://docs.oracle.com/javase/8/docs/api/java/math/BigDecim...
> Compares this BigDecimal with the specified Object for equality. Unlike compareTo, this method considers two BigDecimal objects equal only if they are equal in value and scale (thus 2.0 is not equal to 2.00 when compared by this method).
equals remains object comparisons.
Consider the problem if `2.0` and `2.00` returned true for equals call and they were keys to a HashMap - you'd only be able to put one of them in there.
https://github.com/openjdk/jdk/blob/master/src/java.base/sha...
I feel like if they had simply named it FixedDecimal they would have avoided the confusion. It makes more sense that 1.0 differs from 1.00 as they are different precicions. As another person pointed out, dividing them by 3 will yield different results.
BigFixedDecimal? Sounds like yet another java naming pitfall.
I use bigInt and bigDec when I wrote a pokemonGo bot way back when the cracked api released in like… 2016?
I didn’t bump into the particular issue, but not being much of a Java person, they were very helpful. My bot was pretty consistent at getting lvl30 in a night and usually something fun like a snorlax or charizard.
When they broke the api I stopped playing, writing the code for a bit was more fun (for me) than playing as intended.
As an aside, are there games like that, which require writing code? I imagine the market isn’t big for those, but I sure had a blast for a month or two.
I didn’t bump into the particular issue, but not being much of a Java person, they were very helpful. My bot was pretty consistent at getting lvl30 in a night and usually something fun like a snorlax or charizard.
When they broke the api I stopped playing, writing the code for a bit was more fun (for me) than playing as intended.
As an aside, are there games like that, which require writing code? I imagine the market isn’t big for those, but I sure had a blast for a month or two.
Spacetraders. I've been meaning to get into it but just haven't found the time. Looks really fun.
Zachtronics (https://www.zachtronics.com/) is amazing, but their games are lower level programming. You're typically designing hardware or writing assembly. I haven't played them all, but I can vouch for last call BBS (design hardware), exapunks (software controlling bots), shenzen i/o (you design hardware and write the assembly to power it), and tis-100 (writing assembly for a distributed, cluster of processors).
It sounds a lot more intimidating than reality. The puzzles start off easy enough that you can ramp into the difficulty., and they usually have many different possible solutions. You can "win" with easy solutions, and then later revisit them and try to produce the "best" solution. (Often pretty difficult).
To be fair, I did come at them from somewhat of a background in hardware, but my son also loved them (with no such background).
It sounds a lot more intimidating than reality. The puzzles start off easy enough that you can ramp into the difficulty., and they usually have many different possible solutions. You can "win" with easy solutions, and then later revisit them and try to produce the "best" solution. (Often pretty difficult).
To be fair, I did come at them from somewhat of a background in hardware, but my son also loved them (with no such background).
Kinda makes me think that implementation is probably not very performant or optimized for the obvious case. A big pet peeve of mine is when languages do weird and very unintuitive things with equality. Like in Javascript and PHP you end up needing these crazy charts to determine if something == something else. Personally I think Clojure gets this really well and has some of the fewest surprises.
No need to remember crazy charts to determine equality in Javascript, just blanketly use `===`
JavaScript == comparison isn't particularly strange; the "crazy charts" are mostly cases that you should never encounter if you're writing reasonable code. If you're going to use a language, it's a much better idea to familiarize yourself with its actual semantics than to pepper your codebase with superstitious rituals.
This seems like incredibly bad design. Is there any justification or use case where someone would want the abstraction to leak like this?
It is likely because equals is usually “object equals”, not necessarily “math equals”. Since the internal representation is different, the objects are indeed different. Tricky though, and most 3rd party code would likely choose to make equals true in this kind of situation.
Java has tons of examples where the core behavior is logically correct, but VERY unintuitive - I feel the pain daily.
Java has tons of examples where the core behavior is logically correct, but VERY unintuitive - I feel the pain daily.
I don't think that flies, because the whole point of the .equals() method in regards to string is to avoid "object equals" and make sure the strings say the same thing..
It's supposed to be a mess. We have very little job security anymore.
If one mass is 1.0 kg ± 40 g, and another mass is 1.000 kg ± 0.4 g, are they close enough to equal? Maybe, and if you round them to the same precision you get whichever answer you wanted.
https://en.wikipedia.org/wiki/Significant_figures
Trailing zeroes convey information about the measurement precision of a number. If you're doing math with real-world measurements, errors accumulate, and you're expected to keep track of them.
Trailing zeroes convey information about the measurement precision of a number. If you're doing math with real-world measurements, errors accumulate, and you're expected to keep track of them.
All true, but using a BigDecimal to do that is abusing the implementation and keeping important assumptions implicit. Better to explicitly quantify the uncertainty with a second variable.
Eh just par for the course for Java https://news.ycombinator.com/item?id=21765788
Interesting. I'd say that it's one of the places where using a non-trivial, non-structural comparison method would make sense, as opposed to the infamous URL equals method.
whats funny is the divide between math theory in the 20th century and automated arithmetic.
math world - "wouldn't it be great if we had a thing called a field, or even better a group, a bunch of rules everyone agrees on where operations are closed and you can make all these great discoveries and there is so much consistency and predictability..."
computer world - "yes but wouldn't it also be great if we could add binary fractions with rounding...."
math world - "wouldn't it be great if we had a thing called a field, or even better a group, a bunch of rules everyone agrees on where operations are closed and you can make all these great discoveries and there is so much consistency and predictability..."
computer world - "yes but wouldn't it also be great if we could add binary fractions with rounding...."
This seems like a massive API design mistake
What does? The different numbers being different, or some other part?
> Care should be exercised if BigDecimal objects are used as keys in a SortedMap or elements in a SortedSet since BigDecimal's natural ordering is inconsistent with equals. See Comparable, SortedMap or SortedSet for more information.
https://docs.oracle.com/en%2Fjava%2Fjavase%2F22%2Fdocs%2Fapi...
"Consistency with equals" specifically means a.compareTo(b) == 0 is equivalent to a.equals(b). Java docs explicitly state that this is "strongly recommended (though not required)".
https://docs.oracle.com/en%2Fjava%2Fjavase%2F22%2Fdocs%2Fapi...