In Erlang/OTP 27, +0.0 will no longer be exactly equal to -0.0(erlangforums.com)
erlangforums.com
In Erlang/OTP 27, +0.0 will no longer be exactly equal to -0.0
https://erlangforums.com/t/in-erlang-otp-27-0-0-will-no-longer-be-exactly-equal-to-0-0/2586
234 comments
There is a huge misconception here which really should be clarified in the title. Erlang has an "exactly equal" or strict equality operator, called =:=, which is different from just usual equality, which is ==. The usual equality operator will keep having +0.0 = -0.0, and floating point arithmetic will keep behaving as you would expect... It's that we no longer have +0.0 =:= -0.0, which is a very different thing (and frankly, makes sense - there should be some special operator that can differentiate between these two values).
Honestly - I rather dislike this change. I'd much rather have 0 & -0 be precisely equal to each other and have some obscure library function like `isNegativeZero` for the extremely rare cases they'd need to be differentiated. 0 and -0 being distinct values is honestly just an accident of convenience with how floats are stored - there are two bit patterns that result in a value of 0 but because they're different patterns we've assigned an unwarranted amount of distinction to them.
> 0 and -0 being distinct values is honestly just an accident of convenience with how floats are stored
That's not at all true. The inclusion of negative zero in ieee 754 was a deliberate design choice, and if it were considered desirable that zero should be unsigned, then the bit pattern assigned to -0 would not have been so assigned. See 'much ado about nothing's sign bit'[0], by one of the principal authors of ieee 754.
Whereas -0 and +0 are not operationally equivalent, operational equivalence is an important and useful relation; see the very pattern matching example linked from the OP[1].
0. https://people.freebsd.org/~das/kahan86branch.pdf
1. https://erlangforums.com/t/total-term-order/2477/41
That's not at all true. The inclusion of negative zero in ieee 754 was a deliberate design choice, and if it were considered desirable that zero should be unsigned, then the bit pattern assigned to -0 would not have been so assigned. See 'much ado about nothing's sign bit'[0], by one of the principal authors of ieee 754.
Whereas -0 and +0 are not operationally equivalent, operational equivalence is an important and useful relation; see the very pattern matching example linked from the OP[1].
0. https://people.freebsd.org/~das/kahan86branch.pdf
1. https://erlangforums.com/t/total-term-order/2477/41
That is a phenomenal title.
unfortunately this scan of kahan's chapter is missing some pages
the thread explains in the very first message that the existing semantics caused a subtle compiler bug in a way that makes it clear that you are not just wrong about the semantics of ieee-754 floating point but also about the pragmatics of how this affects real programs
> 0 and -0 being distinct values is honestly just an accident of convenience with how floats are stored
This is actually a valid distinction because these values can represent different values.
Floating point values are not numbers, but ranges of possible values: float -0 is a number in the range (-FLT_TRUE_MIN / 2, 0] and float +0 is a number in the range [0, FLT_TRUE_MIN).
IMO if I were to redesign IEEE 754 I would add a third float 0 value that always represents exactly 0 and redefine float -0 and float +0 as open limits, but that would probably make everything a lot more complicated.
This is actually a valid distinction because these values can represent different values.
Floating point values are not numbers, but ranges of possible values: float -0 is a number in the range (-FLT_TRUE_MIN / 2, 0] and float +0 is a number in the range [0, FLT_TRUE_MIN).
IMO if I were to redesign IEEE 754 I would add a third float 0 value that always represents exactly 0 and redefine float -0 and float +0 as open limits, but that would probably make everything a lot more complicated.
I don't think it's just that.
1/0 is infinity but 1/(-0) is -infinity
1/0 is infinity but 1/(-0) is -infinity
Rigorously division by zero is "undefined" in mathematics.
You can totally define division by zero.
"There are mathematical structures in which [dividing by zero] is defined. [...] However, such structures do not satisfy every ordinary rule of arithmetic (the field axioms)." https://wikipedia.org/wiki/Division_by_zero
You're familiar with the ordinary rules of arithmetic (ORA!) applied to the real numbers. However, there are many number systems that follow different rules.
The rules of arithmetic for floats, wraparound ints, elliptic curves, and asymptotic analysis are all different from ORA! (and each other) but they are useful in their own ways.
"There are mathematical structures in which [dividing by zero] is defined. [...] However, such structures do not satisfy every ordinary rule of arithmetic (the field axioms)." https://wikipedia.org/wiki/Division_by_zero
You're familiar with the ordinary rules of arithmetic (ORA!) applied to the real numbers. However, there are many number systems that follow different rules.
The rules of arithmetic for floats, wraparound ints, elliptic curves, and asymptotic analysis are all different from ORA! (and each other) but they are useful in their own ways.
Shouldn't both of those be NaN? Does Erlang have a NaN?
The reason division by zero can be a number is that floating-point numbers aren't individual numbers but ranges of numbers. So 0.0 isn't really the integer 0, it is the range of numbers between 0 and the smallest representable non-zero number. So division by 0.0 may be division by zero, or it may be division by a very small number, in which case it would approach infinity.
The same goes for the floating-point infinity: it doesn't represent the concept of infinity, it is a placeholder for every number between the largest representable number and infinity. That's how dividing a very large number by a very small number can result in infinity, a number which is really not a number.
This is the philosophy by which IEEE floating-point numbers were designed, and it's the explanation behind which negative zeroes and infinities make sense in floating point.
The way I find it easiest to reason about is by taking a graph of an asymptote and rounding it to the nearest units. You somehow need a way to say "there is an asymptote here!" even though you might not have all the units, and so you introduce infinities and negative zeroes to maintain as much precision as possible.
The same goes for the floating-point infinity: it doesn't represent the concept of infinity, it is a placeholder for every number between the largest representable number and infinity. That's how dividing a very large number by a very small number can result in infinity, a number which is really not a number.
This is the philosophy by which IEEE floating-point numbers were designed, and it's the explanation behind which negative zeroes and infinities make sense in floating point.
The way I find it easiest to reason about is by taking a graph of an asymptote and rounding it to the nearest units. You somehow need a way to say "there is an asymptote here!" even though you might not have all the units, and so you introduce infinities and negative zeroes to maintain as much precision as possible.
this isn't true. floats don't have semantics of internals. they have semantics of exact points that do math with rounding (otherwise the result of subtracting 2 floats would sometimes be more than 1 float)
Moreover the spec defines rounding modes (rtnz, rtg, rtfz rtl) that are designed to exactly do interval arithmetic
IEEE754 has a separate +Infinity and -Infinity to go along with SNaN and QNaN.
For instance:
- as parent said 1/0 is Inf.
- as parent said 1/(-0) and log(0) is -Inf.
- sqrt(-1) and 0/0 is NaN.
[1] https://www.gnu.org/software/libc/manual/html_node/Infinity-...
For instance:
- as parent said 1/0 is Inf.
- as parent said 1/(-0) and log(0) is -Inf.
- sqrt(-1) and 0/0 is NaN.
[1] https://www.gnu.org/software/libc/manual/html_node/Infinity-...
>both of those be NaN
Totally NOT. NaN is defined as 0.0/0.0. That's it.
Totally NOT. NaN is defined as 0.0/0.0. That's it.
IIRC division by zero throws an exception.
There is no negative zero
1/0 is undefined and 1/-0 is also undefined.
what we have is lim(x->0+) 1/x = +inf and lim(x->0-) 1/x = -inf
what we have is lim(x->0+) 1/x = +inf and lim(x->0-) 1/x = -inf
this is wrong. ieee defines them
not in ieee-754, only in reality
Numbers are abstract concepts in general. There is no fundamental reality only "real" numbers.
the question of whether mathematical objects like 53 are more or less real than the historically contingent universe is indeed very interesting
however, it is irrelevant to the fact that a commenter was disagreeing because they had confused ieee-754 division with the division operation on so-called 'real' numbers
however, it is irrelevant to the fact that a commenter was disagreeing because they had confused ieee-754 division with the division operation on so-called 'real' numbers
They're still ==.
=:= (=== in elixir) Somewhat means "and has the same binary form"
Fwiw, julia made the exact same choices.
=:= (=== in elixir) Somewhat means "and has the same binary form"
Fwiw, julia made the exact same choices.
What about the N number of NaN values that are treated the same, but are physically different values?
So, in Erlang it will be the case that:
+0.0 == -0.0 is true, I guess because they are very close
+0.0 =:= -0.0 is false, I guess because they have different bits
A =:= B for two different numbers of course, because they have different bits
How does == work for other very close floating point values in Erlang? Is there some inconsistency here?
+0.0 == -0.0 is true, I guess because they are very close
+0.0 =:= -0.0 is false, I guess because they have different bits
A =:= B for two different numbers of course, because they have different bits
How does == work for other very close floating point values in Erlang? Is there some inconsistency here?
It is not about being very close. IEEE says "Comparisons shall ignore the sign of zero" and erlang (or rather CPU instructions erlang outputs for float == float) respects that
While =:= is not something mandated by IEEE. Erlang implements such an operator on floats and it decides to compare false for 0.0 & -0.0. So it probably compares bits/does integer comparisons on same data
While =:= is not something mandated by IEEE. Erlang implements such an operator on floats and it decides to compare false for 0.0 & -0.0. So it probably compares bits/does integer comparisons on same data
Ah, so == is compliant, and =:= is extra. Makes sense!
The only other distinction between == and =:= is for integers and floats. Integers are not =:= to the corresponding float, but are ==, so it has nothing to do with closeness.
-0.0 is a weird dumb non mathematical value that IEEE 754 put in (for example the standardized value for sqrt(-0.0) is not even justifiable based on their choices). I think they didn't want 1/(1/-Inf) to be Inf, or something.
In the end think of erlang's "==" as "equal in value" and "=:=" as "equal in representation in memory".
Now if you really want to raise the hackles of someone using Erlang, should ask why integers and floats don't have a well ordering, and what you should do when you sort them.
-0.0 is a weird dumb non mathematical value that IEEE 754 put in (for example the standardized value for sqrt(-0.0) is not even justifiable based on their choices). I think they didn't want 1/(1/-Inf) to be Inf, or something.
In the end think of erlang's "==" as "equal in value" and "=:=" as "equal in representation in memory".
Now if you really want to raise the hackles of someone using Erlang, should ask why integers and floats don't have a well ordering, and what you should do when you sort them.
Reading the article is what clarifies things.
HN is full of people who think the world needs their hot take based on a headline, unfortunately.
HN is full of people who think the world needs their hot take based on a headline, unfortunately.
"Please don't sneer, including at the rest of the community." It's reliably a marker of bad comments and worse threads.
https://news.ycombinator.com/newsguidelines.html
https://news.ycombinator.com/newsguidelines.html
I read the first word of your reply and would like to counter you with “no! writing!”
i don't use Erlang (mainly Julia), but this change makes a lot of sense. in Julia, there is the == operator (value equality) and the === operator (value egality). 0.0 == -0.0,because they have the same "value". but they aren't identical (the bits are different), so !(0.0 === -0.0).
> there should be some special operator that can differentiate between these two values).
What for though?
I was expecting as I learned Rust that I would soon find the == operator (which is just the predicate PartialEq::eq) wasn't quite enough and need some === type operator (Rust doesn't have one), but I never did. Turns out actually in most cases equality is what I cared about.
What for though?
I was expecting as I learned Rust that I would soon find the == operator (which is just the predicate PartialEq::eq) wasn't quite enough and need some === type operator (Rust doesn't have one), but I never did. Turns out actually in most cases equality is what I cared about.
There is an example in the article about compiler optimization, and they state that other similar instances may exist.
The example for the compiler just seems like it wants a property that types don't necessarily even have (Equivalence), so my instinct would be that it's probably just broken and this is the tip of the iceberg or if the compiler does care about this property it should know floats don't have Equivalence and thus 0.0 == -0.0 isn't cause to perform this optimisation.
it's using the =:= operator you say it should be using, not the == operator you incorrectly say it is using
however, that operator was originally defined incorrectly, and now they are fixing that bug
however, that operator was originally defined incorrectly, and now they are fixing that bug
So far the only example I've been given of this operator is that it's what the compiler uses to decide equivalence, which is a situation where the correct answer was categorically no. The belief seems to be that this code is fine if we mistakenly unify cases where we think the IEEE floating point number had the same representation, and I don't buy that this is better than unifying -0.0 and 0.0 in the example, I think it'll be the exact same outcome.
i think the belief is rather that the compiler needs to reliably distinguish between -0 and +0 except where it explicitly invokes arithmetic comparison, and so the semantics of both =:= and unification should change to distinguish -0 from +0
i don't know what it would mean to not unify cases where the ieee floating point number had the same representation, assuming that by 'unify' you mean 'paattern-match) (erlang doesn't have full bidirectional unification where run-time data can contain variables which could be instantiated with constants from a pattern). what would the semantics of unification be then? no pattern could ever match a term containing a floating point number?
or perhaps you aren't aware that 'unify' means 'pattern-match' (prolog style) and we are discussing a pattern-matching bug, and by 'unify cases' you just mean 'remove one of two cases'. i think it's very likely that the compiler needs to remove redundant cases that fall out of its optimizer in order to get decent performance, and if the user puts them in their code it's probably a bug
i don't know what it would mean to not unify cases where the ieee floating point number had the same representation, assuming that by 'unify' you mean 'paattern-match) (erlang doesn't have full bidirectional unification where run-time data can contain variables which could be instantiated with constants from a pattern). what would the semantics of unification be then? no pattern could ever match a term containing a floating point number?
or perhaps you aren't aware that 'unify' means 'pattern-match' (prolog style) and we are discussing a pattern-matching bug, and by 'unify cases' you just mean 'remove one of two cases'. i think it's very likely that the compiler needs to remove redundant cases that fall out of its optimizer in order to get decent performance, and if the user puts them in their code it's probably a bug
> no pattern could ever match a term containing a floating point number?
No equivalence matching on floating point numbers seems like a much healthier approach, yes. I don't expect people to have huge problems with "x greater than 0.0" or "z less than 16.35" but "p is exactly 8.9" suggests muddled thinking.
The floats are very strange. While machine integers are already a bit weird compared to the natural numbers we learned in school, the floats are much stranger so I expect code which is trying to test equivalence on these values is going to keep running into trouble.
We know that humans, including human programmers, do not appreciate how weird floats really are, so I think telling programmers they just can't perform these matches will have a better outcome than confusing them by allowing this and then it rarely has totally unexpected consequences because a+epsilon was equivalent to b even though mathematically a+epsilon != b
No equivalence matching on floating point numbers seems like a much healthier approach, yes. I don't expect people to have huge problems with "x greater than 0.0" or "z less than 16.35" but "p is exactly 8.9" suggests muddled thinking.
The floats are very strange. While machine integers are already a bit weird compared to the natural numbers we learned in school, the floats are much stranger so I expect code which is trying to test equivalence on these values is going to keep running into trouble.
We know that humans, including human programmers, do not appreciate how weird floats really are, so I think telling programmers they just can't perform these matches will have a better outcome than confusing them by allowing this and then it rarely has totally unexpected consequences because a+epsilon was equivalent to b even though mathematically a+epsilon != b
as documented in https://news.ycombinator.com/item?id=35889463 this would prevent you from translating lapack into your ideal language
you have to get rid of order comparisons too, though, or people will just replace a == b with a >= b && a <= b
also it seems like a compiler attempting to determine whether two pieces of code are equivalent (so it can throw one away) needs to be able to test whether two constants in them could ever produce different computational results; this is important for code-movement optimizations and for reunifying the profligate results of c++ template expansion
similarly, a dataflow framework like observablehq needs to be able to tell if an observable-variable update needs to propagate (because it could change downstream results) or not; for that purpose it needs to even be able to distinguish different nans. like the compiler, it needs the exact-bitwise-equality relation rather than ordinary arithmetic equality
where i think we agree is that floating-point arithmetic is probably a bad default for most people most of the time because it brings in all kinds of complexity most programmers don't even suspect
your comment reads to me as 'i don't understand floating point equality and therefore no human does so no human should have access to it'
but there are more things in heaven and earth than are dreamed of in your philosophy, tialaramex
you have to get rid of order comparisons too, though, or people will just replace a == b with a >= b && a <= b
also it seems like a compiler attempting to determine whether two pieces of code are equivalent (so it can throw one away) needs to be able to test whether two constants in them could ever produce different computational results; this is important for code-movement optimizations and for reunifying the profligate results of c++ template expansion
similarly, a dataflow framework like observablehq needs to be able to tell if an observable-variable update needs to propagate (because it could change downstream results) or not; for that purpose it needs to even be able to distinguish different nans. like the compiler, it needs the exact-bitwise-equality relation rather than ordinary arithmetic equality
where i think we agree is that floating-point arithmetic is probably a bad default for most people most of the time because it brings in all kinds of complexity most programmers don't even suspect
your comment reads to me as 'i don't understand floating point equality and therefore no human does so no human should have access to it'
but there are more things in heaven and earth than are dreamed of in your philosophy, tialaramex
My fear isn't about floating point equality but about equivalence. As we discussed repeatedly, nothing changed for equality, -0.0 = 0.0 just as -0 = 0 in integers. Equality works fine, though it may be surprising in some cases for floating point values because sometimes a != a and b + 1 == b.
"Exact-bitwise-equality" is yet a further different thing from equivalence. Is this what Erlang's =:= operator does? Erlang's documentation described it as "Exactly equal to" which is a very silly description (implying maybe == is approximately equal to), presumably there's formal documentation somewhere which explains what they actually meant but I didn't find it.
Presumably Exact-bitwise-equality is always defined in Erlang? In a language like Rust or C++ that's Undefined in lots of cases so this would be a terrible idea. I still think it's better not to prod this particular dragon, but you do you.
"Exact-bitwise-equality" is yet a further different thing from equivalence. Is this what Erlang's =:= operator does? Erlang's documentation described it as "Exactly equal to" which is a very silly description (implying maybe == is approximately equal to), presumably there's formal documentation somewhere which explains what they actually meant but I didn't find it.
Presumably Exact-bitwise-equality is always defined in Erlang? In a language like Rust or C++ that's Undefined in lots of cases so this would be a terrible idea. I still think it's better not to prod this particular dragon, but you do you.
I could also imagine dumping binary to a file and expectingb the hashes to be equal.
What’s the use case for different values?
It's a natural consequence of the IEEE 754 standard for floating point numbers.
To be clear, it was added to the standard because there was user demand for it.
Zero is already special cased in IEEE 754 (along with the rest of the denormals, because of the implied leading mantissa bit), the implementation would be no harder if you just only had a single zero. However, back when the standard was being created, when a single zero value was proposed there was pushback from people in the user community who depended on knowing which side of the zero they underflowed on their own floating point implementations.
Many people think that negative zero is just an engineering artefact. This is not true, it is a feature that was asked for, debated, and then added...
... almost certainly because it was an engineering artefact in a previous system that someone managed to depend on, https://xkcd.com/1172/ style.
Zero is already special cased in IEEE 754 (along with the rest of the denormals, because of the implied leading mantissa bit), the implementation would be no harder if you just only had a single zero. However, back when the standard was being created, when a single zero value was proposed there was pushback from people in the user community who depended on knowing which side of the zero they underflowed on their own floating point implementations.
Many people think that negative zero is just an engineering artefact. This is not true, it is a feature that was asked for, debated, and then added...
... almost certainly because it was an engineering artefact in a previous system that someone managed to depend on, https://xkcd.com/1172/ style.
> Many people think that negative zero is just an engineering artefact. This is not true, it is a feature that was asked for, debated, and then added...
Allowing zero to be represented with two bit patterns was a feature that was advocated for to make bit operations easier - it was not decided that both zeroes should be distinguishable. This absolutely is an engineering artifact but you're quite correct that it was quite intentional.
Allowing zero to be represented with two bit patterns was a feature that was advocated for to make bit operations easier - it was not decided that both zeroes should be distinguishable. This absolutely is an engineering artifact but you're quite correct that it was quite intentional.
i think the rationale in https://people.freebsd.org/~das/kahan86branch.pdf goes well above and beyond 'an engineering artefact in a previous system that someone managed to depend on'
unfortunately this scan is missing some pages
1/Infinity in 754 is 0. 1/(-Infinity) is -0. These numbers are not strictly equal in real number terms.
These numbers are inequal in terms of storage - in real number terms zero has no sign and thus -0 and +0 are the same number. The source of their distinction is also important as it's just a pure point of convenience at the bit level - there are two separate bit patterns for 0 but both numbers are the same number.
+/-0 often indicates an underflow on one side of 0 or the other. Certainly in real terms 1/(infinity) != -1/(infinity).
You are overloading the word “real”. In terms of the mathematics of the reals, there is no difference. In terms of real-life pragmatics, 0 and -0 could be used to differentiate between different outcomes in a way that is sometimes useful.
Infinity and -0 are not in the real numbers, so the expression there makes no sense if you are thinking strictly in the real numbers. If you assign real number bounds to what the floating point numbers mean, the expressions make sense.
In floating point terms infinity tends to indicate overflow (any number that is too big to represent) and the 0's indicate underflows. So in more verbose terms,
1/(positive overflow) = (positive underflow)
While
-1/(positive overflow) = (negative underflow)
In this case, since the positive overflow isn't really infinity and the underflows aren't really 0, they are not equal. In practice, -0 and the 0 can both also arise from situations that produce 0 exactly, too, but this is not that case.
You may be thinking about how lim{x->inf}(1/x) = 0 = lim{x->inf}(-1/x), which is true. Infinity in floating point does not necessarily represent that limit, though, just any number that is too big.
You may also notice that the limit is not in the range of the functions inside the limit. For all real x, 1/x != 0
In floating point terms infinity tends to indicate overflow (any number that is too big to represent) and the 0's indicate underflows. So in more verbose terms,
1/(positive overflow) = (positive underflow)
While
-1/(positive overflow) = (negative underflow)
In this case, since the positive overflow isn't really infinity and the underflows aren't really 0, they are not equal. In practice, -0 and the 0 can both also arise from situations that produce 0 exactly, too, but this is not that case.
You may be thinking about how lim{x->inf}(1/x) = 0 = lim{x->inf}(-1/x), which is true. Infinity in floating point does not necessarily represent that limit, though, just any number that is too big.
You may also notice that the limit is not in the range of the functions inside the limit. For all real x, 1/x != 0
We are saying the same thing.
Floating point numbers don't represent all possible values, just a subset of values representable by a number of bits used. Therefore, in some cases it's reasonable to treat floating point numbers not as "exact point on a number scale", but rather a range between a number and the next possible representable number.
In the case of +0.0 and -0.0, they can be treated as values between zero and the smallest representable number (about 5.4E-079 for 32 bit floats). It isn't a very common use case since dealing with such small numbers isn't a very common thing, but it is definitely a possibility.
In the case of +0.0 and -0.0, they can be treated as values between zero and the smallest representable number (about 5.4E-079 for 32 bit floats). It isn't a very common use case since dealing with such small numbers isn't a very common thing, but it is definitely a possibility.
Float stuff like 1.f / (+0.f) = infinity and 1.f / (-0.f) = -infinity.
And maybe complex number shenanigans and multi valued functions? Maybe someone familiar with mathematics can tell us more. :)
And maybe complex number shenanigans and multi valued functions? Maybe someone familiar with mathematics can tell us more. :)
The best example on kahan's webpage is Borda's mouthpiece. Signed zero makes it look right, unsigned creates a singularity
There are actually very clear and technical numerical analysis reasons for all of the weird stuff that happens in IEEE 754. The zero behavior, in particular, is because of this sort of thing.
Preserving the sign in case of arithmetic overflow/underflow.
Because two binary values that are not bit-for-bit equal should have an equality operator that can reflect that without resorting to conversion
there is no such thing as -0.
if you want to say "zero approaching from negative one", fine, say that. but that IS NOT the same thing as -0. zero is just zero, it doesn't have a sign.
some people just want everything to fit in a nice neat box, and sometimes folks, life is not like that. this is an awful change, injecting meaning where there is no room for it.
if you want to say "zero approaching from negative one", fine, say that. but that IS NOT the same thing as -0. zero is just zero, it doesn't have a sign.
some people just want everything to fit in a nice neat box, and sometimes folks, life is not like that. this is an awful change, injecting meaning where there is no room for it.
-0 exists. It's just not a mathematical object. It's an IEEE754 signed-magnitude bit-pattern — one that is distinct from the IEEE754 signed-magnitude bit-pattern we call +0. Both of these bit-patterns encode the semantic meaning of the mathematical object 0; but that doesn't mean that the bit patterns are the same "thing."
Compare/contrast: the JSON document "0" vs the JSON document "0.0". Both encode the same type (JSON only has one Number type) and so the same mathematical object — which is, again, 0 — but these are distinct JSON documents.
And that's the whole point of this change. Erlang == is mathematical equality. Erlang =:= is (encoded bit-pattern) representational equality.
Which is often the kind of equality you care about, if you have canonicalization happening at parse time, and so you want to implicitly assert (usually for added efficiency!) that the values you got after parsing + mathing are not only mathematically equal to some value, but representationally equal to that value.
---
(If this irks you as a mathematician, you could model IEEE754 as its own mathematical group — a lattice, I think? — of which +0 and -0 would be members. Along with with some — but not all — base-2 rationals; and a random assortment of signalling and non-signalling NaNs.)
Compare/contrast: the JSON document "0" vs the JSON document "0.0". Both encode the same type (JSON only has one Number type) and so the same mathematical object — which is, again, 0 — but these are distinct JSON documents.
And that's the whole point of this change. Erlang == is mathematical equality. Erlang =:= is (encoded bit-pattern) representational equality.
Which is often the kind of equality you care about, if you have canonicalization happening at parse time, and so you want to implicitly assert (usually for added efficiency!) that the values you got after parsing + mathing are not only mathematically equal to some value, but representationally equal to that value.
---
(If this irks you as a mathematician, you could model IEEE754 as its own mathematical group — a lattice, I think? — of which +0 and -0 would be members. Along with with some — but not all — base-2 rationals; and a random assortment of signalling and non-signalling NaNs.)
dealing with the same thing in rust, you can't use float in a lot of equality places where you can use ints, and they are moving to change match so you can't use floats in patterns there either.
groups and semilattices are mutually exclusive; semilattices can't have any inverse elements
> JSON document "0" vs the JSON document "0.0"
nope. both those documents are a STRING type, not a number type.
nope. both those documents are a STRING type, not a number type.
I can trump your pedantry :)
A JSON document is itself canonically a string / stream of human-writable bytes. (I.e. the JSON specification specifies JSON documents canonically as a string-encoding grammar, not as a set of formal AST-node types like SGML/XML/etc.) When you mention a JSON document in text, you're mentioning an encoded string, so you put it in quotes.
Backticks to represent what you'd type at a REPL: if `"foo"` is a JSON document, then `foo` is the decoding of that JSON document (which will not itself be a JSON document, but rather a primitive or data structure in your programming language — one which happens to have a valid encoding as a JSON document.)
A JSON document is itself canonically a string / stream of human-writable bytes. (I.e. the JSON specification specifies JSON documents canonically as a string-encoding grammar, not as a set of formal AST-node types like SGML/XML/etc.) When you mention a JSON document in text, you're mentioning an encoded string, so you put it in quotes.
Backticks to represent what you'd type at a REPL: if `"foo"` is a JSON document, then `foo` is the decoding of that JSON document (which will not itself be a JSON document, but rather a primitive or data structure in your programming language — one which happens to have a valid encoding as a JSON document.)
not sure what you hoped to accomplish with that. yes all documents are a byte array. this is not the same thing a the STRING type. this JSON document:
0
0.0
["0", "0.0"]
JSON defines absolutely no semantics for numbers, only the syntax. Some implementations do handle 0 and 0.0 differently.
See also: the problems with stuffing a high magnitude integer like 2^60+7 into a JSON document, and then unpacking it in JS using the standard web browser API. The number can't be represented exactly with the standard JS number type, but it's a valid JSON document, and so the implementation has to make a choice.
See also: the problems with stuffing a high magnitude integer like 2^60+7 into a JSON document, and then unpacking it in JS using the standard web browser API. The number can't be represented exactly with the standard JS number type, but it's a valid JSON document, and so the implementation has to make a choice.
> so your comments have only proven that some strings are different than others
Er... yes? What I'm trying to do here, is to provide an intuition for how IEEE754 bit patterns should be mentally modeled. They're byte-arrays that encode a semantic type, like a JSON document is a byte-array that encodes a semantic type.
And, just like `"0"` and `"0.0"` are different encodings of the same JSON-semantic Number, IEEE754 -0 and IEEE754 +0 are different encodings of the same mathematically-semantic number; namely, zero.
But, unlike with JSON — and I hope this should be obvious — there's no such thing as "decoding" IEEE754 to something else. The bit-patterns in IEEE754 just are... themselves. There's no more-canonical encoding to move them into that will erase the difference between IEEE754 -0 and +0, or between regular numbers and denormals, or between all the different NaNs. They're all distinct IEEE754 objects.
Which means that, unlike with JSON Numbers, where you just decode them to canonicalize them and can forget about the fact that there are multiple equally-canonical JSON encodings for any given decoded value... you can't just forget about there being multiple distinct IEEE754 objects with the same mathematical meaning.
You don't operate on JSON documents in their encoded form. But you do operate on IEEE754 bit-patterns in their encoded form. You don't turn them into "numbers" in some abstract sense, do math, and then turn them back into bit-patterns. They're bit patterns the whole time.
Every time the FPU does an IEEE754-math operation, it has to know what to do with those distinct equally-canonical encodings, applying logic to them such that they produce mathematically-semantically-correct — but potentially still bit-pattern distinct! — results. Your FPU isn't attempting to simulate mathematics over the field of the rationals (in the way that an arbitrary-precision decimal library would); it's doing specific IEEE754 operations that move an IEEE754-bit-pattern-typed register from containing a value representing one position on the IEEE754 lattice, to a value representing a different position on the IEEE754 lattice. Where some of those position-values of the lattice may be interpreted, by humans, to represent a mathematical number. (Non-uniquely, because -0 and denormals!)
There is no other domain to move the math into. You're stuck in encoded non-canonical bit-pattern land. And as such, it's helpful to be able to talk about the particular encoded non-canonical bit-patterns themselves, compare them for bit-pattern equality, and so forth. Because what those bit-patterns are can determine things like how long an operation takes, or how well a vector of said bit-patterns compresses, or whether external systems will be able to decode the results, or whether you'll still have any data at if you try to hard-cast your FP32 to an FP16.
(And don't get me started on the fact that computer "integers" aren't mathematical integers! At least IEEE754 is a standard, such that `float` means "position on the IEEE754 FP32 lattice" on every architecture targeted by programming languages with FP types, and when an arch doesn't have that support, it's shimmed in with a runtime IEEE754-correct FPU support library. The C machine-integer types — not just `int`, all of them — aren't even well-defined as being a position on a two's-complement ring vs a position on a signed-magnitude ring! Any program that ANDs two machine-integers together and expects a particular bit-pattern to result, is completely non-portable to many historical and esoteric architectures! Which comes up more than you'd think, because mathematical proof-assistants are effectively "esoteric architectures.")
Er... yes? What I'm trying to do here, is to provide an intuition for how IEEE754 bit patterns should be mentally modeled. They're byte-arrays that encode a semantic type, like a JSON document is a byte-array that encodes a semantic type.
And, just like `"0"` and `"0.0"` are different encodings of the same JSON-semantic Number, IEEE754 -0 and IEEE754 +0 are different encodings of the same mathematically-semantic number; namely, zero.
But, unlike with JSON — and I hope this should be obvious — there's no such thing as "decoding" IEEE754 to something else. The bit-patterns in IEEE754 just are... themselves. There's no more-canonical encoding to move them into that will erase the difference between IEEE754 -0 and +0, or between regular numbers and denormals, or between all the different NaNs. They're all distinct IEEE754 objects.
Which means that, unlike with JSON Numbers, where you just decode them to canonicalize them and can forget about the fact that there are multiple equally-canonical JSON encodings for any given decoded value... you can't just forget about there being multiple distinct IEEE754 objects with the same mathematical meaning.
You don't operate on JSON documents in their encoded form. But you do operate on IEEE754 bit-patterns in their encoded form. You don't turn them into "numbers" in some abstract sense, do math, and then turn them back into bit-patterns. They're bit patterns the whole time.
Every time the FPU does an IEEE754-math operation, it has to know what to do with those distinct equally-canonical encodings, applying logic to them such that they produce mathematically-semantically-correct — but potentially still bit-pattern distinct! — results. Your FPU isn't attempting to simulate mathematics over the field of the rationals (in the way that an arbitrary-precision decimal library would); it's doing specific IEEE754 operations that move an IEEE754-bit-pattern-typed register from containing a value representing one position on the IEEE754 lattice, to a value representing a different position on the IEEE754 lattice. Where some of those position-values of the lattice may be interpreted, by humans, to represent a mathematical number. (Non-uniquely, because -0 and denormals!)
There is no other domain to move the math into. You're stuck in encoded non-canonical bit-pattern land. And as such, it's helpful to be able to talk about the particular encoded non-canonical bit-patterns themselves, compare them for bit-pattern equality, and so forth. Because what those bit-patterns are can determine things like how long an operation takes, or how well a vector of said bit-patterns compresses, or whether external systems will be able to decode the results, or whether you'll still have any data at if you try to hard-cast your FP32 to an FP16.
(And don't get me started on the fact that computer "integers" aren't mathematical integers! At least IEEE754 is a standard, such that `float` means "position on the IEEE754 FP32 lattice" on every architecture targeted by programming languages with FP types, and when an arch doesn't have that support, it's shimmed in with a runtime IEEE754-correct FPU support library. The C machine-integer types — not just `int`, all of them — aren't even well-defined as being a position on a two's-complement ring vs a position on a signed-magnitude ring! Any program that ANDs two machine-integers together and expects a particular bit-pattern to result, is completely non-portable to many historical and esoteric architectures! Which comes up more than you'd think, because mathematical proof-assistants are effectively "esoteric architectures.")
> there's no such thing as "decoding" IEEE754 to something else.
Huh? You can totally take the raw bytes in an IEEE754 floating point number and parse them into something else (say, a decimal type) the same way you can take the raw bytes of a JSON document and parse it into a struct.
It's a lossy encoding for reals.
Huh? You can totally take the raw bytes in an IEEE754 floating point number and parse them into something else (say, a decimal type) the same way you can take the raw bytes of a JSON document and parse it into a struct.
It's a lossy encoding for reals.
There isn’t in the set of real numbers, but there is in the set of floating point numbers.
[deleted]
As a general rule, if you find yourself comparing floating point numbers, that's probably not what you want.
I'm wondering what they are going to do with other operators, >=, <= etc
I'm wondering what they are going to do with other operators, >=, <= etc
In general, this is good advice.
However, there are cases where comparing floating point numbers for equality works just fine. For example, if you have a variable 'v' and want to update it to a new value 'f', you can do 'if (v == f)' to check if the variable would change. Or if you have a sentinel value -1.0, it is perfectly ok to do 'if (a == -1.0)'
In general, if you look for a number 'f' in variable 'v', you can safely use equality as along as you expect 'v' to be set to 'f', i.e. the value of 'v' is not the result of a computation. This may sound trivial, but this is often omitted when discussing floating point comparison. If someone knows a scenario where the examples above would break, I would be curious to hear! (My main field is audio programming and code like this is omnipresent.)
However, there are cases where comparing floating point numbers for equality works just fine. For example, if you have a variable 'v' and want to update it to a new value 'f', you can do 'if (v == f)' to check if the variable would change. Or if you have a sentinel value -1.0, it is perfectly ok to do 'if (a == -1.0)'
In general, if you look for a number 'f' in variable 'v', you can safely use equality as along as you expect 'v' to be set to 'f', i.e. the value of 'v' is not the result of a computation. This may sound trivial, but this is often omitted when discussing floating point comparison. If someone knows a scenario where the examples above would break, I would be curious to hear! (My main field is audio programming and code like this is omnipresent.)
cesaref's comment is not good advice, it's not advice at all, it's a snarky take on how floating point is trickier than it looks
almost invariably floating-point calculations end up in some kind of comparison; if they didn't we'd probably do the calculations in a galois field or something instead
even equality comparison of floats is useful; there are an abundance of seminumerical algorithms which use it correctly on the results of computations. a trivial example is computing an attractive fixpoint by iterating a function and comparing each hare result to the previous iteration and to a tortoise. you have to be especially careful about x87 extended precision here
ieee-754 requires bit-exact results for the five fundamental operations
here is an example where your 'safe' generalization is not safe:
almost invariably floating-point calculations end up in some kind of comparison; if they didn't we'd probably do the calculations in a galois field or something instead
even equality comparison of floats is useful; there are an abundance of seminumerical algorithms which use it correctly on the results of computations. a trivial example is computing an attractive fixpoint by iterating a function and comparing each hare result to the previous iteration and to a tortoise. you have to be especially careful about x87 extended precision here
ieee-754 requires bit-exact results for the five fundamental operations
here is an example where your 'safe' generalization is not safe:
#include <stdio.h>
int main()
{
float f = 0.7;
printf(f == 0.7 ? "ok\n" : "wtf\n");
return 0;
}
> almost invariably floating-point calculations end up in some kind of comparison
I'm pretty sure OP meant comparison for equality, not comparison in general (which would be absurd, indeed).
> even equality comparison of floats is useful
That's exactly what I tried to point out. However, these are really special cases and you need to be careful and know what you are doing.
I'm pretty sure OP meant comparison for equality, not comparison in general (which would be absurd, indeed).
> even equality comparison of floats is useful
That's exactly what I tried to point out. However, these are really special cases and you need to be careful and know what you are doing.
i think he was snarkily making an absurd comment for the humor value
well you shouldn't expect much if you are comparing floats to doubles
True, but it's an easy mistake to make.
It's an easy mistake to make in C (and C++) because of their lackadaisical approach to type safety.
In Rust if we insist we want to compare an f32 (what C would call float) to an f64 (double), the language says it doesn't have a way to do that, highlighting our mistake. If we just try to compare an f32 with value 0.7 against 0.7, the type inference concludes 0.7 is also an f32 so it's equal, and if we do the same with an f64, this time by inference 0.7 is an f64 and so again it's equal.
In Rust if we insist we want to compare an f32 (what C would call float) to an f64 (double), the language says it doesn't have a way to do that, highlighting our mistake. If we just try to compare an f32 with value 0.7 against 0.7, the type inference concludes 0.7 is also an f32 so it's equal, and if we do the same with an f64, this time by inference 0.7 is an f64 and so again it's equal.
Rust's behavior makes so much more sense! IMO, implicit conversions is one of C++ biggest weaknesses. Yes, it may save some keystrokes, but it can easily lead to subtle (or not so subtle) bugs.
agreed
I know there is a fair bit of division on this point but I've always liked type mixing, in languages where this is available, to handle signal values and preferred separate variables where it is not. I'd prefer to `if (v IS NULL)` a nullable float compared to checking a constant value - or else to use a guard value to test the validity i.e. `if x = FREQUENCY_SET // enter logic involving the frequency stored in v` - that approach is even better if you can use tuples/structs to glue the status and value fields together.
I am a quickly crustifying developer who was an enthusiatic value packer in their younger years but I've changed my tune to be less afraid of using more variables if they are clearer - even if that comes at a cost to data efficiency (and like, an extremely negligible cost in most situations). Though I don't have any objections to value packing for serialization - that's a fundamentally different domain.
I am a quickly crustifying developer who was an enthusiatic value packer in their younger years but I've changed my tune to be less afraid of using more variables if they are clearer - even if that comes at a cost to data efficiency (and like, an extremely negligible cost in most situations). Though I don't have any objections to value packing for serialization - that's a fundamentally different domain.
> For example, if you have a variable 'v' and want to update it to a new value 'f', you can do 'if (v == f)' to check if the variable would change.
Isn't it strictly faster to simply assign? No branching, no branch predictions, only one instruction in all cases.
Isn't it strictly faster to simply assign? No branching, no branch predictions, only one instruction in all cases.
A typically pattern is:
float freq = getParameter(FILTER_FREQUENCY);
float coeff;
if (freq != mFreq) {
coeff = calculateCoeffFromFrequency(freq); // expensive
mFreq = freq;
mCoeff = coeff;
} else {
coeff = mCoeff; // use cached value
}
// use coeffCould you do some threshold-based comparison? What if freq is changed only a little tiny bit?
I can certainly imagine cases where this would make sense.
Assuming that all you want to do is assign, then sure. If you want to log the change or do any additional conditional logic, then that wouldn't work.
It depends on whether you need a set operation or a test-and-set operation.
I've always held that the numbers themselves are perfectly precise. It's the operations that don't do what you expect.
Of course, that observation may be more or less useful, depending on circumstances.
Of course, that observation may be more or less useful, depending on circumstances.
Somewhere in the distant past a second grade me is staring at his math homework and fuming over the frustratingly ambiguous meaning of the minus sign. Is it an operator? A property? Both?
We could make the minus sign go away if we adopted a balanced base (e.g. balanced ternary), that has both negative and positive digits. That way, just looking at the digits would make evident if a number is positive or negative, without any need for a minus sign.
I love that such systems were built once: https://www.wikipedia.org/wiki/Setun
It's only ambiguous if you think it does two things. I say it does one thing.
-4 always removes 4, whether or not there is a number to the left.
-4 always removes 4, whether or not there is a number to the left.
The problem arises with the way that this stuff is taught at a very elementary level, which then needs to be thrown away and relearned long after intuition and habits have formed around the old model. As an adult my reasoning is that it does indeed only do one thing: negation. it doesn't take anything away at all, if anything it actually adds information. Negating a four doesn't take it away, it just specifies a different (inverted) mapping of a given value with respect to zero. And this is still a very naive take, as someone with no background in number theory. But it's much more useful than say, Billy has four apples, he gives three to Mary, etc.
Math notation is to mathematics as poetry is to English.
The only acceptable grammar for math is s-expressions.
The only acceptable grammar for math is s-expressions.
> I've always held that the numbers themselves are perfectly precise. It's the operations that don't do what you expect.
I think that this is an oversimplification, and misses a key point.
Floating point numbers can be interpreted in (at least) two distinct ways.
In one use model, floating point numbers allow a sparse representation of selected points on the (extended) real number line. In this model, the numbers themselves are perfectly precise, and the basic operations do exactly what you'd expect -- but because the result of the expected real-number operation may not be one of the selected representable points, each operation is potentially paired with a second, implicit, rounding operation, and the consequences of the cumulative effects of this rounding operation take some thought.
This first model is often comfortable for those who have done low-level numerical program with sized integers, which are likewise a selected sparse representation of selected points on the integer line. While rounding comes up in fewer operations here, and is simpler, wrap-around becomes an issue.
However, a second model of floating point numbers is that each possible bit sequence represents an /interval/ on the (extended) real number line. In versions of this model, while the set of representable intervals is still very restricted, every point on the real number line falls into a representable interval, which is a property that sometimes eases (and sometimes seems to ease) analysis. In exchange, the operations are much more complex, and their limitations are much more obvious -- in particular, not only can no disjoint intervals be represented, but for any given point on the real line only one interval that contains it is representable -- there's no way to represent intervals of different sizes containing (some of) the same points.
This second model is what people are mostly thinking of when they say that floating point arithmetic is "not precise." In particular, both the inputs and outputs of operations have imprecise interpretations -- that is, the input interval contains infinitely many indistinguishable points, and the output interval contains infinitely many indistinguishable points, even if a real number equivalent of the problem would have a single number in the desired range.
When extended to true interval arithmetic, the second model can be very useful for understanding error propagation and uncertainty, but sadly this is rarely done.
I think that this is an oversimplification, and misses a key point.
Floating point numbers can be interpreted in (at least) two distinct ways.
In one use model, floating point numbers allow a sparse representation of selected points on the (extended) real number line. In this model, the numbers themselves are perfectly precise, and the basic operations do exactly what you'd expect -- but because the result of the expected real-number operation may not be one of the selected representable points, each operation is potentially paired with a second, implicit, rounding operation, and the consequences of the cumulative effects of this rounding operation take some thought.
This first model is often comfortable for those who have done low-level numerical program with sized integers, which are likewise a selected sparse representation of selected points on the integer line. While rounding comes up in fewer operations here, and is simpler, wrap-around becomes an issue.
However, a second model of floating point numbers is that each possible bit sequence represents an /interval/ on the (extended) real number line. In versions of this model, while the set of representable intervals is still very restricted, every point on the real number line falls into a representable interval, which is a property that sometimes eases (and sometimes seems to ease) analysis. In exchange, the operations are much more complex, and their limitations are much more obvious -- in particular, not only can no disjoint intervals be represented, but for any given point on the real line only one interval that contains it is representable -- there's no way to represent intervals of different sizes containing (some of) the same points.
This second model is what people are mostly thinking of when they say that floating point arithmetic is "not precise." In particular, both the inputs and outputs of operations have imprecise interpretations -- that is, the input interval contains infinitely many indistinguishable points, and the output interval contains infinitely many indistinguishable points, even if a real number equivalent of the problem would have a single number in the desired range.
When extended to true interval arithmetic, the second model can be very useful for understanding error propagation and uncertainty, but sadly this is rarely done.
the second model is wishful thinking; the first model describes how ieee-754 arithmetic actually works
as you point out, the second model can be implemented
as you point out, the second model can be implemented
the problem is the second interpretation is wrong. for example, exp(x) would give a higher value than it does since the average value of the exponent of a range is higher than the exponent of it's center.
Yep. For a sequence of operations to work on a range you do actually need to track both ends of the range, not just one of them. Otherwise what you end up implicitly doing is flipping lossily between a range interpretation and a point interpretation and back again.
How do you define numbers in a meaningful way without involving operators?
Without operators, all you have is a set of objects which you can't tell apart.
Without operators, all you have is a set of objects which you can't tell apart.
Oh, you have operators. They're just imprecise.
As a followup to this comment, in Python you have the option of using `isclose()` which is present in both `numpy`[1] and the standard `math` [2] libraries. This has been quite helpful for me in comparing small probability values.
[1] https://numpy.org/doc/stable/reference/generated/numpy.isclo...
[2] https://docs.python.org/3/library/math.html#math.isclose
[1] https://numpy.org/doc/stable/reference/generated/numpy.isclo...
[2] https://docs.python.org/3/library/math.html#math.isclose
Since they're not changing ==, I wouldn't think =< and >= would change. Note, Erlang has a no arrow looking comparison rule, so <= is not a comparison operator; actually it's used in bitstring comprehensions, the bitstring version of list comprehensions which use <- [1]
[1] https://www.erlang.org/doc/reference_manual/expressions.html...
[1] https://www.erlang.org/doc/reference_manual/expressions.html...
We don't have fast hardware decimals, except on IBM mainframes. So, if you need fast financial calculations which are too complicated for integers (e.g. gasp division), you usually use decimally-normalized doubles and do it very very carefully.
This is a sad state of affairs, but nobody was fixing this in the last 30 years, so there's that.
This is a sad state of affairs, but nobody was fixing this in the last 30 years, so there's that.
Are there even any applications where one needs a) decimal precision and b) fast computation? Financial calculations don't need to be fast.
Algorithmic trading, of course. High-frequency trading in particular. But any trading, in fact, when you run simulations for thousands of instruments and search among millions of parameters. Or when you implement an exchange, or route orders between exchanges according to their complicated criteria. Or when you do options pricing. Basically, everything in this field.
[deleted]
So something that produces nothing of value to civilization. Gotcha.
Well, I assume you want to sell your hard-earned stocks of civilization-helping companies sometimes...
High frequency traders effectively charge money to provide liquidity.
But they provide far more than I need. I'm not worried about liquidity when I eventually sell those stocks, and if I could opt out of buying extra liquidity I definitely would.
But they provide far more than I need. I'm not worried about liquidity when I eventually sell those stocks, and if I could opt out of buying extra liquidity I definitely would.
more liquidity in a security drives spreads down, not up, so you have the effect backwards
What in particular do you think I have backwards?
If I have a medium size order, then even though they lower the spread they also front-run and limit how much I can buy at that price. So I'd rather have them not be there.
If I have a tiny order, then I don't care what the spread is within reasonable bounds, and I still don't want them to be there.
If I have a medium size order, then even though they lower the spread they also front-run and limit how much I can buy at that price. So I'd rather have them not be there.
If I have a tiny order, then I don't care what the spread is within reasonable bounds, and I still don't want them to be there.
oh, well, if you're a speculator, often you really do lose money to hfts because they're better speculators than you, but it's unclear why anybody outside your immediate family should care about that
if you're an investor, otoh, the lower spread and greater liquidity means timely execution costs you less, not more. you aren't paying them for liquidity; they're paying you, or rather you're paying them, but much less than the spread you'd've paid an old-style open outcry market maker
(do you even remember markets before decimalization? minimum spread 12.5 cents)
of course you do need to execute intelligently; you can't just plop a million-dollar order in a hundred-million-a-day market and expect the market not to move against you
if you're an investor, otoh, the lower spread and greater liquidity means timely execution costs you less, not more. you aren't paying them for liquidity; they're paying you, or rather you're paying them, but much less than the spread you'd've paid an old-style open outcry market maker
(do you even remember markets before decimalization? minimum spread 12.5 cents)
of course you do need to execute intelligently; you can't just plop a million-dollar order in a hundred-million-a-day market and expect the market not to move against you
> of course you do need to execute intelligently; you can't just plop a million-dollar order in a hundred-million-a-day market and expect the market not to move against you
It's fine for the market to move against big orders, I just want the movement to, like, take one second. I don't want anyone to change their position in response to an order that hasn't even resolved.
> (do you even remember markets before decimalization? minimum spread 12.5 cents)
Well I'm not suggesting we undo that.
> if you're an investor, otoh, the lower spread and greater liquidity means timely execution costs you less, not more. you aren't paying them for liquidity; they're paying you, or rather you're paying them, but much less than the spread you'd've paid an old-style open outcry market maker
In this scenario I'm a long-term investor so the cost means nothing to me. So it's a matter of whether I want HFTs to profit, and I don't, because their actions are often not win-win. If HFT worked somewhat differently I wouldn't mind them the same way.
It's fine for the market to move against big orders, I just want the movement to, like, take one second. I don't want anyone to change their position in response to an order that hasn't even resolved.
> (do you even remember markets before decimalization? minimum spread 12.5 cents)
Well I'm not suggesting we undo that.
> if you're an investor, otoh, the lower spread and greater liquidity means timely execution costs you less, not more. you aren't paying them for liquidity; they're paying you, or rather you're paying them, but much less than the spread you'd've paid an old-style open outcry market maker
In this scenario I'm a long-term investor so the cost means nothing to me. So it's a matter of whether I want HFTs to profit, and I don't, because their actions are often not win-win. If HFT worked somewhat differently I wouldn't mind them the same way.
Nope. I'd rather there not be a thing called a stock market at all.
Do you just plan to save your money in dollar bills hidden inside a pillow until you retire? If you want to invest it in any way, you probably want to avoid floating point numbers for keeping track of it.
I'd prefer living in a world where social security is enough for me not to worry about things like "saving money".
So no pensions, sovereign wealth funds, or scholarship funds? And by what mechanism is capital allocated?
[deleted]
Well, prepare the firing squads! What the world definitely needs is more fire squads, I presume.
liquid markets are a very good thing for civilization
Business apps? Like all of them?
As matter of fact, business apps are by far the longest portion of the apps. Just considering all spreadsheets, almost all RDBMS, etc alone.
Binary Floating-point is what is ACTUALLY niche. And it not look like it just because is the default, similar how in certain niches "0" is "true".
As matter of fact, business apps are by far the longest portion of the apps. Just considering all spreadsheets, almost all RDBMS, etc alone.
Binary Floating-point is what is ACTUALLY niche. And it not look like it just because is the default, similar how in certain niches "0" is "true".
Most business apps don't do anywhere near enough compute that it matters.
Gambling.
I’m not qualified to evaluate it, but Douglas Crockford has at least made a proposal: https://www.crockford.com/dec64.html
Power ISA includes decimal floating point, so it's available beyond mainframes.
https://en.wikipedia.org/wiki/Power_ISA
https://en.wikipedia.org/wiki/Power_ISA
I would add generating large financial reports, enterprise billing etc. There is a huge business effect between for example 2 hours needed to calculate monthly invoices and 24 hours.
Is any of that actually bottlenecked by arithmetic rather than pointer chasing?
Probably more likely inefficient data access patterns, such as N+1 queries, inefficient algorithms, lack of parallelism
no, 2 hours of addition on my cellphone at 2 gigahertz, 4 cores, and conservatively 3 additions per core per cycle would be 170 trillion additions
nobody needs 170 trillion additions for their nightly invoices
nobody needs 170 trillion additions for their nightly invoices
I'm assuming you mean equality comparison, rather than comparison in general!
but is there a better algebraic structure to use when thinking of floats? like, in terms of limits, or something that tracks significant digits and uncertainty? how do formal methods handle these?
but is there a better algebraic structure to use when thinking of floats? like, in terms of limits, or something that tracks significant digits and uncertainty? how do formal methods handle these?
There's more than one way to skin that cat, but interval arithmetic is a good and simple model. You can take the part of the real line which maps to the single float you are thinking about, and then look at the image of that set under the whatever functions you are thinking about.
> if you find yourself comparing floating point numbers, that's probably not what you want
Can you motivate this with an example for me? For instance, I think of games or location data encoded as FP; then, clearly, comparing them is a critical task to know questions like "what is closer" and so on.
What am I missing?
Can you motivate this with an example for me? For instance, I think of games or location data encoded as FP; then, clearly, comparing them is a critical task to know questions like "what is closer" and so on.
What am I missing?
They meant using the equal operator, because floating point is inexact and can produce different representations for the same number depending on how it was obtained. Greater and less than are fine. Equal is usually implemented by seeing if a number fits inside a tight range.
Every finite floating point value precisely represents an exact quantity. Arithmetic isn't lossless though. This isn't a special property of floats. Decimals behave the same way.
I was with you until the end
First point: FPs are unique. Clearly true
Second point: Combing FPs isn't lossless, across most/all arithmetic. Clearly true, the root of our discussion
Third point: decimal arithmetic is lossy too. Strongly disagree. The system of representation is what is lossy - floating point. Arithmetic (between two decimal numbers) is clearly not lossy in and of itself, only as implemented with computers.
First point: FPs are unique. Clearly true
Second point: Combing FPs isn't lossless, across most/all arithmetic. Clearly true, the root of our discussion
Third point: decimal arithmetic is lossy too. Strongly disagree. The system of representation is what is lossy - floating point. Arithmetic (between two decimal numbers) is clearly not lossy in and of itself, only as implemented with computers.
Write 1/3 in decimal. You're required to do it in finite memory.
Decimal is only lossless if you track an indefinitely-growing suffix to your number with a big bar over it to indicate "repeating". It's basically working with rational numbers but more painful.
Decimal numbers with any limit on digits, even a hundred, are lossy.
Decimal numbers as used by humans outside of computers have limits on digits, so they are lossy.
Decimal numbers with any limit on digits, even a hundred, are lossy.
Decimal numbers as used by humans outside of computers have limits on digits, so they are lossy.
Maybe my point is uninteresting, but I feel you haven't really understood it.
Combining two FP numbers with arithmetic leads to possible errors that you can't represent with FP on every operation.
Combining two decimal numbers with non-division arithmetic never leads to that case. With division, sure, things are bad, but that's more of an exception than a rule to me.
This is because decimal numbers don't really come with any inherent limit on digits, and it's a bit strange to add a clause to the other's claims before making a counterargument.
Combining two FP numbers with arithmetic leads to possible errors that you can't represent with FP on every operation.
Combining two decimal numbers with non-division arithmetic never leads to that case. With division, sure, things are bad, but that's more of an exception than a rule to me.
This is because decimal numbers don't really come with any inherent limit on digits, and it's a bit strange to add a clause to the other's claims before making a counterargument.
Well in part I didn't understand your point because you didn't mention division as an exception.
But even then, multiplication causes an explosion in number length if you do it repeatedly.
When you specifically talk about numbers not being in computers, I think it's fair to talk about digit limits. Most real-world use of decimals is done with less precision than the 16 digits we default to in computers. Let alone growing to 50, 100, 200, etc as you keep multiplying numbers together to perform some kind of analysis. Nobody uses decimal like that. Real-world decimal is lossy for a large swath of multiplication.
I agree that if you're doing something like just adding numbers repeatedly in decimal, and those numbers have no repeating digits, then you have a nice purity of never losing precision. That's worth something. But on the other hand if you started with the same kind of numbers in floating point, let's say about 9 digits long, you could still add a million of them without losing precision.
And nobody has said anything about irrational numbers as you dismissed in your other comment.
So in summary: decimal division, usually lossy; decimal multiplication, usually lossy; decimal addition and subtraction, lossless but with the same kind of source numbers FP is usually lossless too
But even then, multiplication causes an explosion in number length if you do it repeatedly.
When you specifically talk about numbers not being in computers, I think it's fair to talk about digit limits. Most real-world use of decimals is done with less precision than the 16 digits we default to in computers. Let alone growing to 50, 100, 200, etc as you keep multiplying numbers together to perform some kind of analysis. Nobody uses decimal like that. Real-world decimal is lossy for a large swath of multiplication.
I agree that if you're doing something like just adding numbers repeatedly in decimal, and those numbers have no repeating digits, then you have a nice purity of never losing precision. That's worth something. But on the other hand if you started with the same kind of numbers in floating point, let's say about 9 digits long, you could still add a million of them without losing precision.
And nobody has said anything about irrational numbers as you dismissed in your other comment.
So in summary: decimal division, usually lossy; decimal multiplication, usually lossy; decimal addition and subtraction, lossless but with the same kind of source numbers FP is usually lossless too
> Combining two decimal numbers with non-division arithmetic never leads to that case.
Most types are subject to overflow that has similar effects. Most of the FP error people encounter is actually division error. For example, the constant 0.1 contains an implicit division. The "tenths" place is defined by division by 10. I think that almost all perceptions of floating point lossiness come from this fact.
Most types are subject to overflow that has similar effects. Most of the FP error people encounter is actually division error. For example, the constant 0.1 contains an implicit division. The "tenths" place is defined by division by 10. I think that almost all perceptions of floating point lossiness come from this fact.
Likewise, bit-limited integers are lossy as they are incomplete representations of ℤ, and are not reciprocal with division. The point being that, just like floating-point representations of ℝ, division will always be lossy in the general case unless you choose a symbolic algebra instead of numeric.
you seem to be confusing the decimal system of representation with platonic abstract numbers, which is what gives rise to the apparent disagreement
Do I?
I think people are speaking shockingly imprecisely in this thread. here's my claim:
> Arithmetic (between two decimal numbers) is clearly not lossy in and of itself, only as implemented with computers
Counterarguments are basically on the level of "you cant represent Pi as a finite decimal number", which I find uninteresting.
I think people are speaking shockingly imprecisely in this thread. here's my claim:
> Arithmetic (between two decimal numbers) is clearly not lossy in and of itself, only as implemented with computers
Counterarguments are basically on the level of "you cant represent Pi as a finite decimal number", which I find uninteresting.
that's slightly incoherent and reflective of the same confusion i identified; there is strictly speaking, no such thing as a decimal number, only a decimal numeral. normally the difference is subtle, but in this case it's the essence of the issue
'decimal' means 'base 10'
numbers, the abstract entities we do arithmetic on, are not decimal or in any other base; it is correct that arithmetic on them is not in any sense lossy
decimal is a system of representation that represents these abstract numbers as finite strings of digits; these are called 'numerals'. it can represent any integer, but only some fractions, those whose denominator has no prime factors other than 2 or 5. such fractions arise non-lossily as the result of the arithmetic operator of division when, for example, the dividend is 3. representing these in decimal requires rounding them, so decimal is lossy
binary floating point is lossy in the same way, with the additional limitation to only being able to represent fractions whose denominator is a power of 2, whose numerator does not have too many significant bits (53 most often), and which are not too positive, too negative, or too small
there are other systems of represention for numbers that are not lossy in this way. for example, improper fractions, mixed numbers, finite continued fractions, and decimal augmented with overbar to indicate repeated digits
'decimal' means 'base 10'
numbers, the abstract entities we do arithmetic on, are not decimal or in any other base; it is correct that arithmetic on them is not in any sense lossy
decimal is a system of representation that represents these abstract numbers as finite strings of digits; these are called 'numerals'. it can represent any integer, but only some fractions, those whose denominator has no prime factors other than 2 or 5. such fractions arise non-lossily as the result of the arithmetic operator of division when, for example, the dividend is 3. representing these in decimal requires rounding them, so decimal is lossy
binary floating point is lossy in the same way, with the additional limitation to only being able to represent fractions whose denominator is a power of 2, whose numerator does not have too many significant bits (53 most often), and which are not too positive, too negative, or too small
there are other systems of represention for numbers that are not lossy in this way. for example, improper fractions, mixed numbers, finite continued fractions, and decimal augmented with overbar to indicate repeated digits
this should read 'when, for example, the divisor is 3 and the dividend is an integer not divisible by 3'
IMHO That was poorly worded. What it should have said is "if you find yourself using equals to compare floating point numbers...". With Floating Point your comparisons should always be less than or greater than. Precision artifacts make the equals unreliable, and you should always be mindful of that when dealing with them.
Caveat: In the following, I use "comparison" to mean "check for equality".
Floating point numbers lose precision because binary arithmetic doesn't represent decimal in all cases ( 1/3 is an easy example). It's not hard to get into a situation where you're asking if a number is 0.0 but due to a precision error the number you have is 0.00000000000001 or whatever and it should have been zero.
If you're dealing with anything where precision is paramount (i.e. money or high precision machining) you should consider using something else that matches the real work precision you are trying to model.
You can, of course, use floats and test differences rather than strict equality (i.e. distance is < 0.01) and that's fine too... if you remember to do it, and account for the potential for small precision differences to propagate throughout your calculations and accumulate into larger errors.
Floating point numbers lose precision because binary arithmetic doesn't represent decimal in all cases ( 1/3 is an easy example). It's not hard to get into a situation where you're asking if a number is 0.0 but due to a precision error the number you have is 0.00000000000001 or whatever and it should have been zero.
If you're dealing with anything where precision is paramount (i.e. money or high precision machining) you should consider using something else that matches the real work precision you are trying to model.
You can, of course, use floats and test differences rather than strict equality (i.e. distance is < 0.01) and that's fine too... if you remember to do it, and account for the potential for small precision differences to propagate throughout your calculations and accumulate into larger errors.
> Floating point numbers lose precision because binary arithmetic doesn't represent decimal in all cases.
That’s not the reason. A counterexample is decimal arithmetic. That represents decimal in all cases, yet loses precision when doing calculations. Once you start doing division that applies even to arbitrarily sized decimal numbers.
The correct reason is because the subset of the reals that is representable as IEEE float isn’t closed under the operations people want to perform on them, including the basic ones of addition, multiplication and division.
That’s not the reason. A counterexample is decimal arithmetic. That represents decimal in all cases, yet loses precision when doing calculations. Once you start doing division that applies even to arbitrarily sized decimal numbers.
The correct reason is because the subset of the reals that is representable as IEEE float isn’t closed under the operations people want to perform on them, including the basic ones of addition, multiplication and division.
> Caveat: In the following, I use "comparison" to mean "check for equality".
You never use the word “comparison” again in the comment!
You never use the word “comparison” again in the comment!
Ideally, you'll want to do distance calculations in fixed point numbers (of desired resolution). Floating point works well as a first approximation, but unless you can make sure that you are not going to end up wandering in the weeds (and ensuring that required understanding of computational numerics), you should probably replace them with fixed point once you understand the problem and the solution and know the limits involved.
It's the hard task of trying to figure out the magnitude of the expected errors (which can accumulate), in the simplest case of a single operation you compare within epsilon distance :
https://en.wikibooks.org/wiki/Floating_Point/Epsilon
https://en.wikibooks.org/wiki/Floating_Point/Epsilon
Common way of comparing floats is to compare the difference to some epsilon value. See Python PEP-485 for example https://peps.python.org/pep-0485/
[deleted]
This is specifically about equality. Comparing FPs for equality is very risky, as your numbers can differ by 0.00000000000001 without anyone noticing.
Strict inequality (> and <) comparison are generally fine as long as you avoid NaNs.
Strict inequality (> and <) comparison are generally fine as long as you avoid NaNs.
Ok, sure, so it was just for a specific case of comparison.
Well, followup, if some error Epsilon can be introduced during any manipulations, when you check for strict inequalities do you check x < y + Epsilon? Does the language implicitly do it for you?
Well, followup, if some error Epsilon can be introduced during any manipulations, when you check for strict inequalities do you check x < y + Epsilon? Does the language implicitly do it for you?
The language just gives you the direct comparison.
If you know numerics, then you can come up with the correct value for epsilon. But that is hard work. There are various things that the language could do for you, like use a dynamic amount of precision, or interval arithmetic where the error bounds are saved—but! Most of the time people just want the answer faster and with less memory used, which is what you get with bare floats. The people who make it their job to care about numerical accuracy can do it better than the language runtime would anyway.
Most of the problems are more easily solvable at a higher level anyway. Like, imagine Clippy saying to you, “It looks like you’re inverting a matrix. Are you sure that’s a good idea?”
If you know numerics, then you can come up with the correct value for epsilon. But that is hard work. There are various things that the language could do for you, like use a dynamic amount of precision, or interval arithmetic where the error bounds are saved—but! Most of the time people just want the answer faster and with less memory used, which is what you get with bare floats. The people who make it their job to care about numerical accuracy can do it better than the language runtime would anyway.
Most of the problems are more easily solvable at a higher level anyway. Like, imagine Clippy saying to you, “It looks like you’re inverting a matrix. Are you sure that’s a good idea?”
This smells like the right way to think about things. Whenever thinking of real life measurements represented as real numbers, equality is always fuzzy — up to some resolution, and it would be better to make that explicit.
Floating point numbers are dangerous but in 2023 we should find a way to improve the side effects.
Playing with Python3:
>>> +0.0==-0.0 True
The C in GCC below also returns 1:
#include <stdio.h>
int main()
{
Playing with Python3:
>>> +0.0==-0.0 True
The C in GCC below also returns 1:
#include <stdio.h>
int main()
{
float a = +0.0;
float b = -0.0;
printf("%d\n", a == b);
}Erlang too, now and in the future will say that minuszero==zero, it's a different more specific operator that's changing.
You'll get the same in python with
>>> -0. is 0.
False
You'll get the same in python with
>>> -0. is 0.
False
The `is` operator is checking object identity by comparing addresses. It is useless, although it may sometimes produce reasonable-looking results due to optimisations.
Python 3.11 gives me this:
>>> -0 is 0 <stdin>:1: SyntaxWarning: "is" with a literal. Did you mean "=="? True
Because (a) -0 is an integer, and there is only one integer zero, (b) small numbers have only one instance in memory as an optimization.
Python 3.11 gives me this:
>>> -0 is 0 <stdin>:1: SyntaxWarning: "is" with a literal. Did you mean "=="? True
Because (a) -0 is an integer, and there is only one integer zero, (b) small numbers have only one instance in memory as an optimization.
this is not about arithmetic comparison, it's about erlang's exact-equality operator
It is 2023 and our tooling still encourages the same mistakes people were making 40 years ago. Can we really not have equality operators that do a comparison with a 1% tolerance or something as a sensible default equivalence instead of blatantly wrong bitwise comparison?
I would even be happy with a default set of compiler warnings or errors, which I don’t believe I have ever seen.
I would even be happy with a default set of compiler warnings or errors, which I don’t believe I have ever seen.
From a user POV, I think == and === on floats should simply be undefined in any language. It should be a compile time or runtime error.
There can be a separate function `float_equals()` with explicit args that does what people want
The only reason to use the same syntax == is for POLYMORPHIC code that is actually correct.
But it's not going to be correct with floats, because they don't obey the same algebraic laws ... So the syntax should be different!
---
I believe the Erlang compiler optimization presented as justification for this change is a good example
The compiler wants to reason about code, independent of types
But that reasoning about the =:= operator is wrong for the float case.
There can be a separate function `float_equals()` with explicit args that does what people want
The only reason to use the same syntax == is for POLYMORPHIC code that is actually correct.
But it's not going to be correct with floats, because they don't obey the same algebraic laws ... So the syntax should be different!
---
I believe the Erlang compiler optimization presented as justification for this change is a good example
The compiler wants to reason about code, independent of types
But that reasoning about the =:= operator is wrong for the float case.
what people want varies, but often what people doing numerical programming want is to write their monomorphic code in infix syntax so they can more easily see bugs in it
they're already familiar with rounding errors
from that perspective you're suggesting taking a step backwards from fortran i toward assembly language
the erlang compiler bug presented is not an example of what you're talking about because you're talking about arithmetic, and the buggy comparison operator it's using is not an arithmetic comparison operator
they're already familiar with rounding errors
from that perspective you're suggesting taking a step backwards from fortran i toward assembly language
the erlang compiler bug presented is not an example of what you're talking about because you're talking about arithmetic, and the buggy comparison operator it's using is not an arithmetic comparison operator
Fortran would be an interesting case, because say C++ is polymorphic, and so are all dynamic programming languages with more than 1 type (x == y could be a string or float comparison)
But I would still say that floating point equality is a vanishingly rare operation
I can't think of any real use case for it, other than maybe some (bad, incomplete) unit tests that assert 1.0 == x and assert 1.0 == y
And that use case is perfectly served by float_equals(), and arguably served better if it has some options. Although probably abs(x - y) < eps is just as good, and you don't even need float_equals()
---
Can you show some real use cases for floating point equality in good, production code? (honest question)
It's similar to hashing floats, which Go and Python do allow. I made an honest request for examples of float hashing:
https://lobste.rs/s/9e8qsh/go_1_21_may_have_clear_x_builtin#...
https://old.reddit.com/r/ProgrammingLanguages/comments/10bm2...
I didn't get any answers that appeared realistic. Some people said you might want to create a histogram of floats -- but that's obviously better served by bucketing floats, so you're hashing integers.
Another person said the same thing for quantizing to mesh.
One person suggested fraud detection for made-up values, but that was clearly addressed by converting the float to a bit pattern, and hashing the bit pattern.
----
This isn't a theoretical question since we're working on this for https://www.oilshell.org.
For that case it seems pretty clearly OK to omit === on floats, since I use awk and R for simple stats and percentages, and that's likely what people would do with a shell.
Though as always I'm open to concrete counterexamples.
But I would still say that floating point equality is a vanishingly rare operation
I can't think of any real use case for it, other than maybe some (bad, incomplete) unit tests that assert 1.0 == x and assert 1.0 == y
And that use case is perfectly served by float_equals(), and arguably served better if it has some options. Although probably abs(x - y) < eps is just as good, and you don't even need float_equals()
---
Can you show some real use cases for floating point equality in good, production code? (honest question)
It's similar to hashing floats, which Go and Python do allow. I made an honest request for examples of float hashing:
https://lobste.rs/s/9e8qsh/go_1_21_may_have_clear_x_builtin#...
https://old.reddit.com/r/ProgrammingLanguages/comments/10bm2...
I didn't get any answers that appeared realistic. Some people said you might want to create a histogram of floats -- but that's obviously better served by bucketing floats, so you're hashing integers.
Another person said the same thing for quantizing to mesh.
One person suggested fraud detection for made-up values, but that was clearly addressed by converting the float to a bit pattern, and hashing the bit pattern.
----
This isn't a theoretical question since we're working on this for https://www.oilshell.org.
For that case it seems pretty clearly OK to omit === on floats, since I use awk and R for simple stats and percentages, and that's likely what people would do with a shell.
Though as always I'm open to concrete counterexamples.
i gave some examples in https://news.ycombinator.com/item?id=35883963, though arguably those are not very concrete. also i pointed out there why unit tests that use exact comparison on floats are not necessarily bad: ieee-784 specifies bit-exact results for the five fundamental arithmetic operations, and if your compiler is introducing rounding errors, you want your tests to fail. (or introducing nonerrors, in the case where it's introducing fmas.)
even hashing is useful for memoization
i think probably you're asking in the wrong places
the lapack source might be a better place to look; we can probably suppose that if something is in lapack it's realistic and also not a bug, and numerical code doesn't get more 'good, production' than lapack
in dgelsx for example i find
and in dlar1v:
but if you ask an audience of type theorists, sysadmins, and web developers, they won't know that; they aren't numerical analysts and probably haven't inverted a submatrix since sophomore math class, if ever
(there are 6550 files in lapack 3.9.0 of which i looked through 15 to find these examples, suggesting that there are on the order of 1000 realistic examples in there; eventually i'd probably find one that wasn't even comparing to zero)
plausibly instead of giving them floating point by default you should give them fixed point with, say, nine digits after the decimal point; that was the approach knuth took in tex, for example, to guarantee reproducibility, but it also gives human-predictable rounding. and in most cases it should be perfectly adequate for simple stats and percentages. as a bonus you get 64 bits of precision instead of 53
even hashing is useful for memoization
i think probably you're asking in the wrong places
the lapack source might be a better place to look; we can probably suppose that if something is in lapack it's realistic and also not a bug, and numerical code doesn't get more 'good, production' than lapack
in dgelsx for example i find
ELSE IF( ANRM.EQ.ZERO ) THEN
*
* Matrix all zero. Return zero solution.
*
CALL DLASET( 'F', MAX( M, N ), NRHS, ZERO, ZERO, B, LDB )
* Determine RANK using incremental condition estimation
*
WORK( ISMIN ) = ONE
WORK( ISMAX ) = ONE
SMAX = ABS( A( 1, 1 ) )
SMIN = SMAX
IF( ABS( A( 1, 1 ) ).EQ.ZERO ) THEN
RANK = 0
CALL DLASET( 'F', MAX( M, N ), NRHS, ZERO, ZERO, B, LDB )
GO TO 100
ELSE
RANK = 1
END IF
and in dlar1v:
WORK( INDS+I ) = S*WORK( INDLPL+I )*L( I )
IF( WORK( INDLPL+I ).EQ.ZERO )
$ WORK( INDS+I ) = LLD( I )
TMP = D( I ) / DMINUS
IF(DMINUS.LT.ZERO) NEG2 = NEG2 + 1
WORK( INDUMN+I ) = L( I )*TMP
WORK( INDP+I-1 ) = WORK( INDP+I )*TMP - LAMBDA
IF( TMP.EQ.ZERO )
$ WORK( INDP+I-1 ) = D( I ) - LAMBDA
IF( ABS(MINGMA).EQ.ZERO )
$ MINGMA = EPS*WORK( INDS+R1-1 )
but if you ask an audience of type theorists, sysadmins, and web developers, they won't know that; they aren't numerical analysts and probably haven't inverted a submatrix since sophomore math class, if ever
(there are 6550 files in lapack 3.9.0 of which i looked through 15 to find these examples, suggesting that there are on the order of 1000 realistic examples in there; eventually i'd probably find one that wasn't even comparing to zero)
plausibly instead of giving them floating point by default you should give them fixed point with, say, nine digits after the decimal point; that was the approach knuth took in tex, for example, to guarantee reproducibility, but it also gives human-predictable rounding. and in most cases it should be perfectly adequate for simple stats and percentages. as a bonus you get 64 bits of precision instead of 53
Thanks for actually going and looking! I would say that this supports the idea of float_equals(x, 0.0) or even float_equals_zero(x) :)
It wouldn't be too surprising to me if 90% of float equalities in real code are comparisons to 0.0 or 1.0 or -1.0 (that's what I meant in my original post, there was a typo)
The point about x >= 0 and x <= 0 is interesting. I suppose to be consistent, you would also remove those operators on floats, although for something awk-like, I don't think it matters either way.
---
The other point is that it looks like the Fortran code isn't actually using == or x.eq.zero polymorphically!
So the first point is that I would want to understand conceptually where float equality would be equal. You mentioned fixed points, although there I also suspect that abs(x - y) < eps is a better test in most situations, though certainly there could be counterexamples.
The second point is -- would any of those situations be polymorphic with int and float, or string and float? It depends on the language, but it seems vanishingly unlikely in most languages, and in particular in say in C++
I would probably go look at the equivalent Eigen code if it mattered with respect to Oil, but it probably doesn't
It wouldn't be too surprising to me if 90% of float equalities in real code are comparisons to 0.0 or 1.0 or -1.0 (that's what I meant in my original post, there was a typo)
The point about x >= 0 and x <= 0 is interesting. I suppose to be consistent, you would also remove those operators on floats, although for something awk-like, I don't think it matters either way.
---
The other point is that it looks like the Fortran code isn't actually using == or x.eq.zero polymorphically!
So the first point is that I would want to understand conceptually where float equality would be equal. You mentioned fixed points, although there I also suspect that abs(x - y) < eps is a better test in most situations, though certainly there could be counterexamples.
The second point is -- would any of those situations be polymorphic with int and float, or string and float? It depends on the language, but it seems vanishingly unlikely in most languages, and in particular in say in C++
I would probably go look at the equivalent Eigen code if it mattered with respect to Oil, but it probably doesn't
yeah, i also wrote a cube root routine using newton's method yesterday and came to the conclusion that |x - x'| < threshold was a better approach in that case. maybe in every case, maybe not
i agree that prefix notation is not as bad for these as i had thought before looking
i think there are plausible cases where your float might be polymorphic with double, complex, a vector, a matrix, or an element of a galois field, but i think it would only be polymorphic with int in cases like stopping change propagation when an observable value gets recomputed to the same value from new inputs, and in those cases what you need is exact bitwise equality, not arithmetic equality
an awk-like thing might be better with fixed-point, like tex and metafont; you could call the data type 'btc` and represent it as a 64-bit integer that's multiplied by 10*-8 for output
i agree that prefix notation is not as bad for these as i had thought before looking
i think there are plausible cases where your float might be polymorphic with double, complex, a vector, a matrix, or an element of a galois field, but i think it would only be polymorphic with int in cases like stopping change propagation when an observable value gets recomputed to the same value from new inputs, and in those cases what you need is exact bitwise equality, not arithmetic equality
an awk-like thing might be better with fixed-point, like tex and metafont; you could call the data type 'btc` and represent it as a 64-bit integer that's multiplied by 10*-8 for output
see also https://news.ycombinator.com/item?id=35902861 for more examples
The tolerance you need depends on how you calculated the number, so there's no way to have a sensible default.
The problem with defining an “epsilon” (1% in your case) is that there is no value that would please everyone. For some, 1% will be fine, but others may need 0.0001%.
The solution is to either use decimal floats if they suit your need (and eat the performance penalty), or to use a linter that flags float comparisons by equality.
The solution is to either use decimal floats if they suit your need (and eat the performance penalty), or to use a linter that flags float comparisons by equality.
That’s bad enough, but it is not the problem with defining an “epsilon”, it’s a problem.
Another one is that you would lose transitivity on equality. With a 1% epsilon, you would have
You also lose that, if a == b and c == d it follows that a + c == b + d.
The behavior around zero also will go against intuition. If you consider 0.99 and 1.0 to be equal, do you really want 1.0E-100 and -1.0E-100 be different?
Another one is that you would lose transitivity on equality. With a 1% epsilon, you would have
1.000 == 1.005
1.005 == 1.010
1.010 == 1.015
1.000 != 1.015
You also lose that, if a == b and c == d it follows that a + c == b + d.
The behavior around zero also will go against intuition. If you consider 0.99 and 1.0 to be equal, do you really want 1.0E-100 and -1.0E-100 be different?
And then there's the question of how you use the epsilon - i.e. whether 1.2e-16 and 1e-20 are "within 1%". Sometimes those are very much different sizes, but if you're comparing sin(π) (i.e. the 1.2e-16) with some other almost-zero computation, they're very much within 1%.
> Can we really not have equality operators that do a comparison with a 1% tolerance or something as a sensible default equivalence instead of blatantly wrong bitwise comparison?
Then it wouldn't really be an equality comparison any more. For instance according to your rule 1.0 and 1.01 are the same number, but they clearly are not. But if you want to do this regularly you could create a special type and overload the equality operator (if your language supports it, otherwise you'll have to call the function directly) that points to 'boolean about_equal(float f1, float f2, float epsilon)' or something equivalent where epsilon is the amount that you would allow f1 and f2 to deviate from each other to still call them equal. And you could define epsilon as ((abs(f1) + abs(f2)) / 100) .
The bitwise comparison isn't 'blatantly wrong', it's the expected behavior in just about every programming language. I'm not aware of any language that has a native float operator that does this but as you can see you can usually simply add one yourself if you really need it. But this need rarely comes up and usually indicates that you are doing something wrong.
Then it wouldn't really be an equality comparison any more. For instance according to your rule 1.0 and 1.01 are the same number, but they clearly are not. But if you want to do this regularly you could create a special type and overload the equality operator (if your language supports it, otherwise you'll have to call the function directly) that points to 'boolean about_equal(float f1, float f2, float epsilon)' or something equivalent where epsilon is the amount that you would allow f1 and f2 to deviate from each other to still call them equal. And you could define epsilon as ((abs(f1) + abs(f2)) / 100) .
The bitwise comparison isn't 'blatantly wrong', it's the expected behavior in just about every programming language. I'm not aware of any language that has a native float operator that does this but as you can see you can usually simply add one yourself if you really need it. But this need rarely comes up and usually indicates that you are doing something wrong.
Negative zero is very, very annoying because it means that many floating point identities flies out of the window. x + 0.0 != x, x * 0.0 != 0.0, etc. It makes it much harder for the compiler to optimize arithmetic. Thus, at least some SIMD circuits do not follow ieee754 to the letter because it would result in performance degradations.
I find it so strange that IEEE-754 SQRT(-0.) is -0., even though (-0.)*(-0.) is +0.
This feels like a bad idea. Treating +0 and -0 as unequal because they have different bit patterns is no more correct than treating two NaNs as equal because they have the same bit pattern. Zero is zero.
In case of floating points, since they are an approximation, zero is not zero. Zero is a quantity approaching to zero. In fact dividing by zero with floats is entirely possible! (it leads to infinite, positive or negative depending on the sign).
For example, the quantity (try it in Python):
Of course for integers -0 doesn't make sense (that is the reason why we invented 2 complement, to not have 2 "zeros")
For example, the quantity (try it in Python):
0.1 ** 100 / 1000**100 evaluates to 0.0
0.1 ** 100 / (-1000**100) evaluates to -0.0
Of course for integers -0 doesn't make sense (that is the reason why we invented 2 complement, to not have 2 "zeros")
> no division can produce a result that is exactly zero
Except 0 / n
Except 0 / n
Or -0.0 / n, for what it's worth, at which point these zeroes will not be equal to one another.
Or 1e-100 / 1e300
Please refer back to "It's clear that neither the two quantities are 0 (no division can produce a result that is exactly zero!) but they approach the zero." because you just gave an example of how that rule is correct, not an exception.
Ok, I should have specified unless the numerator is not exactly zero!
> Zero is a quantity approaching to zero
no, its not. zero is zero, its nothing. "a quantity approaching to zero" is "a quantity approaching to zero". its easy to remember because zero is not a quantity, its a lack of quantity, by definition.
these are two distinct concepts, which some people mistake as the same item.
no, its not. zero is zero, its nothing. "a quantity approaching to zero" is "a quantity approaching to zero". its easy to remember because zero is not a quantity, its a lack of quantity, by definition.
these are two distinct concepts, which some people mistake as the same item.
Not with floating point numbers.
That sounds wrong to me. In IEEE floating point, 0.0 is not an approximation of zero, it is zero exactly. The binary value is literally all zero bits. Negative zero appears simply from the IEEE floating point format reserving a bit for the sign.
IEEE 754’s idea was that if you got a zero in a floating-point calculation (which, at the time, meant one with physical quantities), it’s probably underflow, not a legitimate result, so the FPU might as well try to give you at least a sign even if it can’t give you any significant digits. Though 754 says comparison must treat positive and negative zero as equal nevertheless, you have to deliberately choose to look at the sign (with e.g. C99’s copysign) to see the difference.
Zero is zero, but signs are a direction. There are multiple parts to numbers.
Stand in place and turn around: how far did you go from where you started?
Nonetheless, I agree. It's a bad idea.
Stand in place and turn around: how far did you go from where you started?
Nonetheless, I agree. It's a bad idea.
By that logic there should be e.g. +4.0 and +4.0- (for when you go 4m forwards vs when you go 4m forwards but also turn around at the end).
In vector maths, this would just be +4.0.
And the same is true of zero.
Especially when you consider a two-dimensional vector. How would you encode/interpret facing in a complex zero?
Especially when you consider a two-dimensional vector. How would you encode/interpret facing in a complex zero?
What Every Programmer Should Know About Floating-Point Arithmetic
https://floating-point-gui.de/
https://floating-point-gui.de/
Unless you’re approaching the limit of 0 from one side
But yeah, you’re right
But yeah, you’re right
Can anyone fill me in what the meaning of == vs =:= in Erlang is? Since after the change +0. == -0. but not +0. =:= -0.
=:= is supposed to be "exactly equals". An example given in the docs[1]:
[1]: https://www.erlang.org/doc/reference_manual/expressions.html...
1> 1==1.0.
true
2> 1=:=1.0.
false
[1]: https://www.erlang.org/doc/reference_manual/expressions.html...
This makes sense. They are different entities and you have to have a way to acknowledge that they are. Breaking strict equality seems a fitting way to do so.
In Geometry for Programmers, I reference a case when an automated test system caught a "zero-not-zero" bug, and I couldn't reproduce it with a unit test until I finally realized that the test system was comparing XMLs not numbers. In print, "-0." and "0." are, of course, different, although in runtime they pretend to be the same.
In Geometry for Programmers, I reference a case when an automated test system caught a "zero-not-zero" bug, and I couldn't reproduce it with a unit test until I finally realized that the test system was comparing XMLs not numbers. In print, "-0." and "0." are, of course, different, although in runtime they pretend to be the same.
As someone who programmed assembly on a one's complement machine* 40 years ago, this discussion is interesting, from both the "haven't we learned anything" and the "makes sense to me" camps.
More interesting to me is how they are introducing this change, both in the previous OTP and next, and how they will arm people with tools to hopefully identify and understand the impact. I wonder how many folks will actually be impacted?
* Sperry Univac 1100/62
More interesting to me is how they are introducing this change, both in the previous OTP and next, and how they will arm people with tools to hopefully identify and understand the impact. I wonder how many folks will actually be impacted?
* Sperry Univac 1100/62
What about NaN? Does Erlang have NaN? How does Erlang compare two variables containing a NaN (or expressions evaluating to a NaN)? There are (2^24)-2 different NaNs in a 32bit IEEE 754 float, and there are (2^53)-2 different NaNs in a 64bit IEEE 754 double. The IEEE 754 standard says, that a NaN value should compare to any other number (including any other NaN) as not equal. Yes, the expression `x == x` could return `false`.
Erlang has =:= and ==. Kind of like how Python has is and ==, JavaScript has == and ===, and Lisp has eq, eql, and equal.
The change in the article is about =:=, so NaN =:= NaN does not have to give the same answer as NaN == NaN.
The change in the article is about =:=, so NaN =:= NaN does not have to give the same answer as NaN == NaN.
Quite amazing how so many environments of floating point values make the mistake of treating them as equal. YAML 1.2 does the same thing:
https://yaml.org/spec/1.2.2/#10214-floating-point
https://yaml.org/spec/1.2.2/#10214-floating-point
Is it surprising that so many environments that use IEEE 754 floating point numbers obey IEEE 754 on equality?
IEEE 754-2008 5.11: "Comparisons shall ignore the sign of zero (so +0 = −0)."
IEEE 754-2008 5.11: "Comparisons shall ignore the sign of zero (so +0 = −0)."
Because they are equal in real math
Floats are not real numbers though
They are a way of approximating Real (the set not as an adjective) numbers, though. Real numbers are exact when they are equal to an integer or a rational, because, depending on how you construct them, either that's what the cut contains or that's where the series converges.
And also because that's what the most common spec demands.
And also because that's what the most common spec demands.
[deleted]
Floats are a subset of the reals.
floating point math is math like any other, just with a vastly different number system. There are valid arguments on both side if we should hide or expose those differences in the implementation, but there's always going to be pretty big implications (i.e. the compiler behaviour mentioned here)
Yes but this is floating point - +0.0 represents all numbers between 0 and the smallest +ve FP number, -0.0 represents all numbers between 0 and the smallest -ve FP number - both can represent actual 0 - remember these can be generated by underflow so those meanings can be important in context
One wonders how different the world would be if IEEE754 decided to use two's complement instead of sign-magnitude. Or alternatively, if two's complement was rare and almost all mainstream hardware (due to some historical reason with inertia) was sign-magnitude.
Hot take as a non-user of Erlang. There may be arguments for and against 0.0 =:= -0.0. You could probably do either. But I would be terrified of changing from one to another.
A lot of people are arguing that 0 is not a distinct number from -0 in regular math. But regular math doesn't have +Inf, -Inf, or NaN either.
Obscure Factoid: It is illegal to display -0 on a scale in Canada. Ran into that long ago when designing electronics for a scale.
I would have guessed that a backwards incompatible change like that is a big release thing. Even if it affects a tiny tiny group
It is, hence why it will be in a major release in more than a year, with an intermediate warning mode in releases before that.
-0.0 = 0
+0.0 = 0
Hence -0.0 = +0.0 right ?
+0.0 = 0
Hence -0.0 = +0.0 right ?
Yup - both of the bit patterns for 754 here were assigned the value of 0 and the standard instructed comparison operators to treat the two values as equal. The bit patterns for both -0.0 and +0.0 are assigned to the same real world value - there is no reason to distinguish them outside of circuits that actually execute floating point arithmetic.
it's important to distinguish them in programs written for those circuits and in compilers that transform those programs because they have significantly and intentionally different behavior
Give Erlang/Elixir the full JavaScript!