Magician-turned-mathematician uncovers bias in coin flipping (2004)(news.stanford.edu)
news.stanford.edu
Magician-turned-mathematician uncovers bias in coin flipping (2004)
https://news.stanford.edu/pr/2004/diaconis-69.html
127 comments
The method to get fair flips from an unfair coin is interesting and was recently on hackernews. say a coin is heads(H) 70% of the time. that means it must be tails(T) 30% of the time. If you flip it twice, and calculate the odds of each pair of flips (multiply the probabilities) - HH 49%, HT 21% TH 21% TT 9%. The middle two outcomes are equal probability, aka a fair coin. So ignore HH and TT flips and use HT and TH as Head and Tail of the 'fair' coin.
Just note: you're throwing away 58% of your results! The original coin flip runs 230% the speed of your new RNG (that throws away so many results).
The Von Neuman method is a great way at demonstrating that removal of bias is possible, but not necessarily practical (especially at high bandwidth). In practice, you want to just cryptographic_hash(coin flips), which should maximize your entropy up to the entropy limit of 1/2 the hashsize (assuming you have a perfect cryptographic hash function).
A 512-bit perfect cryptographic hash can only support 256-bits of entropy, due to the birthday attack.
The Von Neuman method is a great way at demonstrating that removal of bias is possible, but not necessarily practical (especially at high bandwidth). In practice, you want to just cryptographic_hash(coin flips), which should maximize your entropy up to the entropy limit of 1/2 the hashsize (assuming you have a perfect cryptographic hash function).
A 512-bit perfect cryptographic hash can only support 256-bits of entropy, due to the birthday attack.
In the general case, you can compress a sequence of N biased coin flips with arithmetic coding.
For a coin with known bias, this compression is (essentially) optimal and will therefore produce ~N*(Shannon information) unbiased bits.
In the "even more general case", the Shannon Theorem suggests that random_code(message) -> reaches the Shannon Limit. :-)
The problem is that perfectly-random codes can't be decoded very easily... so this little factoid is kind of worthless in most cases. Fortunately, we don't actually care what the information looks like when flipping coins. We usually just want "some random message", so no need to decode in this application!
------
Cryptographers assume hash-functions are perfect random codes (they aren't in theory, but they are in practice... at least until they're cracked. Kinda funny how that works out). As such, the cryptohash(message) methodology should also reach the Shannon-limit, as long as your cryptohash remains secure... and you stay within the blocksize restrictions of the hash.
The problem is that perfectly-random codes can't be decoded very easily... so this little factoid is kind of worthless in most cases. Fortunately, we don't actually care what the information looks like when flipping coins. We usually just want "some random message", so no need to decode in this application!
------
Cryptographers assume hash-functions are perfect random codes (they aren't in theory, but they are in practice... at least until they're cracked. Kinda funny how that works out). As such, the cryptohash(message) methodology should also reach the Shannon-limit, as long as your cryptohash remains secure... and you stay within the blocksize restrictions of the hash.
>assuming you have a perfect cryptographic hash function
Except there is no such thing (not in the sense you need it) - it too would be a perfect source of randomness, and you're just pushing the problem down the road.
As an example, NIST publication 800-90B recommends multiple randomness extractors, one of which is SHA. They recommend using twice the amount of entropy in as the entropy out to get random "enough" bits. Thus even SHA is not going to mix bits well enough as you want.
(The things called perfect hash functions in the literature map N items into N slots with no collisions, which is not what is needed here).
As a perhaps surprising counterexample to any simple solution, here [1] is a math paper with a proof that there can be no optimal algorithm that is best for all values of coin bias.
Here's [2] a cool walkthough on some of the ideas in theory that have been investigated.
[1] https://web.eecs.umich.edu/~qstout/abs/AnnProb84.html [2] http://www.eecs.harvard.edu/~michaelm/coinflipext.pdf
Except there is no such thing (not in the sense you need it) - it too would be a perfect source of randomness, and you're just pushing the problem down the road.
As an example, NIST publication 800-90B recommends multiple randomness extractors, one of which is SHA. They recommend using twice the amount of entropy in as the entropy out to get random "enough" bits. Thus even SHA is not going to mix bits well enough as you want.
(The things called perfect hash functions in the literature map N items into N slots with no collisions, which is not what is needed here).
As a perhaps surprising counterexample to any simple solution, here [1] is a math paper with a proof that there can be no optimal algorithm that is best for all values of coin bias.
Here's [2] a cool walkthough on some of the ideas in theory that have been investigated.
[1] https://web.eecs.umich.edu/~qstout/abs/AnnProb84.html [2] http://www.eecs.harvard.edu/~michaelm/coinflipext.pdf
> Except there is no such thing (not in the sense you need it)
???
Okay, lets choose the simplest hash function of them all: a 1-bit XOR (resulting in a 1-bit "hash").
Lets say I flip the 75% biased coin 100 times, count heads as 1 and tails as 0. I XOR all the results together. What's the probability of XOR(all) == 1, and what's the probability that XOR(all) == 0?
You'll find that its quite close to 50/50, which is the limit to the amount of entropy you can pull from a 1-bit hash function.
---------
Now lets see for smaller values:
1 flip: 75% chance of 1, 25% of 0. Which is the limit of information so far.
2 flips: ... you know what? Imma write a program and get back to ya a bit later. I probably will edit my post.
------
2 flips: 62% chance of Parity0 3 flips: 43% chance of Parity0 4 flips: 53% chance of Parity0
Etc. etc. etc.
By 16 flips, its 49.9985% chance of Parity0, pretty close to unbiased. It gets pretty hard to distinguish this coin from an unbiased one.
--------
I'm not sure if you need a cryptographic hash actually. Its just that cryptographic hashes are close enough to random that its easier to use.
Now I'm curious if a CRC32 would efficiently extract 16-bits of entropy from biased coins. CRC32 is clearly not a cryptographic hash, but its pretty good as an "avalanche" due to the nature of GF-arithmetic.
???
Okay, lets choose the simplest hash function of them all: a 1-bit XOR (resulting in a 1-bit "hash").
Lets say I flip the 75% biased coin 100 times, count heads as 1 and tails as 0. I XOR all the results together. What's the probability of XOR(all) == 1, and what's the probability that XOR(all) == 0?
You'll find that its quite close to 50/50, which is the limit to the amount of entropy you can pull from a 1-bit hash function.
---------
Now lets see for smaller values:
1 flip: 75% chance of 1, 25% of 0. Which is the limit of information so far.
2 flips: ... you know what? Imma write a program and get back to ya a bit later. I probably will edit my post.
------
2 flips: 62% chance of Parity0 3 flips: 43% chance of Parity0 4 flips: 53% chance of Parity0
Etc. etc. etc.
By 16 flips, its 49.9985% chance of Parity0, pretty close to unbiased. It gets pretty hard to distinguish this coin from an unbiased one.
--------
I'm not sure if you need a cryptographic hash actually. Its just that cryptographic hashes are close enough to random that its easier to use.
Now I'm curious if a CRC32 would efficiently extract 16-bits of entropy from biased coins. CRC32 is clearly not a cryptographic hash, but its pretty good as an "avalanche" due to the nature of GF-arithmetic.
>It gets pretty hard to distinguish this coin from an unbiased one.
All you're doing is trying to reinvent the Law of Large Numbers - nothing in this post shows a perfect hash function, and in fact is demonstrating my point - by you putting biased data into a hash you're getting biased data out.
To completely remove the bias you need infinite flips. And thus you have improved nothing.
Next, if you're going to flip 100 times to get 1 bit out, simply use the Neumann method - it does work, uses only a few flips on average.
Here's a proof why your method doesn't work: Let p = prob of getting a 1, q = 1-p is prob of getting a 0. Then XORing n of these together is a 1 iff there are an odd number of 1 bits. This happens Sum_{j=1 to N by odd} nCj p^j (1-q)^(n-j) of the time. This simplifies to (1-(1-2p)^n)/2. For this to be exactly 1/2, which is no bias, p must be exactly 1/2. There is no other value that doesn't fail.
This argument can be extended to whatever hash functions you want. You're not going to get the outcome you desire no matter how you structure it.
>I'm not sure if you need a cryptographic hash actually.
It matters not the hash. None will do what you want.
This is why when crpyto uses weak PRNGs at the front, even after all the powerful crypto in the middle, they are weakened. You're trying to do what decades of cryptographers know is impossible.
All you're doing is trying to reinvent the Law of Large Numbers - nothing in this post shows a perfect hash function, and in fact is demonstrating my point - by you putting biased data into a hash you're getting biased data out.
To completely remove the bias you need infinite flips. And thus you have improved nothing.
Next, if you're going to flip 100 times to get 1 bit out, simply use the Neumann method - it does work, uses only a few flips on average.
Here's a proof why your method doesn't work: Let p = prob of getting a 1, q = 1-p is prob of getting a 0. Then XORing n of these together is a 1 iff there are an odd number of 1 bits. This happens Sum_{j=1 to N by odd} nCj p^j (1-q)^(n-j) of the time. This simplifies to (1-(1-2p)^n)/2. For this to be exactly 1/2, which is no bias, p must be exactly 1/2. There is no other value that doesn't fail.
This argument can be extended to whatever hash functions you want. You're not going to get the outcome you desire no matter how you structure it.
>I'm not sure if you need a cryptographic hash actually.
It matters not the hash. None will do what you want.
This is why when crpyto uses weak PRNGs at the front, even after all the powerful crypto in the middle, they are weakened. You're trying to do what decades of cryptographers know is impossible.
The entropy is -0.7log2(0.7)-0.3log2(0.3) = 0.88 .
The post is very confusing, in no case can deterministic hashing increase the entropy. And if it outputs only half of input entropy then it's only 8% more efficient than von neumann in this case.
The post is very confusing, in no case can deterministic hashing increase the entropy. And if it outputs only half of input entropy then it's only 8% more efficient than von neumann in this case.
The Von Neumann method depends on coins being independent.
But the measured bias here is a 51% chance of getting what you started with. Which means that if you flip over and over again, there is a correlation between consecutive entries.
But the measured bias here is a 51% chance of getting what you started with. Which means that if you flip over and over again, there is a correlation between consecutive entries.
What says you can't reset the coin the same side up in between flips?
What about just after the unfair coin sequence toggling every even position?
HTHHTHHTTHHH
HHHTTTHHTTHT
Should be fair. Though there is order now when the coin was unfair.
HTHHTHHTTHHH
HHHTTTHHTTHT
Should be fair. Though there is order now when the coin was unfair.
Fair, but only over time, and not truly random.
Only over time: this doesn’t help if you need _one_ fair flip.
Not random: for any 2 flips in sequence, you want a probability of ¼ to get two heads. This gets you either 70% × 30% or 30% × 70%, both of which are 0.21. That’s only 84% of the expected 0.25. In general, longer sequences of heads or tails are too rare with this method (e.g. for four flips: 70% × 30% × 70% × 30% is 0.0441; only 70.56% of the expected 0.0625. The probability of 6 consecutive heads or tails is less than 60% of what it should be, etc.)
Only over time: this doesn’t help if you need _one_ fair flip.
Not random: for any 2 flips in sequence, you want a probability of ¼ to get two heads. This gets you either 70% × 30% or 30% × 70%, both of which are 0.21. That’s only 84% of the expected 0.25. In general, longer sequences of heads or tails are too rare with this method (e.g. for four flips: 70% × 30% × 70% × 30% is 0.0441; only 70.56% of the expected 0.0625. The probability of 6 consecutive heads or tails is less than 60% of what it should be, etc.)
Good point about it not being random over sequences. Would be fun to find a method that would obey more constraints and requires fewer flips than TH, HT from the other comment.
As https://news.ycombinator.com/item?id=25531532 said look up arithmetic coding (https://en.wikipedia.org/wiki/Arithmetic_coding)
It’s not 100% the same, but understanding it should enable you to use the same idea for this problem.
It’s not 100% the same, but understanding it should enable you to use the same idea for this problem.
Each individual flip is still unfair
For those interested in games of chance, I've been playing around with Betfair's Exchange Hi Lo card game [1]. I came up with an algorithm to, given a game state, quickly compute the odds of all subsequent outcomes in polynomial time.
Based on this algorithm, I've been building a bot [2] to integrate with Betfair's games API to play automatically for me.
After finishing the first iteration, running it for a bit and losing a few Euros, I quickly realised that some of my assumptions as laid out in the README were wrong (I will be updating this soon). Namely, I wasn't accounting for Betfair's commission on winnings (silly me). I currently have an EU-based account, which has a 6.5 per cent commission on winnings. UK-based accounts have the minimum at 5 per cent. The commission has an effect on the cheapest profitable odds you can provide, which has an effect on whether people want to match bets at those odds.
With this in mind, it quickly becomes clear that any unmatched bets you see in the market are far from the maximum profitable odds. This makes sense, and competing based on odds becomes impossible at this point. I am led to believe that the only way to make money is by having a UK-based account plus getting the bets to the market first. I have a few ideas on how to do this.
If I can get the bets to match at odds that guarantee profit in the long-run assuming UK-based commission of 5 per cent, then I'm going to call it a day.
Regardless, I'm not expecting to make too much money from this. I have a lot of free time right now (looking for a job), so if anyone wants to get into anything else available on Betfair together, then please get in touch at the email on my Github through [2]. My discrete probability and programming are okay, but it would be great to have someone who could share their statistics knowledge.
[1] https://games.betfair.com/exchange-hi-lo/standard
[2] https://github.com/jpcooper/betfair-exchange-hi-lo-odds
Based on this algorithm, I've been building a bot [2] to integrate with Betfair's games API to play automatically for me.
After finishing the first iteration, running it for a bit and losing a few Euros, I quickly realised that some of my assumptions as laid out in the README were wrong (I will be updating this soon). Namely, I wasn't accounting for Betfair's commission on winnings (silly me). I currently have an EU-based account, which has a 6.5 per cent commission on winnings. UK-based accounts have the minimum at 5 per cent. The commission has an effect on the cheapest profitable odds you can provide, which has an effect on whether people want to match bets at those odds.
With this in mind, it quickly becomes clear that any unmatched bets you see in the market are far from the maximum profitable odds. This makes sense, and competing based on odds becomes impossible at this point. I am led to believe that the only way to make money is by having a UK-based account plus getting the bets to the market first. I have a few ideas on how to do this.
If I can get the bets to match at odds that guarantee profit in the long-run assuming UK-based commission of 5 per cent, then I'm going to call it a day.
Regardless, I'm not expecting to make too much money from this. I have a lot of free time right now (looking for a job), so if anyone wants to get into anything else available on Betfair together, then please get in touch at the email on my Github through [2]. My discrete probability and programming are okay, but it would be great to have someone who could share their statistics knowledge.
[1] https://games.betfair.com/exchange-hi-lo/standard
[2] https://github.com/jpcooper/betfair-exchange-hi-lo-odds
Heh.
I used to work on those products, way back in 2008 or so. Let me save you some time: you won't get any action at good odds. Or, if you do, it'll be a tiny amount.
Betfair run their own bots on those markets AND they get their bets into the market BEFORE the market opens to the public. Good luck anyway!
I used to work on those products, way back in 2008 or so. Let me save you some time: you won't get any action at good odds. Or, if you do, it'll be a tiny amount.
Betfair run their own bots on those markets AND they get their bets into the market BEFORE the market opens to the public. Good luck anyway!
And they will cannibalize your profits, up to 60%, if you start to make significant money. And they can kick you out anytime they want. It's all in the terms and conditions.
IMO the premium charge is much better than commission as a pricing model. It's less predictable day to day but long term it better aligns incentives between the winning customers and the exchange. They don't really kick people off unless they're exploiting bugs in the exchange or engaged in illegal activity. They certainly won't ban you for winning on the exchange.
Thanks for the heads up. It's good to get confirmation that that actually happens. I'd be surprised if it didn't. I'm not too far from implementing my final idea to get profitable bets in. If that (probably as you say) fails, then it was a fun learning experience.
Thanks for sharing (I recommend taking a look at his GitHub repo [2], where he nicely described the game and his approach).
Some time ago I had a similar project with BetFair soccer matches. I assumed that the number of goals from each team is given by a Poisson process. Then, I used the largest market (Win/Lose/Draw) to estimate this parameter. With this information I could estimate the "fair" odds of more exotics bets (like e.g. team A will score 3 goals more than team B), which were often mispriced, and then use the Kelly criterion to estimate how much to bet.
All in all a nice project, but... at the very end -just before deploying the bot- I realised that the BetFair Exchange is not open to Germans, so I wrote a blog post and open sourced the code. Still, a nice learning opportunity :-)
Some time ago I had a similar project with BetFair soccer matches. I assumed that the number of goals from each team is given by a Poisson process. Then, I used the largest market (Win/Lose/Draw) to estimate this parameter. With this information I could estimate the "fair" odds of more exotics bets (like e.g. team A will score 3 goals more than team B), which were often mispriced, and then use the Kelly criterion to estimate how much to bet.
All in all a nice project, but... at the very end -just before deploying the bot- I realised that the BetFair Exchange is not open to Germans, so I wrote a blog post and open sourced the code. Still, a nice learning opportunity :-)
Thanks. I would like to point out again that the code is out of date, and the margins displayed there are miscalculated due to not accounting for commission. The general spirit remains.
Did you backtest and show potential returns? Can you link to your model? Did you come across "Scoring dynamics across professional team sports: tempo, balance and predictability" [1]? They apply a Poisson model to team sports. I mentioned this paper to a company involved in betting who I interviewed with once, and they told me that they were doing something similar, but way more involved. It demotivated me somewhat regarding implementing it, making me wonder whether there really was any juice left to be squeezed from it.
[1] https://link.springer.com/article/10.1140/epjds29
Did you backtest and show potential returns? Can you link to your model? Did you come across "Scoring dynamics across professional team sports: tempo, balance and predictability" [1]? They apply a Poisson model to team sports. I mentioned this paper to a company involved in betting who I interviewed with once, and they told me that they were doing something similar, but way more involved. It demotivated me somewhat regarding implementing it, making me wonder whether there really was any juice left to be squeezed from it.
[1] https://link.springer.com/article/10.1140/epjds29
I wasn't aware of that paper, but it seems we came to a similar conclusion. From the paper:
> Across all sports, scoring tempo - when scoring events occur - is remarkably well-described by a Poisson process, in which scoring events occur independently with a sport-specific rate at each second on the game clock.
So, yes, basically that was my concept. Estimate its parameters and then you'll have the "real" odds. Any difference with the market ones is a potential of profit (considering the house fees).
I couldn't backtest it since I couldn't find free historical data for all types of markets (then again, maybe I didn't look hard enough). I forward tested it, though, and it seemed to work.
My code is here [1], which also links to a blog post describing the method. Feel free to take a look and contact me to discuss these ideas.
Also, if you're interested in mathematically modelling probabilities, we are looking for team members to develop a product to estimate the risk of default in loans. After all, a loan is another kind of bet, that takes place in a slightly harder to model environment.
[1] https://github.com/ghgr/BetFair_Arbitrer
> Across all sports, scoring tempo - when scoring events occur - is remarkably well-described by a Poisson process, in which scoring events occur independently with a sport-specific rate at each second on the game clock.
So, yes, basically that was my concept. Estimate its parameters and then you'll have the "real" odds. Any difference with the market ones is a potential of profit (considering the house fees).
I couldn't backtest it since I couldn't find free historical data for all types of markets (then again, maybe I didn't look hard enough). I forward tested it, though, and it seemed to work.
My code is here [1], which also links to a blog post describing the method. Feel free to take a look and contact me to discuss these ideas.
Also, if you're interested in mathematically modelling probabilities, we are looking for team members to develop a product to estimate the risk of default in loans. After all, a loan is another kind of bet, that takes place in a slightly harder to model environment.
[1] https://github.com/ghgr/BetFair_Arbitrer
Thanks for the tips. The loans idea sounds interesting. I will send you an email.
Economic reasoning states that the vig will absolutely swamp any small edge you may discover.
These guys did it:
Walsh and Ranogajec took on the might of global gaming markets and managed to consistently win over three decades. Much like the most advanced hedge funds, which employ armies of PhDs to identify patterns in financial markets, Walsh and Ranogajec’s consortium, called the Bank Roll, created statistical models that allowed them to exploit mispricings in betting odds wherever they could be found
https://www.afr.com/opinion/david-walsh-s-wisdom-beats-the-o...
Walsh and Ranogajec took on the might of global gaming markets and managed to consistently win over three decades. Much like the most advanced hedge funds, which employ armies of PhDs to identify patterns in financial markets, Walsh and Ranogajec’s consortium, called the Bank Roll, created statistical models that allowed them to exploit mispricings in betting odds wherever they could be found
https://www.afr.com/opinion/david-walsh-s-wisdom-beats-the-o...
https://en.m.wikipedia.org/wiki/Vigorish For others who don’t know what vig is
Someone has to provide a market. That person has to do it profitably. I am assuming that this is possible, otherwise it would not be possible to play the game. My reasoning could of course be off. Could you elaborate?
> Someone has to provide a market. That person has to do it profitably.
But by this reasoning no one would play blackjack at the casino. That is, how do you know people laying these bets aren't just gambling at a disadvantage? You could view the whole thing as one more novelty game the casino is spreading.
But by this reasoning no one would play blackjack at the casino. That is, how do you know people laying these bets aren't just gambling at a disadvantage? You could view the whole thing as one more novelty game the casino is spreading.
I am assuming that all computer players of this game have access to the algorithm which I have discovered. I am assuming that all computer players have figured out the simple formula which gives them the cheapest profitable odds to back or lay an outcome given the commission and the outcome's probability (thanks to one of your previous comments).
I am assuming that there will be punters who bite at the odds I provide. If there weren't, then the game would not be sustainable.
I came across a "betting guide" for sale for this game which claims to advise on bets based on the "trends" seen in the game. I don't know what this means, but it gives me hope that there are people out there willing to part with their money in my direction.
Failing that, the novelty was an interesting one, and it's all about the journey, dude!
I am assuming that there will be punters who bite at the odds I provide. If there weren't, then the game would not be sustainable.
I came across a "betting guide" for sale for this game which claims to advise on bets based on the "trends" seen in the game. I don't know what this means, but it gives me hope that there are people out there willing to part with their money in my direction.
Failing that, the novelty was an interesting one, and it's all about the journey, dude!
To clarify, I'm not saying your assumptions are necessarily wrong, or that you shouldn't try to test them and see what happens.
I'm just proposing another possibility is that everyone playing on both sides is a "punter" having fun (or deceiving themselves). In any case, that everyone is losing long-term. With 5% vig I'd guess that's the most likely possibility.
I'm just proposing another possibility is that everyone playing on both sides is a "punter" having fun (or deceiving themselves). In any case, that everyone is losing long-term. With 5% vig I'd guess that's the most likely possibility.
[deleted]
You might be right. Also if Betfair has its own bots playing as another commenter claims, then it will be hard to compete on odds. Only one way to find out.
Here is my guess:
45% of players don’t play well enough to win, even at even odds. Kind of mini-Dunning Krueger. They’re hoping for luck, and love to tell stories to their friends when they win.
30% are degenerate gamblers.
The balance play well enough that the loss rate is sufficiently slow so as to pay for the entertainment value.
45% of players don’t play well enough to win, even at even odds. Kind of mini-Dunning Krueger. They’re hoping for luck, and love to tell stories to their friends when they win.
30% are degenerate gamblers.
The balance play well enough that the loss rate is sufficiently slow so as to pay for the entertainment value.
... you’re aware of the adage “the house always wins” right?
Betfair is peer to peer, so the house just takes a rake (similar to poker). This means it is hypothetically possible for both you and the house to win.
On Betfair the house is you. You determine the odds. Let's assume that cards are dealt randomly. If I know the exact probabilities of every outcome I bet on, and I always bet with odds which have a margin over those probabilities, then I win in the long run.
Forgive my ignorance, but: why would anyone bet against you?
I'm a long-time poker player and recent political bettor. Gamblers routinely overestimate their edge or don't care whether they win. Often it's a mix of both.
There are still many many many people who are willing to bet on the results of the US presidential election: https://www.thedailybeast.com/gamblers-bet-big-on-trump-and-...
There are still many many many people who are willing to bet on the results of the US presidential election: https://www.thedailybeast.com/gamblers-bet-big-on-trump-and-...
What do you base your political bets on?
For one-offs, picking the favorite is good enough for most players. Thy are just seeking entertainment and maybe to win a little money. For instance, in a race between two candidates priced at $0.70 and $0.30 (ranging from $0.01 to $0.99 before resolution), many players are happy to pick the favorite indicated by the market.
Of course, if those prices are accurate, then they will lose money in the long-run after fees. For players who are betting more frequently and over a longer term, it's like any other speculation: You might assess the $0.30 underdog to have a 40% chance of winning. She is still the underdog, but you profit over the long run by buying her at her too-low price. Those players are scooping that value wherever they can find it.
For major electoral events that involve a populist with a large, loyal, and misinformed fanbase, the value pretty much finds you. Both of the described groups are making lots of money because the markets are wildly mispriced.
Of course, if those prices are accurate, then they will lose money in the long-run after fees. For players who are betting more frequently and over a longer term, it's like any other speculation: You might assess the $0.30 underdog to have a 40% chance of winning. She is still the underdog, but you profit over the long run by buying her at her too-low price. Those players are scooping that value wherever they can find it.
For major electoral events that involve a populist with a large, loyal, and misinformed fanbase, the value pretty much finds you. Both of the described groups are making lots of money because the markets are wildly mispriced.
I wonder what you base your sentiment on. I can't imagine there's much juice left in Twitter sentiment analysis.
You don’t have to be smarter than Twitter, just smarter than the crowd, who aren’t applying Twitter analysis. Take the above article as an extreme but not exceptional case: The correct price for Biden winning the election was $0.99 for weeks before the market was closed, but he was trading much lower over that time.
We were betting on the results of an event that already happened. No Twitter analysis necessary.
We were betting on the results of an event that already happened. No Twitter analysis necessary.
I need to find more of those.
A recent example. Even after it became apparent that Biden won the election you could still sell 10:1 bets on Trump to win, on betfair, to suckers which refused to accept reality or vastly overestimated the probability for Trump to sabotage the elections and/or stage a coup.
It's basically free money.
It's basically free money.
Consider the case of betting with a traditional bookmaker. The bookmaker lays (bets against) an outcome. You back (bet for) the outcome. Each party is betting against the other.
The don't. They place bets. If they place bets on the opposite odds that you do on the same bet, then they play against you (and you're competing for that).
for fun
The key to breaking this potential advantage is to let someone other than the coin flipper call heads or tails while the coin is in the air.
That’s how I was taught to do it as a kid when we were playing for fun. I never really thought about the reasoning until now.
If you talk to people deep into the performance magic community, there are myths and legends of people who practice dice rolling to the point where they can influence the outcome of a dice roll. I suppose training for hours a day for years on end could have some such influence.
That’s how I was taught to do it as a kid when we were playing for fun. I never really thought about the reasoning until now.
If you talk to people deep into the performance magic community, there are myths and legends of people who practice dice rolling to the point where they can influence the outcome of a dice roll. I suppose training for hours a day for years on end could have some such influence.
It is definitely doable. _Scarne on Dice_ (1945) details a number of methods for controlling dice throws. And Steve Forte's "Gambling Protection Series" videos show what the moves look like in action.
Mind you, none of these moves will fly in a modern casino.
Mind you, none of these moves will fly in a modern casino.
Also the book "Gambling Scams" by Darwin Ortiz.
As I recall, the techniques generally involve not making normal "fair" free-tumbling rolls, but constraining the die movements somehow. Things like tossing them into a corner or making one of the pair spin flat, around only a vertical axis.
As I recall, the techniques generally involve not making normal "fair" free-tumbling rolls, but constraining the die movements somehow. Things like tossing them into a corner or making one of the pair spin flat, around only a vertical axis.
That doesn't break the advantage, it just mitigates it. The advantage is just small enough that it doesn't matter much even before the mitigation. The caller could still look at the coin before it's flipped and have a slight edge in calling it.
It's not terribly hard to influence the outcome of a die roll. You can roll it end over end like a wheel so two of the faces remain on the sides and can't be the result.
It's not terribly hard to influence the outcome of a die roll. You can roll it end over end like a wheel so two of the faces remain on the sides and can't be the result.
There was an episode of the old "Breaking Vegas" series [1] on the History Channel about such "dice dominators". This site seems to have the episode online [2].
[1] https://en.wikipedia.org/wiki/Breaking_Vegas
[2] https://www.goldentouchcraps.com/HistoryChannel.shtml
[1] https://en.wikipedia.org/wiki/Breaking_Vegas
[2] https://www.goldentouchcraps.com/HistoryChannel.shtml
I remember when I was a kid, I had a book called "The Amateur Magician's Handbook", by Henry Hay, which described how lots of magic tricks and sleight of hand with cards, etc. were done, with detailed explanations and pictures. There was a section called "Heartbreakers", which contained tricks and techniques that were especially difficult to master. One of them involved predicting the outcome of a coin toss. The secret to it was that you just practice flipping and catching a coin to an extreme degree, aiming for consistency until you could replicate the action so precisely that the coin flipped the same number of times whenever you performed it. I practiced and practiced, but evidently not enough, because I could never do it. I found a bit of it online:
"18. Heads-or-tails. This I take on the good and sufficient word of Eddie Joseph. He explains how it is possible to flip a borrowed coin, and call heads or tails correctly every time.
"No trick to it -- just mechanical uniformity of action. It is obvious (when pointed out) that which face of the coin comes up depends simply on the number of revolutions the coin makes; and this depends on the weight of the coin, the height of the flip, and the force of the spin.
"By always using one denomination of coin, you fix the weight. The uniform flip of your right thumb comes with practice. To get a uniform height, pick some marked point on the wall, and practice until you can toss the coin just that high every time.
"You must also catch the coin on your left palm at exactly the same height each time -- your waist, naturally, is the easiest.
"Once you attain true uniformity, you will discover that the coin always comes down showing the same face that went up, or else always the opposite face. Experiment with different heights of toss, keeping track of the way the coin comes down. Never vary the force of your thumb flip because that is the hardest to judge.
"In performance, pick a mark such as a molding or picture frame on the wall of the room where you happen to be, make one trial toss to that height, and see whether the coin comes down the same as it went up, or opposite. Every time you toss to this mark afterward, the result should be identical.
"18. Heads-or-tails. This I take on the good and sufficient word of Eddie Joseph. He explains how it is possible to flip a borrowed coin, and call heads or tails correctly every time.
"No trick to it -- just mechanical uniformity of action. It is obvious (when pointed out) that which face of the coin comes up depends simply on the number of revolutions the coin makes; and this depends on the weight of the coin, the height of the flip, and the force of the spin.
"By always using one denomination of coin, you fix the weight. The uniform flip of your right thumb comes with practice. To get a uniform height, pick some marked point on the wall, and practice until you can toss the coin just that high every time.
"You must also catch the coin on your left palm at exactly the same height each time -- your waist, naturally, is the easiest.
"Once you attain true uniformity, you will discover that the coin always comes down showing the same face that went up, or else always the opposite face. Experiment with different heights of toss, keeping track of the way the coin comes down. Never vary the force of your thumb flip because that is the hardest to judge.
"In performance, pick a mark such as a molding or picture frame on the wall of the room where you happen to be, make one trial toss to that height, and see whether the coin comes down the same as it went up, or opposite. Every time you toss to this mark afterward, the result should be identical.
I could do this by playing a lot of chess, the trick is the coin makes a certain sound if you do it right, if you do it wrong its not the same sound
With practice most people can dramatically influence the outcome of a coin flip. I’ve gotten to >80% accurate when I was practicing.
From the article:
> Diaconis himself has trained his thumb to flip a coin and make it come up heads 10 out of 10 times
> Diaconis himself has trained his thumb to flip a coin and make it come up heads 10 out of 10 times
10 out of 10 isn't really that impressive.
If his actual success rate is, say, 70 out 100 10/10 is easy enough to get by chance.
990 out of 1000 would be much more pressive.
If his actual success rate is, say, 70 out 100 10/10 is easy enough to get by chance.
990 out of 1000 would be much more pressive.
It's a popular piece. ‘10 out of 10’ probably just means ‘reliably’.
I calculate that 70/100 gives you a 6 in a million chance of getting 10 in a row.
6 in a million is the chance to fail 10 times in a row with 70% odds. 10 successes is 3%.
[deleted]
Aw dang it, got it backwards!
Flip the coin the same number of times, catch consistently and get good at adjusting the catch. Use a large and heavy coin like a silver dollar. - former magic enthusiast.
Former Pokémon TCG enthusiast (circa 1999) here! Kids back then had the same approach, large and heavy coin, flip from a consistent height, consistent strength. Heads every time.
There were popular coins made specifically for Pokémon TCG then, and they happened to be large and thick and plastic and heavy, so the cheating was widespread.
I was one of the many kids who did this routinely. But there was one kid who took first place nearly every week, out of a field of 50+ entrants. “He must cheat the coin, AND be an amazing deck builder and card player on top of that”, I thought.
Then one week I was matched against him, winner goes to top 8.
Our decks were nearly identical, I could flip heads every time, I was a strong player. “50-50” I thought.
Turn one. Retreat Scyther, in Electabuzz, Thundershock. I flip my giant coin, and before it flattens, he picks it up and hands it back to me. “Flip it higher please.”
He must know I’m cheating. I assumed he must be cheating too though, every other kid is, and he wins this tournament every week!
Guilty, I flipped the coin fair this time, higher. Before it flattens he picks it up again, hands it back to me. “Higher please.”
I lost every coin flip that match. He won, went to the top 8, later won the tournament for the nth week in a row.
It wasn’t until adulthood that I realized how he was winning every tournament. In a field full of cheaters trying to flip heads every time, all he had to do was wait for the coin to enter that terminal spinning motion every coin does right before it flattens. Well practiced at this, if he sees that it’s about to flatten heads, he quickly grabs it and hands it back to you. “Higher please.” Repeat until the coin flattens tails.
His tactic was a silver bullet in a field full of flip cheaters. Nobody ever questioned his intentions when he grabbed the coin and asked for a higher flip.
There were popular coins made specifically for Pokémon TCG then, and they happened to be large and thick and plastic and heavy, so the cheating was widespread.
I was one of the many kids who did this routinely. But there was one kid who took first place nearly every week, out of a field of 50+ entrants. “He must cheat the coin, AND be an amazing deck builder and card player on top of that”, I thought.
Then one week I was matched against him, winner goes to top 8.
Our decks were nearly identical, I could flip heads every time, I was a strong player. “50-50” I thought.
Turn one. Retreat Scyther, in Electabuzz, Thundershock. I flip my giant coin, and before it flattens, he picks it up and hands it back to me. “Flip it higher please.”
He must know I’m cheating. I assumed he must be cheating too though, every other kid is, and he wins this tournament every week!
Guilty, I flipped the coin fair this time, higher. Before it flattens he picks it up again, hands it back to me. “Higher please.”
I lost every coin flip that match. He won, went to the top 8, later won the tournament for the nth week in a row.
It wasn’t until adulthood that I realized how he was winning every tournament. In a field full of cheaters trying to flip heads every time, all he had to do was wait for the coin to enter that terminal spinning motion every coin does right before it flattens. Well practiced at this, if he sees that it’s about to flatten heads, he quickly grabs it and hands it back to you. “Higher please.” Repeat until the coin flattens tails.
His tactic was a silver bullet in a field full of flip cheaters. Nobody ever questioned his intentions when he grabbed the coin and asked for a higher flip.
I can imagine that on a slow, lazy flip with a big coin, but is this realistic with what I think of as a "normal" flip, i.e. a US quarter, flipped rapidly end-over-end with the thumbnail, 2+ feet in the air? It's hard to believe that even decades of practice would get this reliable, and I don't think I've ever seen a stage magician rely on it in a routine.
For comparison, the other method (where the coin rotates about its vertical axis) is something you can learn to do tolerably well in a few minutes, and to do very well after a month or two at most. The motion of the coin will look perfect, and you can even get the "ding!" noise of a real flip. The only giveaway is the hand position; a real flip is done with the thumb under the coin, and this trick (for me, anyway) requires the thumb to be on top.
Source: taught myself to do this many years ago. I'm not a magician and have never used it to win money, I just use this to settle arguments between my kids.
For comparison, the other method (where the coin rotates about its vertical axis) is something you can learn to do tolerably well in a few minutes, and to do very well after a month or two at most. The motion of the coin will look perfect, and you can even get the "ding!" noise of a real flip. The only giveaway is the hand position; a real flip is done with the thumb under the coin, and this trick (for me, anyway) requires the thumb to be on top.
Source: taught myself to do this many years ago. I'm not a magician and have never used it to win money, I just use this to settle arguments between my kids.
In this sense, I found the 51% surprising (surprisingly close to 50%).
If you have a very practiced, consistent flip, you can always get the same result.
If you have a very practiced, consistent flip, you can always get the same result.
The article notes that he got to 100% when intentionally biasing the flip, the 51% was for ordinary people doing ordinary coin flips (no attempt to bias).
To clarify, this is exactly what I found surprising: that any unintentional bias that exists, in those ordinary cases, cancels out almost (but not quite?) perfectly.
Appreciate the much-needed TLDR on this.
What do you do to bias the outcome?
Flip it with the right amount of force, like a juggler with bowling pins.
No. You need to put a lot of horizontal torque on it so it never flips over at all.
That’s not a coin flip! I agree that’s a method to achieve the outcome though. If you get the wobble right it’s almost imperceptible.
no audience is going to fall for that...
They do, occasionally it doesn’t wobble enough and looks obvious but done right it’s not perceptible. As a tip do it physically close to someone and catch it high so they have a worse angle to see the lack of flipping. For a real “performance” there is significantly better methods though.
[deleted]
See also: this short piece about detecting bias in coin tosses by listening to the “ping” they make when tossed. https://cs.stanford.edu/~kach/can-one-hear-the-fate-of-a-coi...
>"Diaconis didn't even own a computer. (He still doesn't. He says he got sick of adjusting to new operating systems and noted, after not using computers for several years, that "nothing bad happened to me. And I said, 'Well, I could either learn the current system or learn differential geometry. I think I'll learn differential geometry this year.'")"
Good for him. He made the right choice I think.
Good for him. He made the right choice I think.
Sounds like he would enjoy GNU Emacs, which hasn't changed its interface in decades. Even the copy-paste shortcuts are the same.
Having spent the majority of my days typing at various applications for the last 30 years, I have developed a great appreciation for UI longevity.
I'm a vi person by muscle-memory, and the stability there means I've learned a thing or two about it. But I've used emacs enough that I'm comfortable there, too, at least to a point.
Don't get me wrong, I love playing with new toys. But I also love not having to think about my tools - I'd rather think about what I'm building.
I'm a vi person by muscle-memory, and the stability there means I've learned a thing or two about it. But I've used emacs enough that I'm comfortable there, too, at least to a point.
Don't get me wrong, I love playing with new toys. But I also love not having to think about my tools - I'd rather think about what I'm building.
The same is true of Notepad. It’s not really the point.
Emacs has calendar, weather, email, games, todo, search, and almost everything else you need in a modern operating system.
Half-assed tosses of a flat object can obviously be biased. We can predictably flip an object to the other side with a light toss, not to mention flip a pancake.
If you flip a coin through the air so that it rapidly rotates (say more than 15-20 times) before being caught, that seems like it would erase the bias.
If there isn't decent spin, then "all bets are off" (quite literally).
As a matter of procedure, there should be a rule: the coin tosser must place the coin onto index finger, which is coiled around the thumb, such that just the edge of the coin is struck from below by the thumbnail, when the thumb is flicked, sending the coin upward in rapid rotation. There should be additional rules that the coin should attain a height of at least one head height above the top of the tosser's head, or something of the sort.
If you flip a coin through the air so that it rapidly rotates (say more than 15-20 times) before being caught, that seems like it would erase the bias.
If there isn't decent spin, then "all bets are off" (quite literally).
As a matter of procedure, there should be a rule: the coin tosser must place the coin onto index finger, which is coiled around the thumb, such that just the edge of the coin is struck from below by the thumbnail, when the thumb is flicked, sending the coin upward in rapid rotation. There should be additional rules that the coin should attain a height of at least one head height above the top of the tosser's head, or something of the sort.
Better yet, just bounce the coin off of two surfaces before it lands. Makes it almost completely unpredictable.
Perci is a fascinating and very accomplished guy. His book "Magical Mathematics" is really interesting. There's actually a long history of math and comp sci folks who were very good magicians. Alex Elmsley, Bob Hummer, Martin Gardner are a few other to check out if you're into that stuff. (says the guy who has spent the last few days alternating between hacking and vainly trying to master the perfect Faro shuffle... ).
Don't forget Ron Graham who cowrote magical mathematics!
Yeah! And Ron was a pro juggler too. (as have been I!) Sadly Ron died this year.
Any of his books worth reading?
Group Representations In Probability And Statistics
Magical Mathematics: The Mathematical Ideas That Animate Great Magic Tricks — winner of the 2013 Euler Book Prize
Group Representations In Probability And Statistics
Magical Mathematics: The Mathematical Ideas That Animate Great Magic Tricks — winner of the 2013 Euler Book Prize
Persi is always worth reading although I have only read his papers.
"Group Reps" is one of my favorite math books of all time. It is a fantastic read, and completely changed the way I think about representation theory. (In short, you can generalized the Fourier transform to functions in groups and take advantage of all the usual Fourier tricks and intuitions.)
Link to pdf: http://personalpages.to.infn.it/~billo/didatt/gruppi/reps.pd...
Caveat emptor: Not sure if it's the right book, though. No author listed.
Caveat emptor: Not sure if it's the right book, though. No author listed.
Magical Mathematics is great! Dense actually, walks the "layperson" through proving how some complex card magic works with combinatorics. Really fun read, some mind bending stuff in there! The math is harder than one would expect the cover...
I'm very surprised that a scientist who previously worked with shaved dice did not take into account or even mentioned what kind of coin they were working with. Many coins are not even close to equal weight or shape on each side.
Slightly related, and of interest to minecraft players and speed runners: https://www.youtube.com/watch?v=1iqpSrNVjYQ
People normally think of probability as a way of understanding games of chance. But historically, games of chance evolved to make probability work, once people cared about that. Roman dice were lopsided. [1]
When probability works, it's because people made it work by removing influences that would throw off the calculations.
[1] https://www.atlasobscura.com/articles/dice-evolution-fate-fa...
When probability works, it's because people made it work by removing influences that would throw off the calculations.
[1] https://www.atlasobscura.com/articles/dice-evolution-fate-fa...
> Preliminary analysis of the video-taped tosses suggests that a coin will land the same way it started about 51 percent of the time.
From 25 trials?
From 25 trials?
They're measuring rotational velocity, not outcomes. As described in the article, actually flipping to determine the bias would have been a) arduous, b) probably prone to a margin of counting error larger than the possible bias (as they discovered for dice).
I did a high school math project that was like this but with dice. After thousands of die rolls, I found out that dice with pips are not very fair, but dice with engraved numbers are better.
Years ago, I went to a talk Persi Diaconis gave on this exact topic. He was the most charismatic mathematician I ever met in person.
Tl;dr: There’s no particular bias to heads or tails, but a coin will land the same way it was when it was tossed 51% of the time.
The article is more interesting than just the finding, to those who want to delve deeper. It’s a short biography of Persi Diaconis, an accomplished mathematician with no high school diploma who didn’t start studying mathematics until he was 24.
Really, really fascinating life and experience.
Really, really fascinating life and experience.
I remembered the result that showed a coin toss was biased that way, but I didn't know that the degree of bias hadn't been nailed down. I guess it makes sense that it varies from person to person. Now I'm interested in an anthropological study of how different people toss coins...
I think it's more random for me because I flip coins onto the floor, not onto my hand, or spin them on the table and see how they land when they stop spinning.
Pretty sure it’s because the heads is heavier.
It’s not, it’s because when flipped the number of flips the coin does in the air is not entirely random.
I thought it was common knowledge that heads are heavier than tails. My head certainly is.
HN is way too harsh on jokes, even when they’re really good - like this one.
I think good jokes can get appreciation here, but we try pretty hard to avoid the reddit-style puns and joke chains that distract from the discussion. They're entertaining sometimes... on reddit.
There are plenty of other sites you can go to if you want to read chains of joke comments.
Some folks have big heads, others big tails.
Not only is this (somewhat obviously given the discovered bias) wrong, it’s physically impossible to bias a coin flip by weighting one side. Weighting a coin will change its center of gravity but to bias a flip you would need to impart angular momentum while it is in the air; at any constant rate of spin (again somewhat obviously once pointed out) each side will spend 50% of its time "up" and 50% "down".
https://www.tandfonline.com/doi/abs/10.1198/000313002605
https://www.tandfonline.com/doi/abs/10.1198/000313002605
There's a huge assumption in this assertion: that you catch the coin.
If you let it bounce on a table, it's much more likely to land heavy-side down. Just imagine a very thick coin one side lead the other side balsa wood. That would land lead-side down much more than 50/50 if allowed to bounce on the table.
If you let it bounce on a table, it's much more likely to land heavy-side down. Just imagine a very thick coin one side lead the other side balsa wood. That would land lead-side down much more than 50/50 if allowed to bounce on the table.
Why does toast land butter side down so often?
If you flip toast rather than drop it, it doesn’t. Also, if you drop it it doesn’t really land “butter side down” but “top side down”. Just butter the bottom of your toast to solve the problem!
https://en.m.wikipedia.org/wiki/Buttered_toast_phenomenon
https://en.m.wikipedia.org/wiki/Buttered_toast_phenomenon
Do we know the reasoning behind the bias?
> Diaconis himself has trained his thumb to flip a coin and make it come up heads 10 out of 10 times.
I had better dexterity as a child, and after hours I was able to flip a coin to 'heads' about 8/10 of the time. Same position, same muscle reflex, same results. I would proudly show my family my skill
Over time, I lost interest and the skill
I had better dexterity as a child, and after hours I was able to flip a coin to 'heads' about 8/10 of the time. Same position, same muscle reflex, same results. I would proudly show my family my skill
Over time, I lost interest and the skill
[deleted]