Irwin – the protector of Lichess from all chess players villainous(github.com)
github.com
Irwin – the protector of Lichess from all chess players villainous
https://github.com/clarkerubber/irwin
121 comments
I see a lot of comments about the general topic, but none on the actual tool, which appears to have no (public-facing) updates in about three years. Presumably it could be a tool that incrementally improves over time with the explosion of data lichess has, but surely there would be a single commit since 2019. Do we know that this is what's actually running on lichess's backend today?
https://github.com/lakinwecker/irwin has more recent commits and I think is what actually runs on Lichess, or at least close to it. Not sure much fundamentally changed though. This isn't "the whole of Lichess' anticheat". It's a tool doing one specific thing which it mostly has been doing that way since it was created. There are a bunch of other parts to cheat detection, some of which are much more recent developments. For example kaladin, which has also been linked in the other thread about Lichess. But there's a fair bit more. Understandably though, Lichess doesn't really talk much about all of them, even though pretty much all of it is open source if you know where to look and how to use it.
Thanks for the context - I haven't spent a ton of time looking around but it does seem like there are several moving parts (and evidently some important stuff isn't in the same github org)
Yes, it is used according to this:
https://lichess.org/source
Thanks! I didn't know about this page.
Recent and related:
Lichess: The free and open source chess server - https://news.ycombinator.com/item?id=32045763 - July 2022 (65 comments)
Lichess: The free and open source chess server - https://news.ycombinator.com/item?id=32045763 - July 2022 (65 comments)
It takes $420k per year to run Lichess -
https://news.ycombinator.com/item?id=29955204 - February 2022 (266 comments)
I’ve seen a video of Magnus Carlsen playing on Lichess before. Does Irwin ever accidentally flag him or people like him? Do these sorts of folks have to be verified in some fashion?
What it looks like to cheat is more than just playing relatively accurate moves. Average move time, centipawn loss over multiple games, blunder/mistake frequency across multiple games, strengh of moves while in time trouble, etc. Cheaters tend to stand out when you look at a short history of games.
That would clearly work for someone who is cheating by letting an engine do all the work, but how about someone who mostly plays the game themselves just using the engine rarely?
Give Nakamura a minute with Stockfish any two times of his choosing in each game, and he would have probably won the Candidates.
Heck, just give a good player a blunder alert that tells them after they have made a blunder that they have done so and it could make a big difference.
There were games in the Candidates where a player would make a blunder that would completely turn the game around if the opponent found the one move that exploited it, but the opponent didn't see it. The first player could have then saved themselves but had not yet realized they blundered so didn't. Then the other player realized what was going on and exploited the blunder.
Give Nakamura a minute with Stockfish any two times of his choosing in each game, and he would have probably won the Candidates.
Heck, just give a good player a blunder alert that tells them after they have made a blunder that they have done so and it could make a big difference.
There were games in the Candidates where a player would make a blunder that would completely turn the game around if the opponent found the one move that exploited it, but the opponent didn't see it. The first player could have then saved themselves but had not yet realized they blundered so didn't. Then the other player realized what was going on and exploited the blunder.
I don't think it's that helpful if you're letting Stockfish make a move or two for you per game, or at least at the level I'm at.
The engine is so good that it often makes moves that are incomprehensible, setting itself up for an attack in n moves where n is often 10+.
If you did want to cheat (but what's the point?) a chrome extension that prevented you from making moves where you lost more then some certain amount of centipawns would be the way to do it.
The engine is so good that it often makes moves that are incomprehensible, setting itself up for an attack in n moves where n is often 10+.
If you did want to cheat (but what's the point?) a chrome extension that prevented you from making moves where you lost more then some certain amount of centipawns would be the way to do it.
Anand talked about that a while back. He said even 1 bit of information from Stockfish per game would result in a significant amount of rating points, and be quite hard to detect. I.e. you are in a position where you could choose to play a solid move, or alternatively to launch a risky combination. Stockfish explores the combination and gives you a 1 or 0 saying whether it will work.
I believe cheat detection is done partly by humans. Computers flag something as suspicious and then they show the suspicious moves to some grandmasters who might immediately say "no human would do that", or else "yeah that move looks weird but I could imagine someone making it", that type of thing. There was a youtube of Nakamura looking at such a position a while back. The person had a chance to sac some material in order to simplify to a trivially winning endgame, but instead carried out a ridiculously complicated maneuver that kept the material. Just the sort of thing a computer would do, and Naka pointed it out.
I believe cheat detection is done partly by humans. Computers flag something as suspicious and then they show the suspicious moves to some grandmasters who might immediately say "no human would do that", or else "yeah that move looks weird but I could imagine someone making it", that type of thing. There was a youtube of Nakamura looking at such a position a while back. The person had a chance to sac some material in order to simplify to a trivially winning endgame, but instead carried out a ridiculously complicated maneuver that kept the material. Just the sort of thing a computer would do, and Naka pointed it out.
> That would clearly work for someone who is cheating by letting an engine do all the work, but how about someone who mostly plays the game themselves just using the engine rarely?
Thats exactly what those engines try to detect. If you are average player, but every time you start loosing you get significantly better, then there is great possibility that you are cheating. Thats whats looking through history gives you. It's about actually finding deviations from your usual behaviour.
Thats exactly what those engines try to detect. If you are average player, but every time you start loosing you get significantly better, then there is great possibility that you are cheating. Thats whats looking through history gives you. It's about actually finding deviations from your usual behaviour.
Subtle cheating is always hard to automatically detect. You can't tell with high confidence that someone used an engine for one move every couple games for instance because sometimes humans find those rare moves on their own. Thankfully a lot cheaters seem to have an ego that pushes them to cheat more and more though so they tend to get weeded out.
personjerry(3)
High level humans and chess engines play differently. You can see commentary on chess YouTube videos when they run across cheaters.
Also the chess engines people use are accessible, you can compare what a suspected cheater does with what the cheater does and if they’re exactly the same consistently, you have a pretty strong signal.
One of the bigger tells are strange moves that set up a many move series resulting in a victory, things that humans just can’t find quickly.
Also the chess engines people use are accessible, you can compare what a suspected cheater does with what the cheater does and if they’re exactly the same consistently, you have a pretty strong signal.
One of the bigger tells are strange moves that set up a many move series resulting in a victory, things that humans just can’t find quickly.
It's not that simple. I'm an amateur player (2200 at lichess) but there are plenty of situations where I simply _know_ the best variation. Plenty of chess players analyze their own games using an engine and then use their memory when they are confronted with the same position. The same with opening theory: when I'm using my opening preparation, I'm playing at GM level as these are just the moves played by GMs in that position, and I did not need to find/calculate them, I just know them.
This is a very strong claim that is almost certainly false.
Would you be willing to reveal your account so that it can be independently verified? For example I'm 2116 on lichess and looking over the last 10 opponents who are in the neighborhood of 2200, it is never the case that their moves are optimal compared to a chess engine. For the first 10 or so moves yeah sure, just play a book opening, but beyond that people at 2200 make plenty of mistakes every single game including blunders.
The idea that you can consistently make optimal moves over the course of a 30-40 move game beyond the book opening requires some kind of evidence because in examining the last 10 games of 10 accounts arbitrarily picked, there isn't a single one that isn't absolutely full of inaccuracies and mistakes.
Would you be willing to reveal your account so that it can be independently verified? For example I'm 2116 on lichess and looking over the last 10 opponents who are in the neighborhood of 2200, it is never the case that their moves are optimal compared to a chess engine. For the first 10 or so moves yeah sure, just play a book opening, but beyond that people at 2200 make plenty of mistakes every single game including blunders.
The idea that you can consistently make optimal moves over the course of a 30-40 move game beyond the book opening requires some kind of evidence because in examining the last 10 games of 10 accounts arbitrarily picked, there isn't a single one that isn't absolutely full of inaccuracies and mistakes.
Sure, I blunder and I never claimed it to be always the case, but sometimes I know a tree of variations really deep, just because I remembered it from looking at it together with a computer.
Anyway, here's my account on lichess:
toolslive
Your explanation doesn't really distinguish high-level human play (not obvious cheating) from engine play (obvious cheating). There are probably plenty of games in which your first 20 moves (or 5-10 for me) are in GM databases and evaluated favorably by the engine, ass a 2200 you're probably booked up pretty well. But you know computer moves aren't so common in openings; they're much more common in complex middlegames and late games when the computer can calculate more combinations that we can and is able to produce lines that break intuition and principles but are strictly best.
My point is that high level human play online can be caused by computer analysis offline. One cannot observe a difference, as the moves are exactly the same.
But for the vast majority of players there will be a difference in computer lines and human evaluation; even if you're playing a strong game there is still a world of difference between 2500 lichess and ~3300 FIDE stockfish. This is more true for the median ~1500 lichess player. Even if you put two GMs up against each other and give one a computer, some portion of games would include an obvious computer line that a ~2800 FIDE human wouldn't evaluate the same way as an engine.
> The same with opening theory: when I'm using my opening preparation, I'm playing at GM level as these are just the moves played by GMs in that position, and I did not need to find/calculate them, I just know them.
I wouldn't count that as playing the opening at GM level unless you understand why GMs play those moves.
Around 1990 there was a chess teacher and coach named Richard Shorman that would come to a public chess club that met weekly in Sunnyvale and give free advice to people who had hit plateaus and just couldn't seem to get better no matter how much they played and studied and analyzed. People would show him their games from recent tournaments and he'd analyze them and give advice to get unstuck. This was all out in the open so even those of us who had not brought games could watch and learn.
The people attending these sessions typically ranged from beginners who if they were rated were somewhere under 1000 USCF all the way to people in the 2000-2200 range who had been stuck in that range for years.
On of the big problems Shorman found with pretty much everyone there was that everyone wanted to play like a Karpov or a Kasparov. They studied the opening such players played, memorized all the variants of those openings from ECO, bought and read books on those openings, and studied the games with those openings from top tournaments.
So yeah...they might play 25 moves of a game just like Karpov or Kasparov would have because they are copying from a Karpov of Kasparov game that followed the same line. But what happens when their opponent plays a bad move? If it is so bad there is an immediate tactical refutation maybe they find that (especially if they are around 2100). But at the Karpov/Kasparov level there are a lot of bad moves where the move isn't bad for some short term tactical reason. It's bad because it gives some small weakness that a GM over the course of the next 20 or 30 moves can exploit to eventually allow some winning tactic.
And if your opponent does stay in your opening book to the end...then you just find yourself in a position that is supposed to be good for you, but without knowing why. A GM would know why they are better and how to use that.
It always does eventually come down to tactics, but the deeper you understand tactics the more you start to understand positional concepts and how they make it so certain tactics will or will not work. You can't really understand the positional stuff until you understand the tactical stuff. When you try to play like a GM too soon, you don't yet have the tactical skill to understand the positional stuff, and you get stuck.
One way Shorman put it was something like "Before you can play good chess you have to be good at playing bad chess".
For the lower rated players Shorman would tell them to play gambits. They might not be sound against high rated players but that's not who lower rated players are playing. They should be aiming for unbalanced positions and playing the most aggressive moves that they can't see a tactical refutation for.
As for books, what he'd tell the lower rated players to get was a collection of Morphy's games and skip to the end where it has the games where he gave odds or was playing simuls against amateurs--the games where Morphy needed to crush people.
That's what he meant by "bad chess"...the kind of chess people played in the 19th century.
For higher rated people, like a couple friends of mine who were stuck around 2000-2100, he'd still tell them to play unbalanced openings and play aggressively, but not in the balls out channeling Morphy way that worked for the lower rated people. Gambits were still recommended, but now ones that were not played at top level because the other side could equalize or get a slight advantage too easy rather than because they might actually be unsound.
That got both my friends off the long standing plateaus.
I was only around 1600, and wasn't playing tournament chess anymore (I was instead playing in Go events at the Palo Alto Go Club), so never got a chance to see if Shorman could get me unstuck, but I brought a list of my chess books to Shorman to see which if any I should actually read.
Younger chess players might not realize just how many chess books even casual chess players would accumulate back in the days before internet. Here was my list [1], and this was by no means a large collection for someone around 1600.
Shorman praised a few of them as good books that could teach a lot--and then told me to set them aside and read specific ones of them such as "My System" whenever I hit 2200, and in the meantime go get Morphy's games and skip to the odds games. (I never did get back into tournament chess, except for a couple of events).
I think Shorman's points and methods are still sound, but now with internet and online chess and computers that can automatically generate tactics training from real games we can probably go about applying them more efficiently.
We don't have to seek unbalanced positions and play aggressively in them in order to get tactical practice now--we've got tactics trainers. And now we can play more serious games against good opponents in a week then we might be able to get in a whole year of tournaments in 1990.
[1] https://pastebin.com/mw3q1784
I wouldn't count that as playing the opening at GM level unless you understand why GMs play those moves.
Around 1990 there was a chess teacher and coach named Richard Shorman that would come to a public chess club that met weekly in Sunnyvale and give free advice to people who had hit plateaus and just couldn't seem to get better no matter how much they played and studied and analyzed. People would show him their games from recent tournaments and he'd analyze them and give advice to get unstuck. This was all out in the open so even those of us who had not brought games could watch and learn.
The people attending these sessions typically ranged from beginners who if they were rated were somewhere under 1000 USCF all the way to people in the 2000-2200 range who had been stuck in that range for years.
On of the big problems Shorman found with pretty much everyone there was that everyone wanted to play like a Karpov or a Kasparov. They studied the opening such players played, memorized all the variants of those openings from ECO, bought and read books on those openings, and studied the games with those openings from top tournaments.
So yeah...they might play 25 moves of a game just like Karpov or Kasparov would have because they are copying from a Karpov of Kasparov game that followed the same line. But what happens when their opponent plays a bad move? If it is so bad there is an immediate tactical refutation maybe they find that (especially if they are around 2100). But at the Karpov/Kasparov level there are a lot of bad moves where the move isn't bad for some short term tactical reason. It's bad because it gives some small weakness that a GM over the course of the next 20 or 30 moves can exploit to eventually allow some winning tactic.
And if your opponent does stay in your opening book to the end...then you just find yourself in a position that is supposed to be good for you, but without knowing why. A GM would know why they are better and how to use that.
It always does eventually come down to tactics, but the deeper you understand tactics the more you start to understand positional concepts and how they make it so certain tactics will or will not work. You can't really understand the positional stuff until you understand the tactical stuff. When you try to play like a GM too soon, you don't yet have the tactical skill to understand the positional stuff, and you get stuck.
One way Shorman put it was something like "Before you can play good chess you have to be good at playing bad chess".
For the lower rated players Shorman would tell them to play gambits. They might not be sound against high rated players but that's not who lower rated players are playing. They should be aiming for unbalanced positions and playing the most aggressive moves that they can't see a tactical refutation for.
As for books, what he'd tell the lower rated players to get was a collection of Morphy's games and skip to the end where it has the games where he gave odds or was playing simuls against amateurs--the games where Morphy needed to crush people.
That's what he meant by "bad chess"...the kind of chess people played in the 19th century.
For higher rated people, like a couple friends of mine who were stuck around 2000-2100, he'd still tell them to play unbalanced openings and play aggressively, but not in the balls out channeling Morphy way that worked for the lower rated people. Gambits were still recommended, but now ones that were not played at top level because the other side could equalize or get a slight advantage too easy rather than because they might actually be unsound.
That got both my friends off the long standing plateaus.
I was only around 1600, and wasn't playing tournament chess anymore (I was instead playing in Go events at the Palo Alto Go Club), so never got a chance to see if Shorman could get me unstuck, but I brought a list of my chess books to Shorman to see which if any I should actually read.
Younger chess players might not realize just how many chess books even casual chess players would accumulate back in the days before internet. Here was my list [1], and this was by no means a large collection for someone around 1600.
Shorman praised a few of them as good books that could teach a lot--and then told me to set them aside and read specific ones of them such as "My System" whenever I hit 2200, and in the meantime go get Morphy's games and skip to the odds games. (I never did get back into tournament chess, except for a couple of events).
I think Shorman's points and methods are still sound, but now with internet and online chess and computers that can automatically generate tactics training from real games we can probably go about applying them more efficiently.
We don't have to seek unbalanced positions and play aggressively in them in order to get tactical practice now--we've got tactics trainers. And now we can play more serious games against good opponents in a week then we might be able to get in a whole year of tournaments in 1990.
[1] https://pastebin.com/mw3q1784
> everyone wanted to play like a Karpov or a Kasparov.
I wonder if there's a modern analog to this with how super GM styles have mostly converged. Trying to play like Magnus is as silly as trying to play like an engine. Even the most aggressive players (Nepo, Shak?, Rapport) aren't so wildly different in style.
Maybe the 2010-2020s version of this is bandwagoning onto popular theory, like all the Najdorf lines I know I'll never understand.
I wonder if there's a modern analog to this with how super GM styles have mostly converged. Trying to play like Magnus is as silly as trying to play like an engine. Even the most aggressive players (Nepo, Shak?, Rapport) aren't so wildly different in style.
Maybe the 2010-2020s version of this is bandwagoning onto popular theory, like all the Najdorf lines I know I'll never understand.
> High level humans and chess engines play differently. You can see commentary on chess YouTube videos when they run across cheaters.
That's similar to people describing how to catch other frauds, such as fake Amazon comments or bots. It's medieval 'science': They usually have no evidence of their accuracy, either false negatives (frauds who they overlook) and false positives (people falsely accused of fraud). So it's easy to say, 'this is how to identify them' - nobody will ever test your claim.
Regarding false negatives, for example, there is reason to believe that people detect only the obvious frauds, and that our detection becomes tuned for the obvious. Regarding false positives, people will cite the 'obvious' positives - e.g., some humanly impossible property - but even if they are correct, the problem is the cases in the grey area. False accusations are no joke.
Ironically, now we want a bot to solve our problems. What data do we have to say that it's accurate, or any more accurate than we are?
That's similar to people describing how to catch other frauds, such as fake Amazon comments or bots. It's medieval 'science': They usually have no evidence of their accuracy, either false negatives (frauds who they overlook) and false positives (people falsely accused of fraud). So it's easy to say, 'this is how to identify them' - nobody will ever test your claim.
Regarding false negatives, for example, there is reason to believe that people detect only the obvious frauds, and that our detection becomes tuned for the obvious. Regarding false positives, people will cite the 'obvious' positives - e.g., some humanly impossible property - but even if they are correct, the problem is the cases in the grey area. False accusations are no joke.
Ironically, now we want a bot to solve our problems. What data do we have to say that it's accurate, or any more accurate than we are?
> It's medieval 'science': They usually have no evidence of their accuracy,
I mean, not really. The difference between human and computer chess play-styles is well-documented, to the extent that in the earlier days of chess engines, human chess disciplines were developed to counter the way computers play ("anti-computer chess").
I mean, not really. The difference between human and computer chess play-styles is well-documented, to the extent that in the earlier days of chess engines, human chess disciplines were developed to counter the way computers play ("anti-computer chess").
If it makes you feel better (or worse), signal "fingerprinting" is used in laboratory science to verify things like purity and identity. Detecting a lack of divergence from a known chess bot seems like a good fingerprint to me.
People have been asking for a transparency report to verify accuracy for various countermeasures for awhile. The platforms simply won't release that kind of information (chess.com & lichess.com)
They have an incentive to show their game isn't a den of cheaters, and yet they don't release which means there is a stronger incentive to hide that information.
Makes you wonder what kind of incentives are preventing them from releasing that information. Marketing says 100's of millions of games. Are they games between two people, or potentially a lot of matches against computers (where you don't know they are computers up front). Food for thought.
They have an incentive to show their game isn't a den of cheaters, and yet they don't release which means there is a stronger incentive to hide that information.
Makes you wonder what kind of incentives are preventing them from releasing that information. Marketing says 100's of millions of games. Are they games between two people, or potentially a lot of matches against computers (where you don't know they are computers up front). Food for thought.
Thing is... if your methods work well enough to help you avoid the thing you wish to avoid; does it matter if it's right or wrong? It works, right?
> if your methods work well enough to help you avoid the thing you wish to avoid; does it matter if it's right or wrong?
You don't know if it's helping you at all; that's the issue. The latter question is a bit bizarre.
You don't know if it's helping you at all; that's the issue. The latter question is a bit bizarre.
Not lichess, but Alireza Firouzja (World Rank #3) was banned from chess.com when he was younger.
It was some time ago so probably their cheat engine detection, and also lichess's should give less false positives.
It was some time ago so probably their cheat engine detection, and also lichess's should give less false positives.
> Not lichess, but Alireza Firouzja (World Rank #3) was banned from chess.com when he was younger.
This has turned into a bit of an urban legend. The automated system flagged him based on both his rapid rise in rating and reports from several verified titled players.
On inspection by a human, he was cleared.
Danny exaggerates when he tells the story cause it's a funny anecdote.
This was also a very old version of the anti-cheat like you mentioned. Personally, part of the reason I prefer chess.com is their much better cheat detection than Lichess.
This has turned into a bit of an urban legend. The automated system flagged him based on both his rapid rise in rating and reports from several verified titled players.
On inspection by a human, he was cleared.
Danny exaggerates when he tells the story cause it's a funny anecdote.
This was also a very old version of the anti-cheat like you mentioned. Personally, part of the reason I prefer chess.com is their much better cheat detection than Lichess.
My understanding is that that was not about his actual play, but instead about his rapid rise in rating.
I would assume they verify titled players to avoid any potential liability or defamation legal issues but they aren't the most professional bunch so who really knows. There are a lot of questionable practices they've done over the years as an organization.
As for Irwin, its detection rate, and thresholds, it has fairly high false-positive rates inherent in the model.
From my experience, if you get banned don't expect any kind of due process. They aren't professional, they don't respond. Not even the legal contact on their charity.
Not everyone gets banned for cheating.
They do ban people from the lichess site for many other unprofessional reasons such as contributing to the project (when they don't like what you had to say on an issue and you weren't spamming).
I have to wonder how many similar-named accounts got axed alongside mine when they decided to go after me for what I disclosed. Its not like my github username was connected in any way to my lichess account (or that similarly named).
Needless to say, I don't use their platform anymore because its more bots than anything else, and I don't volunteer my time to people that don't deserve it.
As a side note, their lack of standard professional practices make me wonder what kind of fraud is actually going on behind the scenes.
As a business, the only arguably beneficial reason for not following GAAP+ other standard business practices is to commit a fraud.
As for Irwin, its detection rate, and thresholds, it has fairly high false-positive rates inherent in the model.
From my experience, if you get banned don't expect any kind of due process. They aren't professional, they don't respond. Not even the legal contact on their charity.
Not everyone gets banned for cheating.
They do ban people from the lichess site for many other unprofessional reasons such as contributing to the project (when they don't like what you had to say on an issue and you weren't spamming).
I have to wonder how many similar-named accounts got axed alongside mine when they decided to go after me for what I disclosed. Its not like my github username was connected in any way to my lichess account (or that similarly named).
Needless to say, I don't use their platform anymore because its more bots than anything else, and I don't volunteer my time to people that don't deserve it.
As a side note, their lack of standard professional practices make me wonder what kind of fraud is actually going on behind the scenes.
As a business, the only arguably beneficial reason for not following GAAP+ other standard business practices is to commit a fraud.
> Do these sorts of folks have to be verified in some fashion?
As far as I understand, yes. In Chess.com at least (not sure about Lichess), there will be some kind of human verification for very high-level players, asking to prove that they are IMs or GMs. They can stay anonymous, but they will have the rank appear IIRC. I'm not sure on all the details, I'm very (very very) far from that level :)
As far as I understand, yes. In Chess.com at least (not sure about Lichess), there will be some kind of human verification for very high-level players, asking to prove that they are IMs or GMs. They can stay anonymous, but they will have the rank appear IIRC. I'm not sure on all the details, I'm very (very very) far from that level :)
[deleted]
Unless they've made significant improvements, this thing is borderline useless.
About all it does is act as a nice PR/gimmick. Magic thinking.
When last I checked about 5 years ago, they were including client-based stats in the analysis (time per move, focus, and timing between clicks to pick a piece up and drop a piece).
The issue with doing that is the client is fungible, you can set any state you want locally; and so by manipulating these stats you could skew the input going into these models.
To give people a layman's short overview of the model:
It uses a several CNN Pooling Layers for feature detection and LSTM (attention embeddings) in a siamese network architecture.
CNN models notoriously fail to detect features that that are larger than its kernel size.
The LSTM embeddings can only be as useful as the features that it is trying to detect.
There were a number of problems with the model at the time, a high false positive rate, overfitting, and too much weight was being given to client-controllable input.
When the client-controllable input was held stable (constant). Certain book openings would be skewed with additional weight towards a false positive, and earlier book moves activated more often than lesser seen positions.
I brought the issues up to ornicar years ago in an issue, but they closed the issue without comment. The posts are mostly gone now.
There were a number of disagreements (mostly about certain people with admin privileges abusing their authority, not sure if it was ornicar or one of their flunkies; either way not important just completely unprofessional).
I guess it was good enough for them despite the high false positive rate (driving account churn).
About all it does is act as a nice PR/gimmick. Magic thinking.
When last I checked about 5 years ago, they were including client-based stats in the analysis (time per move, focus, and timing between clicks to pick a piece up and drop a piece).
The issue with doing that is the client is fungible, you can set any state you want locally; and so by manipulating these stats you could skew the input going into these models.
To give people a layman's short overview of the model:
It uses a several CNN Pooling Layers for feature detection and LSTM (attention embeddings) in a siamese network architecture.
CNN models notoriously fail to detect features that that are larger than its kernel size.
The LSTM embeddings can only be as useful as the features that it is trying to detect.
There were a number of problems with the model at the time, a high false positive rate, overfitting, and too much weight was being given to client-controllable input.
When the client-controllable input was held stable (constant). Certain book openings would be skewed with additional weight towards a false positive, and earlier book moves activated more often than lesser seen positions.
I brought the issues up to ornicar years ago in an issue, but they closed the issue without comment. The posts are mostly gone now.
There were a number of disagreements (mostly about certain people with admin privileges abusing their authority, not sure if it was ornicar or one of their flunkies; either way not important just completely unprofessional).
I guess it was good enough for them despite the high false positive rate (driving account churn).
Related to this, there's a really good talk by the founder of lichess that includes an overview of the cheating problem, and the techniques they use to detect and manage it.
The relevant section of the video on YouTube is: https://www.youtube.com/watch?v=LZgyVadkgmI&t=1080s
The relevant section of the video on YouTube is: https://www.youtube.com/watch?v=LZgyVadkgmI&t=1080s
If the system suspects a cheater, maybe it can just match them up against a GM (or beyond GM) bot to confirm?
You normally specify a rating range of the opponent you're searching for, so if the cheater isn't targetting GM level himself (in which case it's happening already), the system can't really do it.
Of course the system could go ahead anyway, displaying a fake rating - within the search criteria - for this GM / bot, but given that there are always false positives (and it would have to be done for merely suspected accounts, as you said yourself), it would essentially mean deliberately pairing some innocent players against, well, effectively cheaters. Which kind of defeats the core purpose.
Of course the system could go ahead anyway, displaying a fake rating - within the search criteria - for this GM / bot, but given that there are always false positives (and it would have to be done for merely suspected accounts, as you said yourself), it would essentially mean deliberately pairing some innocent players against, well, effectively cheaters. Which kind of defeats the core purpose.
This is really cool. Rampant cheating is the reason I completely abandoned playing chess, so nice to see something happening.
Where did you experience rampant cheating? I don't remember any particular problems back when I played on ICC in my middle/high school days, and modern equivalents like chess.com and lichess seem to do a relatively good job at catching cheaters. I might have just gotten lucky though.
I never notice cheaters too, but I only play around 1500 FIDE rating. The cheaters are around 2300 FIDE or even higher.
I'm of similar strength, and I have occasionally been informed that some account got banned for cheating (you receive a notification because the system restores your lost rating points after that).
But it's definitely affected less than 1 in a 1000 games for me. Of course
I might have still gotten beaten by cheaters that went undetected of course, especially if they stopped cheating or lost interest in the whole thing leaving the site before they'd get caught.
But it's definitely affected less than 1 in a 1000 games for me. Of course
I might have still gotten beaten by cheaters that went undetected of course, especially if they stopped cheating or lost interest in the whole thing leaving the site before they'd get caught.
Same reason why I stopped playing Scrabble apps.
I'm in a similar boat, I used to play ~ 50 games a day and I'm down to about 3 games a week now. No weekend play.
Once you get to ~1900 FIDE level of play and rework your repertoire for anti-engine strategy you realize just how much rampant cheating is happening.
This continues until at some point you have to ask yourself, do you want to waste your time volunteering to be a victim to satisfy someone else's severe mental illness.
They get their dopamine hit for psychologically abusing you with a tool they didn't create, often with some amount of delusion to justify it.
What do you get out of playing a game against a Super-GM rated computer? Nothing but time theft.
The current state of the game is without a guarantee that the game is between two humans, the game isn't actually chess. The providers for online chess don't make that guarantee.
So based on the incentives, the game you play on their services is more likely to be not-chess but they still call it chess. Deceitful bunch.
Once you get to ~1900 FIDE level of play and rework your repertoire for anti-engine strategy you realize just how much rampant cheating is happening.
This continues until at some point you have to ask yourself, do you want to waste your time volunteering to be a victim to satisfy someone else's severe mental illness.
They get their dopamine hit for psychologically abusing you with a tool they didn't create, often with some amount of delusion to justify it.
What do you get out of playing a game against a Super-GM rated computer? Nothing but time theft.
The current state of the game is without a guarantee that the game is between two humans, the game isn't actually chess. The providers for online chess don't make that guarantee.
So based on the incentives, the game you play on their services is more likely to be not-chess but they still call it chess. Deceitful bunch.
Perhaps you could clarify what you mean by cheating?
People using an engine to tell them what moves to make against their opponent.
Failing to follow the rules set forth for chess.
Specifically, in this context receiving outside assistance directly or indirectly, that in any way impacts your decisions made during the currently on-going game.
This includes more than just lines and moves, such as the exact evaluation score of a given position.
Specifically, in this context receiving outside assistance directly or indirectly, that in any way impacts your decisions made during the currently on-going game.
This includes more than just lines and moves, such as the exact evaluation score of a given position.
Probably isn't very good and chalks it up to cheating. Chess takes years to get good at. It's a slow slow grind.
Playing people who take 10 seconds every move— whether an obvious “only move” or the most complicated situations… and that pick the move that stockfish makes almost all of the time… are cheating.
Depends I like to slow play. It can frustrate someone to play quicker and make mistakes
Yah, I know about slow play, too, and it's effective against me when I'm in the mode "Eh, I didn't really have time for this game, but one more!"
But -- if you're over 2000, surely you've encountered cheaters and understand the difference that I'm driving at.
It's rather blatant, and on Chess.com I almost always get an email a couple weeks later that they were removed for cheating and that I got rating points back, and when I look back on Lichess I often find them banned. I don't even bother reporting anymore.
If it's early enough in the game that I realize it, I can just get into a closed, very-slow-to-progress position and watch them clock flag. But that's still a waste of 10 minutes of my life.
But -- if you're over 2000, surely you've encountered cheaters and understand the difference that I'm driving at.
It's rather blatant, and on Chess.com I almost always get an email a couple weeks later that they were removed for cheating and that I got rating points back, and when I look back on Lichess I often find them banned. I don't even bother reporting anymore.
If it's early enough in the game that I realize it, I can just get into a closed, very-slow-to-progress position and watch them clock flag. But that's still a waste of 10 minutes of my life.
The point is not that they take their time, the point is that they spend roughly the same fixed amount of time for difficult moves (coming up with seemingly brilliant ones now and then) as trivial ones (even in positions when there's physically only one legal move). This is one of the red flags implying somebody is cheating, and doesn't even understand the position on the board. Their pace is monotonous because the time they need for copying the move over to their engine and then the engine's response is more or less constant.
> nice to see something happening
the latest ("real") commit is from 3 years ago
the latest ("real") commit is from 3 years ago
This isn't the repository which runs on Lichess. The actual one does have much newer commits. Though not sure much fundamentally changed. Ultimately, I guess it does what it's supposed to do (catch obvious cheaters to save human time). But this is only one part of a lot of anti-cheating measures, including a lot more automated ones (some, like kaladin, which actually is a recent development, were also linked in the other thread about Lichess) but also a huge amount of human work. Most of the people here or in the other thread don't really know anything about cheating, how to properly detect it, and the difficulty of it.
Though ofc you're certainly right that this isn't a recent development at all and at the same time it's certainly also not perfect yet. And I doubt it ever will be. I don't think cheating in online chess is something that can ever really be solved completely.
Though ofc you're certainly right that this isn't a recent development at all and at the same time it's certainly also not perfect yet. And I doubt it ever will be. I don't think cheating in online chess is something that can ever really be solved completely.
How do you cheat at chess? Serious question… there are fixed rules that govern the game, certainly a piece of software or in the human case 3rd party observer should be able to enforce them? In other words, given a list of moves, you can write a program that returns `valid` if the set of moves is allowed or `invalid` if it is not, no?
Simple: don’t play moves you discover independently, and instead use an advanced chess AI to tell you what to play. You play your opponent’s moves against an alternative AI program, and then play its moves back in your game against the human.
I don’t know why people do this. It’s not like it makes them better at chess.
I don’t know why people do this. It’s not like it makes them better at chess.
The same reasons cheating is rampant in all online videogames, I imagine
Oh, wait, this must be cheating in the sense of “I used a computer to assist my brain in determining my move”. I guess that is a new type of problem in online chess…
Open up high level bot in another tab. Play their moves and copy what the bot does in response.
I have no idea why people do it, it's not even like video game cheats where it gives people a advantage but you're still in control - you're literally just copying.
I have no idea why people do it, it's not even like video game cheats where it gives people a advantage but you're still in control - you're literally just copying.
They don't even need to open another tab. There is at least one Google Chrome extension specifically for cheating in chess (which the creators pretty much openly admit - their YT channel features a blatant cheating tutorial video).
I've reported this extension for abuse - since it's obviously unethical, goes against the rules of the websistes it's supposed to be used on, plus the description in the Store is misleading, as it's deliberately vague and unlike their YT profile, it cautiously doesn't mention cheating at all. Obviously it hasn't been taken down though.
I've reported this extension for abuse - since it's obviously unethical, goes against the rules of the websistes it's supposed to be used on, plus the description in the Store is misleading, as it's deliberately vague and unlike their YT profile, it cautiously doesn't mention cheating at all. Obviously it hasn't been taken down though.
Simple. You use an engine to decide what moves you make instead of your own knowledge and skill. It's depressingly common.
[deleted]
There are multiple permissible moves. Cheaters use a computer to identify the best one.
Now we have a computer effectively accusing people of fraud (if I understand correctly). How does Irwin address the obvious risks and dangers?
What obvious risks and dangers?
Not sure if it's the same at Lichess, but chess.com flags potential cheaters, and they get reviewed by experts to determine if they are just a very strong player or a cheater.
A few of younger up and coming players were flagged.
A few of younger up and coming players were flagged.
[deleted]
I love chess. At my absolute best, I had a over-the-board USCF rating of 2156. I never made it to master no matter how hard I tried. So, ticked off, I gave up serious play. Years go by and while I'm no where near as good these days, I started enjoying playing again. And, then I start running into people online who were clearly using engines instead of their own skills and wits. Now, I only pay with friends and over-the-board. If this can help wipe out the cheats, I may finally go back to playing online again.
What time controls do you play? Cheating is much more prevalent in rapid than blitz, with correspondence probably being the worst of all. It's unlucky for people who don't like speed chess.
The only solution to this problem I'm aware of is developing/becoming a part of a community of people you trust. Of course, that's hard and comes with plenty of limitations.
Lastly, please don't get your expectations up. This is a hugely unsolved problem on lichess and the history of its and chesscom's efforts do not inspire confidence. I noticed this repo has no updates in the past three years, during which lichess has maintained its reputation of having tons of cheaters. Chesscom has a team and probably more tools (proprietary, larger company, etc.) but the best case scenario still involves cheaters getting several games in, wasting your time along the way.
The only solution to this problem I'm aware of is developing/becoming a part of a community of people you trust. Of course, that's hard and comes with plenty of limitations.
Lastly, please don't get your expectations up. This is a hugely unsolved problem on lichess and the history of its and chesscom's efforts do not inspire confidence. I noticed this repo has no updates in the past three years, during which lichess has maintained its reputation of having tons of cheaters. Chesscom has a team and probably more tools (proprietary, larger company, etc.) but the best case scenario still involves cheaters getting several games in, wasting your time along the way.
I have been playing daily for about two years now on Lichess and over-the-board whenever the opportunity presents itself. I find people who burn the clock when their game takes a turn for the worse far more frustrating than those who cheat their way to victory. At least with the cheaters, the game is over pretty quickly, so I can move onto a game which is actually enjoyable.
I got into Lichess myself about a year ago, after not really having played much chess in the past 20 or 20 years. Oddly enough, it was people on HN who persuaded me to give online chess a chance. I'd never bothered previously, as I couldn't see how you would ever know the other player wasn't getting their moves from a chess app. HNers persuaded me to give it a go because Lichess had algorithms in place which detected cheating.
Having now played nearly 2000 games on Lichess, I'm pretty happy with their ability to weed out cheaters. I've very rarely had cause to suspect my opponent wasn't playing fair. Maybe once or twice someone with a lower ranking than me seemed to be a bit too good --but then we all have our flashes of inspiration, as well as our off days.
However, on a couple of occasions, I've logged onto the site to see a notification that I've had some ranking points restored, as a previous opponent had cheated. But, unfortunately, Lichess doesn't tell you which opponent or which game, so I'm still none the wiser.
PS: Agree with you about the people who run the clock down, when they're losing. Really annoying. When this happens, I usually switch to another browser tab and read intarwebs instead, switching back to the Lichess tab every 5 mins or so, to see if they've made a move yet.
Having now played nearly 2000 games on Lichess, I'm pretty happy with their ability to weed out cheaters. I've very rarely had cause to suspect my opponent wasn't playing fair. Maybe once or twice someone with a lower ranking than me seemed to be a bit too good --but then we all have our flashes of inspiration, as well as our off days.
However, on a couple of occasions, I've logged onto the site to see a notification that I've had some ranking points restored, as a previous opponent had cheated. But, unfortunately, Lichess doesn't tell you which opponent or which game, so I'm still none the wiser.
PS: Agree with you about the people who run the clock down, when they're losing. Really annoying. When this happens, I usually switch to another browser tab and read intarwebs instead, switching back to the Lichess tab every 5 mins or so, to see if they've made a move yet.
> At least with the cheaters, the game is over pretty quickly
Yeah, unless you want to play a longer game than a 5 minute blitz.
Yeah, unless you want to play a longer game than a 5 minute blitz.
I don’t understand this sentiment.
The rating of a player will reflect their strength, regardless of whether they are an unaided human, a full computer, or a human with some heuristic to consult an engine. Whatever the player is doing, their rating will reflect their average strength, and the website matches us to players of similar rating. Whether or not a player is pure human, pure engine, or centaur, we get matched to a player that we have a roughly even chance of beating.
Pure cheaters will quickly skyrocket to the top of the ratings and I will never see them. Hybrid cheaters, who do not have a perfect rating, have a probability of losing because their heuristic to switch to an engine is imperfect, and their rating reflects this imperfection.
Whatever the player is doing, their numeric rating reflects their average strength, and if you have a similar rating you have roughly even chances of beating them.
The rating of a player will reflect their strength, regardless of whether they are an unaided human, a full computer, or a human with some heuristic to consult an engine. Whatever the player is doing, their rating will reflect their average strength, and the website matches us to players of similar rating. Whether or not a player is pure human, pure engine, or centaur, we get matched to a player that we have a roughly even chance of beating.
Pure cheaters will quickly skyrocket to the top of the ratings and I will never see them. Hybrid cheaters, who do not have a perfect rating, have a probability of losing because their heuristic to switch to an engine is imperfect, and their rating reflects this imperfection.
Whatever the player is doing, their numeric rating reflects their average strength, and if you have a similar rating you have roughly even chances of beating them.
I disagree. The centaur might have a rating of 2150, but they will sporadically play a 3300 rated engine move. This completely breaks the rating system's probabilities.
This is like boxing in the lightweight division, but your opponent can, at will, use Mike Tyson in his prime to hit you in the face.
This is like boxing in the lightweight division, but your opponent can, at will, use Mike Tyson in his prime to hit you in the face.
> Pure cheaters will quickly skyrocket to the top of the ratings
Unless they cheat in unrated games, or discard their account and create another before the rating ascends too high (not necessarily for that purpose, but to avoid detection).
Unless they cheat in unrated games, or discard their account and create another before the rating ascends too high (not necessarily for that purpose, but to avoid detection).
[deleted]
You should be happy with that! That's like candidate master level. Real impressive
I don’t understand what someone gets out of winning chess games by cheating. It would actually make me feel worse about myself.
There are people who’s sense of self-confidence is so fragile that they cannot allow themselves the possibility of losing or feeling “lesser than” in any context. Cheating and being banned is much better than showing the world your actual skill level and confronting the possibility of failure and judgment. The thought process goes something like “well if I beat them by cheating they’re a sucker for actually trying. And if I get banned then that’s the system out to get me”
Such a person often ends up making an absolute fool of themselves from the perspective of someone with a little bit more self integrity. But maybe it is worth extending a little compassion, after all a lack of compassion is probably how they ended up like that anyhow.
Such a person often ends up making an absolute fool of themselves from the perspective of someone with a little bit more self integrity. But maybe it is worth extending a little compassion, after all a lack of compassion is probably how they ended up like that anyhow.
There are people who’s sense of self-confidence is so fragile that they cannot allow themselves the possibility of losing or feeling “lesser than” in any context.
Happened to a good friend of mine up in Seattle. Good fella, incredibly smart, graduated top marks, a bit ostentatious sometimes but you can tell he really wants to do right by others in spite of his sometimes high falutin tendencies.
Anyway.
Lost a game of chess to his blue collar dad, guy couldn’t handle it. Fell into a hole of depression that lasted for at least an entire episode of Cheers
Happened to a good friend of mine up in Seattle. Good fella, incredibly smart, graduated top marks, a bit ostentatious sometimes but you can tell he really wants to do right by others in spite of his sometimes high falutin tendencies.
Anyway.
Lost a game of chess to his blue collar dad, guy couldn’t handle it. Fell into a hole of depression that lasted for at least an entire episode of Cheers
Happens to mainly two sorts of people, in my experience:
* Overperforming kids (whether smart, in athletics, etc). They don't have to confront failure and work hard like everyone else. They get a lot of praise and self-worth gets tied to it, and it all feels kind of fake/too easy. Then, one day, the work ethic and tolerance of failure everyone else has been learning become critical...
* People raised in abusive or quasi-abusive environments, where success at any cost, appearances over all, etc, are emphasized. The ends justify the means to get the required outcome.
* Overperforming kids (whether smart, in athletics, etc). They don't have to confront failure and work hard like everyone else. They get a lot of praise and self-worth gets tied to it, and it all feels kind of fake/too easy. Then, one day, the work ethic and tolerance of failure everyone else has been learning become critical...
* People raised in abusive or quasi-abusive environments, where success at any cost, appearances over all, etc, are emphasized. The ends justify the means to get the required outcome.
In a sibling comment there is already a very adequate answer.
To give the opposite answer, why you should not cheat. Chess is fun if you play against players of similar strengt. Mostly a score of around 50% will be what you get. If you have a 1400 rating, you can have fun games against 1400 players. If you cheat and go towards 2300 rating, you will still have a score of 50% against other 2300 players. But all the fun of the game won't be there, since it is just cheating. You will lose something (fun), but not really win anything within the game itself.
Or simpler said, in a long gone past I got bored with playing Doom. I hunted down cheat codes and got bored even faster.
To give the opposite answer, why you should not cheat. Chess is fun if you play against players of similar strengt. Mostly a score of around 50% will be what you get. If you have a 1400 rating, you can have fun games against 1400 players. If you cheat and go towards 2300 rating, you will still have a score of 50% against other 2300 players. But all the fun of the game won't be there, since it is just cheating. You will lose something (fun), but not really win anything within the game itself.
Or simpler said, in a long gone past I got bored with playing Doom. I hunted down cheat codes and got bored even faster.
>Or simpler said, in a long gone past I got bored with playing Doom. I hunted down cheat codes and got bored even faster.
This is a life hack if ever you find yourself too distracted with a game (for me, Rimworld). Mod / cheat to the point of ruining the game, and you'll likely never play it again. That being said, I do not advocate doing this in multiplayer games.
This is a life hack if ever you find yourself too distracted with a game (for me, Rimworld). Mod / cheat to the point of ruining the game, and you'll likely never play it again. That being said, I do not advocate doing this in multiplayer games.
The only game this doesn't seem to work on for me is Civilization; since I am often fixing the game now to work better or how it should have to begin with.
Most cheaters probably do get bored pretty quickly, but even if they only cheat for, say, 5 games, by the time they're over and done with it another curious cheater takes their place to see what it's like.
So yes the turnover may be pretty high, but it doesn't make much difference from the perspective of a regular user.
So yes the turnover may be pretty high, but it doesn't make much difference from the perspective of a regular user.
This assumes that you can't have fun while cheating
You can, but the fun will be different because the challenges are different.
The cheaters aren't playing chess, but some other unnamed game
You can, but the fun will be different because the challenges are different.
The cheaters aren't playing chess, but some other unnamed game
in competitive games it's all about the social credit
the game might a big part of their life, and being perceived as a good player puts you higher in the pecking order within your friend group
the game might a big part of their life, and being perceived as a good player puts you higher in the pecking order within your friend group
Having cheated at a few competitive multiplayer video games over the years, this is one of a few good answers, particularly when it comes to FPS. That, and many such video games induce boredom so quickly, and almost always the reward at the end is not worth the time spent getting there. Sometimes I'm glad I didn't waste my time getting to end-game content just to find out it was more of the same (or garbage) anyway. I don't play multiplayer games anymore, at all really, unless it's cooperative/integrated like Elden Ring.
I think something like chess is different; cheating in video games feels more similar to hacking around some software just because you can and it's interesting.
I think something like chess is different; cheating in video games feels more similar to hacking around some software just because you can and it's interesting.
I cheated in Arma3. Moneyhax - just wanted to practice flyin` some jets in a decent combat environment without grinding endlessly. Glitched invisibility - real hillarious for couple hours. Occasional item duping to jumpstart combat readyness of our group (somewhat hated this one). Cliff glitching to enter inside of them and store loot on persistent servers. Most of the cheating came out of curiosity - to explore and push the boundaries of game. Nevertheless, the very best Arma3 gaming experience came out of fair and square pwnage operation of our group on public server. Infil via chopper, 2 snipers @ south / east hills, 1 AT/spotter, 4 people assault squad, myself as a commander miles away on an island watching through UAV. We cleared like 20 people in couple minutes and got a thropy tank without taking (!) any casualties. Planned and prepared for about an hour (+ ~3h to build up necessary supplies).
I did maybe 10-15 years ago. There was a chess game on Xbox live and I found a chess program (I forget what it was now) and wanted to try it out. I think I played 2 or 3 games like that, got bored and never did it again. It was more of a "let's see if I can do it" sort of thing, sort of funny and probably more novel of an idea at the time.
I have no idea why you would do it on a ongoing basis. It would be so boring, all you do is transcribe moves back and forth.
I have no idea why you would do it on a ongoing basis. It would be so boring, all you do is transcribe moves back and forth.
Same, and I don't get why cheaters don't just play in a "cheating league" where there are no anti-cheating rules or anything.
What does winning do for someone when they didn't actually achieve mastery of the game?
What does winning do for someone when they didn't actually achieve mastery of the game?
I personally wouldn't have any issue with this. At that point it's all down to who's more clever in their ruses and cheats.
I think it’s an “I am miserable so let’s make other people miserable” thing.
Cheaters often have the mindset that "everyone else is cheating, so it's only fair that I do too".
Just for chess specifically?
just hire Kamsky