Robocalypse Now: Using Deep Learning to Combat Cheating in CSGO [video](gdcvault.com)
gdcvault.com
Robocalypse Now: Using Deep Learning to Combat Cheating in CSGO [video]
http://www.gdcvault.com/play/1024994/Robocalypse-Now-Using-Deep-Learning
32 comments
It should be noted, though, that this ML model is based on Overwatch (that's where they get their training data), which is only good at detecting blatant cheats as well. So they probably are fairly sure already that this system won't be able to catch 100 % of cheaters.
However, as John noted in the beginning, those blatant cheats mostly have one reason to exist: to actively and obviously spoil the fun of others involved, so to improve things to players of that game, it's a very good starting point.
However, as John noted in the beginning, those blatant cheats mostly have one reason to exist: to actively and obviously spoil the fun of others involved, so to improve things to players of that game, it's a very good starting point.
To clarify for anyone reading, "Overwatch" is not referring to the Blizzard game, but an in-game Counterstrike tool in which players review recorded gameplay of a suspected cheater.
When enough reviewers agree that a player is cheating, that person is banned. Each player has a hidden successful conviction rating and on occasion the system will send you professional players or non-cheating players in an effort to verify your ability.
This is only meant to convict the most blatant of cheaters. It's fairly successful for what it's intended to do, but of course, it does not even remotely catch every cheater.
When enough reviewers agree that a player is cheating, that person is banned. Each player has a hidden successful conviction rating and on occasion the system will send you professional players or non-cheating players in an effort to verify your ability.
This is only meant to convict the most blatant of cheaters. It's fairly successful for what it's intended to do, but of course, it does not even remotely catch every cheater.
Thank you. I was very confused by the comment.
> However, as John noted in the beginning, those blatant cheats mostly have one reason to exist: to actively and obviously spoil the fun of others involved, so to improve things to players of that game, it's a very good starting point.
This is very true (and very important), but I don't think you need AI to detect blatant cheats. IMO, simple heuristics can do the job better.
This is very true (and very important), but I don't think you need AI to detect blatant cheats. IMO, simple heuristics can do the job better.
Perhaps with a worse conviction rate of the reports or by catching fewer cheaters. One major part of what makes this successful is that the reports by VACnet are high-quality and very often lead to convictions (i.e. they're not false positives). This keeps the humans who're actually judging engaged. From experience I found Overwatch fairly tedious if there's nothing fishy going on at all for ten cases or so, so participation rate has dropped over time.
gameref looks like a really clever and simple solution as a hardware device.
> After some research and a few POCs I am 100% certain a ML solution will not work in detecting cheats other than completely trivial "spinbots" or "aim-lock" aimbots.
I'm curious why you say this. I'm a machine learning noob, but it seems pretty straightforward that it would work. Relating to wallhacks, they can use your last 1,000 matches, and if you're never/rarely surprised by a camping enemy, you're probably cheating. Your camera position moments before interaction with an obscured enemy is probably another indicator.
> After some research and a few POCs I am 100% certain a ML solution will not work in detecting cheats other than completely trivial "spinbots" or "aim-lock" aimbots.
I'm curious why you say this. I'm a machine learning noob, but it seems pretty straightforward that it would work. Relating to wallhacks, they can use your last 1,000 matches, and if you're never/rarely surprised by a camping enemy, you're probably cheating. Your camera position moments before interaction with an obscured enemy is probably another indicator.
I'm failing to see a use case for gameref. At the big tournaments the computer is provided by the organizer.
For online tournaments, wallhacks and maphacks won't be detected by this device.
> I'm curious why you say this. I'm a machine learning noob, but it seems pretty straightforward that it would work. Relating to wallhacks, they can use your last say 1,000 matches, and if you're never/rarely surprised by a camping enemy, you're probably cheating.
Exactly. And not just that, wouldn't ML in theory also be able to detect subtle reactions to changes that the player shouldn't see? For example if a cheater has a map hack that only shows dots on the minimap, and does some small action like twitch with the mouse when an enemy he's following and can't see or hear ingame, changes direction?
> I'm curious why you say this. I'm a machine learning noob, but it seems pretty straightforward that it would work. Relating to wallhacks, they can use your last say 1,000 matches, and if you're never/rarely surprised by a camping enemy, you're probably cheating.
Exactly. And not just that, wouldn't ML in theory also be able to detect subtle reactions to changes that the player shouldn't see? For example if a cheater has a map hack that only shows dots on the minimap, and does some small action like twitch with the mouse when an enemy he's following and can't see or hear ingame, changes direction?
There was a huge scandal in CSGO a few years ago where people started thinking a top professional player had been cheating in a major LAN, and that was about when he came up with the gameref. The player was banned for cheating, but whether that was on LAN or in online games is still unknown.
The cheat was allegedly injected into the game through an exploit in a custom map which you could download through steam.
The cheat was allegedly injected into the game through an exploit in a custom map which you could download through steam.
> I'm failing to see a use case for gameref. At the big tournaments the computer is provided by the organizer. For online tournaments, wallhacks and maphacks won't be detected by this device.
There have been cases of people smuggling aimbots in onboard mice drivers. There was also a Steam oversight where hacks could be loaded via the workshop feature. A semi-pro German player was caught in 2015 I believe.
There have been cases of people smuggling aimbots in onboard mice drivers. There was also a Steam oversight where hacks could be loaded via the workshop feature. A semi-pro German player was caught in 2015 I believe.
There has never been a proven case of either as far as I know, only theories and "insiders" claiming these things.
The german guy who was caught was saying a lot of these things which obviously weren't true, like claiming that a lot of other players in top teams were cheating on LAN, but in hindsight that hardly seems believable.
A driver cheat which would auto-inject code on insertion would as far as I know require a 0-day exploit on Windows, and I have a hard time believing one of the cheat devs was sitting on one of those.
The german guy who was caught was saying a lot of these things which obviously weren't true, like claiming that a lot of other players in top teams were cheating on LAN, but in hindsight that hardly seems believable.
A driver cheat which would auto-inject code on insertion would as far as I know require a 0-day exploit on Windows, and I have a hard time believing one of the cheat devs was sitting on one of those.
> A driver cheat which would auto-inject code on insertion would as far as I know require a 0-day exploit on Windows, and I have a hard time believing one of the cheat devs was sitting on one of those.
Nowadays there's definitely a market for it. eSports is turning in to a multi-billion dollar industry.
0day's can be cheap enough for the type of people willing to risk five figures on CSGO betting, etc.
Nowadays there's definitely a market for it. eSports is turning in to a multi-billion dollar industry.
0day's can be cheap enough for the type of people willing to risk five figures on CSGO betting, etc.
> The german guy who was caught was saying a lot of these things which obviously weren't true, like claiming that a lot of other players in top teams were cheating on LAN, but in hindsight that hardly seems believable.
I mean, there were 3 pro players banned in like a week. I doubt this was coincidence [1].
> A driver cheat which would auto-inject code on insertion would as far as I know require a 0-day exploit on Windows, and I have a hard time believing one of the cheat makers was sitting on one of those
It's not completely out of the question. Check out this DEF CON hack which illustrates something similar[2].
[1] https://www.pcgamer.com/csgo-competitive-scene-embroiled-in-...
[2] https://theoutline.com/post/2032/how-to-hack-a-mouse-to-win-...
I mean, there were 3 pro players banned in like a week. I doubt this was coincidence [1].
> A driver cheat which would auto-inject code on insertion would as far as I know require a 0-day exploit on Windows, and I have a hard time believing one of the cheat makers was sitting on one of those
It's not completely out of the question. Check out this DEF CON hack which illustrates something similar[2].
[1] https://www.pcgamer.com/csgo-competitive-scene-embroiled-in-...
[2] https://theoutline.com/post/2032/how-to-hack-a-mouse-to-win-...
About [2], yeah, that's the way it could happen without a 0-day, but it really does seem very, very impractical, risky and unlikely.
Out of sf, KQLY and Emilio, it does seem likely that both sf and Emilio would cheat online in order to qualify, but KQLY whose team almost always passed online qualifiers? I have no idea.
If I recall correctly, Emilio was banned months prior to the others though, and he hardly played any major LANs. sf and KQLY were using the same cheat, but there's nothing which makes it seem like it happened on LANs.
Either way, it's impossible to say what happened years ago, but if it did happen, I doubt it kept happening.
Out of sf, KQLY and Emilio, it does seem likely that both sf and Emilio would cheat online in order to qualify, but KQLY whose team almost always passed online qualifiers? I have no idea.
If I recall correctly, Emilio was banned months prior to the others though, and he hardly played any major LANs. sf and KQLY were using the same cheat, but there's nothing which makes it seem like it happened on LANs.
Either way, it's impossible to say what happened years ago, but if it did happen, I doubt it kept happening.
> I'm curious why you say this.
Well the answer is surprisingly simple. Suppose you only have one person that's wallhacking on your team. He's calling out positions for everyone else. He can play terribly, but since the calls are going out, your team still wins. (This is not a hypothetical and has actually happened in high level matches before.)
Well the answer is surprisingly simple. Suppose you only have one person that's wallhacking on your team. He's calling out positions for everyone else. He can play terribly, but since the calls are going out, your team still wins. (This is not a hypothetical and has actually happened in high level matches before.)
How high level can a team get in CS (in terms of national or world rankings) without playing in a controlled environment where this sort of cheating would be more easily detected by local human observers?
Could my team be ranked in the top ten in the US or Korea or wherever if we only ever play via the internet?
Could my team be ranked in the top ten in the US or Korea or wherever if we only ever play via the internet?
> How high level can a team get in CS (in terms of national or world rankings) without playing in a controlled environment where this sort of cheating would be more easily detected by local human observers?
Very high. You can see for yourself here[1]. Anecdotally, I had a player on my team that cheated in an (online) semi-final match. We had no idea, but were moved down a league, were disgraced, and obviously cut him. A year later, he was one of the people that toured the country with the XFX team. I couldn't believe it. Every one knew this guy cheated just a few months prior, now he was offered a contract. I was very active in the Socal LAN scene, and know at least 3-4 players that played professionally and cheated.
As far as your second point is concerned, some aimbots are so tricky, even local human observers couldn't catch them. I saw this proof of concept made by a famous cheat maker where the mouse would "tug" at the direcction of the player through a wall. The movie would be almost imperceptible by a spectator, but definitely "felt" by the player.
[1] https://www.hltv.org/blog/13538/cheating-on-professional-lev...
Very high. You can see for yourself here[1]. Anecdotally, I had a player on my team that cheated in an (online) semi-final match. We had no idea, but were moved down a league, were disgraced, and obviously cut him. A year later, he was one of the people that toured the country with the XFX team. I couldn't believe it. Every one knew this guy cheated just a few months prior, now he was offered a contract. I was very active in the Socal LAN scene, and know at least 3-4 players that played professionally and cheated.
As far as your second point is concerned, some aimbots are so tricky, even local human observers couldn't catch them. I saw this proof of concept made by a famous cheat maker where the mouse would "tug" at the direcction of the player through a wall. The movie would be almost imperceptible by a spectator, but definitely "felt" by the player.
[1] https://www.hltv.org/blog/13538/cheating-on-professional-lev...
> Relating to wallhacks, they can use your last 1,000 matches, and if you're never/rarely surprised by a camping enemy, you're probably cheating. Your camera position moments before interaction with an obscured enemy is probably another indicator.
Do you actually play CS:GO proficiently? Because it doesn't sound like you do.
Do you actually play CS:GO proficiently? Because it doesn't sound like you do.
> (a) move smoothly/naturally and (b) introduce random noise.
Moving naturally is not that easy to code. Otherwise Google would not be able to tell if you are a robot or not simply by analyzing how you click a checkbox.
Moving naturally is not that easy to code. Otherwise Google would not be able to tell if you are a robot or not simply by analyzing how you click a checkbox.
Google can't. They use your history of online activity across their services to decide if you're a robot. If you click the checkbox the exact same way as usual, but do so in an incognito window without any established Google cookies, they require you to go through a captcha.
Off topic: but why the fuck does Google operate two different captchas? One that is useable, and for internal services one that is incredibly difficult top get right on the first 9 attemps
it's also hard to define "move naturally" - if you look for smoothness you'd get a ton of false positive. check this out https://streamable.com/swz3y - twitchiness is a byproduct of scouting for target in motions.
Well, Google can't. Or maybe I am a robot.
what team if you don't mind me asking?
I’d be curious to know if there was a difference in engagement from Overwatchers. I basically stopped doing cases because there was so much junk in queue (rage reporting, skilled players playing against unskilled players, aka ‘smurfing’, etc). But after the introduction of VACnet the case quality went through the roof, i no longer felt like i was wasting my time doing cases and thus do more of them now.
Now cheat by using a generative network to combat the deep learning that combats cheaters.
What a time to be alive
I have long said that the future will be our AI bots vs their AI bots with the humans stuck in the middle of it all just trying to have a life.
I haven't watched this talk yet, but as an avid CS:GO player, this isn't doing much to combat cheaters.
This is a really amusing comment after watching the 9-12 minute segment of the video. :)
For what it's worth, I very rarely run into cheaters. I couldn't specifically identify the last time I ran into one, in fact.
For what it's worth, I very rarely run into cheaters. I couldn't specifically identify the last time I ran into one, in fact.
After some research and a few POCs I am 100% certain a ML solution will not work in detecting cheats other than completely trivial "spinbots" or "aim-lock" aimbots. For example, it by definition cannot catch an entire class of cheats (wall hacks), but it even falters when trying to catch aimbots; the presentation here argues for a "heuristic-based" method, but, again, these methods often fail when (if you look at the source), aimbots are often coded to (a) move smoothly/naturally and (b) introduce random noise.
In my case, I tried to at the very least combat one class of cheats with close to 100% confidence. These kinds of talks are great at conferences, but, as they say, the proof is in the pudding. Cheaters are very hard to catch, and cheat coders are much smarter than generalized AI methods.