AI Learns to Play CS:Go via Large-Scale Behavioural Cloning(arxiv.org)
arxiv.org
AI Learns to Play CS:Go via Large-Scale Behavioural Cloning
https://arxiv.org/abs/2104.04258
40 comments
What they did with Dota 2 was really surprising, the AI did a technique where if it was ahead it would go behind the enemy tower to get 100% of the gold while the enemy would struggle with last hits under the tower and take creep damage. I'd never seen anyone do this before and pro's started copying this after. The way I remember them explaining it was that the AI has something it's trying to maximize for, and it plays millions of games trying slightly different variations which can lead this kind of creativity, pretty wild.
Gameplay examples, found in the PDF:
https://youtu.be/KTY7UhjIMm4
https://youtu.be/KTY7UhjIMm4
Fascinating! You can see it is trying to mimic many gameplay tactics used by CS:GO players, but it's just barely missing the mark. It's like a monkey trying to mimic human behaviour.
Examples:
- Follows the wall contour using the crosshairs when moving around the corners. The only thing wrong here is that the AI is aiming at the wall instead of at the open space next to the wall where the enemy would be.
- After going through a door, it checks all corner to clear the room. But it's very slow in doing this, and would get shot instantly in a real game.
- Adjusts the crosshair constantly to not look at the floor when walking. The problem is that the crosshair is never at headshot level!
I am not bashing on this AI btw, just pointing out that it's actually getting close to real human behaviour! Fascinating!
Something I've noticed is going well is the fact that the AI doesn't spray n' pray but controls the spray pattern and drags the crosshair down. Shoots in bursts!
Examples:
- Follows the wall contour using the crosshairs when moving around the corners. The only thing wrong here is that the AI is aiming at the wall instead of at the open space next to the wall where the enemy would be.
- After going through a door, it checks all corner to clear the room. But it's very slow in doing this, and would get shot instantly in a real game.
- Adjusts the crosshair constantly to not look at the floor when walking. The problem is that the crosshair is never at headshot level!
I am not bashing on this AI btw, just pointing out that it's actually getting close to real human behaviour! Fascinating!
Something I've noticed is going well is the fact that the AI doesn't spray n' pray but controls the spray pattern and drags the crosshair down. Shoots in bursts!
> The only thing wrong here is that the AI is aiming at the wall instead of at the open space next to the wall where the enemy would be.
This is actually how well-practiced humans play, aiming instead where the enemies are likely to be (behind the wall) instead of trying to perfectly aim at the corner all the time.
This is actually how well-practiced humans play, aiming instead where the enemies are likely to be (behind the wall) instead of trying to perfectly aim at the corner all the time.
It depends on how you’re planning to peek. Generally you’d want to smoothly track edges to clear out everything, including off-angles. Only if you’re quite certain of an enemy’s position you’ll “aim at the wall”, then peek out. Your cross hair will then be spot on immediately. This method always runs into the risk of overpeeking and getting killed easily. Smoothly tracking edges has the risk of getting swung out wide on, instead of getting peeked at tightly, such that sometimes you want to place your cross hair a bit away from edges, but only if you’re the one holding an angle statically.
The AI ignores enemy players running behind him. Because he can't see them.
I'd love to see the same method, but with audio inputs too.
I'd love to see the same method, but with audio inputs too.
Wow it even learned to spawn camp lol
Excellent. Believably plays like a pro with regard to spray patterns but maybe slightly slower. This is doom for any cheat detect that relies on outliers. Doesn't look aimbotty in the slightestt.
If by "spray pattern" you mean bursts of three shots, sure. But this thing is not playing like a pro in any respect.
Well, it's not public knowledge but it seems that VAC does not care about how it looks. Also while it doesn't look aimbotty, according to modern standards it also sucks at the game. In contrast, many pro players look very aimbotty, but are legit. But ultimately yeah, anticheat will be totally lost.
Interesting. What's stopping the AI from more or less instantly detecting human shapes and then dishing out headshots instantly as well? Technically, that should be possible. I suppose its learning material is strictly actual humans, so it'd never be able to learn that, even if technically feasible.
There's already bots that are capable of this (often used for cheating in games) - but this research is trying something more difficult and interesting - which is teaching an AI how to play a game, with nothing but a ton of learning material.
Things like moving around the map in a rational way, learning how to counterplay enemies, tactics when entering a room, all based on solely visual data, are extremely difficult. These skills are much more intuitive to humans, but much harder to learn than "there is a human head at {523,1021}"
Things like moving around the map in a rational way, learning how to counterplay enemies, tactics when entering a room, all based on solely visual data, are extremely difficult. These skills are much more intuitive to humans, but much harder to learn than "there is a human head at {523,1021}"
Aimbots have been around in FPS games for a very long time, including purely out-of-band approaches (observe video output, send input) as they cannot be detected by anti-cheat software.
Valve has been at this for a long time...
They started with a system that lets players review games and try to understand if a player is a cheat or not. Games are anonymized, and you get to rewatch the replay to try to make a determination
They can then use that data to train their NN to help detect which players / games should be fed into this manual review process
I assume now the product is probably good enough that it's ranking reviewers too to determine how good reviewers are at detecting if there's cheats or not.
It's pure genius
Here's a GDC talk you may find interesting: > In this 2018 GDC session, Valve's John McDonald discusses how Valve has utilized Deep Learning to combat cheating in Counter-Strike: Global Offensive.
https://www.youtube.com/watch?v=kTiP0zKF9bc
They started with a system that lets players review games and try to understand if a player is a cheat or not. Games are anonymized, and you get to rewatch the replay to try to make a determination
They can then use that data to train their NN to help detect which players / games should be fed into this manual review process
I assume now the product is probably good enough that it's ranking reviewers too to determine how good reviewers are at detecting if there's cheats or not.
It's pure genius
Here's a GDC talk you may find interesting: > In this 2018 GDC session, Valve's John McDonald discusses how Valve has utilized Deep Learning to combat cheating in Counter-Strike: Global Offensive.
https://www.youtube.com/watch?v=kTiP0zKF9bc
Out-of-band cheats can sometimes be detected based on behavioral analysis.
"matches the performance of the medium difficulty built-in AI on the deathmatch game mode"
You might not realize how bad that actually is. Built in bots on medium difficulty can't even challenge a complete beginner.
You might not realize how bad that actually is. Built in bots on medium difficulty can't even challenge a complete beginner.
kinda a missed opportunity they scaled the whole screen down to 280x150. They should have left the cross hair area larger--so like 10x10 around the crosshair unscaled, and instead of doing weird warping geometry, just pass it in as a second 2d image. I'm guessing the focus of the research is on the behavior cloning, which is really awesome!
> Whilst RL research often aims to maximise reward, we emphasise that this is not the exclusive ob- jective of this paper – perfect aim can be achieved through simple geometry and accessing backend information about enemy locations (hacks and built-in bots exploit this). Rather, we aim to produce an agent that plays in a humanlike fashion, that is fun and challenging to play with and against.
love it!
https://github.com/TeaPearce/Counter-Strike_Behavioural_Clon...
Also so awesome, I've been wondering how people automate games. it looks super easy & simple, no longer a big mystery :D
> Whilst RL research often aims to maximise reward, we emphasise that this is not the exclusive ob- jective of this paper – perfect aim can be achieved through simple geometry and accessing backend information about enemy locations (hacks and built-in bots exploit this). Rather, we aim to produce an agent that plays in a humanlike fashion, that is fun and challenging to play with and against.
love it!
https://github.com/TeaPearce/Counter-Strike_Behavioural_Clon...
Also so awesome, I've been wondering how people automate games. it looks super easy & simple, no longer a big mystery :D
I both do and don't want it to include the use of chat.
0.68 K/D ratio to start off with is pretty good against a real person.
Wasn't Google Stadia's weird AI latency improvement system based off of something similar?
I’m very ignorant when it comes to AI but I’d like to have a machine watch me select some sub strings out of different sequences of text and have it find what is the pattern behind all the choices I make. Is there something like that I can train?
That sounds a bit like the SQuAD 2.0 challenge, but simpler.
In SQuAD 2.0 a model is given a question, and a passage of information. It has to select the start and end points of where the answer is contained in the passage.
Your problem is simpler in that there is no question - you just have to have a model that can identify the start and end positions that _you_ will select.
In SQuAD 2.0 a model is given a question, and a passage of information. It has to select the start and end points of where the answer is contained in the passage.
Your problem is simpler in that there is no question - you just have to have a model that can identify the start and end positions that _you_ will select.
Would it work if the text is not words? It’s just a string of chars
Sounds like excel flash fill might be similar to what you’re interested in. It’s a well documented feature described in papers and talks, it comes from a pretty cool group in MS research.
My data is a 600 char string of letters and following some very loose rules you slice 7-10 substrings of the string. The string is an RNA sequence and the substrings vary depending on the genus/species/etc. I know loosely how far down the string each might be, but it’s not perfect and it varies. I definitely know in what order they are, so that helps. Right now I have a script that works 90% of the time, but I’d love to make it learn from the data it reads so it can guess the other 10% or be ready for new discoveries. So I’m not sure how flash fill would work in this scenario.
it has already been a couple days, but here is my $0.02 (for context, i have played cs:go for 3k+ hours):
this game has much more going on than games like mario, and even dota 2. for one, audio matters! also, there are more inputs to train, such as walk, jump and crouch.
this is a great start, but a lot more needs to be done before it can be called a complete solution.
this game has much more going on than games like mario, and even dota 2. for one, audio matters! also, there are more inputs to train, such as walk, jump and crouch.
this is a great start, but a lot more needs to be done before it can be called a complete solution.
I've always wanted to train a CS:GO AI/bot but am terrified of VAC bans.
Is this related to the undetectable AI based AIM bot?
what could possibly go wrong ? lol, rip mankind
This is starting to feel like a medium takeoff.
> Hey what should we train our AI to play? Cities: Skylines so it can fix traffic?
> Nah bro, let's teach it to shoot people in CS:Go!
> Nah bro, let's teach it to shoot people in CS:Go!
[deleted]
They could take a 20 year old that has never played fps's and perform similar tests. Coordination, twitch muscle reflexes, peripheral vision and other variables will never exist with AI. GLHF as they say.
Hopefully not too off-topic, but AI playing chess, doing this or that "better than humans" is not a 1st world problem.
I never followed the chess scene and was intrigued with the Han Niemann angle, but noticed the chess engines being overblown with a lot of it (play ping pong against a wall type of stuff).
The 1st thing I thought was a chess format called "turn the tables" where chess players perform the 1st 3-5 moves, then "turn the board" and compete with the oposing players setup (did not research and this may already exist).
A Deep Blue versus Garry Kasparov rematch using "turn the tables" gameplay would be cool.
I did laugh at Magnus Carlsen winning in 1 move when the opponent resigned after a crappy knight move by Magnus at the recent tournament. TtT chess takes a lot of the memorization strategies out of play for the natural GM's.
With much of this AI stuff, nobody is chasing AI because of its severe limitations, impediments and attribute deficiencies. Just play ping-pong against a wall.
Hopefully not too off-topic, but AI playing chess, doing this or that "better than humans" is not a 1st world problem.
I never followed the chess scene and was intrigued with the Han Niemann angle, but noticed the chess engines being overblown with a lot of it (play ping pong against a wall type of stuff).
The 1st thing I thought was a chess format called "turn the tables" where chess players perform the 1st 3-5 moves, then "turn the board" and compete with the oposing players setup (did not research and this may already exist).
A Deep Blue versus Garry Kasparov rematch using "turn the tables" gameplay would be cool.
I did laugh at Magnus Carlsen winning in 1 move when the opponent resigned after a crappy knight move by Magnus at the recent tournament. TtT chess takes a lot of the memorization strategies out of play for the natural GM's.
With much of this AI stuff, nobody is chasing AI because of its severe limitations, impediments and attribute deficiencies. Just play ping-pong against a wall.
Finally, you just do dead-simple behavior cloning (predict "expert" action from current observation) on the large labeled dataset. It's still surprising to me how well this works! Behavior cloning has some theoretical issues since it doesn't address the sequential nature of decision problems at all, but apparently with enough scale it can still do fairly well - super cool.
[1]: https://openai.com/blog/vpt/