Using Computer Vision to Win at Duck Hunt(blog.roboflow.com)
blog.roboflow.com
Using Computer Vision to Win at Duck Hunt
https://blog.roboflow.com/computer-vision-win-games-duck-hunt/
29 comments
Images are broken; 403 from googleusercontent.com
Hey thanks, someone just let me know that this post from a while back ended up on HN; let me look and see if we have those broken images saved anywhere!
Edit: fixed!
Edit: fixed!
Appreciate you fixing the links, though is less than a year actually "a while back"? Seems like a relatively short time frame for links to atrophy. </grump>
Yeah, not ideal. We typically re-host images we share on our blog but had apparently hyperlinked these from something the user had shared.
I was hoping that this would be a tongue-in-cheek post about how a training a CV model and a robot with enough range of motion would eventually figure out that aiming the light gun off-screen towards a white wall would result in 100% accuracy.
Tangential, but fascinating to me: I realized a few years ago that you cannot play duck hunt with an original light gun and original NES connected to a flatscreen TV. I’m sure there are people on HN who can explain this more eloquently than I can, but basically the flatscreen TVs have some latency that the original NES hardware cannot handle. You pull the trigger and the gun looks for the white square immediately, but the TV has some digital processing going on that takes some fraction of a second before the image is rendered, and it’s enough that the NES doesn’t think you hit the target. I seem to recall some thing about the CRT refresh rate as well but I don’t remember what that had to do with it.
There are apparently after market mods and stuff that people have come up with to work around this, but it takes some effort.
As you can imagine I discovered this by trying to play duck hunt and being perplexed when I was literally unable to hit any target, which made a very poor demo for my kids haha.
There are apparently after market mods and stuff that people have come up with to work around this, but it takes some effort.
As you can imagine I discovered this by trying to play duck hunt and being perplexed when I was literally unable to hit any target, which made a very poor demo for my kids haha.
The fix is actually simple: shoot at a light bulb. It will think you always hit. I figured this out accidentally as a kid one day when I was randomly firing around the room and hit a duck.
Mind = Blown!
[deleted]
I haven’t tested this with Duck Hunt, but I believe most NES Zapper games actually detect this exploit by blanking the screen for one frame before drawing the target in white, and checking that the gun saw one dark frame followed by one light frame.
[deleted]
I think what you meant was the CRT refresh rate was used for determining WHICH (multi)target was hit, explained pretty well @ https://www.youtube.com/watch?v=cu83tZIAzlA
I'm not sure this solves the hard part of the problem. I was assuming there would be an actual robot arm involved.
This is obviously not a difficult CV problem. I don't want to disparage the post too much, though - the Roboflow tools to make this process accessible for a non-expert look very good, and it's a cool demo of those which is the real point of the blog.
This is obviously not a difficult CV problem. I don't want to disparage the post too much, though - the Roboflow tools to make this process accessible for a non-expert look very good, and it's a cool demo of those which is the real point of the blog.
> I annotated the images with Roboflow and was able to easily export the data
> Roboflow's Dataset Health Check helped me
> I found Roboflow’s Model Library to be the best
> Roboflow would be my go-to platform
> Matt Brems, Growth Manager @ Roboflow
Please call this a demo or “how I use Roboflow to win a duck hunt”
> Roboflow's Dataset Health Check helped me
> I found Roboflow’s Model Library to be the best
> Roboflow would be my go-to platform
> Matt Brems, Growth Manager @ Roboflow
Please call this a demo or “how I use Roboflow to win a duck hunt”
You missed one:
> The user experience for preprocessing is amazing
Agreed, this was a bit over the top, but I could have improved my expectations by paying more attention to the domain name. :)
> The user experience for preprocessing is amazing
Agreed, this was a bit over the top, but I could have improved my expectations by paying more attention to the domain name. :)
I don't understand this example.
Duck Hunt has only a few sprites, maybe four or six. It's really easy to just scan the whole image and match those sprites, exactly pixel-by-pixel. This doesn't really need anything that could reasonably be called machine learning or computer vision. Synthetic 8-bit images can be handled with 8-bit algorithms. We don't need much art and we certainly don't need its state either.
Why pick such an example to advertise roboflow?
The article itself also has pretty much no details on how Roboflow works. Not a single code sample! Just links to Roboflow docs.
This is blogspam.
Duck Hunt has only a few sprites, maybe four or six. It's really easy to just scan the whole image and match those sprites, exactly pixel-by-pixel. This doesn't really need anything that could reasonably be called machine learning or computer vision. Synthetic 8-bit images can be handled with 8-bit algorithms. We don't need much art and we certainly don't need its state either.
Why pick such an example to advertise roboflow?
The article itself also has pretty much no details on how Roboflow works. Not a single code sample! Just links to Roboflow docs.
This is blogspam.
Agree. Detecting things that move doesn’t require rocket surgery.
When all you have is a hammer
Don't really agree with this. It like saying "why did you use a phone to do that math when a simple calculator or even pen and paper would work"
Sure it would work, but the phone also works and does way more. You might be able to use some trivial processing on duck hunt but it won't work on anything slightly more complex so why would you bother learning a method that only works on the most basic of games when you can develop something that can be applied everywhere.
Sure it would work, but the phone also works and does way more. You might be able to use some trivial processing on duck hunt but it won't work on anything slightly more complex so why would you bother learning a method that only works on the most basic of games when you can develop something that can be applied everywhere.
I agree with you for the most part. This problem definitely doesn't require ML to achieve.
However, if doing it with a classical approach (matching sprites) takes a person half a day, and doing it with the new fangled proprietary ml takes a person 20 minutes, I do see that doing it with the new fangled ml approach does have some merits. People want to get stuff done.
I'm a barely passable programmer, so not exactly a shining benchmark, so maybe my view is distorted. I don't think I can do duck detection flawlessly with classical programming in half a day and it would be a total pain in the ass to do it that way, to be completely honest with you.
However, if doing it with a classical approach (matching sprites) takes a person half a day, and doing it with the new fangled proprietary ml takes a person 20 minutes, I do see that doing it with the new fangled ml approach does have some merits. People want to get stuff done.
I'm a barely passable programmer, so not exactly a shining benchmark, so maybe my view is distorted. I don't think I can do duck detection flawlessly with classical programming in half a day and it would be a total pain in the ass to do it that way, to be completely honest with you.
You can do sprite matching using new fangled ML tools and save the same amount of developer time.
Yeah, but in the classical approach your users don't need to have a GPU installed.
This seems to be a case of "if you're holding a hammer, everything looks like a nail".
This seems to be a case of "if you're holding a hammer, everything looks like a nail".
Edit: I'm pretty sure I missclicked to which comment I wanted to answer and now I can't find it... It said something along the ML approach being more developer-time efficient.
You can definitely do it in 20 minutes if you know how (which is also applicable to the ML version):
You can definitely do it in 20 minutes if you know how (which is also applicable to the ML version):
import cv2
import numpy as np
red_duck = cv2.imread("red_duck.png", cv2.IMREAD_GRAYSCALE)
# boilertplate etc, up to the point where you want to match your ducks on screen:
res = cv2.matchTemplate(img_screen, red_duck, cv2.TM_CCOEFF_NORMED)
positions = np.zeros_like(img_screen)
positions[res > 0.7] = 1 # we found a duck
# now do whatever you want with each position
# on real life images you may need to do some additional post-processing, on 8-bit rendered images you probably don't need to> This is blogspam
Looking at the domain and the author, I'd say it was a straight-up advert
Looking at the domain and the author, I'd say it was a straight-up advert
Compare current screen with previous screen, if something is moving upwards you shoot it. That's the only algorithm you need here.
Being charitable, Duck Hunt is a well know game and this might be considered a tutorial for the software's use using a simple problem. It does look like an advert blog post, but simple examples let people get a feel for the code.
> simple examples let people get a feel for the code.
There's no code in there. Just links to other articles and whatnot.
I vote for blogspam, too.
There's no code in there. Just links to other articles and whatnot.
I vote for blogspam, too.
That is not the real Duck Hunt, it's a replica.
The fonts are different, the UI is different and the UI elements are misaligned, the scenario is larger.
The fonts are different, the UI is different and the UI elements are misaligned, the scenario is larger.
I experimented with using super basic computer vision to cheat at videogames but It never worked quite right. My target was venge.io, pretty bad game but I can tamper with the files freely with some userscripts. I injected bright purple textures for all the characters and used a simple color range matching to find the target.
My choice of language was python with pywin32,numba and a screen duplication library which name I forgot. The biggest problem was that by the time I finished scanning the screenbuffer contents the enemies on screen have already moved.
If I had paid attention in university math classes I could've come up with a direction tracking and prediction system. :/
But then I threw the whole thing out the window and rewrote everything with good old fashioned readmemory()/writememory() :^)
My choice of language was python with pywin32,numba and a screen duplication library which name I forgot. The biggest problem was that by the time I finished scanning the screenbuffer contents the enemies on screen have already moved.
If I had paid attention in university math classes I could've come up with a direction tracking and prediction system. :/
But then I threw the whole thing out the window and rewrote everything with good old fashioned readmemory()/writememory() :^)