A Game You Can Control with Your Mind(nytimes.com)
nytimes.com
A Game You Can Control with Your Mind
https://www.nytimes.com/2017/08/27/technology/thought-control-virtual-reality.html
47 comments
I've spent a lot of time working with EEG and pulling my hair out over garbage SNR, finnicky setups, and artifacts galore.
I don't understand why so many folks try to go the route of using EEG as mouse/keyboard/generic HID for the masses, when (ignoring para/quadraplegics) most humans have no problem with computer input. There are tons of applications where the EEG can be used for what it's (sorta) good at: attentional and mood states, which, in the time of distraction firehose, seems like it would have a use case. But no, we have to make yet another pick-up-a-virtual-object interface.
(not to imply X-aplegics are not worthy of tech. it's just that as mentioned above, eye tracking is WAY more robust)
I don't understand why so many folks try to go the route of using EEG as mouse/keyboard/generic HID for the masses, when (ignoring para/quadraplegics) most humans have no problem with computer input. There are tons of applications where the EEG can be used for what it's (sorta) good at: attentional and mood states, which, in the time of distraction firehose, seems like it would have a use case. But no, we have to make yet another pick-up-a-virtual-object interface.
(not to imply X-aplegics are not worthy of tech. it's just that as mentioned above, eye tracking is WAY more robust)
>pulling my hair out over garbage SNR
I bought into the OCZ NIA about a decade ago. Nice unit, but I was using it within about 2m of a set of AC/DC plugpack transformers. It had a SNR visualiser, and you could see it go increasingly ballistic as I moved my head towards and away from the powerboard.
I have my sincere doubts that EEG devices can ever function satisfactorily in residential homes. Too many microwaves and fridges, turns out there's a reason hospital EEGs are done in faraday cage rooms.
I bought into the OCZ NIA about a decade ago. Nice unit, but I was using it within about 2m of a set of AC/DC plugpack transformers. It had a SNR visualiser, and you could see it go increasingly ballistic as I moved my head towards and away from the powerboard.
I have my sincere doubts that EEG devices can ever function satisfactorily in residential homes. Too many microwaves and fridges, turns out there's a reason hospital EEGs are done in faraday cage rooms.
Yes the primary commercial application found in the EEG literature seems to be fatigue monitoring for professional drivers. Certainly "control" BCI applications won't be the first Neurotech killer app.
Yup. IN fact, our application was locked-in patients. SSVEP BCI "works", but only if you can foveate on the target, which means you can move your eyes, which means you should use an eye tracker.
There are quite a lot of papers that show you can actually modulate SSVEP response even without eye movement. Their protocol was quite fascinating, they used a swarm of targets with four frequencies all intermingled. You had vertical and horizontal (short) lines which were either blue or red. One would control the system by concentrating on blue horizontal stripes for example. Of course the throughput was quite low even for a BCI. I think more research in this vein could reveal some interesting possibilities.
You can. In our experience, we were unable to get reliable performance with SSVEP or m-sequences with targets in the periphery, though - the subject had to be foveating on the target to get a good response.
Given that constraint, an eye tracker would be just as applicable to the patient population, so SSVEP was a bag of disadvantages with no obvious upside.
Given that constraint, an eye tracker would be just as applicable to the patient population, so SSVEP was a bag of disadvantages with no obvious upside.
Do you know of any viable setups for using eye tracking in daily computer use? I think it would be pretty nifty to be able to control my mouse while having both hands on my keyboard.
Tobii's entry level eye tracker bar does this. "Blink twice to jump the mouse to where I'm looking", too.
Can you recommend any one in particular? Tobii EyeX Gaming vs Tobii Eye Tracker 4C
the technology is pretty robust now. I used the eyeX, but any of them are going to be good performers.
Good points.
You say an order of magnitude, do you have any information about generational increases? Like was last generation an order of magnitude worse than what you worked on?
You say an order of magnitude, do you have any information about generational increases? Like was last generation an order of magnitude worse than what you worked on?
our control paradigm was steady state visually evoked responses -- these are low-frequency components of a scalp EEG rig that correspond to the visual cortex locking on to a blinking test pattern. On a good day with a bald subject and fresh gel, decoding time is between 6 and 11 seconds.
A tobii eye tracker has a latency of about 30 ms.
Spatial discrimination is oodles better, too.
A tobii eye tracker has a latency of about 30 ms.
Spatial discrimination is oodles better, too.
[deleted]
Interesting! I was big on this stuff seven years ago, when I found a "Star Wars Force Training" toy [1] -- you put the brain wave sensor on, which passes a signal that sets the strength of a fan that's blowing upward on a ball in a tube, and therefore its height. So it makes a neat brain-ball-height link. Unfortunately, it was really unreliable -- only for brief periods could I (or anyone) consistently set the height.
I also bought the OCZ Neural Impulse Actuator that you could use as a PC peripheral and it could detect alpha/beta brains waves as well as tension and eye movement, but which I really only found reliable for the last two. (You control tension by "gritting your teeth".)
[1] https://www.amazon.com/Star-Wars-Science-Force-Trainer/dp/B0...
I also bought the OCZ Neural Impulse Actuator that you could use as a PC peripheral and it could detect alpha/beta brains waves as well as tension and eye movement, but which I really only found reliable for the last two. (You control tension by "gritting your teeth".)
[1] https://www.amazon.com/Star-Wars-Science-Force-Trainer/dp/B0...
I have long since offered to volunteer myself to be able to have a direct brain interface with a computer and the Internet. I'll even shave my hair and let them install a wireless nub on the back of my head.
In my imagination, this means everything from authoring my own custom programs to thwarting hackers using the power of my mind.
I'm pretty sure this would be just about the most awesome thing ever created. I'd be surfing at brain-speed and able to access all information that is out there.
Trivially related: I'd want an ad blocking mechanism, but I might be able to just will it into place.
Yes, I'd be willing to take the risks if someone had a reasonable plan to trial this.
In my imagination, this means everything from authoring my own custom programs to thwarting hackers using the power of my mind.
I'm pretty sure this would be just about the most awesome thing ever created. I'd be surfing at brain-speed and able to access all information that is out there.
Trivially related: I'd want an ad blocking mechanism, but I might be able to just will it into place.
Yes, I'd be willing to take the risks if someone had a reasonable plan to trial this.
You'd still have to deal with connecting to servers and grabbing whatever data you need. Then you'd have to deal with the delays, which might not be so fun.
I am not scared. I will take thus tech, as soon as they ask for volunteers.
I've been onlne for a long time. I'd happily take these risks. I'd risk death, with nary a question asked. I will sign up today, for such a tech.
I've been onlne for a long time. I'd happily take these risks. I'd risk death, with nary a question asked. I will sign up today, for such a tech.
Cheap EEGs have interesting applications for the quantified-self people. For example, Gwern uses a similar device to get data about their sleep pattern.
> https://www.gwern.net/Zeo
> https://www.gwern.net/Zeo
this has been around for a few years, https://www.emotiv.com/ and never really got any good.
sceptical to say the least.
You just don't get enough resolution via eegs.
sceptical to say the least.
You just don't get enough resolution via eegs.
What I really want is a highly sensitive position tracking set of breathable gloves, probably with some mild electric feedback. We have have loads of sensitivity and control in our hands. Why bother to dangerously wire to the brain when you can input/output via the hands? There would be a learning curve, but if it's possible to read braille and use sign language, I'd think it's possible to relay information to the hand via electric stimulus and from the hand by tracking movement.
Wiring stuff directly to the brain has the nice side effect of improving the tools we use to understand how the brain works. It also helps us develop better prostheses (for example for people who lost their hands).
Would interpreting the EEG in terms of commands given work more reliably if there were more sensors? If so, why are there so few sensors? If not, why not?
It mostly depends on what paradigm you are using. They seem to use the P300 response (your brain's response to an "anticipated surprise") which indeed benefits quite a lot from additional sensors because the response can be found all over the brain. If they were using motor imagery then you only need electrodes on the sensory-motor cortex.
First concern is cost, additional sensors of good quality are expensive (for example, a single g.Tec dry electrode costs about 100 euros). If you use gel electrodes (which are vastly superior to dry electrodes) then you add preparation time.
One of the biggest hurdles of EEG for games is practicality. People were annoyed when they had to spend a few seconds calibrating their Kinect, now imagine you would have to spend 5-15 minutes donning and calibrating the system each time you wished to play.
First concern is cost, additional sensors of good quality are expensive (for example, a single g.Tec dry electrode costs about 100 euros). If you use gel electrodes (which are vastly superior to dry electrodes) then you add preparation time.
One of the biggest hurdles of EEG for games is practicality. People were annoyed when they had to spend a few seconds calibrating their Kinect, now imagine you would have to spend 5-15 minutes donning and calibrating the system each time you wished to play.
For real, this is one step before full VR reality and loss of humanity. Once people will have a choice between living there life and going full fantasy-mod of there choice there will be problems. It's nice that tech. goes forward, but we should step gently. Otherwise ''Roy: A Life Well Lived'' arcades will pop up everywhere.
This is one argument as to why we don't see any ET life.
Once they reach a sufficient level of technology, instead of going outward, they go inward. Creating Matrioshka Brains and living in virtual worlds.
Here's a video by the excellent Isaac Arthur: https://www.youtube.com/watch?v=Ef-mxjYkllw
Another one by Joe Scott: https://www.youtube.com/watch?v=EJOKb1_aBM0
If it indeed turns out that we are living in a simulation, then I guess the question then begs. Why not live in a simulation of your own rules and making? What would be so wrong with that?
Once they reach a sufficient level of technology, instead of going outward, they go inward. Creating Matrioshka Brains and living in virtual worlds.
Here's a video by the excellent Isaac Arthur: https://www.youtube.com/watch?v=Ef-mxjYkllw
Another one by Joe Scott: https://www.youtube.com/watch?v=EJOKb1_aBM0
If it indeed turns out that we are living in a simulation, then I guess the question then begs. Why not live in a simulation of your own rules and making? What would be so wrong with that?
living in constant subtile fear of the janitor who trips over cables also know as g.o.d.
Why do you think most CEO's or your fav celeb jumps out of bed every morning? Most of them are addicted to whatever real world "game" they are involved in. No VR required.
Or in other words, the future of The Matrix or other lotus eater machines isn't being tricked into using them, but being offered the chance to play your favourite online games/watch your favourite films/take part in your fantasies for eternity.
If you want to experiment with an EEG device, that works look at Muse. They emphasize meditation feedback, but you can access your EEG raw data.
http://www.choosemuse.com/research/
http://www.choosemuse.com/research/
I've been wanting to get one to try some ideas out, but trying to decide if the $200 is worth it. Do you have a muse? What do you use it for?
next generation will love this, while us old fart will scream about the ethical issues of recording and uploading brain scans to the private advertisement companies that'll run billboard in the virtual worlds...
Get off my mind lawn!
I realize you were joking but "get of my lawn" is normally something that the rich say to the poor. In this situation it's the opposite. Also, I'm young enough to be in the "next generation" and, at least in my case, the GP is wrong. The previous generation assumes they can get lost in the noise. They imagine all our data flowing into a human machine for processing. They don't understand the power of machine learning and fail to grasp the degree to which this data will increasingly determine outcomes in their individual lives.
>I realize you were joking but "get of my lawn" is normally something that the rich say to the poor.
No it's not. A lawn is a very middle-class thing in the US. It is stereotypically an older person that spends a bunch of time keeping their lawn pristine since they have no small children.
https://en.wikipedia.org/wiki/You_kids_get_off_my_lawn!
No it's not. A lawn is a very middle-class thing in the US. It is stereotypically an older person that spends a bunch of time keeping their lawn pristine since they have no small children.
https://en.wikipedia.org/wiki/You_kids_get_off_my_lawn!
Exactly. The rich have signs that say "Get off my lawn"... The middle class have to get off the couch to say it.
I'm not sure I'd make absolute statements based on arbitrary birth dates. Some of us have created the very things you use and are well aware of the implications. I'd also make no such assumptions based on someone being younger than myself.
I didn't intend for my statements to be absolute. I could have thrown in some seem tos and generallys but I felt like the "at least in my case" covered it. I've spoken to older people who are very aware of what's coming and/or already here. It's just not the common case. I'm not making a hard assumption; It's more like a branch prediction.
I think there was a Star Trek episode about this...
[deleted]
more seriously, i worked on a brain-computer interface (BCI) system a few years ago that was supposed to do directional control of a cursor from scalp EEG recordings. A $90 eye tracker worked an order of magnitude better and three orders faster.