Touch Typing on a Gamepad(darkshadow.io)
darkshadow.io
Touch Typing on a Gamepad
https://darkshadow.io/2020/07/07/touch-typing-on-a-gamepad.html
59 comments
That looks very similar to MessagEase keyboard available for Android and iOS. It is pretty cool, I was an avid user when I had an Android phone and I was barely making any typos. It’s very customizable and I particularly enjoyed clipboard integration. On iPhone third party keyboards don’t work as good so I had to stop using it.
On consoles (including the Switch), Scribblenauts mega pack has a similar interface! https://cdn1-www.playstationlifestyle.net/assets/uploads/gal...
You hold the analog in the appropriate direction and press the button. At first its strange, but I got used to it kind of fast.
You hold the analog in the appropriate direction and press the button. At first its strange, but I got used to it kind of fast.
Soft- or hard-modded original (2001) Xbox consoles had a community developed web browser called 'linksboks' with a similar analog stick and button pad keyboard concept.
There were groups of four characters in screen's corners and centered on the edges, eight groups in all, with the characters of each group color coded to match the A, B, X, and Y buttons on the controller[1]. You picked a group with the analog stick, and pressed the color-matched button to type. It was great!
Sadly, I can't find any screenshots that feature the linksboks onscreen keyboard.
[1]: https://www1-lw.xda-cdn.com/files/2013/01/Xbox_controller_s_...
There were groups of four characters in screen's corners and centered on the edges, eight groups in all, with the characters of each group color coded to match the A, B, X, and Y buttons on the controller[1]. You picked a group with the analog stick, and pressed the color-matched button to type. It was great!
Sadly, I can't find any screenshots that feature the linksboks onscreen keyboard.
[1]: https://www1-lw.xda-cdn.com/files/2013/01/Xbox_controller_s_...
Ha, I was literally about to post the same thing. During the PSP days I could type without looking using that exact keyboard. The LYNX browser also had the same keyboard implemented.
Unfortunately I can't find a direct reference on any Valve official channels, but I'd say that typing on a Steam Controller is quite good; I'd even go so far to say that it's going to be hard to beat it for keyboard input on a controller.
Anecdotally, most people hated the Steam Controller's touchpads, and I'll admit I wasn't a fan at first either. But the ability to remap them to be: sticks, trackballs, d-pads, bring up (multi-level) radial menus, etc. is an absolutely killer feature.
I think one of the reasons they "failed" is because they absolutely were not plug-and-play, because to truly utilize their full capabilities, you had to configure them per game.
Anecdotally, most people hated the Steam Controller's touchpads, and I'll admit I wasn't a fan at first either. But the ability to remap them to be: sticks, trackballs, d-pads, bring up (multi-level) radial menus, etc. is an absolutely killer feature.
I think one of the reasons they "failed" is because they absolutely were not plug-and-play, because to truly utilize their full capabilities, you had to configure them per game.
As someone who actually enjoys tinkering with these settings. I grew to love the controllers quite quickly. But it's exactly as you said.
Steam Controllers are not plug-and-play. They're really a niche thing.
I was able to set up a reasonable competetive setup for Metal Gear Online. I had the left pad for walking. Then set up the right one with extremely sensitive camera control, so I could turn around really quickly. But configured it in such a way that it slowed down a bit when I pressed down the aim button. After which I also enabled gyro controls for fine-aiming.
For Fallout 4 it was a more casual layout. But here I made the right trigger hipfire if you fully pressed it immediately. But a half-press would go into iron-sights first, while also enabling gyro aiming. This felt really intuitive for me.
Steam Controllers are not plug-and-play. They're really a niche thing.
I was able to set up a reasonable competetive setup for Metal Gear Online. I had the left pad for walking. Then set up the right one with extremely sensitive camera control, so I could turn around really quickly. But configured it in such a way that it slowed down a bit when I pressed down the aim button. After which I also enabled gyro controls for fine-aiming.
For Fallout 4 it was a more casual layout. But here I made the right trigger hipfire if you fully pressed it immediately. But a half-press would go into iron-sights first, while also enabling gyro aiming. This felt really intuitive for me.
>Steam Controllers are not plug-and-play
I gotta say, for old games I had already played it was a great experience since I knew what I wanted to configure and how. However, new games were a pain since I was still trying to figure everything out
I gotta say, for old games I had already played it was a great experience since I knew what I wanted to configure and how. However, new games were a pain since I was still trying to figure everything out
Valve created an implementation of the keyboard experiment we see here (a keyboard with letters accessed with direction + face button[0]), but actually scrapped it when they released the Steam Controller (the Big Picture keyboard is now a split keyboard, where each stick/touchpad controls one side of the split[1]).
[0]: http://img.gawkerassets.com/img/17ynanc1qocy1jpg/original.jp...
[1]: https://images.techhive.com/images/article/2015/10/my-great-...
[0]: http://img.gawkerassets.com/img/17ynanc1qocy1jpg/original.jp...
[1]: https://images.techhive.com/images/article/2015/10/my-great-...
I have a htc vive which uses the same trackpads and I think radial menus on them work so well.
I think all the "optimization" was a mistake. Instead of optimizing for the most-used keys, I think it's better to optimize to make it easy to remember.
Phone typing was easy to learn because the letters were both predictable (alphabetical) and because you could look at the letters when you forgot, especially while learning.
Since looking at the keys isn't possible here, I think it's even more important that being able to predict the locations of letters is possible and easy.
Phone typing was easy to learn because the letters were both predictable (alphabetical) and because you could look at the letters when you forgot, especially while learning.
Since looking at the keys isn't possible here, I think it's even more important that being able to predict the locations of letters is possible and easy.
T9 was amazing. Any upcoming human interface industrial engineer ought to study it. When driving or in class, I used to regularly send SMS messages without looking at my phone’s screen. It was THE biggest thing I missed when I got my first iPhone 3G.
Only recently have Google's and Apple's TTS come close to matching it. Occasionally I still miss T9 like when in a movie theater where I can't speak to type and don't want to look at my screen.
Only recently have Google's and Apple's TTS come close to matching it. Occasionally I still miss T9 like when in a movie theater where I can't speak to type and don't want to look at my screen.
I'm a bit of a keyboard maniac. I don't know why I am this way, but I am. On my computer I have a lovely split mechanical keyboard which I've remapped to Colemak, but I've always wished I could get a good typing experience on my phone.
I like being able to type while walking and without looking, and by far the best phone keyboard for that is 8pen [0]. A masterclass in innovative UX imo, and clearly a ton of work went into it. Unfortunately, they pulled it from the play store (and deleted all the tweets on their twitter) some years ago for reasons unknown to me.
Related is MessageEase [1], which is not as good but still passable. But it's incredibly ugly and takes up a ton of space on my screen, so I don't like to use it.
[0]: http://www.8pen.com/
[1]: https://www.exideas.com/ME/index.php
I like being able to type while walking and without looking, and by far the best phone keyboard for that is 8pen [0]. A masterclass in innovative UX imo, and clearly a ton of work went into it. Unfortunately, they pulled it from the play store (and deleted all the tweets on their twitter) some years ago for reasons unknown to me.
Related is MessageEase [1], which is not as good but still passable. But it's incredibly ugly and takes up a ton of space on my screen, so I don't like to use it.
[0]: http://www.8pen.com/
[1]: https://www.exideas.com/ME/index.php
8pen was definitely the most interesting/promising alternative input methods I've ever seen.
> When driving
Even if you aren't looking at the phone. You're still distracted as you're thinking about typing. As well has having your hand occupied.
Also, why would you want to be texting in a movie theater anyway?
Even if you aren't looking at the phone. You're still distracted as you're thinking about typing. As well has having your hand occupied.
Also, why would you want to be texting in a movie theater anyway?
Fwiw, I've had decent luck blind typing with Minuum on Android. Very different solution, but it works.
The old steam gamepad keyboard was also predictable in this way; I don't know if it's still used, I just remember being impressed by the concept. With two sticks and shoulder buttons I think you could really get somewhere with this idea
https://lh3.googleusercontent.com/proxy/ACAiL9UXh-hN-rMMc5J5...
https://lh3.googleusercontent.com/proxy/ACAiL9UXh-hN-rMMc5J5...
Steam's keyboard was mentioned in the Gamasutra article linked to in the article[1], which I found more interesting than the original article. I'd recommend reading it; the two-stick entry method detailed by the end of the article makes basically every game-pad entry system I've ever used feel archaic.
[1]: https://www.gamasutra.com/blogs/CharlieDeck/20170721/301392/...
[1]: https://www.gamasutra.com/blogs/CharlieDeck/20170721/301392/...
That's really interesting! The new steam controller keyboard takes advantage of the dual touchpads to give you the split hunt and peck as described in the study. You can see how there are larger targets for certain letters, and more total inputs that can be picked. However, you have to look at the keyboard to type, it's not a solution for the touch typing problem.
https://i.stack.imgur.com/XDNKX.png
Having experience with precise angles on a gamecube controller in Super Smash Bros Melee, I can't help but feel like I would prefer the penultimate solution in the article you linked, with appropriate zones that can be targeted directly. Locking you to a quadrant and forcing most letters to be a compound input doesn't seem worth it to me. You could also definitely improve the quadrant system by allowing you to break into a new quadrant without returning to the center, maybe by passing the vertical or horizontal center line, depending on the quadrant. But, I obviously haven't tried it myself.
https://i.stack.imgur.com/XDNKX.png
Having experience with precise angles on a gamecube controller in Super Smash Bros Melee, I can't help but feel like I would prefer the penultimate solution in the article you linked, with appropriate zones that can be targeted directly. Locking you to a quadrant and forcing most letters to be a compound input doesn't seem worth it to me. You could also definitely improve the quadrant system by allowing you to break into a new quadrant without returning to the center, maybe by passing the vertical or horizontal center line, depending on the quadrant. But, I obviously haven't tried it myself.
TIL you can tilt the PS4 controller for text entry. This is amazing, thanks for the link!
It was the Big Picture "Daisy Wheel" keyboard. They've since removed it to focus on what I think is a much weaker controller keyboard designed primarily for the Steam Controller that is pretty garbage for standard xbox controllers.
That would be helpful in the very beginning but even with just a little practice the gestures become muscle memory and remembering the gestures a non-issue. I believe making the gestures predictable would have greatly decreased efficiency which is my priority here as my goal is to be able to use this system for work not just for the occasional text message.
If you're the only one using it, then it doesn't matter as much. Do whatever you like.
But if you want others to use it, remember that they are different than you and even learning such a system is a big investment for many people.
Qwerty keyboards work because people can see the keys and they're everywhere. When you come across a keyboard of another layout, even just alphabetical, it's a big pain.
If people are going back and forth from your layout to any other kind, it's going to be a huge headache. And if they put down your layout for more than a couple weeks, they're going to have to re-learn it. Having it be predictable is a lot more desirable if there's ever a time you have to re-learn it.
But if you want others to use it, remember that they are different than you and even learning such a system is a big investment for many people.
Qwerty keyboards work because people can see the keys and they're everywhere. When you come across a keyboard of another layout, even just alphabetical, it's a big pain.
If people are going back and forth from your layout to any other kind, it's going to be a huge headache. And if they put down your layout for more than a couple weeks, they're going to have to re-learn it. Having it be predictable is a lot more desirable if there's ever a time you have to re-learn it.
That's not the optimisation used for Qwerty, Dvorak, etc. By optimising for memorisation you're sacrificing ease of use for power users for the sake of beginners who will mostly give up. That makes sense for a product you're trying to sell but not for a niche, hobbyist thing like this where you're trying to supercede keyboards.
That's exactly my reasoning!
So basically, typing via Street Fighter moves.
Feh. Everybody knows Real Programmers type by manipulating the locations of specific koopa troopas so that when the glitch is triggered, exactly what they intended to type will be written into memory.
Feh. Everybody knows Real Programmers type by manipulating the locations of specific koopa troopas so that when the glitch is triggered, exactly what they intended to type will be written into memory.
As someone with RSI-strained fingers, there's no way I am concentrating all fingers' worth of input movement into just my two thumbs. Sounds like a recipe for carpal tunnel/tendinitis/miscellaneous other flavors of RSI.
You do have to be careful and stop when symptoms occur especially in the beginning but I believe the issue can be avoided. The key is to stay relaxed and focus on minimizing mistakes rather than trying to type very fast from the get-go. Typing with this system is actually not that different from playing a console game.
So I guess the question is, do people who play LOTS of console games (e.g. professional streamers) get RSI?
Some do but as with RSI from keyboard/mouse usage it affects everyone differently. One thing they usually have in common is excessive repetition for example by constantly using shortcuts with CTRL. While practicing typing (both keyboard and gamepad) I noticed that highly repetitive lessons with lots of text like "jf jf jf fj fj fj jfjfjfj" cause much more strain than lessons with more realistic text. In the end it all comes down to responsible usage, nothing is completely safe.
A chorded keyboard would also map well to gamepads
https://en.wikipedia.org/wiki/Chorded_keyboard
https://en.wikipedia.org/wiki/Chorded_keyboard
Good idea, I will give it a try!
I was a bit disappointed at how limited the linked Gamasutra's article was. For example, it didn't even mention helical keyboards that arrange letters and numbers into a 3D helix that allows very fast character selection. For example, when you can immediately see that the character you need is one and a half loops away ending on the 3 o'clock position, twirling your analog stick to that position takes a fraction of a second.
You mean this type of keyboard? https://www.youtube.com/watch?v=euquOpUmUyk I had so much fun playing beyond good and evil on the PS2!
It's not the best but it is both easy and fun to use so if you want to avoid frustrating your players it may be the safest choice.
It's not the best but it is both easy and fun to use so if you want to avoid frustrating your players it may be the safest choice.
> Whenever one of the sticks left the center by crosssing the threshold, an input sequence was started. As soon as both sticks were back in the center within the threshold, the sequence was considered completed and transformed into a pair of tuples that described the movement of the sticks.
I wonder if it was easier in practice to move both sticks simultaneously, or one after the other and then release them simultaneously. Because the currently most commonly accepted psychological model two-handedness[0] would suggest the latter, and that you start with your off-hand and finish with your main hand.
[0] http://cogprints.org/625/ Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model
I wonder if it was easier in practice to move both sticks simultaneously, or one after the other and then release them simultaneously. Because the currently most commonly accepted psychological model two-handedness[0] would suggest the latter, and that you start with your off-hand and finish with your main hand.
[0] http://cogprints.org/625/ Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model
Indeed, when entering combined inputs one by one starting with my off-hand my accuracy was higher and I found myself using that technique occasionally during practice, especially when I had made a mistake and needed to repeat an input. However, as my speed increased, eventually I tended to move the sticks simultaneously.
Cool idea, but wouldn't it make sense to remove the letter J from the most used letters?
If you don't use that letter for Vim it drops from the top 8 all the way to the bottom of the frequency chart (assuming you only type English). If you than map whatever J does in Vim to a more frequently used letter your typing speed should go up.
Alternatively since J seems to be used mostly for navigation (not sure, I don't use Vim) use the d-pad to do that instead.
---
> After I upgraded my gamepad to the DualShock4, I realized I could make diagonal inputs relatively accurately. Integrating diagonal inputs would reduce the number of more complex inputs required thus increasing speed.
Any plans to do that?
If you don't use that letter for Vim it drops from the top 8 all the way to the bottom of the frequency chart (assuming you only type English). If you than map whatever J does in Vim to a more frequently used letter your typing speed should go up.
Alternatively since J seems to be used mostly for navigation (not sure, I don't use Vim) use the d-pad to do that instead.
---
> After I upgraded my gamepad to the DualShock4, I realized I could make diagonal inputs relatively accurately. Integrating diagonal inputs would reduce the number of more complex inputs required thus increasing speed.
Any plans to do that?
You're right of course about removing the letter j from the most used letters. My strategy is to continue logging the inputs made with this system and adjust the mapping as frequencies change. If I indeed use the d-pad more often than j when using this system, the letter j will eventually be flagged for remapping. By making only one adjustment at a time the relearning effort is manageable.
As for diagonal inputs, I've actually done it and I'm still experimenting with it. I can accurately target 8 sectors when there is no dialing involved but with dialing I would stick to 4. I think a hybrid system that allows for 8 sectors but switches to 4 when I start rotating the stick would be the best of both.
As for diagonal inputs, I've actually done it and I'm still experimenting with it. I can accurately target 8 sectors when there is no dialing involved but with dialing I would stick to 4. I think a hybrid system that allows for 8 sectors but switches to 4 when I start rotating the stick would be the best of both.
Single hand chording inputs are supposed to best touch typing, I'm surprised they can't do similar things with the gamepad and its shoulder buttons and other inputs.
Yeah, I was kind of really surprised to see this instead of a chording system.
I will give it a try! You might still have to use both hands since you can't reach as many keys on a gamepad single-handedly as you can on a chorded keyboard.
I think the point of this project was to do a project, not to make something you can use.
Reminds me of the chorded keyset from Doug Engelbart (inventor of the mouse)[1]
The idea of simultaneously using a mouse with the right hand and keyboard with the left is intriguing.
[1]: https://www.dougengelbart.org/content/view/273/309/
The idea of simultaneously using a mouse with the right hand and keyboard with the left is intriguing.
[1]: https://www.dougengelbart.org/content/view/273/309/
There is actually a gamepad-type keyboard as a real product.
It doesn't work with gestures but with dedicated buttons.
From the looks of it, you can type pretty fast on it, too.
It's called AlphaGrip http://www.alphagrips.com/
Never used one, though.
It's called AlphaGrip http://www.alphagrips.com/
Never used one, though.
This reminds me of the orbitouch, which is a keyless keyboard for people with disabilities. It works by having one stick chose a subset of letters, and the other to letter within that subset. https://orbitouch.com/
Nice, but I still don't know why Dasher[0] didn't get more "mainstream"?
[0]: https://en.wikipedia.org/wiki/Dasher_(software)
[0]: https://en.wikipedia.org/wiki/Dasher_(software)
This seems like it could be really useful for text entry in VR! Using a keyboard is impractical in that scenario, but you typically are holding two controllers with joysticks. A system like this could make VR workspaces more practical.
Reminds me of Hadouken and Shoryuken moves :)
https://streetfighter.fandom.com/wiki/Hadoken
https://streetfighter.fandom.com/wiki/Hadoken
Selfless plug. Have been obsessing over this for a few years.
Do not hesitate to contact me via contact form if you share this passion and or wants to take it further.
tbf-rnd.life/
Do not hesitate to contact me via contact form if you share this passion and or wants to take it further.
tbf-rnd.life/
[deleted]
Are there any usable 1 handed typing systems out there? I just want an media remote with a touchpad+trigger and T9 input.
For a traditional style remote, held in a pointing grasp with thumb on top, I have a hard time imagining something much better than a T9 layout.
Personally, I'm kind of enamored with the idea of a phone or tablet for media selection & control.
On touchscreen based devices, I'm a big fan of https://en.wikipedia.org/wiki/MessagEase.
When I started looking into alternative text entry for a phone, the big thing I was looking for was the ability to completely disable predictive text entry. Messagease allows for that - I'm able to input text quickly, accurately, and without the use of predictive input.
I've used it while holding my phone in either hand and using the thumb of that same hand (though it works better in my dominant hand). I've also used while holding the phone in my non-dominant hand and using the index finger of my dominant hand.
I can almost, but not quite, use it without looking.
So, if you're thinking of a custom remote project to scratch that itch, that layout may provide inspiration.
Personally, I'm kind of enamored with the idea of a phone or tablet for media selection & control.
On touchscreen based devices, I'm a big fan of https://en.wikipedia.org/wiki/MessagEase.
When I started looking into alternative text entry for a phone, the big thing I was looking for was the ability to completely disable predictive text entry. Messagease allows for that - I'm able to input text quickly, accurately, and without the use of predictive input.
I've used it while holding my phone in either hand and using the thumb of that same hand (though it works better in my dominant hand). I've also used while holding the phone in my non-dominant hand and using the index finger of my dominant hand.
I can almost, but not quite, use it without looking.
So, if you're thinking of a custom remote project to scratch that itch, that layout may provide inspiration.
Looks like it’s patented? That might inhibit use a lot.
The keyboard, though, was great:
http://localhost.geek.nz/afkim/docs/usingafkim.html
It's a 3x3 matrix. You use the analog stick to pick a square, then hit one of the face buttons to enter a character. Left and right shoulder buttons shifted the keyboard to uppercase, numbers, specials, etc.
IIRC it was a Lua module that any PSP homebrewer could drop into their application for a pretty decent OSK.