Meta Is Researching Turning Any Flat Surface Into a Virtual Keyboard (uploadvr.com) 46
Mark Zuckerberg posted a video to his personal Instagram profile showing clips of himself and Meta CTO Andrew Bosworth trying a surface-locked virtual keyboard in a Quest 2 headset. UploadVR reports: Zuckerberg claims he was able to achieve around 100 words per minute, while Bosworth says he reached 119 words per minute. The average person types at around 40 words per minute on a traditional keyboard, whereas professional typists reach between 70 and 120 words per minute depending on their skill level. If future headsets could turn any flat surface into a virtual keyboard, it would bring partial haptic feedback and allow you to rest your wrists as with physical keyboards, without the need to carry around a physical keyboard.
Developers can technically already build surface-locked virtual keyboards on Quest today, by using hand tracking and getting the user to tap the surface to calibrate its position. But in practice, even the slightest deviation of the virtual surface height from the real surface results in false key presses. Meta didn't share yet exactly how its research solves this issue. However, fiducial markers can be seen on the desk in the clip. If the system is preprogrammed with the exact dimensions of these markers, this may act as a robust dynamic calibration system.
Quest 3 will add a depth sensor to a Meta headset for the first time, and a leaked setup clip shows the headset generating a 3D mesh of its environment. If this mesh is precise enough, Quest 3 could potentially eventually support this kind of virtual keyboard. For now though, Meta is solely describing this as research, not a demo of a near term product experience. Meta will likely show off more of its VR and AR research at Meta Connect, which starts September 27 this year.
Developers can technically already build surface-locked virtual keyboards on Quest today, by using hand tracking and getting the user to tap the surface to calibrate its position. But in practice, even the slightest deviation of the virtual surface height from the real surface results in false key presses. Meta didn't share yet exactly how its research solves this issue. However, fiducial markers can be seen on the desk in the clip. If the system is preprogrammed with the exact dimensions of these markers, this may act as a robust dynamic calibration system.
Quest 3 will add a depth sensor to a Meta headset for the first time, and a leaked setup clip shows the headset generating a 3D mesh of its environment. If this mesh is precise enough, Quest 3 could potentially eventually support this kind of virtual keyboard. For now though, Meta is solely describing this as research, not a demo of a near term product experience. Meta will likely show off more of its VR and AR research at Meta Connect, which starts September 27 this year.
Obvious bullshit (Score:5, Insightful)
Keyboards need to do tactile feedback to work well. Anything else is asking for trouble, longer-term.
Re: (Score:3)
Indeed. I often wonder how so many people manage to do all their online activities on a phone or tablet. I find both android and ios virtual keyboards borderline unusable for any input longer than a short phrase and as for touch typing, forget it. Give me a proper PC keyboard (the clickier the better frankly) any day.
Re: (Score:3)
Exactly. I find that on non-tactile projected keyboards I hit and press too hard and will very likely get nerve damage in my fingertips and probably carpal tunnel syndrome as well. That was just from 15 minutes of testing a borrowed early laser-projected keyboard on a table surface, but the tingling in my fingertips was a clear warning to stay away from this bad tech idea.
Re: (Score:2)
As far as clicky keyboards, I find a good medium ground are tactile switches. They give enough feedback that I can type without bottoming out, but I barely feel the bump while playing games. I could probably type faster with a clicky keyboard, but then I'd need a linear switch for gaming.
Re: (Score:2)
Indeed. I often wonder how so many people manage to do all their online activities on a phone or tablet.
The natural gesture for complex interactions on mobile is the swipe, not the tap. Swipe keyboards with predictive correction work reasonably well most of the time, with maybe just one or two words per paragraph where you need to delete the word and repeat the gesture; not unlike deleting and retyping a few letters with a keyboard, for anyone but the most precise typists.
Re: (Score:2)
Predictive text is utter shit for specialist topics.
Re: (Score:2)
Well, I think current iPhones use the tap engine to simulate a "press", but for short sprints you don't need as much feedback.
Also, you have to remember the feedback is visual, and since your e
Not really (Score:2)
Jail/prison cells.
Areas that have lots of hazardous/radioactive chemicals and you want to avoid contamination
The real question is, I thought that this was already done before? IOW, Meta is simply duplicating work done by others
Re: (Score:1)
Keyboards need to do tactile feedback to work well.
Absolutely.
They have this thing exactly backwards.
I don't want a virtual keyboard for use in VR - I want my real keyboard to be also shown in VR so I can find it and use it with my half touch type, half hunt and peck style of keyboard input I have been using my whole life.
If I am just typing a few words I can usually do that reasonably error free without looking - but as soon as I start to need symbols, or if I'm not typing whole words that I mostly have in muscle memory, I really need to be able to look a
100 words per minute? (Score:5, Funny)
Is that anything like 6'3" and 215 pounds?
Re: (Score:3)
Good one. Thanks for the laugh.
Re: (Score:2)
Only if HR got some CVs confused and hired a maintance guy as your secretary by mistake.
Keyboard? That's all you came up with? (Score:3)
Spoiler alert (Score:5, Informative)
It's already been done. [wired.com]
And I actually used one once. I hated it. I can type a sustained 100+ wpm, but with this thing, I barely breached 20. Granted, it was buggy as hell, but even if it had worked flawlessly, I have no tactile feedback at my fingertips to confirm that I'm pressing on a key, which slowed me down. More importantly, there was absolutely no tactile reference point to anchor my fingers (unlike the raised dimples on the 'f' and 'j' keys), so my fingers would drift off the keys if I didn't keep a constant look at them, which also slowed me down terribly.
No way these guys were getting over 100 wpm.
Death to Meta. What a waste of money.
Re:Spoiler alert (Score:4, Informative)
Virtual keyboards were invented in the early 1990s. You can buy laser based ones like the the one in the article you linked to for under $50. "It's already been done" is an understatement.
Far as I can tell, the only thing different here is the visual aspect is via VR headset rather than projecting it onto the surface you're tapping, and you can't see your real hands.
=Smidge=
Re:Spoiler alert (Score:4, Funny)
> rather than projecting it onto the surface you're tapping, and you can't see your real hands.
Perfect! That means the AI can just move the key I meant to press right under my finger, even if I moved my finger to slightly the wrong place. No more typos!
"The quick brown fox jumped over the lazy dog. Meta and Zuck are the greatest. Yes! Please sign me up for a Facebook account. Keep the regulators out of Big Tech. Did I mention Zuck was the greatest?"
Re: (Score:2, Insightful)
Parent can't be modded up enough.
It's literally pre-dot-com-bubble tech. You can buy cheap chinese shit that does it.
And the verdict is in: Outside a small number of very specific use cases, it's a bad idea that hasn't taken off in 30 years for good reasons.
Re: (Score:2)
so my fingers would drift off the keys if I didn't keep a constant look at them, which also slowed me down terribly.
It would seem that a proper implementation would use the position of your hands as a reference for where the virtual keyboard is, so it wouldn't matter if your hands drifted. Perhaps, make it work like a split keyboard and track each hand separately.
Re: (Score:2)
so my fingers would drift off the keys if I didn't keep a constant look at them, which also slowed me down terribly.
It would seem that a proper implementation would use the position of your hands as a reference for where the virtual keyboard is, so it wouldn't matter if your hands drifted. Perhaps, make it work like a split keyboard and track each hand separately.
But how would you differentiate the hand drifting from it moving to press another row?
Re: (Score:2)
It's already been done. [wired.com]
And I actually used one once. I hated it. I can type a sustained 100+ wpm, but with this thing, I barely breached 20. Granted, it was buggy as hell, but even if it had worked flawlessly, I have no tactile feedback at my fingertips to confirm that I'm pressing on a key, which slowed me down. More importantly, there was absolutely no tactile reference point to anchor my fingers (unlike the raised dimples on the 'f' and 'j' keys), so my fingers would drift off the keys if I didn't keep a constant look at them, which also slowed me down terribly.
No way these guys were getting over 100 wpm.
Death to Meta. What a waste of money.
They just hit the "A"key 100 times. A is a word, amirite?
Finger drift is real, and it's a pain in the ass if you don't have a nice deep cup in the keys. If I don't have decent cupped keys, I'm typing out yjr in short order.
note for the phone and pad users - that's "the" shifted to the right by one key.
This is so old (Score:4, Informative)
// Checking how old:
https://www.google.com/search?q=virtual+keyboard [google.com]
Oh yeah! Projection keyboard [wikipedia.org]
IBM ... 1992.
Someone else ... Canesta ... 2002
Reinventing the wheel (Score:3)
Touch typing doesn't need a keyboard image? (Score:2)
Re: (Score:2)
I'd argue by the drivel spewed that most can bear lying comprehend their native language let alone type out complete thoughts with any speed.
Emphasis mine, hilarity yours
How fast are they with their toes? (Score:2)
Do they have legs, yet?
Tactile feedback anyone ? (Score:2)
Won't have to carry keyboards? WTAF? (Score:4, Interesting)
Facepalm moment there - no, you'll have to carry around a headset instead.
You'll also be limited to the battery life that has, so you'll need to be able to charge it - frequently.
A 10keyless bluetooth keyboard battery will last weeks if not longer between charges.
You can get them in exceptionally small form factor and even get slim mechanical ones.
You can attach them to _any_ computer or mobile device. They have haptic feedback built-in.
I get it - the idea that you won't need to carry a laptop, because your AR/VR goggles _are_ the computer.
The convergence of tech to bring super lightweight comfortable "glasses" with day long battery life is some time away - it'll probably happen.
Thing is, it's a solution looking for a problem. And the problems the solution currently bring are ... just problems.
Re: (Score:2)
Facepalm moment there - no, you'll have to carry around a headset instead.
This needs to be at +5 insightful.
What's more we'll look pretty stupid sitting there with a headset on, wiggling our fingers. "Is this tool typing, or feeling up miss Trixie on Pornhub VR?" You'll also be limited to the battery life that has, so you'll need to be able to charge it - frequently.
A keyless bluetooth keyboard battery will last weeks if not longer between charges.
I have a small form factor Logitek BT keyboard that seems to last forever. A couple Triple A's and it is good for months - altho
Just as pleasant as typing on an iPad screen (Score:2)
That didn't work either but still Apple got a nice bunch of patents for that feature.
But can it� (Score:2)
But can it turn my wife into a virtual keyboard?
Just what I wanted (Score:2)
Touchscreen keyboards for everybody!
Fractures are a risk (Score:2)
These type of hard-surface keyboards have been associated with stress fractures in the fingers when used extensively. As an occasional device, entering in password credentials or filling in the occasional data form they aren't a problem, but sustained use is generally not recommended.
It's a convenience keyboard at best.
fix layout first... (Score:3)
If you want to do anything keyboard-wise, before wasting money on new tech, address the craptastic QWERTY layout first. It's terrible in all ways, and is the 800 lb gorilla of user input in almost all cases. I personally switched to Dvorak some 25 years ago. Colemak may also be a good alternative.
A common misconception is that people claim Dvorak to be faster. There are plenty of people who master QWERTY and are incredibly fast. The issue is the amount of hand/finger muscle effort required. Dvorak is roughly 1/3 the effort of QWERTY. I'm on a keyboard daily without the slightest hint of carpal tunnel syndrome.
Re: (Score:2)
Bad designs never die once they become widely used standards
It's a sad law of reality
Another dumb idea from Zuckerberg (Score:3)
Touch typing? (Score:3)
How the f*** are you meant to type on a flat surface without looking at your hands?
How do you locate keys by feel?
So (Score:1)
Stucking thinking inside the box again.... (Score:2)
If you're going to do a VR experience with a headset that simulates the entire reality you experience around you? How backwards is it to still have to type on a keyboard to interact with the computer?
I know it's a far bigger "ask" -- but I feel like the real solution here is figuring out a whole new UI for this to eliminate the keyboard concept as your input/communications device to the computer.
This has already been done (Score:2)
input (Score:2)
feel is part of it, duh? (Score:2)
I get the convenience, but the point of touch typing isn't just a 'sense' of precisely where your fingers should be, it's a constant process of self-correction, where you feel the edge of a key or a group of keys and self-edit your next finger-stroke.
The idea that you can do this on a smooth featureless surface is silly.
This sounds like RSI waiting to happen (Score:2)
Holding your hands up in the air above the "keyboard" the whole time? I hope you don't need to type anything longer than a paragraph. I'm going to keep my keyboard that allows resting my fingers on the keys.