Meta recently showed the press his first personal look at the progress towards a wristband that can sense your intentions.
In 2019, the company, formerly known as Facebook, acquired a startup called CTRL-Labs that was working on a wrist-worn device to read electrical activity at your fingertips. It does this with EMG, or electromyography, which reads activity with sensors strapped around the circumference of your arm.
Before CTRL-Labs was picked up by Mark Zuckerberg, only a few news outlets tried the prototype, including The Verge’s Adi Robertson to write “I tried the wristband that lets you control computers with your brain.”
We still haven’t worn the wristband — it requires significant training time in its current condition — but Zuckerberg and those he employs believe Thomas Reardon’s team at CTRL-Labs is on to something that could hold the key. to unlock for the 21st century what the QWERTY keyboard layout did in the 19th century and the graphical user interface (GUI) in the 20th century.
They have been looking for an input system that makes sense to combine with Augmented Reality glasses and with the technology they obtained from CTRL-Labs they think they have found it.
Why a wristband?
Voice input only gets you so far, because there are plenty of places where you’d rather be quiet than audibly say what you’re thinking. Large gestures recognized by hand tracking are a no-go for the same reason. You just can’t do it everywhere without looking like a fool. Touch controllers, on the other hand, don’t make sense to take them everywhere.
The idea is that this wristband, watch, bracelet – whatever it is marketed as in a consumer product – can be combined with future eyewear and “co-adjust” with the wearer to understand and translate intent faster than the current phones and PCs.
“The two main benefits of this are… how responsive it can be initially,” Zuckerberg replied to a question from UploadVR. “If you want to type something on your phone, you have to take this whole activation out of your phone, unlock it… and then you’re good to go. And on this, it’s extremely fast. So the activation is one.”
Zuckerberg wore the wristband, demonstrating how small gestures — even those practically unnoticeable to an onlooker — could translate into the kind of simple actions we do hundreds of times a day for things like scrolling through a TV’s menu via the remote or from one place to another. screen to the other. the next on your phone.
“Then…what’s your stable communication bandwidth? And this,” he says, referring to the bulky device on his wrist, “I think it will be superior initially at activation, while having a relatively lower communication bandwidth for very complex stuff. But in the end…say within the next five or six years…that’s not an unreasonable time frame to say that this is really the highest-bandwidth form of communication that you have.”
Friction and focus
As Zuckerberg put it, Quest VR headsets and Horizon with its avatar-populated 3D realms are two of what he sees as four interrelated platforms he’s prioritizing for the future of Reality Labs. The other two are clearly still in the roughest research and product development stages: AR glasses and neural interfaces like the wristband.
“We’re really going to focus on those four areas and probably prioritize other things in Reality Labs,” Zuckerberg said. “There is clearly a trade-off between them, but I also think they will each be on their own. By the time you have neural interfaces, there will still be a lot more people using phones and computers than VR or AR devices, even if it was initially invented as some sort of input mechanism for augmented reality.”
If you’ve owned a VR headset in the last ten years, you’re familiar with this “activation problem.” John Carmack, a leading tech guide to Oculus-then-Facebook-then-Meta, has called the headset problem “friction.” That is, everything from charging your battery to completely blocking the physical world to the seconds-long wait at startup robs the wearer of precious moments that could be better spent on something else.
Some point to this friction as one reason why “VR is dead” and Meta has mis-spent tens of billions of dollars it earned by targeting ads to the personal details of much of the planet. If there’s a more useful story to watch out for than this chorus, it’s that Meta’s teams are obsessed with paying attention to data and using it to let people slip further into spaces they control for extended periods of time. In other words, they are experts at reducing friction.
Examples:
– Quest Pro is more comfortable than any headset that Meta has delivered before, while keeping the wearers aware of their physical environment as often as they want.
– Horizon Worlds eliminates the need for large downloads before visiting 3D virtual worlds accessible over the Internet.
– Horizon Home opens Quest headsets to visitors from the moment the headset is unlocked.
If you put the neural wristband effort into this larger context, and if it can actually be faster than unlocking your phone, then you can begin to understand the end goal of Zuckerberg’s metaverse effort.
Faster than the QWERTY keyboard?
The biggest claim of the day about the impact of this work came from the CEO of Meta himself. As Zuckerberg said, “right now the bit rate is lower than what you would get for typing fast”.
Meta’s live demos of the wristband seemed imperfect, even when worn by Zuckerberg and his CTRL-Labs researchers. Still, there are more than hints that the top of Zuckerberg’s leadership team see a much bigger picture.
In an effort to contextualize the weight of this work, Meta CTO Andrew Bosworth cited the invention of the QWERTY keyboard layout, while the company’s top researcher, Michael Abrash, pointed to Doug Engelbart and the invention of the graphical user interface. The leaders of Meta seem to be delving into the foundations of the first principles of modern technology. Perhaps looking for the right starting point and strategy to lay a foundation for the next 50 years of personal computing.
“Mark alludes to this idea of bit rate,” Reardon said. “And we really think it’s almost impossible not to get faster. This is simply a result of the way your motor nervous system works, that you adapt quickly to the task and if you get the right feedback you just get faster at it. You can’t help but get faster at it if you minimize the amount of exercise you do… I don’t want to talk about products here, but I would just like to say that we have really, really compelling research results on that result. “
When pressed about what usage scenarios the input system is ready to replace in its current state, Bosworth replied, “Reardon could replace his keyboard.”
“We intend to go beyond any experience you would have today with a mechanical controller,” Reardon said. “Something like the Touch controller – there’s no reason we can’t go much further than that.”
“You can imagine we have to figure out what the language and interface will be to deal with this,” he said. “But you have a very large number of gestures and you have this complex system where you communicate through a combination of these, or we make you just write things down.”