Medicine and the Metaverse: New Technology Allows Doctors to Travel Inside Your Body

The world of technology is rapidly shifting from flat media viewed in the third person to immersive media experienced in the first person. Recently called “the metaverseThis major transition in mainstream computing has sparked a new wave of excitement about the core technologies of virtual and augmented reality. But there is a third area of ​​technology known as: telepresence that is often overlooked, but will become an important part of the metavers.

As virtual reality brings users into simulated worlds, telepresence (also called telerobotics) uses remote robots to take users to faraway places so they can look around and perform complex tasks. This concept harks back to 1940s science fiction and a groundbreaking short story by Robert A. Heinlein titled Waldo. If we combine that concept with another classic science fiction story, Fantastic trip (1966), we can imagine small robotic vessels going inside the body and swimming around under the control of doctors diagnosing patients from the inside and even performing surgical tasks.

I know that sounds like pure fiction, but a startup company in Hayward California recently “flown” a tiny robot into human test subjects’ digestive tracts. The company is endiatx, and I had the chance to discuss their technology and vision with CEO Torrey Smith. As a technologist who has been involved in telepresence inquiry from the start I was impressed with the progress Endiatx has made. But before I get into that, let’s jump back in time a few decades and provide some context as to why their breakthrough seems like such unexpected progress to me.

The first telerobotic prototypes

My first experience with an immersive telerobotic system was more than thirty years ago. I can still remember the first time I peered into a headset, grabbed a set of periscope-like handles, and looked around a room where I wasn’t sitting. It was 1991 and I was working in a lab at NASA that had some of the first prototypes of a mobile telerobotic system. It was developed by Fake space labs and Telepresence Research and you could control a mobile robot with a camera system that returned stereoscopic images in real time. Below is a photo of that early system of a academic paper to those efforts, which show the state of telepresence research thirty years ago.

That early system was extremely impressive at the time and was developed by some of the top researchers in the field at the time. But if you want to take the hardware to a trade show, you’ll probably need a hefty van or maybe even a U-Haul truck. The idea of ​​reducing such a system to a size that a person could swallow and be monitored by a doctor was unimaginable. So what were the proposed areas of application during the early days of telepresence?

In the beginning, the focus was on enabling human operators to perform work in dangerous places, such as cleaning up nuclear accidents, repairing satellites and even repairing leaking oil wells on the ocean floor. My personal focus in the early 1990s was adding augmented reality and haptic feedback to telepresence with the aim of improving operator performance. This was groundbreaking research at the time, but I’ve never considered making the technology big enough to “fly” into the human body as a means of diagnosing and treating patients from the inside out.

‘Fantastic Journey’ realized

For these reasons, I was surprised to hear about the ambitious vision and recent technical success of the team at Endiatx. Founded in 2019, they have already created a small robotic drone that can be swallowed by human patients and controlled remotely in the stomach and other parts of the digestive tract. Their prototype system, known as the PillBot™, is basically a small multivitamin-sized remote-controlled submarine that sends real-time video back to a doctor’s computer or phone. And it works – they did tests on cadavers and living people.

During a conversation with CEO Torrey Smith, he told me that he was inspired by the sci-fi film Innerspace (1987) as a child and has been thinking about that concept ever since. A few years ago, he finally took the plunge and founded Endiatx to bring this concept to life. And so far it’s going well. Not only is his company on track to bring this capability to healthcare as a shipping product, but Torrey himself was the first person in the world to own a robotic drone. flew around in his stomach. He volunteered and swallowed the first prototype that was a real… fantastic trip.

Since that first test, others at the company have often swallowed working prototypes, capturing live video footage of the type that will one day be used to screen patients for ulcers, gastritis, cancer and other potential conditions. And that day is not far away — the team is currently conducting cadaver tests with doctors at the Mayo Clinic and planning trials for FDA approval. If all goes smoothly, the PillBot could diagnose patients around the world by 2024.

This can be a huge benefit for people who go to the doctor with a stomachache. Rather than a standard endoscopy procedure, which generally requires sedation and involves multiple visits, the small swallowable robot can save time, money and complexity, giving doctors a quick and easy way to look around their patient. And it can provide more flexible control than a traditional endoscope, because the untethered Pillbot has the full 3D mobility of a small robotic submarine. It even looks like a tiny sub – with micromotors and tiny propellers, along with a video camera, battery and wireless link to send images back to doctors in real time.

Faster, cheaper and more accurate screenings

Currently, doctors can control the tiny drone with a standard Xbox game controller, but the company plans to enable touchscreen control of any mobile phone. That’s because their vision is for a disposable unit to be shipped to your home and swallowed during a telemedicine consultation with your doctor, who watches the camera feed in real time from their PC or phone. Endiatx believes the robotic pills could be manufactured for $25 each, sold for hundreds of dollars per unit, and saved many thousands of dollars in medical costs that endoscopic procedures cost to perform under anesthesia. More importantly, the company believes PillBots will save many lives by enabling faster and cheaper screening that finds serious conditions earlier than would otherwise be practical.

The company expects to begin initial clinical trials later this year, roll out an in-clinic version shortly after, and launch a home version not long after. And once they ship products that allow doctors to see inside the body, their next goal is to enable the device to take tissue samples and perform other surgical tasks. In the long run, their plan is to shrink their robotic drone to the size of a grain of rice, opening up possibilities beyond the digestive tract. And their goal is to bring all of these possibilities for home use, with doctors controlling bots through telemedicine.

The future of telemedicine?

I was a little skeptical about the telemedicine angle for a little robot like this, because we’re talking about a powerful diagnostic tool that will be delivered by mail. Will the medical profession embrace such a change, or will they insist on maintaining such capabilities in clinics and hospitals? Then I saw this week’s press release from Amazon announcement of the acquisition from healthcare provider One Medical for nearly $4 billion and says Amazon wants to reinvent the healthcare experience.

Neil Lindsay, SVP of Amazon Health Services is quoted as saying in the release: “Make an appointment, wait weeks or even months to be seen, take time off work, drive to a clinic, find a parking space, wait in the waiting room, and then the exam room for what is too often a rushed few minutes with a doctor, and then another trip to a pharmacy – we see many opportunities to both improve the quality of the experience and give people back valuable time in their days.

In that context, the sci-fi vision of tiny drones being swallowed by patients and controlled remotely by doctors during telemedicine visits could really become part of our mainstream medical experience. As a lifelong technologist who often writes about the impending dangers of the metaversethis is an emerging field of application that really impresses me.

Author Bio: Louis Rosenberg, PhD is a pioneer in virtual reality, augmented reality and artificial intelligence. Thirty years ago, he developed the first functional mixed reality system at the Air Force Research Laboratory. He then founded the early virtual reality company Immersion Corporation (1993) and the early augmented reality company Outland Research (2004). He is currently CEO of Unanimous AI, a company that amplifies the intelligence of human groups. Rosenberg received his PhD from Stanford University, was a professor at California State University and has been awarded more than 300 patents for his work in VR, AR and AI.