How edge data trains AI for accurate, real-time response

Autonomous driving is seen as the future of mobility, thanks to companies like Tesla that have developed AI-powered advanced driver assistance systems (ADAS) to help users navigate from one point to another under certain conditions.

The progress has been astonishing to many, but the fact remains: we are a long way from truly autonomous vehicles. To achieve true autonomy, self-driving vehicles must be able to outperform human drivers in all conditions, be it a densely populated urban area, a village or an unexpected scenario on the road.

“Autonomous driving is often actually quite easy. Sometimes it’s as simple as driving on an empty road or following a vehicle in front. However, since we’re dealing with the real world, there’s a wide variety of ‘edge cases’ that can arise,” said Kai Wang, the forecasting director at Amazon’s mobility company. Zooxsaid at VentureBeat’s Transform 2022 Conference.

These edge cases cause problems for algorithms. Imagine a group of people stepping out onto the street from a blind spot or a pile of rubble in the way.

Zoox training effort

Humans are pretty good at recognizing and responding to almost all kinds of edge cases, but machines find the task difficult because there are so many possibilities of what can happen down the road. To solve this, Zoox, which builds fully autonomous driving software and a purpose-built autonomous robotic axi, has taken a multi-layered approach.

“There is not really one solution that solves all these cases. So we’re trying to build in different kinds of mitigation at our entire system level, at every layer to give us the best chance of addressing these things,” Wang said.

First, as the director explained, Zoox enables the observation of various conditions/objects by taking in data from the sensor pods on all four corners of the vehicle.

Each pod features multiple sensor modalities – RGB cameras, Lidar sensors, radars and thermal sensors – that complement each other. For example, RGB cameras can sense details in images but cannot measure depth, which is handled by Lidar.

“The job of our perception system is to use all these sensors together and fuse them together to produce just a single representation for all the objects around us. This gives the best chance of recognizing all the things in the world around us.” Wang said.

Once the surrounding agents are identified, the system models where they will end up in the next few seconds. This is happening with data-driven deep learning algorithms that devise a distribution of future potential trajectories. Post this, it takes into account all dynamic entities and their predicted trajectories and makes a decision on what to do or how to navigate safely through the current scenario to the target destination.

Tele-guidance

While the system basically models and handles edge cases, it may encounter certain new situations along the way. In those cases, the system will stop and use teleguidance functions to get a human expert for help (while simultaneously checking for collisions and obstacles with other agents).

“We called a human operator to suggest a route to get through the blockade. So far we have received teleguides for less than 1% of our total mission time in complex environments. And as our system matures, this percentage should continue to decline,” Wang said.

After moving forward, the data associated with the edge case goes to the company through a feedback loop, allowing it to use the scenario and its variants in simulations to make the software system more robust.

Miss the . not full session on how edge data trains AI to be more accurate and responsive.