Engineers program marine robots to take calculated risks

We know far less about the Earth's oceans than we do about the surface of the moon or Mars. The sea floor is carved with expansive canyons, towering seamounts, deep trenches, and sheer cliffs, most of which are considered too dangerous or inaccessible for autonomous underwater vehicles (AUV) to navigate.

But what if the reward for traversing such places was worth the risk?

MIT engineers have now developed an algorithm that lets AUVs weigh the risks and potential rewards of exploring an unknown region. For instance, if a vehicle tasked with identifying underwater oil seeps approached a steep, rocky trench, the algorithm could assess the reward level (the probability that an oil seep exists near this trench), and the risk level (the probability of colliding with an obstacle), if it were to take a path through the trench.

"If we were very conservative with our expensive vehicle, saying its survivability was paramount above all, then we wouldn't find anything of interest," says Benjamin Ayton, a graduate student in MIT's Department of Aeronautics and Astronautics. "But if we understand there's a tradeoff between the reward of what you gather, and the risk or threat of going toward these dangerous geographies, we can take certain risks when it's worthwhile."

Ayton says the new algorithm can compute tradeoffs of risk versus reward in real time, as a vehicle decides where to explore next. He and his colleagues in the lab of Brian Williams, professor of aeronautics and astronautics, are implementing this algorithm and others on AUVs, with the vision of deploying fleets of bold, intelligent robotic explorers for a number of missions, including looking for offshore oil deposits, investigating the impact of climate change on coral reefs, and exploring extreme environments analogous to Europa, an ice-covered moon of Jupiter that the team hopes vehicles will one day traverse.

"If we went to Europa and had a very strong reason to believe that there might be a billion-dollar observation in a cave or crevasse, which would justify sending a spacecraft to Europa, then we would absolutely want to risk going in that cave," Ayton says. "But algorithms that don't consider risk are never going to find that potentially history-changing observation."

Ayton and Williams, along with Richard Camilli of the Woods Hole Oceanographic Institution, will present their new algorithm at the Association for the Advancement of Artificial Intelligence conference this week in Honolulu.

A bold path

The team's new algorithm is the first to enable "risk-bounded adaptive sampling." An adaptive sampling mission is designed, for instance, to automatically adapt an AUV's path, based on new measurements that the vehicle takes as it explores a given region. Most adaptive sampling missions that consider risk typically do so by finding paths with a concrete, acceptable level of risk. For instance, AUVs may be programmed to only chart paths with a chance of collision that doesn't exceed 5 percent.

But the researchers found that accounting for risk alone could severely limit a mission's potential rewards.

"Before we go into a mission, we want to specify the risk we're willing to take for a certain level of reward," Ayton says. "For instance, if a path were to take us to more hydrothermal vents, we would be willing to take this amount of risk, but if we're not going to see anything, we would be willing to take less risk."

The team's algorithm takes in bathymetric data, or information about the ocean topography, including any surrounding obstacles, along with the vehicle's dynamics and inertial measurements, to compute the level of risk for a certain proposed path. The algorithm also takes in all previous measurements that the AUV has taken, to compute the probability that such high-reward measurements may exist along the proposed path.

If the risk-to-reward ratio meets a certain value, determined by scientists beforehand, then the AUV goes ahead with the proposed path, taking more measurements that feed back into the algorithm to help it evaluate the risk and reward of other paths as the vehicle moves forward.

The researchers tested their algorithm in a simulation of an AUV mission east of Boston Harbor. They used bathymetric data collected from the region during a previous NOAA survey, and simulated an AUV exploring at a depth of 15 meters through regions at relatively high temperatures. They looked at how the algorithm planned out the vehicle's route under three different scenarios of acceptable risk.

In the scenario with the lowest acceptable risk, meaning that the vehicle should avoid any regions that would have a very high chance of collision, the algorithm mapped out a conservative path, keeping the vehicle in a safe region that also did not have any high rewards -- in this case, high temperatures. For scenarios of higher acceptable risk, the algorithm charted bolder paths that took a vehicle through a narrow chasm, and ultimately to a high-reward region.

The team also ran the algorithm through 10,000 numerical simulations, generating random environments in each simulation through which to plan a path, and found that the algorithm "trades off risk against reward intuitively, taking dangerous actions only when justified by the reward."

A risky slope

Last December, Ayton, Williams, and others spent two weeks on a cruise off the coast of Costa Rica, deploying underwater gliders, on which they tested several algorithms, including this newest one. For the most part, the algorithm's path planning agreed with those proposed by several onboard geologists who were looking for the best routes to find oil seeps.

Ayton says there was a particular moment when the risk-bounded algorithm proved especially handy. An AUV was making its way up a precarious slump, or landslide, where the vehicle couldn't take too many risks.

"The algorithm found a method to get us up the slump quickly, while being the most worthwhile," Ayton says. "It took us up a path that, while it didn't help us discover oil seeps, it did help us refine our understanding of the environment."

In their long-term vision, the researchers hope to use such algorithms to help autonomous vehicles explore environments beyond Earth.

"If we went to Europa and weren't willing to take any risks in order to preserve a probe, then the probability of finding life would be very, very low," Ayton says. "You have to risk a little to get more reward, which is generally true in life as well."

This research was supported, in part, by Exxon Mobile, as part of the MIT Energy Initiative, and by NASA.