Reinforcement learning allows underwater robots to locate and track objects underwater

Scientists from the Institut de Ciències del Mar (ICM-CSIC) in Barcelona, in collaboration with the Monterey Bay Aquarium Research Institute (MBARI) in California, the Universitat Politècnica de Catalunya (UPC), and the Universitat de Girona (UdG), have achieved a groundbreaking development in underwater robotics. They have demonstrated, for the first time, that reinforcement learning, a type of machine learning where a neural network learns the best actions to perform based on rewards, enables autonomous vehicles and underwater robots to locate and track marine objects and animals effectively.

The use of underwater robotics has become increasingly important for exploring the depths of the ocean, as these vehicles can reach depths of up to 4,000 meters and provide valuable in-situ data that complements satellite observations. This technology is instrumental in studying various phenomena, including CO2 capture by marine organisms, which plays a role in climate change regulation.

Reinforcement learning, commonly employed in control, robotics, and natural language processing applications like ChatGPT, allows neural networks to optimize specific tasks that would otherwise be challenging to achieve. By training the robots with this learning method, the researchers successfully optimized the trajectory of the vehicles, enabling them to locate and track moving underwater objects with precision.

Ivan Masmitjà, the lead author of the study, emphasizes the significance of this learning approach in advancing ecological research, such as studying migration and movement of marine species at different scales, using autonomous robots. Additionally, the technology’s progress will facilitate real-time monitoring of oceanographic instruments through a network of robots, with some operating on the surface and others on the seabed, transmitting data via satellite.

The team employed range acoustic techniques to estimate the position of objects based on distance measurements taken from different points. However, the accuracy of object localization depended on where the acoustic range measurements were taken. To address this issue, artificial intelligence, specifically reinforcement learning, was crucial in identifying the best points and determining the optimal trajectory for the robot.

The neural networks were trained using the computer cluster at the Barcelona Supercomputing Center (BSC-CNS), which houses one of Europe’s most powerful supercomputers. This significantly accelerated the parameter adjustments for different algorithms compared to conventional computers.

Overall, this breakthrough in underwater robotics and the successful application of reinforcement learning pave the way for more in-depth ecological studies, as well as enhanced oceanographic monitoring, through a network of autonomous underwater robots.