Multi-sensor deep learning for autonomous population monitoring of marine species
Date
2022
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of Delaware
Abstract
Monitoring marine species, both at an individual and population level, is a critical part of protecting the marine ecosystem and managing modern fisheries. The development and utilization of Autonomous Underwater Vehicles (AUV) provides a more efficient and safer way for the task of population assessment than traditional dredging methods, as well as yielding greater data collection for biological and oceanographic surveys. However current settings of the AUV still have limitations such that it has to follow preset searching patterns for data collection and often requires offline processing with manual analysis. Therefore we are motivated to develop a vision-based deep learning approach to bring on-board intelligence to the vehicle for autonomous population monitoring of marine species. ☐ Our contributions in this work are the following: we first implemented deep learning based frameworks on the task of image-based scallop detection and further analyzed the performance of different architecture settings to demonstrate their capability of detection on low-contrast images in real-time. We also explored several ways to automatically upgrade the groundtruth annotation process. With our preliminary results, we extended our work on multi-class classification for scallop mortality rate estimation as well as analyzing the dynamics of predation. Next we experimented with optical flow for temporal analysis on sequential data. Furthermore we investigated deep learning based image registration and mosaicing methods to remove overlapping areas of successive images and therefore achieved a more precise scallop population census. Finally we proposed multi-sensor terrain analysis that combined information from optical images and side-scan sonar imagery in order to gain detailed representations of various substrate types. We also established the scallop-habitat relationship utilizing the results from our terrain classifier associated with scallop density distribution.
Description
Keywords
Monitoring marine species, Autonomous population monitoring, Mosaicing methods, Deep learning-based image registration