Elements: Development of cyberinfrastructure to establish a scalable application of self-supervised machine learning for over a decade of NOAA's water column sonar data

The health of the ocean is vital to economies throughout the world due to the global importance of ship traffic, commercial and recreational fisheries, tourism, and energy exploration and extraction. One technology that scientists use to understand and predict changes in the ocean is sonar, which allows them to ‘see’ into the ocean to observe its inhabitants and features such as ocean currents, sunken vessels, vents, and ocean bottom.

By investigating the water column – the area of the ocean from the surface to the seafloor – using sonar technology, foundational information on the condition of the ocean ecosystems can be learned. Sonars produce vast amounts of data and much of it is interpreted by the trained eye of expert scientists. Unfortunately, modern sonar systems produce more data than scientists can interpret, and fast and accurate ways to extract information are needed.

This project’s innovative approach efficiently processes decades of publicly available water column sonar data, which adds up to hundreds of terabytes. This project focuses on economically critical fisheries, and the results show how the patterns of fish schools and small swimming animals called zooplankton change over time and location. The project’s methods are being shared widely so scientists across the world can more easily use water column sonar in their research and interpretation is simplified since the results are directly comparable.

The processing of existing data in new ways provides new information about ocean health, and rapid sharing of that information will lead to quicker answers for management decisions. These methods can also be applied to real-time sonar data collected on global fishing vessels and integrated into swarms of scientific ocean robots. When combined with other ocean data, the team can understand why the distribution of essential critters like zooplankton and fish changes, and how climate change can affect global fisheries.

The project’s team is also training the next generation of scientists and engineers by using the information learned throughout the project in undergraduate courses for a diversity of students.

Water column sonars provide foundational information on the condition of ocean ecosystems and inform marine resource conservation decisions. This project is developing the cyberinfrastructure (CI) required to apply self-supervised machine learning (SSL) to decades (and thus tens of terabytes) of water column sonar data to discover patterns that reflect the spatio-temporal physical and biological structure of aquatic environments.

The SSL model is built using multi-frequency echosounder data collected from 1998 to 2022 by the NOAA Northeast Fisheries Science Center. These data are archived at the NOAA National Centers for Environmental Information and accessible as analysis-ready zarr stores on Amazon Web Services. This effort explores different scales of data in different regions of the Northwest Atlantic, evaluates the latency of pattern analysis, and validates the accuracy of the patterns found with domain experts.

The project will deliver a CI proof of concept for a new, self-learning, and extensible method to classify acoustic signal patterns from large volumes of data. In combination with climate indicators, it enables advanced understanding of how the distribution of ecosystem essential critters like zooplankton and fish have been changing over time and space, and why.

This award by the NSF Office of Advanced Cyberinfrastructure is jointly supported by the NSF Division of Biological Infrastructure (BIO/DBI).

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.