Project: CoastVision

Project summary:

It is now common to use underwater cameras to study and monitor coastal fish populations. Currently, human experts manually identify, size and count fish, frame by frame. This represents a bottleneck for upscaling deployment and data analysis.

CoastVision will apply deep learning to develop automated detection and sizing of coastal fish caught on camera. The computer vision will also be trained to identify fish in the wild by their natural “barcodes” that distinguish species, sexes and individuals, such as differences in body shape and skin coloration patterns. Individual identification and reliable re-identification is the most innovative and novel aspect of CoastVision and will open new opportunities to study behaviour, growth and survival of fish in their natural habitat.

CoastVision will focus on Atlantic cod, ballan wrasse and corkwing wrasse, all commercially important species with complex, high-contrast skin patterns. This feature will be the final step in a fully automated video analysis pipeline that will identify, track, size and count fish in video feeds from long term monitoring stations. The pipeline will be integrated into ongoing surveys and case studies whose main objective is to better understand the factors that affect the reproduction, recruitment and survival of commercially and ecologically important coastal fishes.

Further, CoastVision will support studies on short- and long-term temporal dynamics of fish communities, including detecting as the arrival of invasive species, distribution shifts and altered animal behaviour associated with climate change or other environmental stressors. Widespread adoption of camera-based monitoring with integrated computer vision will revolutionize our ability to observe, understand and respond to ecological change at scales far more refined than is currently possible.

Key research questions: 

1. How to solve the data-deficiency problem? A lack of labelled data of sufficient quality and quantity for training currently limits the rapid development and adoption of computer vision in marine ecology. Improved procedures and new methods for data collection is, therefore, needed. This challenge receives full attention in WP1
2. Can we fully replace the current manual video analysis procedure of RUV/BRUV data? This requires that multiple key species are precisely identified, counted and accurately sized. This is the goal of WP2. 
3. Is it possible to re-ID fish with high precision, and over what time scales? Our three-focal species, cod, corkwing, and ballan wrasse have complex, high-contrast skin patterns that can easily be discriminated on RUVs (Figure 1). The key question is how temporally stable these patterns are. That stability, or the lack thereof, will determine the scope of applications for visual re-ID. WP3 will develop the models to explore and answer this. 
4.  How can computer vision improve monitoring and assessments of fish stocks? High-end computers can analyse large volumes of video much faster than manual analysis and eliminate any human error and inconsistency. With the development of re-ID as a viable alternative to tagging we can obtain key knowledge for assessments much faster and at lower cost than is currently the case. We will demonstrate how in WP4.