Tries to use facial recognition technology on fish

It’s not that easy to distinguish a salmon from another salmon, or one cod from another. To us they all look pretty similar, but even fish have their own facial features. Now scientists will start using facial recognition in a pioneering project to find out who’s who.

Cameras, antennae and algorithms

This summer Kim Halvorsen and his team put out cameras and antennae around the nests of the corkwing wrasse that live at Saltskjærholmane in Austevoll. The films of the nests will be combined with information sent to the antennae from tagged corkwing wrasse, before being fed into computers to be interpreted by advanced algorithms.

“Corkwing wrasse look the same at first glance, but their patterns are different, which makes them an ideal species for testing this technology. In addition, they’re sedentary, and we’ve already tagged several thousand individuals”, says Halvorsen.

This is the first time that machine learning has been used to recognise fish in the wild, although automatic individual identification has been developed for land animals. If the technology works, it could be used to analyse many species of fish more effectively than is currently possible.

^{Daniel Nyqvist holds a ring that will be placed around a nest of a corkwing wrasse.  (Foto: Torhild Dahl/ Havforskningsinstituttet)}

Provides precise information

“When we use videos to count fish, we don’t know whether we are counting the same fish several times. If we manage to train the computers to recognise individual fish, it will open up a range of applications. For example, surveying wild salmon as they migrate up and down our rivers, or calculating the exact number of fish in a cage”, says Halvorsen.

The researchers are also interested in getting more precise information about which habitats fish use, and exactly what they do there. Do individuals return to exactly the same place each year to spawn? Facial recognition could also be used to study how the fish behave before mating, or whether a warming climate affects how fish care for their young.

^{Kim Halvorsen and colleague Caroline Durif are on their way out to the corkwing nests. (Foto: Torhild Dahl/ Havforskningsinstituttet)}

Need a large image library

One of the advantages of this method is that you can find out a lot about the fish without taking them out of the water. Physical tagging is currently used to estimate populations and to study how fish move around. As well as being expensive and time-consuming, this can affect both the welfare and behaviour of the fish. Automatic individual recognition has the potential to replace tagging experiments, and would be cheaper, better for the fish and quicker for the scientists.

The project is still in its infancy, and the researchers are by no means certain that it will succeed. One prerequisite is to build up a large number of pictures and videos of the same individuals over time and in different settings.

“We need a large image library to find out if this works, and training the computers is a big job. This is basic research that we hope will be of benefit at some point in the future”, says Halvorsen.