Artificial intelligence has been taught to "read" salmon scales

Like trees have growth rings, fish have information stored in their scales. In the case of salmon, experts can interpret this information to ascertain the age of the fish, how long it has lived in the sea, how many times it has spawned and whether it comes from a fish farm. However, this can be a time-consuming process. 

That’s why scientists at the Institute of Marine Research (IMR) have taught a type of artificial intelligence known as a neural network how to interpret images of salmon scales seen through a magnifying glass. The results have been published in a scientific paper. 

Easy to distinguish wild and farmed salmon, but harder to determine their age

“The neural network had a 97 percent hit rate when asked to distinguish between farmed and wild salmon. But this is a relatively simple task that experts can also perform quickly”, says Endre Moen, a computer scientist at the IMR.

“The most interesting questions are how long the salmon have lived in the ocean and river respectively”, he continues.

To measure how well the neural network performed these tasks, the scientists looked at how often it agreed with six human experts.

Second best at ascertaining sea age

“The network agreed with the experts 94 percent of the time when estimating the sea age of the fish. In fact it was second best – only beaten by one human expert who was 97 percent in agreement with the others”, says Moen. 

When estimating the river age of the salmon, the network scored 66 percent. In this case the experts scored between 75 and 85 percent, but they had access to additional information as well as the image. For example, which river the fish came from. 

This matters because salmon have local habits in terms of the age at which they leave rivers as smolts. It can be anywhere from two to six years, depending on the local conditions, such as the water temperature and size of the river. 

^{Here is one of several thousand scales used in the study. The dark grey section at the bottom of the picture is the part of the scale that is visible on the outside of the salmon. The age has been determined by an expert.} 

In any case, the river age was hardest to estimate for both artificial and human intelligence.

The quality of the model answer matters

“Interpreting the river age is a more difficult task, and the one where there is most disagreement between the experts. This is reflected in the neural network”, explains Endre Moen. 

To train the neural network, the researchers fed in over 6,000 images of salmon scales, along with the experts’ interpretations. Afterwards, it took an “exam” by interpreting a further 150 pictures by itself. 

This is called supervised machine learning (see fact box).

“If the experts are uncertain or disagree, it makes the artificial intelligence more uncertain when it starts operating independently.”

Not fully understood how it thinks

We cannot be sure that the neural network notices the same characteristics as the humans. The researchers are not entirely sure it reaches the right answer. 

“We know that it first looks for simple characteristics in the images, such as short horizontal and vertical lines. It may be that the network gives importance to both the shape and size of the scales, or it might be completely different things”, says Moen. 

The type of neural network they used was originally developed to recognise objects in images, such as dogs, cats, bikes and houses, but has been modified to be applicable to ageing of fish scales.

AI can join the expert panel

The initial purpose of the study is to demonstrate the possibilities offered by the new technology. 

“For the moment, it is not safe to leave the whole analysis of the salmon scales to a neural network. But as a cautious first step, the network can be included as one of several experts on a panel. This could help to free up human resources or increase reliability”, says Endre Moen. 

Reference

Vabø, Rune, Endre Moen, Szymon Smoliński, Åse Husebø, Nils Olav Handegard, and Ketil Malde. "Automatic interpretation of salmon scales using deep learning." Ecological Informatics 63 (2021): 101322.

Links: https://www.sciencedirect.com/science/article/pii/S1574954121001138#f0015