Beyond transfer learning: Leveraging ancillary images in automated classification of plankton

We assess whether a supervised machine learning algorithm, specifically a convolutional neural network (CNN), achieves higher accuracy on planktonic image classification when including non-plankton and ancillary plankton during the training procedure. We focus on the case of optimizing the CNN for a single planktonic image source, while considering ancillary images to be plankton images from other instruments. We conducted two sets of experiments with three different types of plankton images (from a Zooglider, Underwater Vision Profiler 5, and Zooscan), and our results held across all three image types. First, we considered whether single-stage transfer learning using non-plankton images was beneficial. For this assessment, we used ImageNet images and the 2015 ImageNet contest-winning model, ResNet-152. We found increased accuracy using a ResNet-152 model pretrained on ImageNet, provided the entire network was retrained rather than retraining only the fully connected layers. Next, we combined all three plankton image types into a single dataset with 3.3 million images (despite their differences in contrast, resolution, and pixel pitch) and conducted a multistage transfer learning assessment. We executed a transfer learning stage from ImageNet to the merged ancillary plankton dataset, then a second transfer learning stage from that merged plankton model to a single instrument dataset. We found that multistage transfer learning resulted in additional accuracy gains. These results should have generality for other image classification tasks.