CNN-based binary classification of 3D optical microscopic images

In the production of display screen modules, multi-faceted quality control is performed. One of the processes is detection of defects on and between module components such as particles, scratches and air bubbles using a 3D optical microscope. Technicians view a stack of images of potential defect areas and make a qualitative assessment of the sample. However, this is made difficult by the artifacts in the unfocused image layers. Moreover, there is a large discrepancy in the detection tendencies of the technicians. In order to standardize and automate the classification of major and minor defects in products, we propose a convolutional neural network based binary classification that makes use of the normal angle and oblique angle images. The decision factors affecting the classification of the sample include defect position, size, and shape. In order to reflect these factors, the microscopic images of the sample are taken in varying focal depths from normal and oblique angles. Then, the maximum intensity projection (MaxIP) and minimum intensity projection (MinIP) in the xy, yz, xz plane are created. The set of MaxIP and MinIP are used to train a modified VGG-network. Each plane differs in size, so MaxIP and MinIP of every plane was independently added as input to the network and were concatenated in the fully connected layer. Being that the dataset used for this work composed of 185 major defect samples and 2036 minor defect samples, augmentation was essential. In order to even out the major and minor defect sample ratio, random affine transformation was performed on the major defect sample images. The proposed method of binary classification performs with a total accuracy of 98.6%.