An Automatic Method for Sublingual Image Segmentation and Color Analysis

Automatic segmentation of sublingual images and color quantification of sublingual vein are of great significance to disease diagnosis in traditional Chinese medicine. With the development of computer vision, automatic sublingual image processing provides a noninvasive way to observe patients' tongue and is convenient for both doctors and patients. However, current sublingual image segmentation methods are not accurate enough. Besides, the differences in subjective judgments by different doctors bring more difficulties in color analysis of sublingual veins. In this paper, we propose a method of sublingual image segmentation based on a modified UNet++ network to improve the segmentation accuracy, a color classification approach based on triplet network, and a color quantization method of sublingual vein based on linear discriminant analysis to provide intuitive one-dimensional results. Our methods achieve 88.2% mean intersection over union (mIoU) and 94.1% pixel accuracy on tongue dorsum segmentation, and achieves 69.8% mIoU and 82.7% pixel accuracy on sublingual vein segmentation. Compared with the state-of-art methods, the segmentation mIoUs are improved by 5.8% and 5.3% respectively. Our sublingual vein color classification method has the highest overall accuracy of 81.2% and the highest recall for the minority class of 77.5%, and the accuracy of color quantization is 90.5%. Clinical Relevance- The methods provide accurate and quantified information of the sublingual image, which can assist doctors in diagnosis.