The distribution characteristics and geometric morphology characteristics of defects within RFC are important factors affecting the strength properties and rupture morphology of RFC. However, the excessive size of commonly used aggregates for RFC leads to difficulties in conducting in-depth experimental studies indoors. Based on the improved U-Net and image processing technology, this research establishes an integrated model for the identification, classification, and extraction of defects inside the RFC, quantitatively counts and analyzes the acquired defect distribution characteristics and geometrical morphology characteristics, and establishes a defect characteristic distribution function that can be used for the numerical reconstruction of defects. In order to realize the acceleration of U-Net training using training weights, use VGG-16 with the fully connected layer removed instead of the Encoder part of the U-Net. The integrated model in this research can realize automatic identification, classification, and extraction of multiple types of defects at the same time, and the established distribution function of defect characteristics provides a data basis and new ideas for the establishment of RFC three-dimensional numerical models containing real defects.