Vehicle load is important for condition assessment, maintenance and reinforcement of bridges. In recent years, computer vision technology has been applied to the identification of vehicle loads on bridges. According to the latest reports, the vehicle can be detected as a whole to get its spatiotemporal information. However, the wheelbase and number of axles of vehicle loads are difficult to obtain accurately. In addition, the high cost of WIM equipment makes it difficult to be used in practice. Therefore, it is necessary to explore an economical identification approach that can provide more complete and accurate data for subsequent mechanics analysis and other decisions making. In this paper, an approach for fine-grained identification of vehicle loads on bridges was proposed. Based on the deep convolutional neural networks, a vehicle detector was obtained to achieve the detection of vehicles and tires at two different scales. Using the results from vehicle detection and camera calibration, an accurate 3D bounding box reconstruction algorithm was proposed to obtain the vehicle sizes, position, wheelbase and axle number. Then the vehicle was tracked using the optimized Kalman filter algorithm and the trajectory and speed were obtained. Finally, the gross vehicle weight and axle weight were obtained according to the axle information and statistical distribution model of the vehicle weight. To test the accuracy and reliability, the algorithm for vehicle load identification was developed and tested on a bridge in operation, and the results demonstrated that it was capable of identify vehicle loads at the fine-grained level.