Just-In-Time Defect Prediction for Intellignet Computing Frameworks

被引：0
作者：
Ge J. [1 ]
Yu H.-Q. [1 ]
Fan G.-S. [1 ]
Tang J.-H. [1 ]
Huang Z.-J. [1 ]
机构：
[1] School of Computer Science and Engineering, East China University of Science and Technology, Shanghai
来源：
Ruan Jian Xue Bao/Journal of Software | 2023年 / 34卷 / 09期
关键词：
empirical software engineering; explainable artificial intelligence; intelligent computing framework; just-in-time defect prediction;
D O I：
10.13328/j.cnki.jos.006874
中图分类号：
学科分类号：
摘要：
In recent years, Intelligent Computing frameworks have been widely applied as Artificial Intelligence (AI) engineering implementation tools, and the reliability of Intelligent Computing framework is a key factor to the effectiveness of AI implementation. However, the reliability assurance of the Intelligent Computing Framework is challenging. On the one hand, the code iteration of Intelligent Computing Framework is fast, and testing such code is difficult. On the other hand, unlike traditional software, Intelligent Computing Framework involves a large number of tensor calculations, and its code specification lacks the guidance of software engineering theory. Existing research mostly employs fuzzy testing to localize defects in order to address this issue. However, such method can only accurately discover specific types of faults, and it is difficult to guide developers and let them focus on software quality in the development process. Therefore, this paper predicts the defects of the Intelligent Computing Framework at the code commit level. We use popular Intelligent Computing frameworks (Tensorflow, Baidu PaddlePaddle, etc.) and build a variety of commit-level features to construct datasets. Furthermore, we use LDA to mine code and commit semantic information as new features, and then use Random Forest as a classifier to perform Just-In-Time defect prediction, Results show that the average AUC-ROC performance of 0.77, and introducing semantic features slightly increases the performance. Finally, we use a machine learning model explanation technique called SHAP to analyze the influence of each feature on the prediction output of the model. We discover that (1) the influence of basic features on the model conforms to the characteristics of traditional software development, (2) code and commit semantic features are important in Just-In-Time Defect Prediction of Intelligent Computing frameworks, and (3) the contribution of different features in different systems to the output of the prediction model is also quite different. © 2023 Chinese Academy of Sciences. All rights reserved.
引用
收藏
相关论文
共 59 条
[1]  
Tensorflow
[2]  
MXNet
[3]  
PaddlePaddle
[4]  
Pei K, Cao Y, Yang J, Et al., Deepxplore: Automated whitebox testing of deep learning systems, Proc. of the 26th Symposium on Operating Systems Principles, pp. 1-18, (2017)
[5]  
Jia L, Zhong H, Wang X, Et al., The symptoms, causes, and repairs of bugs inside a deep learning library, Journal of Systems and Software, 177, (2021)
[6]  
Zhang X, Liu J, Sun N, Et al., Duo: Differential fuzzing for deep learning operators, IEEE Trans. on Reliability, 70, 4, pp. 1671-1685, (2021)
[7]  
Lessmann Stefan Baesens, Mues Bart, Pietsch Christophe, Swantje, Benchmarking Classification Models for Software Defect Prediction: A Proposed Framework and Novel Findings, IEEE transactions on pattern analysis and machine intelligence, 34, 4, pp. 485-496, (2008)
[8]  
Shihab E, Hassan A E, Adams B, Et al., An industrial study on the risk of software changes, Proc. of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering, pp. 1-11, (2012)
[9]  
Hassan A E., Predicting faults using the complexity of code changes, Proc. of the 2009 IEEE 31st International Conference on Software Engineering, pp. 78-88, (2009)
[10]  
Cai L, Fan YR, Yan M, Xia X., Just-in-time software defect prediction: Literature review, Ruan Jian Xue Bao/Journal of Software, 30, 5, pp. 1288-1307, (2019)
123456