An Empirical Analysis on Just-In-Time Defect Prediction Models for Self-driving Software Systems

Just-in-time (JIT) defect prediction has been used to predict whether a code change is defective or not. Existing JIT prediction has been applied to different kind of open-source software platform for cloud computing, but JIT defect prediction has never been applied in self-driving software. Unlike other software systems, self-driving system is an AI-enabled system and is a representative system to which edge cloud service is applied. Therefore, we aim to identify whether the existing JIT defect prediction models for traditional software systems also work well for self-driving software. To this end, we collect and label the dataset of open-source self-driving software project using SZZ (Sliwerski, Zimmermann and Zeller) algorithm. And we select four traditional machine learning methods and state-of-the-art research (i.e., JIT-Line) as our baselines and compare their prediction performance. Our experimental results show that JITLine and logistic regression produce superior performance, however, there exists a room to be improved. Through XAI (Explainable AI) analysis it turned out that the prediction performance is mainly affected by experience and history-related features among change-level metrics. Our study is expected to provide important insight for practitioners and subsequent researchers performing defect prediction in AI-enabled system.