An Empirical Study of IR-based Bug Localization for Deep Learning-based Software

被引:3
作者
Kim, Misoo [1 ]
Kim, Youngkyoung [2 ]
Lee, Eunseok [3 ]
机构
[1] Sungkyunkwan Univ, Inst Software Convergence, Suwon, South Korea
[2] Sungkyunkwan Univ, Dept Elect & Comp Engn, Suwon, South Korea
[3] Sungkyunkwan Univ, Coll Comp & Informat, Suwon, South Korea
来源
2022 IEEE 15TH INTERNATIONAL CONFERENCE ON SOFTWARE TESTING, VERIFICATION AND VALIDATION (ICST 2022) | 2022年
基金
新加坡国家研究基金会;
关键词
Empirical study; Deep learning-related software; Information retrieval-based bug localization; !text type='Python']Python[!/text] bugs; CLASSIFIER CONFIGURATION; IMPACT;
D O I
10.1109/ICST53961.2022.00024
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
As the impact of deep-learning-based software (DLSW) increases, automatic debugging techniques for guaranteeing DLSW quality are becoming increasingly important. Information-retrieval-based bug localization (IRBL) techniques can aid in debugging by automatically localizing buggy entities (tiles and functions). The low-cost advantage of IRBL can alleviate the difficulty of identifying bug locations due to the complexity of DLSW. However, there are significant differences between DI SW and traditional software, and these differences lead to differences in search space and query quality for IRBL. That is, IRBL performance must be validated in DLSW. We empirically validated IRBL performance for DLSW from the following four perspectives: 1) similarity model, 2) query generation, 3) ranking model for buggy file localization, and 4) ranking model for buggy function localization. Based on four research questions and a large-scale experiment using 2,365 bug reports from 136 DLSW projects, we confirmed the salient characteristics of DLSW from the perspective of IRBL and derived four recommendations for practical IRBL usage in DLSW from the empirical results. Regarding IRBL performance, we validated that IRBL performance midi the combination of bug-related features outperformed that of using only file similarity by 15% and IRBL ranked buggy files and functions on average of 1.6th and 2.9th, respectively. Our study is valuable as a baseline for IRBL researchers and as a guideline for DLSW developers who wish to apply IRBL to ensure DLSW quality.
引用
收藏
页码:128 / 139
页数:12
相关论文
共 59 条
[1]  
Ahasanuzzaman M, 2018, 2018 25TH IEEE INTERNATIONAL CONFERENCE ON SOFTWARE ANALYSIS, EVOLUTION AND REENGINEERING (SANER 2018), P244, DOI 10.1109/SANER.2018.8330213
[2]   A Large-Scale Comparative Evaluation of IR-Based Tools for Bug Localization [J].
Akbar, Shayan A. ;
Kak, Avinash C. .
2020 IEEE/ACM 17TH INTERNATIONAL CONFERENCE ON MINING SOFTWARE REPOSITORIES, MSR, 2020, :21-31
[3]   Method-level bug localization using hybrid multi-objective search [J].
Almhana, Rafi ;
Kessentini, Marouane ;
Mkaouer, Wiem .
INFORMATION AND SOFTWARE TECHNOLOGY, 2021, 131
[4]   Software Engineering for Machine Learning: A Case Study [J].
Amershi, Saleema ;
Begel, Andrew ;
Bird, Christian ;
DeLine, Robert ;
Gall, Harald ;
Kamar, Ece ;
Nagappan, Nachiappan ;
Nushi, Besmira ;
Zimmermann, Thomas .
2019 IEEE/ACM 41ST INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING: SOFTWARE ENGINEERING IN PRACTICE (ICSE-SEIP 2019), 2019, :291-300
[5]  
Amodei D, 2016, Arxiv, DOI arXiv:1606.06565
[6]   Using bug descriptions to reformulate queries during text-retrieval-based bug localization [J].
Chaparro, Oscar ;
Florez, Juan Manuel ;
Marcus, Andrian .
EMPIRICAL SOFTWARE ENGINEERING, 2019, 24 (05) :2947-3007
[7]  
Corder G.W, 2014, NONPARAMETRIC STAT S
[8]   Ariadne: Analysis for Machine Learning Programs [J].
Dolby, Julian ;
Shinnar, Avraham ;
Allain, Allison ;
Reinen, Jenna .
MAPL'18: PROCEEDINGS OF THE 2ND ACM SIGPLAN INTERNATIONAL WORKSHOP ON MACHINE LEARNING AND PROGRAMMING LANGUAGES, 2018, :1-10
[9]   Combining Query Reduction and Expansion for Text-Retrieval-Based Bug Localization [J].
Florez, Juan Manuel ;
Chaparro, Oscar ;
Treude, Christoph ;
Marcus, Andrian .
2021 IEEE INTERNATIONAL CONFERENCE ON SOFTWARE ANALYSIS, EVOLUTION AND REENGINEERING (SANER 2021), 2021, :166-176
[10]   What do Programmers Discuss about Deep Learning Frameworks [J].
Han, Junxiao ;
Shihab, Emad ;
Wan, Zhiyuan ;
Deng, Shuiguang ;
Xia, Xin .
EMPIRICAL SOFTWARE ENGINEERING, 2020, 25 (04) :2694-2747