Overfitting in semantics-based automated program repair

被引:0
作者
Xuan Bach D. Le
Ferdian Thung
David Lo
Claire Le Goues
机构
[1] Singapore Management University,School of Information Systems
[2] Carnegie Mellon University,School of Computer Science
来源
Empirical Software Engineering | 2018年 / 23卷
关键词
Automated program repair; Program synthesis; Symbolic execution; Patch overfitting;
D O I
暂无
中图分类号
学科分类号
摘要
The primary goal of Automated Program Repair (APR) is to automatically fix buggy software, to reduce the manual bug-fix burden that presently rests on human developers. Existing APR techniques can be generally divided into two families: semantics- vs. heuristics-based. Semantics-based APR uses symbolic execution and test suites to extract semantic constraints, and uses program synthesis to synthesize repairs that satisfy the extracted constraints. Heuristic-based APR generates large populations of repair candidates via source manipulation, and searches for the best among them. Both families largely rely on a primary assumption that a program is correctly patched if the generated patch leads the program to pass all provided test cases. Patch correctness is thus an especially pressing concern. A repair technique may generate overfitting patches, which lead a program to pass all existing test cases, but fails to generalize beyond them. In this work, we revisit the overfitting problem with a focus on semantics-based APR techniques, complementing previous studies of the overfitting problem in heuristics-based APR. We perform our study using IntroClass and Codeflaws benchmarks, two datasets well-suited for assessing repair quality, to systematically characterize and understand the nature of overfitting in semantics-based APR. We find that similar to heuristics-based APR, overfitting also occurs in semantics-based APR in various different ways.
引用
收藏
页码:3007 / 3033
页数:26
相关论文
共 29 条
[1]  
Alur R(2015)Syntax-guided synthesis Dependable Software Systems Engineering 40 1-25
[2]  
Bodik R(2012)Genprog: a generic method for automatic software repair IEEE Trans Softw Eng 38 54-72
[3]  
Juniwal G(2015)The ManyBugs and IntroClass benchmarks for automated repair of C programs IEEE Trans Softw Eng 41 1236-1256
[4]  
Martin MM(2012)Modular and verified automatic program repair SIGPLAN Not 47 133-146
[5]  
Raghothaman M(2015)Mining software repair models for reasoning on the search space of automated program fixing Empir Softw Eng 20 176-205
[6]  
Seshia SA(2010)Automatic program repair with evolutionary computation Commun ACM 53 109-116
[7]  
Singh R(undefined)undefined undefined undefined undefined-undefined
[8]  
Solar-Lezama A(undefined)undefined undefined undefined undefined-undefined
[9]  
Torlak E(undefined)undefined undefined undefined undefined-undefined
[10]  
Udupa A(undefined)undefined undefined undefined undefined-undefined