An Empirical Study of Greedy Test Suite Minimization Techniques Using Mutation Coverage

被引:2
作者
Jehan, Seema [1 ]
Wotawa, Franz [2 ]
机构
[1] Natl Univ Sci & Technol, Sch Elect Engn & Comp Sci, Islamabad 44000, Pakistan
[2] Graz Univ Technol, Inst Software Technol, Christian Doppler Lab Qual Assurance Methods Auton, A-8010 Graz, Austria
关键词
Test suite minimization; mutation testing; regression testing; !text type='Java']Java[!/text]Script; REDUCTION; PRIORITIZATION; ALGORITHMS; SELECTION;
D O I
10.1109/ACCESS.2023.3289073
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Test suite minimization is the task of finding a smaller test suite that still fulfills the properties of the original test suite but which comprises fewer test cases. It is important in practice, especially in the context of regression testing, where test suites are re-executed. However, test suite minimization as a set covering problem is known as an NP-complete problem, which requires applications of heuristics. Although many test suite minimization techniques have been applied previously but obtained conflicting results primarily due to inherent differences in underlying programming languages and experimental setup. In this respect, we study traditional greedy-based algorithms for test suite minimization that allow to remove test cases in a way such that the reduced test suite satisfies all requirements. Specifically, we evaluated commonly discussed approaches on publicly available JavaScript applications using mutation coverage. We show that the discussed algorithms reduce the test suite size of the studied example programs on average to 70% without compromising the fault-detection capability of the original test suite. The suggested approach not only minimizes the test suite's size, thereby reducing the regression testing cost, but also ensures that the reduced test suite catches the same number of faults as that of the original test suite. Further, we also examine their performance in scenarios when meeting all testing requirements is not feasible due to time and budget constraints.
引用
收藏
页码:65427 / 65442
页数:16
相关论文
共 61 条
[1]   Fault coverage-based test suite optimization method for regression testing: learning from mistakes-based approach [J].
Agrawal, Arun Prakash ;
Choudhary, Ankur ;
Kaur, Arvinder ;
Pandey, Hari Mohan .
NEURAL COMPUTING & APPLICATIONS, 2020, 32 (12) :7769-7784
[2]   Establishing Theoretical Minimal Sets of Mutants [J].
Ammann, Paul ;
Delamaro, Marcio E. ;
Offutt, Jeff .
2014 IEEE SEVENTH INTERNATIONAL CONFERENCE ON SOFTWARE TESTING, VERIFICATION AND VALIDATION (ICST), 2014, :21-30
[3]   A Survey of Dynamic Analysis and Test Generation for Java']JavaScript [J].
Andreasen, Esben ;
Gong, Liang ;
Moller, Anders ;
Pradel, Michael ;
Selakovic, Marija ;
Sen, Koushik ;
Staicu, Cristian-Alexandru .
ACM COMPUTING SURVEYS, 2017, 50 (05)
[4]  
[Anonymous], 2007, P 2007 INT S SOFTW T
[5]  
Artzi S, 2011, 2011 33RD INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING (ICSE), P571, DOI 10.1145/1985793.1985871
[6]  
Async, 2020, ASYNC
[7]   Bi-criteria models for all-uses test suite reduction [J].
Black, J ;
Melachrinoudis, E ;
Kaeli, D .
ICSE 2004: 26TH INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING, PROCEEDINGS, 2004, :106-115
[8]  
Body-Parser, 2019, BODY PARSER FRAMEWOR
[9]  
Cheerio, 2018, ABOUT US
[10]   A new heuristic for test suite reduction [J].
Chen, TY ;
Lau, MF .
INFORMATION AND SOFTWARE TECHNOLOGY, 1998, 40 (5-6) :347-354