A Comparative Study of Whole Issues and Challenges in Mutation Testing

Mutation testing is a powerful and expensive method in software testing context. It is used as a test criterion to assess the quality of test suites or generate test suites which are able to kill mutants created by seeding artificial defects in the original program code. These test suites are high-quality tests with good ability to find probable faults in the program under test. Mutation testing has gained high attention recently through its superiority over other testing criteria in different aspects and is the main topic of researches in software testing area. As well as powerfulness of mutation testing, it has challenges and difficulties which are great barriers for industrial utilization. This paper provides a comprehensive classification of mutation testing issues and challenges. Furthermore, a comparative study is conducted on researches that made an effort to investigate issues and resolve challenges of mutation testing along with presenting a classification of these proposed approaches. This comparative structure suggests different research topics and makes a guideline for researchers interested in mutation testing context. This also can be used for the purpose of comparing the existing methods, selecting the best one and improving selected methods.