Towards Mutation Testing of Embedded Software: A Framework and Case Study

Embedded software commonly executes safety-critical tasks and thus is expected to be highly reliable, which calls for stronger quality assurance techniques. As a fault-based testing technique, mutation testing is widely recognized to have stronger fault detection capability than those coverage or function-based testing techniques. However, some unique features of embedded software, such as limited hardware resources, real-time requirements, and differences between development and execution environments, pose challenging issues for the application of mutation testing. In this study, we explore the application of mutation testing to embedded software. As a result, we propose a mutation testing framework for embedded software, and evaluate the simulation capability of mutation operators for real-life faults in embedded software. A case study was conducted on a realistic open-source embedded software to evaluate the effectiveness and efficiency of the proposed technique. The experimental results demonstrate that mutation testing is able to detect various faults of embedded software, thus delivering a promising supplement for embedded software testing.