Using deep learning for selenium web UI functional tests: A case-study with e-commerce applications

The application of test cases for detecting the faults within the software is called software testing. Manual testing is laborious and time-consuming hence automation tools to test software were introduced. Despite the use of automation tools at the User Interface (UI) level of the test pyramid, the limitations of current automation tools like automated test case generation and automated repairing of fragile tests still force us to carry out a large amount of manual testing. In this paper, we propose a novel method using AI to address the given challenges. With our proposed method test cases are automatically generated from the structure of the UI using a pipelined architecture of object detection, text detection and NLP models. We show that the test cases generated by the proposed framework can be translated into executable test scripts using a simple parser. The proposed method generates an average of 98.8% correct executable test cases for the applications under study. We also show the capability of the proposed method in generating new tests automatically when the application is modified. The proposed method generates an average of 98.605% correct executable test cases when the UI is modified for the applications under study. We also empirically prove that a GPU implementation of the proposed framework results in just an additional average runtime of 0.92 seconds per test case which is significantly low given the benefits of automated generation of test scripts and automated repairing of fragile tests.