Accelerated Testing and Evaluation of Autonomous Vehicles Based on Dual Surrogates

Testing and evaluation plays a critical role in the research and development (R&D) of autonomous vehicles (AVs). Due to the black-box property of AVs and the restraint of test resources, how to quickly test and evaluate AVs' safety remains a major challenge. To address this problem, a novel search-based ADOE testing and evaluation method is proposed, which improves the efficiency of testing and evaluation through a two-stage acceleration. In Accelerated Testing stage, the proposed ADOE-based testing method (DUSGAT) is used to accelerate testing, which adopts dual surrogates (result surrogate MR\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\varvec{M}}}_{{\varvec{R}}}$$\end{document} and behavior surrogate MB\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\varvec{M}}}_{{\varvec{B}}}$$\end{document}). This method indicates not only the previous searching results but also the behavior pattern of the system under test (SUT) during previous tests, to accelerate the whole process of unveiling all critical scenarios for the SUT. In the Accelerated Evaluation stage, the trained MB\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\varvec{M}}}_{{\varvec{B}}}$$\end{document} is used to quickly predict and evaluate the safety performance of SUT in the logical scenario. Experimental results show that DUSGAT has the best search performance compared with baseline methods, and the F2-score of DUSGAT is higher than TuRBO. In the 2-dimensional car-following scenario, the relative error between the output SUT's hazard ratio of our method and that of grid search is only 0.68%. In the 3-dimensional cut-in scenario, the relative error is only 1.32%. What's more, compared with grid search, this study's method is 3.217 x faster in the car-following scenario, and 22.116 x faster in the cut-in scenario. Therefore, the method can accurately test and evaluate the safety performance of SUT and has the potential to be used in high-dimensional scenarios.