Toward a Methodology for the Verification and Validation of AI-Based Systems

被引:1
作者
Paardekooper, Jan-Pieter [1 ,2 ]
Borth, Michael [1 ]
机构
[1] TNO, Integrated Vehicle Safety, The Hague, Netherlands
[2] Radboud Univ Nijmegen, Donders Inst Brain, Nijmegen, Netherlands
来源
SAE INTERNATIONAL JOURNAL OF CONNECTED AND AUTOMATED VEHICLES | 2025年 / 8卷 / 01期
关键词
Artificial intelligence; AI; safety; Automated driving; Safety standards;
D O I
10.4271/12-08-01-0006
中图分类号
U [交通运输];
学科分类号
08 ; 0823 ;
摘要
Verification and validation (V&V) is the cornerstone of safety in the automotive industry. The V&V process ensures that every component in a vehicle functions according to its specifications. Automated driving functionality poses considerable challenges to the V&V process, especially when data-driven AI components are present in the system. The aim of this work is to outline a methodology for V&V of AI-based systems. The backbone of this methodology is bridging the semantic gap between the symbolic level at which the operational design domain and requirements are typically specified, and the sub-symbolic, statistical level at which data-driven AI components function. This is accomplished by combining a probabilistic model of the operational design domain and an FMEA of AI with a fitness-for-purpose model of the system itself. The fitness-for-purpose model allows for reasoning about the behavior of the system in its environment, which we argue is essential to determine whether the system meets its requirements. While this work only provides an outline of such a methodology, we point out future research directions toward a full methodology for the V&V of AI-based systems.
引用
收藏
页数:12
相关论文
共 33 条
  • [1] Putz A., Zlocki A., Bock J., Eckstein L., System Validation of Highly Automated Vehicles with a Database of Relevant Traffic Scenarios, The in ITS Europe, (2017)
  • [2] Elrofai H., Paardekooper J.P., De Gelder E., Kalisvaart S., Et al., StreetWise: Scenario-Based Safety Validation of Connected Automated Driving, Technical Report, (2018)
  • [3] Borth M., Oliveira Filho J., Van Der Ploeg C., Fitness Assessment of AI-Based Systems, The in 2024 Prognostics and System Health Management Conference (PHM 2024), (2024)
  • [4] Seshia S.A., Sadigh D., Sastry S.S., Toward Verified Artificial Intelligence, Commun. ACM, 65, 7, pp. 46-55, (2022)
  • [5] (2022)
  • [6] Koopman P., UL 4600: What to Include in an Autonomous Vehicle Safety Case, Computer, 56, 5, pp. 101-104, (2023)
  • [7] Burton S., Hellert C., Huger F., Mock M., Et al., Safety Assurance of Machine Learning for Perception Functions, Deep Neural Networks and Data for Automated Driving: Robustness, Uncertainty Quantification, and Insights Towards Safety, pp. 335-358, (2022)
  • [8] Borg M., Et al., Ergo, SMIRK Is Safe: A Safety Case for a Machine Learning Component in a Pedestrian Automatic Emergency Brake System, Software Qual J, 31, 2, pp. 335-403, (2023)
  • [9] Fremont D.J., Et al., Scenic: A Language for Scenario Specification and Data Generation, (2020)
  • [10] Bergenhem C., Johansson R., Soderberg A., Nilsson J., Tryggvesson J., Et al., How to Reach Complete Safety Requirement Refinement for Autonomous Vehicles, CARS 2015 - Critical Automotive Applications: Robustness & Safety, (2015)