Evaluating the reliability of machine-learning-based predictions used in nuclear power plant instrumentation and control systems

被引:10
作者
Chen, Edward [1 ]
Bao, Han [1 ]
Dinh, Nam [2 ]
机构
[1] Idaho Natl Lab, Idaho Falls, ID 83415 USA
[2] North Carolina State Univ, Raleigh, NC USA
关键词
Machine learning; Reliability; Trustworthiness; Out -of -distribution detection;
D O I
10.1016/j.ress.2024.110266
中图分类号
T [工业技术];
学科分类号
08 ;
摘要
The field of data-driven, neural-network-based machine learning (ML) has seen significant growth, with applications in various information and control systems. Despite promising real-world uses, the reliability of models remains questionable. Conventionally, reliability is assessed based on predictive fidelity, accuracy, and training effectiveness; however, quality developmental procedures and excellent training performance metrics do not guarantee operational reliability. Instead, an ML model's predictive performance depends on the training set's representativeness to the intended operational space. It is known that ML algorithms excel at interpolation but struggle with extrapolation tasks. Anomalies and feature drift can also reduce operational performance. Determining whether a new sample is an interpolation or extrapolation task involves out-of-distribution (OOD) detection for assessing its proximity to the existing training data. Thus, we present a real-time, model-agnostic individual prediction reliability evaluation method called Data Auditing for Reliability Evaluation (DARE) for applying OOD detection to the training dataset. We demonstrate on a feedforward neural network ML-integrated digital twin for predicting fuel centerline temperatures during loss-of-flow transients. DARE acts as a "data supervisor" in determining the model's applicability under different operating conditions. In this manner, we demonstrate how training data can serve as inductive evidence to support the reliability of ML predictions.
引用
收藏
页数:16
相关论文
共 56 条
[41]   Machine learning and model driven bayesian uncertainty quantification in suspended nonstructural systems [J].
Qin, Zhiyuan ;
Naser, M. Z. .
RELIABILITY ENGINEERING & SYSTEM SAFETY, 2023, 237
[42]  
Roelofs R, 2019, ADV NEUR IN, V32
[43]  
Saunders C, 1999, IJCAI-99: PROCEEDINGS OF THE SIXTEENTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOLS 1 & 2, P722
[44]  
SHEATHER SJ, 1991, J ROY STAT SOC B, V53, P683
[45]  
Smith DJ., 2005, Reliability, Maintability and Risk, V7th
[46]   Predictive maintenance enabled by machine learning: Use cases and challenges in the automotive industry [J].
Theissler, Andreas ;
Perez-Velazquez, Judith ;
Kettelgerdes, Marcel ;
Elger, Gordon .
RELIABILITY ENGINEERING & SYSTEM SAFETY, 2021, 215
[47]   Robust deep Gaussian process-based trustworthy fog-haze-caused pollution flashover prediction approach for overhead contact lines [J].
Wang, Jian ;
Liu, Huiyuan ;
Gao, Shibin ;
Yu, Long ;
Liu, Xingyang ;
Zhang, Dongkai ;
Kou, Lei .
RELIABILITY ENGINEERING & SYSTEM SAFETY, 2024, 243
[48]   Modeling the bathtub shape hazard rate function in terms of reliability [J].
Wang, KS ;
Hsu, FS ;
Liu, PP .
RELIABILITY ENGINEERING & SYSTEM SAFETY, 2002, 75 (03) :397-406
[49]   Trustworthiness modeling and evaluation for a nearly autonomous management and control system [J].
Wang, Longcong ;
Lin, Linyu ;
Dinh, Nam .
RELIABILITY ENGINEERING & SYSTEM SAFETY, 2024, 245
[50]  
Wang S, 2019, Exploring the mechanism of crashes with automated vehicles using statistical modeling apporaches