A framework for assessment of predictive capability maturity and its application in nuclear thermal hydraulics

This work presents a formalized and computerized framework for the assessment of decision regarding the adequacy of a simulation tool for a nuclear reactor application. The decision regarding a code's adequacy for an application is dependent on the assessment of different attributes that govern verification, validation, and uncertainty quantification of the code. In this work, the focus is on code validation. Therefore, the framework is developed and illustrated from the perspective of decision regarding the validation assessment of a code. Code validation assessment is performed based on the validation test results, data applicability, and process quality assurance factors. The process quality assurance factors warrant the trustworthiness of the evidence and help in checking people and process compliance with respect to the standard requirements. The proposed framework is developed using an argument modeling technique called Goal Structuring Notation (GSN). Goal structuring notation facilitates structural knowledge representation, information abstraction, evidence incorporation, and provides a skeletal structure for quantitative maturity assessment. The decision schema for the development of the decision model is based on the Predictive Capability Maturity Model (PCMM) and Analytic Hierarchy Process (AHP) and formalized using Goal structuring notation. Each decision attribute is formulated as a claim, where the degree of validity of the claim (attribute's assessment) is expressed using different maturity levels. The GSN representation of the decision model is transformed into a confidence network to provide evidence-based quantitative maturity assessment using the Bayesian network. A metric based on the expected utility of maturity levels, called expected distance metric, is proposed to measure the distance between target maturity and achieved maturity on a scale of zero to one. Expected distance metric helps in comparing the assessment of different attributes and identification of major areas of concern in terms of modeling capability, data needs, and quality of assessment process. The practical application of the framework is demonstrated by a case study on validation assessment of a thermal-hydraulic code for a challenge problem called Departure from Nucleate Boiling (DNB).