The Advanced Simulation and Computing (ASC) initiative on Advanced Machine Learning (AML) aims to maximize near and long-term impact of Artificial Intelligence (AI) and Machine Learning (ML) technologies on Sandia’s Nuclear Deterrence (ND) program. In this ASC-AML funded project, the team is developing new approaches for assessing the quality of machine learning predictions based on expensive/limited experimental data, with focus on problems in the nuclear deterrence (ND) mission domain. Guided by the classical Predictive Capability Maturity Model (PCMM) workflow for Verification, Validation, and Uncertainty Quantification (V&V/UQ), the project aims to rigorously assess the statistical properties of the input features when training a Scientific Machine Learning (SciML) model and examine the associated sources of noise, e.g., measurement noise. This will, in turn, enable the decomposition of output uncertainty into unavoidable aleatory part versus reducible model-form uncertainty. To improve Sandia’s stockpile surveillance analysis capabilities, the team uses signature waveforms collected using non-destructive functioning and identify the most-discriminative input features, in order to assess the quality of a training dataset. By further decomposing uncertainty into its aleatoric and epistemic components, the team will guide computational/sampling resources towards reducing the treatable parts of uncertainty. This workflow will enhance the overall credibility of the resulting predictions and open new doors for SciML models to be credibly deployed to costly or high stakes ND problems.

Workflow illustration of a machine learning classifier, trained on labeled time- or frequency-domain signals in order to classify new measurements, with uncertainty metrics.

Contact: Rushdi, Ahmad
June 2021
2021-7207 S