Rapid Calibration of AI Models to High-Fidelity Benchmarks
Request a Meeting
Challenge
High-fidelity, physics-based simulations (e.g., 6DOF trajectory models) are often too slow for large-scale analysis. Faster AI-based surrogate models are needed, but they must be accurately calibrated to match the “truth” performance of the accredited high-fidelity model.
Our Solution
We use OptDef to automate the calibration and tuning of AI surrogate models. OptDef systematically adjusts the AI’s hyperparameters (like learning rate, network architecture, etc.) with the explicit objective of minimizing the difference between the AI model’s output and the 6DOF benchmark trajectory.
Impact
This process rapidly finds the optimal set of AI tuning parameters to produce a fast-running AI model that most accurately replicates the behavior of the high-fidelity system. This provides analysts with a validated, high-speed surrogate model for use in larger simulations, wargames, and training data generation.
More Case Studies
Stress Testing to Harden Simulation Models and Uncover "Brittle" Behavior
Simulation models can exhibit unexpected or “brittle” behavior at the edges of their performance envelope, creating vulnerabilities that are difficult to find with standard testing. Proactively identifying these failure points is critical for building trust in the model.
Missile Defense Agency: Efficient Test & Evaluation (T&E) via Adaptive Sampling Simulation Optimization
The MDA needs to generate “heatmap” (response surface) analyses to understand the effectiveness of the Missile Defense System across a wide range of scenarios. Traditional approaches, like full enumeration or dense sampling, would require orders of magnitude more simulation runs than are computationally feasible.