In-Situ Training and Active Learning for Surrogates

State of the art in online/in-situ training of ML surrogates during FEM simulation, active learning for simulation data, and uncertainty quantification for neural operators.

Online/in-situ training during simulation

Key approaches

Yan & Yan (2025): Surrogate models trained in-situ using data generated at each simulation timestep, with direct data flow from simulation into the training pipeline. Defines spatio-temporal characteristics for selective data extraction.

Citation: Yan, K. & Yan, L. (2025). “Applying Surrogate Modeling to Decouple Data Collection and Analysis from Simulation for Accelerated In-Situ Analysis.” Proc. SC’25 Workshops, ACM.

Alsaadi et al. (2025): ROSE (RADICAL Orchestrator for Surrogate Exploration). Distinguishes online training (continuous model updates during simulation) from offline training. Provides orchestration framework for HPC surrogate deployment.

Citation: Alsaadi, R. et al. (2025). “ROSE: RADICAL Orchestrator for Surrogate Exploration.” Proc. SC’25, ACM.

Lee et al. (2026): Catalyst framework for in-situ analysis and visualization in exascale computing. Demonstrates in-situ surrogate model training with online learning that adapts ML models dynamically as simulation progresses.

Citation: Lee, S. et al. (2026). “Catalyst: Expanding In Situ Analysis and Visualization Workflows for Exascale Computing.” AIAA SCITECH 2026.

Ricks & Arnold (2025): Documents practical integration of ML surrogates replacing physics-based ply models within traditional FE software. Key finding: the surrogate replaces individual component models, not the entire solver.

Citation: Ricks, T.M. & Arnold, P.M. (2025). “Lessons Learned from Integrating a Surrogate Model into a Finite Element Software.” NASA Technical Report.

Triggering strategies

• Error threshold: Retrain when surrogate prediction error vs FEM residual exceeds tolerance
• Fixed iteration count: Retrain every N timesteps
• Adaptive intervals: Based on solution change rate
• Residual-based: FEM residual norm divergence from surrogate prediction exceeds tolerance

Memory-efficient pipelines

• Ring buffers for training data
• Importance-weighted sampling to retain only high-information samples
• Gradient accumulation for mini-batch training without storing full datasets
• Progressive data eviction with summary statistics

C++ FEM implementations

Roth, Schröder & Wick (2022) is the only published deal.II + LibTorch integration. deal.II solves the primal FEM problem; LibTorch solves the adjoint problem with a feedforward neural network. Demonstrates NN-guided goal-oriented a posteriori error estimation.

Citation: Roth, M., Schröder, J. & Wick, T. (2022). “Neural network guided adjoint computations in dual weighted residual error estimation.” SN Applied Sciences (Springer). Cited 20 times.

No published work on in-situ training within deal.II specifically.

Active learning for simulation data

Uncertainty-driven sampling

Jiang, Chen & Zhao (2024): Uses MC dropout to separate aleatoric and epistemic uncertainty for active sample selection in FEA contexts.

Citation: Jiang, Y., Chen, W. & Zhao, J. (2024). “Monte Carlo Dropout Based Active Learning for Deep Learning in Structural Simulation.” ASME.

Xiang, Chen, Bao & Li (2020): Combines DNN with weighted sampling that balances uncertainty and space-filling. Cited 156 times.

Citation: Xiang, Z., Chen, W., Bao, Y. & Li, Q. (2020). “An active learning method combining deep neural network and weighted sampling for structural reliability analysis.” Mechanical Systems and Signal Processing.

Query-by-committee

Shoghi, Morand, Helm & Hartmaier (2024): QBC for constitutive model training with FE-generated data, with dynamic stopping criterion.

Citation: Shoghi, A. et al. (2024). “Optimizing machine learning yield functions using query-by-committee for support vector classification.” Computational Mechanics.

Shoghi et al. (2026): Diversity-aware batch QBC for maximum information content at minimum cost.

Citation: Shoghi, A. et al. (2026). “Diversity-Aware Batch-Mode Active Learning for Efficient Sampling in Data-Driven Constitutive Modeling.” arXiv.

Adaptive mesh refinement guided by surrogate error

Yang, Dzanic, Petersen, Kudo et al. (2023): RL-trained policies for AMR using surrogate reward signals. Cited 85 times.

Citation: Yang, Y. et al. (2023). “Reinforcement learning for adaptive mesh refinement.” Proc. ICML (MLR Press).

Baiges, Codina & Castanar (2020): ANN-based ROM with AMR driven by error estimators. Cited 77 times.

Citation: Baiges, J., Codina, R. & Castanar, I. (2020). “A finite element reduced-order model based on adaptive mesh refinement and artificial neural networks.” Int. J. Numerical Methods in Engineering.

Multi-fidelity approaches

Pellegrini et al. (2023): Multi-fidelity surrogate with active learning for shape optimization using coarse/fine FE grids. Cited 54 times.

Citation: Pellegrini, R. et al. (2023). “A multi-fidelity active learning method for global design optimization problems with noisy evaluations.” Engineering with Computers.

Wen et al. (2026): Review covering multi-fidelity fused networks and transfer learning approaches for composites.

Citation: Wen, Y. et al. (2026). “Multi-fidelity surrogates for mechanics of composites: from co-kriging to multi-fidelity neural networks.” arXiv.

Error estimation and uncertainty quantification

Ensemble methods for epistemic uncertainty

Song et al. (2026): Directly addresses epistemic UQ for neural operators on PDEs. Compares deep ensembles, Laplace approximation, and MC dropout for neural operator architectures.

Citation: Song, Y. et al. (2026). “Structure-Aware Epistemic Uncertainty Quantification for Neural Operator PDE Surrogates.” arXiv.

Pensoneault & Zhu (2025): Uses Ensemble Kalman Inversion for UQ in operator learning. Cited 11 times.

Citation: Pensoneault, C. & Zhu, Y. (2025). “Uncertainty quantification for DeepONets with ensemble Kalman inversion.” J. Computational Physics.

Sharma & Shankar (2024): Ensemble DeepONet architectures with improved generalization. Cited 12 times.

Citation: Sharma, A. & Shankar, V. (2024). “Ensemble and mixture-of-experts DeepONets for operator learning.” arXiv.

Conformal prediction

Gopakumar, Gray, Oskarsson et al. (2026): Model-agnostic conformal prediction framework with statistically guaranteed marginal coverage for spatio-temporal surrogate models including PDE operators. Cited 10 times.

Citation: Gopakumar, A. et al. (2026). “Uncertainty quantification of surrogate models using conformal prediction.” Machine Learning: Science and Technology (IOP).

Yu, Ho & Wang (2025): Conformal prediction adapted for PINNs with local adaptiveness for spatio-temporal predictions.

Citation: Yu, X., Ho, D. & Wang, Z. (2025). “A conformal prediction framework for uncertainty quantification in physics- informed neural networks.” arXiv.

A posteriori error estimation

Qiu, Dahmen & Chen (2025): Rigorous a posteriori error bounds for neural operator surrogates of parametric PDEs using reduced basis methods.

Citation: Qiu, W., Dahmen, W. & Chen, H. (2025). “Variationally correct operator learning: Reduced basis neural operator with a posteriori error estimation.” arXiv.

De Ryck & Mishra (2022): Foundational theoretical result on a posteriori error for PINNs and operator learning. Cited 112 times.

Citation: De Ryck, T. & Mishra, S. (2022). “Generic bounds on the approximation error for physics-informed (and) operator learning.” NeurIPS 2022.

Comprehensive UQ library

Zou, Meng, Psaros & Karniadakis (2024): NeuralUQ library covering both neural ODEs/PDEs and neural operators with multiple UQ methods. Cited 119 times.

Citation: Zou, X. et al. (2024). “NeuralUQ: A comprehensive library for uncertainty quantification in neural differential equations and operators.” SIAM Review.

Error propagation through chained surrogates

Wang & Tartakovsky (2024): Addresses total uncertainty (aleatoric + epistemic) when chaining surrogate models for inverse problems.

Citation: Wang, N. & Tartakovsky, A.M. (2024). “Total uncertainty quantification in inverse PDE solutions obtained with reduced-order deep learning surrogate models.” arXiv.

Explicit solvers with time scaling for training data generation

Motivation

Faster FEM simulations mean faster data generation for surrogate training. Explicit solvers (central difference) avoid nonlinear solves per timestep and are trivially parallelizable on GPUs.

Deposition time scaling: proven approach

Time scaling (also called event time scaling or velocity scaling) artificially compresses the simulated event duration by multiplying the physical time increment by a scaling factor (SF). This is distinct from mass scaling (which increases density). Time scaling adjusts the loading rate and thermal cycle rate directly.

The key insight: AM and welding are quasi-static processes where inertial effects are negligible. The mechanical response is dominated by thermal expansion/contraction and plastic strain accumulation, not stress wave propagation. The explicit solver can tolerate much larger time steps than the Courant limit as long as kinetic energy remains negligible relative to internal energy.

ORNL GPU explicit FEM: the key reference

Huang, Ma, Chen, Feng & Murakawa (2020): GPU-based explicit FEM with time scaling for WAAM residual stress and distortion prediction. Validated against neutron diffraction measurements on 4-layer and 20-layer thin-wall WAAM builds.

Citation: Huang, H., Ma, N., Chen, J., Feng, Z. & Murakawa, H. (2020). “Toward large-scale simulation of residual stress and distortion in wire and arc additive manufacturing.” Additive Manufacturing, 34: 101248. DOI: 10.1016/j.addma.2020.101248

Calibration table (equivalent plastic strain error):

Time SF	Max equiv. epsilon_p	Error
80	0.0266	0.00%
800	0.0266	0.00%
8,000	0.0267	0.38%
10,000	0.0269	1.13%
26,000	0.0271	1.88%
40,000	0.0273	2.63%
80,000	0.0280	5.26%

Safe SF limit: ~30,000 for <2% error. Beyond that, inertial artifacts corrupt the plastic strain field.

Speedup achieved:

Method	20-Layer Model	Speedup vs Abaqus
Abaqus Standard (4 CPU)	142.9 h	1x
GPU Explicit + Time Scaling (1 GPU)	2.0 h	70x

The 20-layer build took 2.7 h of actual printing time. The simulation completed in 2.0 h — near real-time.

Additional key papers

Ma (2016): ACEXP (Accelerated Explicit) method with GPU parallelization for welding deformation and residual stress.

Citation: Ma, N. (2016). “An accelerated explicit method with GPU parallel computing for thermal stress and welding deformation of large structure models.” Int. J. Advanced Manufacturing Technology, 87: 2195—2211. DOI: 10.1007/s00170-016-8542-3

Ikushima & Shibahara (2014): Idealized Explicit FEM (IEFEM) with GPU for multi-pass welding residual stress.

Citation: Ikushima, K. & Shibahara, M. (2014). “Prediction of residual stresses in multi-pass welded joint using idealized explicit FEM accelerated by a GPU.” Computational Materials Science, 93: 62—67. DOI: 10.1016/j.commatsci.2014.06.024

Huang et al. (2026): High-fidelity explicit coupled thermo-mechanical FEA for metal AM.

Citation: Huang, H. et al. (2026). “A high-fidelity explicit coupled thermo-mechanical FEA model to predict stress and distortion in metal additive manufacturing.” Additive Manufacturing, 121: 105153. DOI: 10.1016/j.addma.2026.105153

Mozaffar et al. (2019): GPU-accelerated explicit thermal FEM for powder-bed AM. 100—150x speedup for transient heat transfer.

Citation: Mozaffar, M. et al. (2019). “Acceleration strategies for explicit finite element analysis of metal powder-based additive manufacturing processes using graphical processing units.” Computational Mechanics, 64: 879—894. DOI: 10.1007/s00466-019-01685-4

Recommended hybrid approach

The ORNL code uses a sequential explicit/implicit framework:

1. Heating/deposition phase: Explicit FEM with time scaling (fast)
2. Cooling and fixture relaxation: Implicit FEM (accurate)

This preserves accuracy where it matters most (final residual stress state) while exploiting explicit speed during the computationally expensive deposition phase.

GPU speedup benchmarks

Solver	Problem	Speedup
ORNL GPU explicit	Auto panel brazing, 2M DOF	126x vs 8 CPU Abaqus
ORNL GPU explicit	Nuclear pipe DMW, 1.3M nodes	~2000x vs estimated implicit
Northwestern GPU explicit	AM thermal, various sizes	100—150x vs single CPU
Ikushima GPU explicit	Multi-pass weld	12x vs CPU implicit

MOOSE explicit with time scaling on HPC: viable approach

The ORNL GPU speedups are impressive, but MOOSE explicit with time scaling on a multi-core HPC is also viable. The key factors:

• Time scaling (SF ~26,000) reduces timestep count from ~10^10 to ~10^6 for a few hours of WAAM process time
• Explicit per-step cost is low: no Newton iterations, no linear solve, just element loop + node assembly
• Trivial parallelism: Element computations are independent; only ghost node communication needed. Scales well with MPI.
• 192 CPU threads on HPC compensates for CPU-per-step being slower than GPU-per-step

Estimated performance: With SF=26,000 and 192 MPI ranks, MOOSE explicit should be competitive with the ORNL GPU code for moderate- size WAAM models (~500K—2M elements). The per-step cost is higher on CPU, but the thread count and absence of GPU porting effort make this the practical choice.

Advantages of staying in MOOSE:

• Single codebase for physics development, data generation, and surrogate deployment
• No need to port to CUDA or maintain separate GPU code
• MOOSE’s ADKernel infrastructure works with explicit integrators
• LibTorch surrogate integration already available in MOOSE
• In-situ training loop stays within the same MPI job

Recommended workflow:

1. Develop and validate JMAK/KM/TRIP physics in MOOSE with implicit solver (easier debugging, no scaling artifacts)
2. Switch to CentralDifference or ExplicitMixedOrder with time scaling (SF=10,000—26,000) for batch data generation on 192 threads
3. Validate time-scaled explicit against implicit baseline (check KE/IE < 5%, plastic strain error < 2%)
4. Generate training dataset: parametric sweep over process parameters
5. Train GNO offline, deploy via MOOSE’s LibTorch infrastructure
6. In-situ training: MOOSE explicit generates data, GNO predicts, uncertainty triggers retraining

Has anyone trained surrogates on time-scaled explicit data?

No published work specifically documents using time-scaled explicit FEM data to train neural operator surrogates. This is a clear research gap and a potential novel contribution of the TMM surrogate project.

The ORNL group mentions building a “digital twin” of WAAM using these fast simulations. The real-time simulation capability makes this approach ideal for generating large training datasets across parametric variations (scan strategies, geometries, process parameters).

Practical recommendations

• Use SF = 10,000—26,000 for WAAM/DED with steel. Calibrate by running a short segment at multiple SFs and comparing plastic strain.
• Monitor KE/IE ratio (<5%) as a runtime check.
• Hybrid explicit/implicit: Explicit for deposition, implicit for cooling and fixture release.
• Batch generation: A single A100/H100 GPU can produce 10—50 full 3D AM simulations per week for moderate-size parts (~500K—2M elements).

Research gaps

In-situ training of chained neural operators

No published work combines concurrent in-situ training with chained neural operators. In-situ training exists for single surrogate models. Chained operators exist for multi-physics but are always trained offline.

deal.II + LibTorch for in-situ training

Roth et al. only use LibTorch for adjoint approximation, not for surrogate training during time-stepping. No framework exists for training thermal surrogates concurrently with the FEM thermal solve.

Active learning for AM thermal simulation

Active learning has been applied to structural FEM but not to transient thermal problems with neural operator surrogates. No work addresses WAAM-specific challenges (moving heat sources, phase changes, layer deposition).

Error-aware triggering for in-situ training

Existing in-situ frameworks use fixed schedules or simple thresholds. No work uses surrogate uncertainty estimates (ensembles, conformal bounds) to decide when to retrain during simulation.

Chained operator training with error budget allocation

When a thermal surrogate feeds a mechanical surrogate (TMM pipeline), there is no framework for allocating error budgets across the chain or training both operators concurrently with coupled loss functions.

Connection to TMM surrogate project

See TMM Surrogates. In-situ training is the core innovation target: training neural operators concurrently with the MALAMUTE FEM solver during WAAM simulation.

The recommended pipeline:

1. MALAMUTE solves thermal FEM
2. At each timestep, GNO predicts thermal field (surrogate)
3. Compare surrogate vs FEM; compute uncertainty
4. When uncertainty exceeds threshold, add FEM data to training set
5. Retrain GNO in background (ring buffer, gradient accumulation)
6. Use trained GNO for subsequent timesteps where FEM is not needed

See also:

• Neural Operators for PDE Surrogates for the surrogate architecture
• deal.II + LibTorch Integration for the implementation framework