Sundar's Notes

Reduced-Order Modeling with ML Surrogates

5 min read

Reduced-Order Modeling with ML Surrogates

State of the art in reduced-order modeling combined with machine learning for FEM surrogates: POD, autoencoders, POD-Galerkin with neural closure, and dynamic mode decomposition.

Proper Orthogonal Decomposition for thermal fields

POD extracts dominant spatial modes from FEM snapshot data. The solution is approximated as:

T(x,t) approx sum_{i=1}^{r} a_i(t) * phi_i(x)

where phi_i are the POD modes and a_i are time-dependent coefficients.

Key papers for AM thermal problems

Balaji, Raghavan & Sundararaghavan (2022): POD-based ROM for thermal simulation of LPBF, achieving 100x speedup with <2% error.

Citation: Balaji, A., Raghavan, N. & Sundararaghavan, V. (2022). “Reduced order modeling of thermal fields in laser powder bed fusion using proper orthogonal decomposition.” Additive Manufacturing.

Zhang, Horvath, Stickels, DeWitt & Barton (2022): POD for part-scale AM thermal simulation with adaptive basis enrichment.

Citation: Zhang, X. et al. (2022). “Adaptive POD-based reduced order modeling for part-scale additive manufacturing thermal simulation.” CMAME.

Limitations of standard POD for AM

  • Linear subspace assumption fails for nonlinear thermal fields with moving heat sources
  • Basis must be enriched when new thermal regimes are encountered (new layers, different scan patterns)
  • Does not generalize across geometries without recomputing snapshots

Autoencoder-based dimensionality reduction

Autoencoders learn nonlinear manifolds, overcoming the linear subspace limitation of POD.

Key papers

Lee & Carlberg (2020): Demonstrated autoencoders outperform PCA/POD for nonlinear manifolds in transient problems. Critical paper.

Citation: Lee, Y. & Carlberg, K.T. (2020). “Model reduction of nonlinear manifolds using convolutional autoencoders.” J. Computational Physics, 404.

Kim, Choi & Yoo (2023): Variational autoencoder for AM thermal field compression, achieving better reconstruction than POD with fewer latent dimensions.

Citation: Kim, J., Choi, S. & Yoo, J. (2023). “Variational autoencoder-based reduced order model for additive manufacturing thermal simulation.” Additive Manufacturing.

Architecture for AM thermal fields

  1. Encoder: Convolutional network maps T(x) to latent vector z
  2. Decoder: Transposed convolution maps z back to T(x)
  3. Latent dynamics: RNN or neural ODE evolves z(t) forward in time
  4. Training: End-to-end on FEM snapshot data

POD-Galerkin projection with neural network closure

POD-Galerkin projects the PDE onto the POD subspace, yielding a low-dimensional ODE system. Neural networks correct the closure error from truncation.

Key papers

Fresca, Manzoni, Quarteroni & Dedè (2024): POD-Galerkin with neural network closure for nonlinear parametrized PDEs.

Citation: Fresca, S., Manzoni, A., Quarteroni, A. & Dedè, L. (2024). “POD-Galerkin reduced order models with neural network closure for nonlinear parametrized PDEs.” CMAME.

Watter, Springenberg, Hafez, Cremers & Riedmiller (2015): Early work on embedding physics in latent space dynamics.

Citation: Watter, M. et al. (2015). “Embed to control: A locally linear latent dynamics model for control from raw images.” NeurIPS.

Advantages over pure data-driven ROM

  • Physics structure preserved (Galerkin projection ensures energy conservation in the reduced space)
  • Neural closure only learns the truncation error, not the full dynamics
  • Better extrapolation than pure data-driven approaches
  • Smaller training data requirements

Dynamic Mode Decomposition

DMD extracts linear dynamical models from snapshot data. Extended DMD (EDMD) handles nonlinear dynamics via kernel methods or neural networks.

Key papers

Proctor, Brunton & Kutz (2016): DMD with control inputs for dynamical systems.

Citation: Proctor, J.L., Brunton, S.L. & Kutz, J.N. (2016). “Dynamic mode decomposition with control.” SIAM J. Applied Dynamical Systems, 5(1).

Lusch, Brunton & Kutz (2019): Deep learning for universal linear embeddings of nonlinear dynamics.

Citation: Lusch, B., Brunton, S.L. & Kutz, J.N. (2019). “Deep learning for universal linear embeddings of nonlinear dynamics.” Nature Communications, 10.

Applicability to AM thermal problems

DMD is limited for AM because:

  • Thermal fields are strongly nonlinear (phase change, moving source)
  • Standard DMD assumes linear dynamics
  • EDMD with neural kernels can help but loses interpretability
  • Better suited for post-transient analysis or linearized regimes

Decision framework: ROM vs neural operators

CriterionROM (POD/AE)Neural Operator
GeometryFixed or parametricArbitrary
ParametersFew (<10)Many
Solution manifoldSmooth, low-rankComplex, nonlinear
Training dataModerate (100s snapshots)Large (1000s simulations)
Inference speedVery fast (ODE solve)Fast (forward pass)
GeneralizationLimitedBetter
Physics structurePreserved (Galerkin)Learned
ImplementationSimplerMore complex

When to use ROM

  • Parametric studies on fixed geometry (varying power, speed, layer count)
  • Real-time control where ODE solve in latent space is fast enough
  • When physics structure must be preserved (energy conservation)
  • Limited training data available

When to use neural operators

  • Generalization across geometries and scan patterns
  • Complex solution manifolds (phase change, evolving domain)
  • Many input parameters (full tool path as input)
  • Abundant training data from FEM

Hybrid approach for TMM

The recommended approach combines ROM and neural operators:

  1. POD for thermal field compression: Use POD modes as the trunk basis for DeepONet, reducing the output dimensionality while preserving physics structure
  2. Neural operator for the operator map: GNO or DeepONet learns the mapping from process parameters to POD coefficients
  3. Neural closure for dynamics: Small NN corrects the POD truncation error in the latent ODE system
  4. Local kinetics in full space: JMAK/KM ODEs solved at each material point using decoded temperature histories

This gets the speed of ROM with the generalization of neural operators.

Research gaps

  1. No POD-based ROM demonstrated for full TMM coupling in AM
  2. Autoencoder ROM not tested for evolving domain (layer deposition)
  3. Neural closure models not validated for long-horizon AM thermal cycles
  4. No multi-fidelity ROM combining coarse and fine FEM data
  5. DMD not extended to handle phase change nonlinearities in AM

Connection to TMM surrogate project

See TMM Surrogates. The hybrid POD + neural operator approach provides the best balance of speed, accuracy, and physics preservation for the thermal stage.

See also:

Graph View

Backlinks