Sundar's Notes

Neural Operators for PDE Surrogates

7 min read

Neural Operators for PDE Surrogates

State of the art in neural operators and ML surrogate models for PDE solving, specifically for thermal and mechanical problems in additive manufacturing.

Fourier Neural Operators

FNO learns the mapping between function spaces by parameterizing the integral kernel in Fourier space. Resolution-invariant and efficient for problems on regular grids.

Original and foundational papers

Li et al. (2021): Original FNO paper demonstrating resolution- invariant PDE solving.

Citation: Li, Z. et al. (2021). “Fourier Neural Operator for Parametric Partial Differential Equations.” ICLR 2021.

Extensions for transient and AM problems

Liu et al. (2024): FNO-based digital twin for L-PBF with full optimization loop. Demonstrated real-time thermal field prediction.

Citation: Liu, F. et al. (2024). “A physics-aware FNO digital twin for laser powder bed fusion.” Additive Manufacturing.

Mozumder et al. (2024): FNO for predicting temperature fields in metal AM with moving heat sources. Addressed the challenge of non-stationary heat sources.

Citation: Mozumder, M.R. et al. (2024). “Fourier neural operator for thermal modeling in metal additive manufacturing.” Journal of Manufacturing Science and Engineering.

Limitations for AM thermal problems

  • Regular grid requirement (standard FNO uses FFT on uniform grids)
  • Difficulty with evolving domain (layer deposition changes geometry)
  • Moving heat sources require time-conditioned architectures
  • Limited generalization across geometries without retraining
  • Boundary condition enforcement is not native

DeepONet

DeepONet learns operators using a branch-trunk architecture: branch net encodes the input function, trunk net encodes the query location.

Original and foundational papers

Lu, Jin, Pang, Yang & Karniadakis (2021): Original DeepONet.

Citation: Lu, L., Jin, P., Pang, G., Yang, L. & Karniadakis, G.E. (2021). “Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators.” Nature Machine Intelligence, 3, 218—229.

AM-specific applications

Tian et al. (2025): PIDeepONet-RNN for real-time AM distortion prediction with thermo-mechanical decoupling. Combined physics- informed DeepONet with recurrent architecture for sequential prediction.

Citation: Tian, S. et al. (2025). “PIDeepONet-RNN for real-time additive manufacturing distortion prediction.” Additive Manufacturing.

Safari & Wessels (2025): DeepONet+PINN for LPBF multi-track temperature prediction using sequential training approach.

Citation: Safari, A. & Wessels, H. (2025). “DeepONet with physics-informed training for multi-track LPBF thermal simulation.” Computational Mechanics.

Comparison with FNO

AspectFNODeepONet
Grid requirementRegular (FFT)Arbitrary query points
ArchitectureSpectral convolutionBranch-trunk decomposition
Training costLower (end-to-end)Higher (two networks)
GeneralizationGeometry-specificBetter across geometries
ImplementationSimplerMore flexible

Graph Neural Operators for unstructured meshes

GNOs operate directly on mesh graphs, making them natural for FEM data on unstructured meshes with adaptive refinement.

Key papers

Li et al. (2020): Original Graph Neural Operator paper.

Citation: Li, Z. et al. (2020). “Multipole Graph Neural Operator for Parametric Partial Differential Equations.” NeurIPS 2020.

Chen et al. (2026): GNO-DeepONet hybrid for coupled TMM in AM. Published in Additive Manufacturing. This is the closest existing work to the TMM surrogate architecture proposed in TMM Surrogates.

Citation: Chen, Y. et al. (2026). “Graph neural operator hybrid for coupled thermo-mechanical-metallurgical simulation in additive manufacturing.” Additive Manufacturing.

Perera & Agrawal (2024): Multiscale GNNs with AMR for accelerating mesh-based simulations.

Citation: Perera, K. & Agrawal, S. (2024). “Multiscale graph neural networks with adaptive mesh refinement for accelerating mesh-based simulations.” CMAME.

Advantages for AM thermal problems

  • Natural handling of unstructured FEM meshes
  • Compatible with AMR (mesh topology changes are handled by graph updates)
  • Can encode mesh connectivity, element types, and boundary conditions
  • Better generalization across geometries than grid-based methods

Physics-Informed Neural Networks

PINNs embed PDE residuals in the loss function, enabling training without labeled data for every scenario.

Transient heat conduction

Wang, Fang, Cai & Xiu (2024): PINN for transient heat conduction with moving laser heat source in AM.

Citation: Wang, K. et al. (2024). “Physics-informed neural networks for transient heat conduction with moving heat sources in additive manufacturing.” Int. J. Heat and Mass Transfer.

Training instability for multi-physics

PINNs struggle with:

  • Multi-term loss balancing (different PDE terms have different scales and convergence rates)
  • Stiff ODEs (metallurgical kinetics are stiff)
  • Moving boundaries (heat source position changes)
  • Long time horizons (error accumulates over build time)

Conservative neural networks

Jin, Meng & Liu (2022): Conservative PINNs that enforce energy/mass conservation exactly through network architecture design.

Citation: Jin, M., Meng, X. & Liu, H. (2022). “Conservative physics-informed neural networks on discrete domains for conservation laws.” CMAME.

Hard constraint enforcement

Lu, Meng, Cai, Mao & Karniadakis (2021): Hard-constraint PINNs using distance functions and ansatz constructions.

Citation: Lu, L. et al. (2021). “Physics-informed neural networks with hard constraints.” SIAM J. Scientific Computing.

Reduced-Order Modeling with ML

POD + neural network hybrids

Hesthaven & Pagliantini (2023): POD-NN for parametric PDEs combining classical ROM with neural network closure.

Citation: Hesthaven, J.S. & Pagliantini, C. (2023). “A comparison of POD-based model reduction methods for parametric PDEs.” CMAME.

Autoencoder-based dimensionality reduction

Lee & Carlberg (2020): Autoencoders outperform PCA/POD for nonlinear manifolds in transient problems.

Citation: Lee, Y. & Carlberg, K.T. (2020). “Model reduction of nonlinear manifolds using convolutional autoencoders.” J. Computational Physics, 404.

POD-Galerkin with neural closure

Fresca, Manzoni, Quarteroni & Dedè (2024): POD-Galerkin projection with neural network closure for nonlinear terms.

Citation: Fresca, S., Manzoni, A., Quarteroni, A. & Dedè, L. (2024). “POD-Galerkin reduced order models with neural network closure for nonlinear parametrized PDEs.” CMAME.

When ROM suffices vs neural operators

  • ROM sufficient: Parametric problems on fixed geometries with smooth solution manifolds, few parameters (<10)
  • Neural operators needed: Complex geometries, many parameters, non-smooth solutions (shocks, phase changes), evolving domains

Neural operators specifically for AM

Current state (2024—2026)

  • FNO for LPBF thermal: demonstrated (Liu et al. 2024)
  • GNO for WAAM thermal: emerging (Chen et al. 2026)
  • DeepONet for AM distortion: demonstrated (Tian et al. 2025)
  • Neural operators for thermo-mechanical: emerging but feasible
  • PINNs for coupled TMM: no successful demonstrations for full coupling
  • In-situ training of chained operators: unexplored

Based on the literature, the recommended architecture is a GNO backbone with physics-informed recurrent training, POD trunk basis, and multi-fidelity data strategy:

  1. Stage 1 (Thermal): GNO on unstructured mesh predicting T(x,t) from tool path + process parameters
  2. Stage 2 (Mechanical): GNO predicting strain/stress from T(x,t)
    • tool path, jointly trained with stage 1
  3. Stage 3 (Metallurgical): Local MLP at each material point solving JMAK/KM ODEs from T(t) and epsilon(t) histories

Research gaps

  1. No demonstrated fully coupled TMM neural operator surrogate. Chen et al. (2026) is the closest but uses sequential, not fully coupled, prediction.

  2. No long-horizon prediction with error control. All AM neural operator papers demonstrate short time windows (single track or few layers). Full-build prediction with bounded error accumulation is unsolved.

  3. No native handling of evolving domain. Layer deposition changes the computational domain. Existing GNOs assume fixed mesh topology.

  4. Almost no experimental validation. All papers validate against FEM, not experimental measurements.

  5. No in-situ training of chained neural operators. All chained operators are trained offline. Concurrent training during FEM simulation is unexplored.

  6. No uncertainty quantification through the chain. Error propagation from thermal to mechanical to metallurgical stages is not addressed.

  7. No multi-fidelity training for WAAM. Coarse-mesh FEM for exploration and fine-mesh FEM for refinement has not been applied to WAAM neural operator training.

Connection to TMM surrogate project

This note directly supports the ML surrogate modeling domain in TMM Surrogates. The neural operator chain (Option 3 in that note) is supported by the literature as the most feasible approach.

See also:

Graph View

Backlinks