RoPINN: Region Optimized Physics-Informed Neural Networks. See Paper or Slides.
This paper proposes and theoretically studies a new training paradigm of PINNs as region optimization and presents RoPINN as a practical algorithm, which can bring the following benefits:
- Better generalization bound: Introducing "region" can theoretically decrease generalization error and provide a general theoretical framework that first reveals the balance between generalization and optimization.
- Efficient practical algorithm: We present RoPINN with a trust region calibration strategy, which can effectively accomplish the region optimization and reduce the gradient estimation error caused by sampling.
- Boost extensive backbones: RoPINN consistently improves various PINN backbones (i.e. PINN, KAN and PINNsFormer) on a wide range of PDEs (19 different tasks) without extra gradient calculation.
Unlike conventional point optimization, our proposed region optimization extends the optimization process of PINNs from isolated points to their continuous neighborhood region.
Figure 1. Comparison between previous methods and RoPINN.
We present RoPINN for PINN training based on Monte Carlo sampling, which can effectively accomplish the region optimization without extra gradient calculation. A trust region calibration strategy is proposed to reduce the gradient estimation error caused by sampling for more trustworthy optimization.
Figure 2. RoPINN algorithm.
- Install Python 3.8 or Python 3.9 and Pytorch 1.13.0. For convenience, execute the following command.
pip install -r requirements.txt- Train and evaluate model. We provide the experiment scripts of all benchmarks under the folder
./scripts/. You can reproduce the experiment results as the following examples:
bash scripts/1d_reaction_point.sh # canonical point optimization
bash scripts/1d_reaction_region.sh # RoPINN: region optimization
bash scripts/1d_wave_point.sh # canonical point optimization
bash scripts/1d_wave_region.sh # RoPINN: region optimization
bash scripts/convection_point.sh # canonical point optimization
bash scripts/convection_region.sh # RoPINN: region optimizationSpecifically, we have included the following PINN models in this repo:
- PINN (Journal of Computational Physics 2019) [Paper]
- FLS - (IEEE Transactions on Artificial Intelligence 2022) [Paper]
- QRes - (SIAM 2021) [Paper]
- KAN - (arXiv 2024) [Paper]
- PINNsFormer - (ICLR 2024) [Paper]
We have experimented with 19 different PDE tasks. See our paper for the full results.
Figure 3. Part of experimental results of RoPINN.
If you find this repo useful, please cite our paper.
@inproceedings{wu2024ropinn,
title={RoPINN: Region Optimized Physics-Informed Neural Networks},
author={Haixu Wu and Huakun Luo and Yuezhou Ma and Jianmin Wang and Mingsheng Long},
booktitle={Advances in Neural Information Processing Systems},
year={2024}
}
If you have any questions or want to use the code, please contact wuhx23@mails.tsinghua.edu.cn.
We appreciate the following GitHub repos a lot for their valuable code base or datasets: