Fourier Neural Operator

Traditional Partial Differential Equations (PDE) solvers, such as finite element methods (FEM) and finite difference methods (FDM), depend on discretizing the space into a mesh. This imposes a trade-off on resolution: coarse grids offer speed but less accuracy, while fine grids provide accuracy but are slow. Solving complex systems of PDEs, which model real-life physical phenomena, often requires very fine discretization. This task is challenging and time-consuming for traditional solvers.

However, data-driven methods can directly learn the trajectory of the family of equations from the data. Consequently, learning-based methods can be significantly faster than traditional solvers..

A recent line of work has proposed the learning of mesh-free, infinite-dimensional operators using neural networks.¹ The neural operator addresses the mesh-dependent nature of finite-dimensional operator methods. It accomplishes this by producing a single set of network parameters applicable to various discretizations. Importantly, the neural operator does not require knowledge of the underlying PDE, only data.

The Fourier Neural Operator (FNO) is a unique, data-driven deep learning architecture. It can learn mappings between infinite-dimensional spaces of functions. Its key feature, known as spectral convolutions, involves operations that function in the Fourier space with an integral kernel.

Operator learning can be taken as an image-to-image problem. The Fourier layer can be viewed as a substitute for the convolution layer.²

The inputs and outputs of PDEs are continuous functions. It is more efficient to represent them in Fourier space and do global convolution. Then, for the Fourier layer, we can use the fast Fourier transform (FFT). Activation functions will then be applied on the spatial domain. They help to recover the Higher frequency modes and non-periodic boundary which are left out in the Fourier layers. Therefore it's necessary to the Fourier transform and its inverse at each layer.²

References:

1. Neural Ordinary Differential Equations (article) (opens in a new tab)
2. Fourier Neural Operator - blogpost by co-creator, Zongyi Li (opens in a new tab)