Recent years have witnessed growing interests in solving partial differential equations by deep neural networks, especially in the high-dimensional
case. Unlike classical numerical methods, such as finite difference method and
finite element method, the enforcement of boundary conditions in deep neural networks is highly nontrivial. One general strategy is to use the penalty
method. In the work, we conduct a comparison study for elliptic problems with
four different boundary conditions, i.e., Dirichlet, Neumann, Robin, and periodic boundary conditions, using two representative methods: deep Galerkin
method and deep Ritz method. In the former, the PDE residual is minimized
in the least-squares sense while the corresponding variational problem is minimized in the latter. Therefore, it is reasonably expected that deep Galerkin
method works better for smooth solutions while deep Ritz method works better for low-regularity solutions. However, by a number of examples, we observe that deep Ritz method can outperform deep Galerkin method with a clear
dependence of dimensionality even for smooth solutions and deep Galerkin
method can also outperform deep Ritz method for low-regularity solutions.
Besides, in some cases, when the boundary condition can be implemented in
an exact manner, we find that such a strategy not only provides a better approximate solution but also facilitates the training process.