Image inverse problem aims to reconstruct or restore high-quality images from observed samples or degraded images, with wide applications in imaging sciences. The traditional methods rely on mathematical models to invert the process of image sensing or degradation. But these methods require good design of image prior or regularizer that is hard to be hand-crafted. In recent years, deep learning has been introduced to image inverse problems by learning to invert image sensing or degradation process. In this paper, we will review a new trend of methods for image inverse problem that combines the imaging/degradation model with deep learning approach. These methods are typically designed by unrolling some optimization algorithms or statistical inference algorithms into deep neural networks. The ideas combining deep learning and models are also emerging in other fields such as PDE, control, etc. We will also summarize and present perspectives along this research direction.

We develop Banach spaces for ReLU neural networks of finite depth $L$ and infinite width. The spaces contain all finite fully connected $L$-layer networks and their $L^2$-limiting objects under bounds on the natural path-norm. Under this norm, the unit ball in the space for $L$-layer networks has low Rademacher complexity and thus favorable generalization properties. Functions in these spaces can be approximated by multi-layer neural networks with dimension-independent convergence rates.
The key to this work is a new way of representing functions in some form of expectations, motivated by multi-layer neural networks. This representation allows us to define a new class of continuous models for machine learning. We show that the gradient flow defined this way is the natural continuous analog of the gradient descent dynamics for the associated multi-layer neural networks. We show that the path-norm increases at most polynomially under this continuous gradient flow dynamics.

In this paper we study the effect of the artificial regularization term for the second order accurate (in time) numerical schemes for the no-slope-selection (NSS) equation of the epitaxial thin film growth model. In particular, we propose and analyze an alternate second order backward differentiation formula (BDF) scheme, with Fourier pseudo-spectral spatial discretization. The surface diffusion term is treated implicitly, while the nonlinear chemical potential is approximated by a second order explicit extrapolation formula. A second order accurate Douglas-Dupont regularization term, in the form of −$A$∆$t$$∆^2_N$($u^{n+1}$−$u^n$), is added in the numerical scheme to justify the energy stability at a theoretical level. Due to an alternate expression of the nonlinear chemical potential terms, such a numerical scheme requires a minimum value of the artificial regularization parameter as A=$\frac{289}{1024}$, much smaller than the other reported artificial parameter values in the existing literature. Such an optimization of the artificial parameter value is expected to reduce the numerical diffusion, and henceforth improve the long time numerical accuracy. Moreover, the optimal rate convergence analysis and error estimate are derived in details, in the $ℓ^∞$(0,$T$;$ℓ^2$)∩$ℓ^2$(0,$T$;$H^2_h$) norm, with the help of a linearized estimate for the nonlinear error terms. Some numerical simulation results are presented to demonstrate the efficiency and accuracy of the alternate second order numerical scheme. The long time simulation results for $ε$ =0.02 (up to $T$ =3×$10^5$) have indicated a logarithm law for the energy decay, as well as the power laws for growth of the surface roughness and the mound width.

Recently a novel family of eigensolvers, called spectral indicator methods (SIMs), was proposed. Given a region on the complex plane, SIMs first compute an indicator by the spectral projection. The indicator is used to test if the region contains eigenvalue(s). Then the region containing eigenvalues(s) is subdivided and tested. The procedure is repeated until the eigenvalues are identified within a specified precision. In this paper, using Cayley transformation and Krylov subspaces, a memory efficient multilevel eigensolver is proposed. The method uses less memory compared with the early versions of SIMs and is particularly suitable to compute many eigenvalues of large sparse (non-Hermitian) matrices. Several examples are presented for demonstration.

There exists a well defined energy for classical phase-field equations under which the dissipation law is satisfied, i.e., the energy is non-increasing with respect to time. However, it is not clear how to extend the energy definition to time-fractional phase-field equations so that the corresponding dissipation law is still satisfied. In this work, we will try to settle this problem for phase-field equations with Caputo time-fractional derivative, by defining a nonlocal energy as an averaging of the classical energy with a time-dependent weight function. As the governing equation exhibits both nonlocal and nonlinear behavior, the dissipation analysis is challenging. To deal with this, we propose a new theorem on judging the positive definiteness of a symmetric function, that is derived from a special Cholesky decomposition. Then, the nonlocal energy is proved to be dissipative under a simple restriction of the weight function. Within the same framework, the time fractional derivative of classical energy for time-fractional phase-field models can be proved to be always nonpositive.

We proposed a piecewise quadratic reconstruction method in multiple dimensions, which is in an integrated style, for finite volume schemes to scalar conservation laws. This integrated quadratic reconstruction is parameter-free and applicable on flexible grids. We show that the finite volume schemes with the new reconstruction satisfy a local maximum principle with properly setup on time step length. Numerical examples are presented to show that the proposed scheme attains a third-order accuracy for smooth solutions in both 2D and 3D cases. It is indicated by numerical results that the local maximum principle is helpful to prevent overshoots in numerical solutions.

In most convolution neural networks (CNNs), downsampling hidden layers is adopted for increasing computation efficiency and the receptive field size. Such operation is commonly called pooling. Maximization and averaging over sliding windows ($max/average$ $pooling$), and plain downsampling in the form of strided convolution are popular pooling methods. Since the pooling is a lossy procedure, a motivation of our work is to design a new pooling approach for less lossy in the dimensionality reduction. Inspired by the spectral pooling proposed by Rippel et al. [1], we present the Hartley transform based spectral pooling method. The proposed spectral pooling avoids the use of complex arithmetic for frequency representation, in comparison with Fourier pooling. The new approach preserves more structure features for network's discriminability than max and average pooling. We empirically show the Hartley pooling gives rise to the convergence of training CNNs on MNIST and CIFAR-10 datasets.

A continuous interior penalty finite element method (CIP-FEM) is proposed to solve high-frequency Helmholtz scattering problem by an impenetrable obstacle in two dimensions. To formulate the problem on a bounded domain, a Dirichlet-to-Neumann (DtN) boundary condition is proposed on the outer boundary by truncating the Fourier series of the original DtN mapping into finite terms. Assuming the truncation order $N≥kR$, where $k$ is the wave number and $R$ is the radius of the outer boundary, then the $H^j$-stabilities, $j$ = 0,1,2, are established for both original and dual problems, with explicit and sharp estimates of the upper bounds with respect to $k$. Moreover, we prove that, when $N≥λkR$ for some $λ$>1, the solution to the DtN-truncation problem converges exponentially to the original scattering problem as $N$ increases. Under the condition that $k^3$$h^2$ is sufficiently small, we prove that the pre-asymptotic error estimates for the linear CIP-FEM as well as the linear FEM are $C_1$$kh$+$C_2$$k^3$$h^2$. Numerical experiments are presented to validate the theoretical results.