• Abstract

Federated learning (FL) has been becoming a popular interdisciplinary research area in both applied mathematics and information sciences. Mathematically, FL aims to collaboratively optimize aggregate objective functions over distributed datasets while satisfying a variety of privacy and system constraints. Different from conventional distributed optimization methods, FL needs to address several specific issues (e.g. non-i.i.d. data and differential private noises), which pose a set of new challenges in the problem formulation, algorithm design, and convergence analysis. In this paper, we will systematically review existing FL optimization research including their assumptions, formulations, methods, and theoretical results. Potential future directions are also discussed.

• General Summary

• Abstract

$ℓ_1$ based sparse regularization plays a central role in compressive sensing and image processing. In this paper, we propose $ℓ_1$DecNet, as an unfolded network derived from a variational decomposition model, which incorporates $ℓ_1$ related sparse regularizations and is solved by a non-standard scaled alternating direction method of multipliers. $ℓ_1$DecNet effectively separates a spatially sparse feature and a learned spatially dense feature from an input image, and thus helps the subsequent spatially sparse feature related operations. Based on this, we develop $ℓ_1$DecNet+, a learnable architecture framework consisting of our $ℓ_1$DecNet and a segmentation module which operates over extracted sparse features instead of original images. This architecture combines well the benefits of mathematical modeling and data-driven approaches. To our best knowledge, this is the first study to incorporate mathematical image prior into feature extraction in segmentation network structures. Moreover, our $ℓ_1$DecNet+ framework can be easily extended to 3D case. We evaluate the effectiveness of $ℓ_1$DecNet+ on two commonly encountered sparse segmentation tasks: retinal vessel segmentation in medical image processing and pavement crack detection in industrial abnormality identification. Experimental results on different datasets demonstrate that, our $ℓ_1$DecNet+ architecture with various lightweight segmentation modules can achieve equal or better performance than their enlarged versions respectively. This leads to especially practical advantages on resource-limited devices.

• General Summary

• Abstract

Previous research has shown that fully-connected neural networks with small initialization and gradient-based training methods exhibit a phenomenon known as condensation [T. Luo et al., J. Mach. Learn. Res., 22(1), 2021]. Condensation is a phenomenon wherein the weight vectors of neural networks concentrate on isolated orientations during the training process, and it is a feature in the non-linear learning process that enables neural networks to possess better generalization abilities. However, the impact of neural network architecture on this phenomenon remains a topic of inquiry. In this study, we turn our focus towards convolutional neural networks (CNNs) to investigate how their structural characteristics, in contrast to fully-connected networks, exert influence on the condensation phenomenon. We first demonstrate in theory that under gradient descent and the small initialization scheme, the convolutional kernels of a two-layer CNN condense towards a specific direction determined by the training samples within a given time period. Subsequently, we conduct systematic empirical investigations to substantiate our theory. Moreover, our empirical study showcases the persistence of condensation under broader conditions than those imposed in our theory. These insights collectively contribute to advancing our comprehension of the non-linear training dynamics inherent in CNNs.

• General Summary

• Abstract

In this work, we consider the dynamic framework of the ocean-atmosphere (O-A) coupled model with physical boundary conditions at the ocean-atmosphere interface, and this coupled model can be viewed as atmosphere general circulation model coupled with ocean general circulation model. As the initial data and boundary conditions are assumed to meet certain assumptions, by taking advantage of energy estimates method and compactness arguments, we addressed the existence and stability of global weak solutions, the existence and uniqueness of global strong solution to the O-A coupled model.

• General Summary

• Abstract

The Blahut-Arimoto (BA) algorithm has played a fundamental role in the numerical computation of rate-distortion (RD) functions. This algorithm possesses a desirable monotonic convergence property by alternatively minimizing its Lagrangian with a fixed multiplier. In this paper, we propose a novel modification of the BA algorithm, wherein the multiplier is updated through a one-dimensional root-finding step using a monotonic univariate function, efficiently implemented by Newton’s method in each iteration. Consequently, the modified algorithm directly computes the RD function for a given target distortion, without exploring the entire RD curve as in the original BA algorithm. Moreover, this modification presents a versatile framework, applicable to a wide range of problems, including the computation of distortion-rate (DR) functions. Theoretical analysis shows that the outputs of the modified algorithms still converge to the solutions of the RD and DR functions with rate $\mathcal{O}(1/n),$ where $n$ is the number of iterations. Additionally, these algorithms provide $ε$-approximation solutions with $\mathcal{O}((MN{\rm log}N/ε)(1+{\rm log}|{\rm log}ε|))$ arithmetic operations, where $M,$ $N$ are the sizes of source and reproduced alphabets respectively. Numerical experiments demonstrate that the modified algorithms exhibit significant acceleration compared with the original BA algorithms and showcase commendable performance across classical source distributions such as discretized Gaussian, Laplacian and uniform sources.

• General Summary

• Abstract

We propose a novel two-stage model for image denoising. With the group sparse representations over local singular value decomposition stage (locally), one can remove the noise effectively and keep the texture well. The final denoising by a first-order variational model stage (globally) can help us to remove artifacts, maintain the image contrast, suppress the staircase effect, while preserving sharp edges. The existence and uniqueness of global minimizers of the low-rank problem based on group sparse representations are analyzed and proved. Alternating direction method of multipliers is utilized to minimize the associated functional, and the convergence analysis of the proposed optimization algorithm are established. Numerical experiments are conducted to showcase the distinctive features of our method and to provide a comparison with other image denoising techniques.

• General Summary

• Abstract

In this paper, we develop a set of efficient methods to compute stationary states of the spherical Landau-Brazovskii (LB) model in a discretization-then-optimization way. First, we discretize the spherical LB energy functional into a finite-dimensional energy function by the spherical harmonic expansion. Then five optimization methods are developed to compute stationary states of the discretized energy function, including the accelerated adaptive Bregman proximal gradient, Nesterov, adaptive Nesterov, adaptive nonlinear conjugate gradient and adaptive gradient descent methods. To speed up the convergence, we propose a principal mode analysis (PMA) method to estimate good initial configurations and sphere radius. The PMA method also reveals the relationship between the optimal sphere radius and the dominant degree of spherical harmonics. Numerical experiments show that our approaches significantly reduce the number of iterations and the computational time.

• General Summary

• Abstract

In this paper, we study numerical approximations of the ground states in finite temperature density functional theory. We formulate the problem with respect to the density matrices and justify the convergence of the finite dimensional approximations. Moreover, we provide an optimal a priori error estimate under some mild assumptions and present some numerical experiments to support the theory.

• General Summary

• Abstract

Using newly developed $\mathbf{H}$(${\rm curl}^2$) conforming elements, we solve the Maxwell’s transmission eigenvalue problem. Both real and complex eigenvalues are considered. Based on the fixed-point weak formulation with reasonable assumptions, the optimal error estimates for numerical eigenvalues and eigenfunctions (in the $\mathbf{H}$(${\rm curl}^2$))-norm and $\mathbf{H}$(${\rm curl}$)-semi-norm) are established. Numerical experiments are performed to verify the theoretical assumptions and confirm our theoretical analysis.

• General Summary

• Abstract

In this paper, we present the Gauss’s law-preserving spectral methods and their efficient solution algorithms for curl-curl source and eigenvalue problems arising from Maxwell’s equations. Arbitrary order $\boldsymbol{H}(curl)$-conforming spectral basis functions in two and three dimensions are firstly proposed using compact combination of Legendre polynomials. A mixed formulation involving a Lagrange multiplier is then adopted to preserve the Gauss’s law in the weak sense. To overcome the bottleneck of computational efficiency caused by the saddle-point nature of the mixed scheme, we present highly efficient algorithms based on reordering and decoupling of the linear system and numerical eigen-decomposition of 1D mass matrix. The proposed solution algorithms are direct methods requiring only several matrix-matrix or matrix-tensor products of $N$-by-$N$ matrices, where $N$ is the highest polynomial order in each direction. Compared with other direct methods, the computational complexities are reduced from $\mathcal{O}(N^6)$ and $\mathcal{O}(N^9)$ to $\mathcal{O}(N^{log_2 7})$ and $O(N^{1+log^2 7})$ with small and constant pre-factors for 2D and 3D cases, respectively. Moreover, these algorithms strictly obey the Helmholtz-Hodge decomposition, thus totally eliminate the spurious eigen-modes of non-physical zero eigenvalues for convex domains. Ample numerical examples for solving Maxwell’s source and eigenvalue problems are presented to demonstrate the accuracy and efficiency of the proposed methods.

• General Summary

• Abstract

A two-layer dual strategy is proposed in this work to construct a new family of high-order finite volume element (FVE-2L) schemes that can avoid main common drawbacks of the existing high-order finite volume element (FVE) schemes. The existing high-order FVE schemes are complicated to construct since the number of the dual elements in each primary element used in their construction increases with a rate $\mathcal{O}((k+1)^2 ),$ where k is the order of the scheme. Moreover, all k-th-order FVE schemes require a higher regularity $H^{k+2}$ than the approximation theory for the $L^2$ theory. Furthermore, all FVE schemes lose local conservation properties over boundary dual elements when dealing with Dirichlet boundary conditions. The proposed FVE-2L schemes has a much simpler construction since they have a fixed number (four) of dual elements in each primary element. They also reduce the regularity requirement for the $L^2$ theory to $H^{k+1}$ and preserve the local conservation law on all dual elements of the second dual layer for both flux and equation forms. Their stability and $H^1$ and $L^2$ convergence are proved. Numerical results are presented to illustrate the convergence and conservation properties of the FVE-2L schemes. Moreover, the condition number of the stiffness matrix of the FVE-2L schemes for the Laplacian operator is shown to have the same growth rate as those for the existing FVE and finite element schemes.

• General Summary