Federated learning (FL) has been becoming a popular interdisciplinary research area in both applied mathematics and information sciences. Mathematically, FL aims to collaboratively optimize aggregate objective functions over distributed datasets while satisfying a variety of privacy and system constraints. Different from conventional distributed optimization methods, FL needs to address several specific issues (e.g. non-i.i.d. data and differential private noises), which pose a set of new challenges in the problem formulation, algorithm design, and convergence analysis. In this paper, we will systematically review existing FL optimization research including their assumptions, formulations, methods, and theoretical results. Potential future directions are also discussed.

$ℓ_1$ based sparse regularization plays a central role in compressive sensing and image processing. In this paper, we propose $ℓ_1$DecNet, as an unfolded network derived from a variational decomposition model, which incorporates $ℓ_1$ related sparse regularizations and is solved by a non-standard scaled alternating direction method of multipliers. $ℓ_1$DecNet effectively separates a spatially sparse feature and a learned spatially dense feature from an input image, and thus helps the subsequent spatially sparse feature related operations. Based on this, we develop $ℓ_1$DecNet+, a learnable architecture framework consisting of our $ℓ_1$DecNet and a segmentation module which operates over extracted sparse features instead of original images. This architecture combines well the benefits of mathematical modeling and data-driven approaches. To our best knowledge, this is the first study to incorporate mathematical image prior into feature extraction in segmentation network structures. Moreover, our $ℓ_1$DecNet+ framework can be easily extended to 3D case. We evaluate the effectiveness of $ℓ_1$DecNet+ on two commonly encountered sparse segmentation tasks: retinal vessel segmentation in medical image processing and pavement crack detection in industrial abnormality identification. Experimental results on different datasets demonstrate that, our $ℓ_1$DecNet+ architecture with various lightweight segmentation modules can achieve equal or better performance than their enlarged versions respectively. This leads to especially practical advantages on resource-limited devices.

Previous research has shown that fully-connected neural networks with small initialization and gradient-based training methods exhibit a phenomenon known as condensation [T. Luo et al., J. Mach. Learn. Res., 22(1), 2021]. Condensation is a phenomenon wherein the weight vectors of neural networks concentrate on isolated orientations during the training process, and it is a feature in the non-linear learning process that enables neural networks to possess better generalization abilities. However, the impact of neural network architecture on this phenomenon remains a topic of inquiry. In this study, we turn our focus towards convolutional neural networks (CNNs) to investigate how their structural characteristics, in contrast to fully-connected networks, exert influence on the condensation phenomenon. We first demonstrate in theory that under gradient descent and the small initialization scheme, the convolutional kernels of a two-layer CNN condense towards a specific direction determined by the training samples within a given time period. Subsequently, we conduct systematic empirical investigations to substantiate our theory. Moreover, our empirical study showcases the persistence of condensation under broader conditions than those imposed in our theory. These insights collectively contribute to advancing our comprehension of the non-linear training dynamics inherent in CNNs.

In this work, we consider the dynamic framework of the ocean-atmosphere (O-A) coupled model with physical boundary conditions at the ocean-atmosphere interface, and this coupled model can be viewed as atmosphere general circulation model coupled with ocean general circulation model. As the initial data and boundary conditions are assumed to meet certain assumptions, by taking advantage of energy estimates method and compactness arguments, we addressed the existence and stability of global weak solutions, the existence and uniqueness of global strong solution to the O-A coupled model.

The Blahut-Arimoto (BA) algorithm has played a fundamental role in the numerical computation of rate-distortion (RD) functions. This algorithm possesses a desirable monotonic convergence property by alternatively minimizing its Lagrangian with a fixed multiplier. In this paper, we propose a novel modification of the BA algorithm, wherein the multiplier is updated through a one-dimensional root-finding step using a monotonic univariate function, efficiently implemented by Newton’s method in each iteration. Consequently, the modified algorithm directly computes the RD function for a given target distortion, without exploring the entire RD curve as in the original BA algorithm. Moreover, this modification presents a versatile framework, applicable to a wide range of problems, including the computation of distortion-rate (DR) functions. Theoretical analysis shows that the outputs of the modified algorithms still converge to the solutions of the RD and DR functions with rate $\mathcal{O}(1/n),$ where $n$ is the number of iterations. Additionally, these algorithms provide $ε$-approximation solutions with $\mathcal{O}((MN{\rm {\rm log}}N/ε)(1+{\rm log}|{\rm log}ε|))$ arithmetic operations, where $M,$ $N$ are the sizes of source and reproduced alphabets respectively. Numerical experiments demonstrate that the modified algorithms exhibit significant acceleration compared with the original BA algorithms and showcase commendable performance across classical source distributions such as discretized Gaussian, Laplacian and uniform sources.

We propose a novel two-stage model for image denoising. With the group sparse representations over local singular value decomposition stage (locally), one can remove the noise effectively and keep the texture well. The final denoising by a first-order variational model stage (globally) can help us to remove artifacts, maintain the image contrast, suppress the staircase effect, while preserving sharp edges. The existence and uniqueness of global minimizers of the low-rank problem based on group sparse representations are analyzed and proved. Alternating direction method of multipliers is utilized to minimize the associated functional, and the convergence analysis of the proposed optimization algorithm are established. Numerical experiments are conducted to showcase the distinctive features of our method and to provide a comparison with other image denoising techniques.

In this paper, we study numerical approximations of the ground states in finite temperature density functional theory. We formulate the problem with respect to the density matrices and justify the convergence of the finite dimensional approximations. Moreover, we provide an optimal a priori error estimate under some mild assumptions and present some numerical experiments to support the theory.