• Abstract

In this work, we investigate the mechanism underlying loss spikes observed during neural network training. When the training enters a region with a lower-loss-as-sharper structure, the training becomes unstable, and the loss exponentially increases once the loss landscape is too sharp, resulting in the rapid ascent of the loss spike. The training stabilizes when it finds a flat region. From a frequency perspective, we explain the rapid descent in loss as being primarily influenced by low-frequency components. We observe a deviation in the first eigendirection, which can be reasonably explained by the frequency principle, as low-frequency information is captured rapidly, leading to the rapid descent. Inspired by our analysis of loss spikes, we revisit the link between the maximum eigenvalue of the loss Hessian ($λ_{{\rm max}}$), flatness and generalization. We suggest that $λ_{{\rm max}}$ is a good measure of sharpness but not a good measure for generalization. Furthermore, we experimentally observe that loss spikes can facilitate condensation, causing input weights to evolve towards the same direction. And our experiments show that there is a correlation (similar trend) between $λ_{{\rm max}}$ and condensation. This observation may provide valuable insights for further theoretical research on the relationship between loss spikes, $λ_{{\rm max}}$, and generalization.

• General Summary

• Abstract

The h-version analysis technique developed in [Banjai et al., SIAM J. Numer. Anal., 55 (2017)] for Trefftz discontinuous Galerkin (DG) discretizations of the second order isotropic wave equation is extended to the time-dependent Maxwell equations in anisotropic media. While the discrete variational formulation and its stability and quasi-optimality are derived parallel to the acoustic wave case, the derivation of error estimates in a mesh-skeleton norm requires new transformation stabilities for the anisotropic case. The error estimates of the approximate solutions with respect to the condition number of the coefficient matrices are proved. Furthermore, we propose the global Trefftz DG method combined with local DG methods to solve the time-dependent nonhomogeneous Maxwell equations. The numerical results verify the validity of the theoretical results, and show that the resulting approximate solutions possess high accuracy.

• General Summary

• Abstract

The weak convergence analysis plays an important role in error estimates for stochastic differential equations, which concerns with the approximation of the probability distribution of solutions. In this paper, we investigate the weak convergence order of a splitting-up method for stochastic differential equations. We first construct a splitting-up approximation, based on which we also set up a splitting-up numerical solution. We prove both of these two approximation methods are of first order of weak convergence with the help of Malliavin calculus. Finally, we present several numerical experiments to illustrate our theoretical analysis.

• General Summary

• Abstract

In this paper, we propose and analyze a multiphysics finite element method for a nonlinear poroelasticity model. To more effectively capture the deformation and diffusion processes, we reformulate the original nonlinear fluid-solid coupling problem into a fluid-fluid coupling problem using a multiphysics approach. We then establish the growth, coercivity, and monotonicity properties of the nonlinear stress-strain relation, derive energy estimates, and use Schaefer’s fixed point theorem to prove the existence and uniqueness of the weak solution. Furthermore, we design a fully discrete time-stepping scheme – multiphysics finite element method with $P_2−P_1−P_1$ elements for spatial variables and the backward Euler method for time variable. To handle nonlinearity, we employ the Newton iterative method, establish discrete energy laws and give the optimal convergence order error estimates. Finally, we show some numerical examples to verify the rationality of theoretical analysis and the proposed method has no “locking phenomenon”.

• General Summary

• Abstract

Numerical simulation of time-periodic problems is a special area of research, since the time periodicity modifies the problem structure, and then it is desirable to use parallel methods to solve such problems. The classical parareal algorithm for time-periodic problems, which is parallel in time, solving an initial value coarse problem, called the periodic parareal algorithm with initial value coarse problem (PP-IC), usually converges very slowly, and even diverges for wave propagation problems. In this paper, we first present a new PP-IC algorithm based on a diagonalization technique proposed recently. In this new algorithm, we approximate the coarse propagator $G$ in the classical PP-IC algorithm with a head-tail coupled condition such that $G$ can be parallelized using diagonalization in time. We analyze the convergence factors of the diagonalization-based PP-IC algorithm for both the linear and nonlinear cases. Then, we further design and analyze a new parallel-in-time algorithm for time-periodic problems by combining the Krylov subspace method with the diagonalization-based PP-IC algorithm to accelerate the convergence. Finally, we also determine an appropriate choice of the parameter $α$ in the head-tail coupling condition, and illustrate our theoretical results with several numerical experiments, both for model problems and the realistic application of Maxwell’s equations.

• General Summary

• Abstract

The rigorous error analysis of a class of serendipity virtual element methods applied to numerically solve semilinear parabolic integro-differential equations on curved domains is the focus of this study. Different from the standard virtual element method, the serendipity virtual element method eliminates all the internal-moment degrees of freedom only under certain conditions of the mesh and the degree of approximation. Consequently, if the interpolation operators are utilized to approximate the nonlinear terms, the implementation of Newton’s iteration algorithm can be simplified. Nonhomogeneous Dirichlet boundary conditions are considered in this paper. The strategy of approximating curved domains with polygonal domains is taken into consideration, and to overcome the issue of suboptimal convergence caused by enforcing Dirichlet boundary conditions strongly, Nitsche-based projection method is employed to impose the boundary conditions weakly. For time discretization, Crank-Nicolson scheme incorporating trapezoidal quadrature rule is adopted. Based on the concrete formulation of Nitsche-based projection method, a Ritz-Volterra projection is introduced and its approximation properties are rigorously analyzed. Building upon these approximation properties, error estimates are derived for the fully discrete scheme. Additionally, the extension of the fully discrete scheme to 3D case is also included. Finally, we present two numerical experiments to corroborate the theoretical findings.

• General Summary

• Abstract

In this paper, we present a generalized Jacobi spectral Galerkin method for fractional Volterra integro-differential equations (FVIDEs). The basis functions of the proposed method are generalized Jacobi functions, which serve as natural basis functions for appropriately designed spectral methods for FVIDEs. We establish a convergence analysis of the generalized Jacobi spectral Galerkin method under reasonable assumptions. Numerical experiments are provided to demonstrate the effectiveness of the proposed method.

• General Summary

• Abstract

In this research article, we present convenient analytical-approximate solutions for fluid flow models known as multi-dimensional Navier-Stokes equations containing time-fractional order by using a relatively new analytical method called modified generalized Mittag-Leffler function method. The Caputo fractional derivative is used to describe fractional mathematical formalism. The approximate solutions for five problems are implemented to demonstrate the validity and accuracy of the proposed method. It is also demonstrated that the solutions obtained from our method when $α = 1$ coincide with the exact solutions, this is displayed by using some 2D and 3D plots for each problem. Moreover, the comparison between our outcomes with given exact solutions and results obtained by other methods in the literature besides absolute error is provided in some tables. Additionally, we offer some plots when $α$ has different values to present the effect of fractional order on the solution of each suggested problem. The numerical simulation presented in this work indicates that the proposed method is efficient, reliable, accurate and easy which has less computational ability to give analytical-approximate solution form. So, this method can be extended to implement on different related problems arising in various areas of innovation and research.

• General Summary

• Abstract

Achieving linear complexity is crucial for demonstrating optimal convergence rates in adaptive refinement. It has been shown that the existing linear complexity local refinement algorithm for ${\rm T}$-splines generally produces more degrees of freedom than the existing greedy refinement, which lacks linear complexity. This paper introduces a novel greedy local refinement algorithm for analysis-suitable ${\rm T}$-splines, which achieves linear complexity and requires fewer control points than existing algorithms with linear complexity. Our approach is based on the observation that confining refinements around each ${\rm T}$-junction to a preestablished feasible region ensures the algorithm’s linear complexity. Building on this constraint, we propose a greedy optimization local refinement algorithm that upholds linear complexity while significantly reducing the degrees of freedom relative to previous linear complexity local refinement methods.

• General Summary

• Abstract

In this paper, we introduce a nonlinear finite volume scheme preserving discrete strong extremum principle (DSEP) for diffusion equations on tetrahedral meshes. In the construction of our nonlinear scheme, the key is to reformulate a discrete normal flux with local extremum principle structure, which is based on a modification of a second order linear scheme. In the construction of existing cell-centered finite volume schemes that maintain the discrete maximum principle, it is required to represent auxiliary unknowns as convex combinations of primary unknowns, which results in strong constraints on the smoothness of the mesh and diffusion coefficient. By contrast, our new scheme avoids this kind of constraints. Moreover, we will prove that there holds the DSEP for any solution of our scheme and there exists at least one solution preserving DSEP for our scheme. Furthermore, a modified Picard iteration with the Anderson acceleration (mP-AA) for solving the nonlinear scheme is proposed, and the nonlinear convergence of the modified Picard iteration is also proved. Finally, numerical examples are presented to show that the new scheme preserves DSEP and obtains second order accuracy, as well as the mP-AA method is effective.

• General Summary

• Abstract

In this paper, our focus is on examining the robustness of the central scheme in two dimensions. Although stability analyses are available in the literature for the scheme’s solution of scalar conservation laws, the associated Courant-Friedrichs-Lewy (CFL) number is often notably small, occasionally degenerating to zero. This challenge is traced back to the initial data reconstruction. The interface value limiter used in the reconstruction proves insufficient to maintain the invariant region of the updated solutions. To overcome this limitation, we introduce the vertex value limiter, resulting in a more suitable CFL number that is half of the one-dimensional value. We present a unified analysis of stability applicable to both types of limiters. This enhanced stability condition enables the utilization of larger time steps, offering improved resolution to the solution and ensuring faster simulations. Our analysis extends to general conservation laws, encompassing scalar problems and nonlinear systems. We support our findings with numerical examples, validating our claims and showcasing the robustness of the enhanced scheme.

• General Summary

• Abstract

Many inverse problems that appear in applications can be modeled as an operator equation. In practice, most of these problems are ill-posed, and computing solutions to such problems in an efficient manner is challenging and has been of greatest interest among researchers in the recent past. While many approaches are developed within infinite-dimensional Hilbert space settings, practical applications often require solutions in finite-dimensional spaces, and we need to discretize the problem. In this manuscript, we study a novel discretization scheme along with a class of regularization techniques for solving linear ill-posed problems and obtain the optimal order error estimates under an a priori parameter choice strategy. We illustrate the computational efficacy of the proposed scheme through numerical examples, and the results demonstrate that the proposed scheme is more economical due to the amount of discrete information needed to solve the problem is significantly lower than the traditional finite-dimensional approach.

• General Summary

• Abstract

In this paper, a three-term conjugate gradient projection method that employs both inertial and relaxed techniques is proposed to find approximate solutions for unconstrained nonlinear pseudo-monotone equations. The search direction generated at each iteration possesses the sufficient descent and trust region properties independent of the line search technique used. The global convergence of the proposed method is shown without the Lipschitz continuity of the underlying mapping. Moreover, the asymptotic and non-asymptotic convergence rates in terms of iteration complexity are established under the local Lipschitz continuity assumption. To our knowledge, this is the first time in the literature that an iteration-complexity analysis has been conducted for inertial-relaxed gradient-type projection methods. Numerical experiments on large-scale benchmark test problems are conducted to demonstrate the effectiveness and efficiency of the proposed algorithm. Furthermore, the applicability and practicality of the proposed method are also verified by applying it to solve sparse signal restoration problems.

• General Summary

• Abstract

This paper investigates an interface-coupled fractional subdiffusion model, featuring two subdiffusion equations in adjacent domains connected by an interface allowing bidirectional energy transfer. The fractional derivative, accounting for long-term medium effects, introduces challenges in theoretical analysis and computational efficiency. We propose a partitioned time-stepping algorithm using higher-order extrapolations on the interface term to decouple the system with improved temporal accuracy, combined with finite element spatial approximations. Rigorous theoretical analysis demonstrates unconditional stability and optimal $L^2$ norm error estimates, supported by several numerical experiments.

• General Summary

• Abstract

In this paper, we propose an accelerated stochastic variance reduction gradient method with a trust-region-like framework, referred as the NMSVRG-TR method. Based on NMSVRG, we incorporate a Katyusha-like acceleration step into the stochastic trust region scheme, which improves the convergence rate of the SVRG methods. Under appropriate assumptions, the linear convergence of the algorithm is provided for strongly convex objective functions. Numerical experiment results show that our algorithm is generally superior to some existing stochastic gradient methods.

• General Summary

• Abstract

In this paper, the novel optimization model for solving tensor completion with noise is proposed, its objective function is a convex combination of the minimum nuclear norm and maximum nuclear norm. The necessary condition and sufficient condition of the stationary point and optimal solution are discussed. Based on the proximal gradient algorithm and feasible direction method, we design the new algorithm for solving the proposed nonconvex and nonsmooth optimization problem and prove that the sub-sequence generated by the new algorithm converges to the stationary point. Finally, experimental results on the random sample completions and images show that the proposed optimization and algorithm are superior to the compared algorithms in CPU time or precision.

• General Summary

• Abstract

In this paper, we propose an adaptive algorithm for L1-fidelity color image restoration by using saturation-value total variation. The main contribution of this paper is to employ the generalized cross validation method efficiently and automatically to estimate the regularization parameter in a saturation-value total variation plus L1-fidelity color image restoration model. We consider Poisson noise and mixed noise in this paper, and the experimental results show that the visual quality and the SSIM/PSNR/SAM values of the restored images by using the proposed algorithm are competitive with other tested existing methods, which makes the proposed algorithm to be comparable both quantitatively and qualitatively.

• General Summary

• Abstract

In practical applications, the Allen-Cahn (AC) equation is commonly used to model microstructure evolutions, including alloy solidification, crystal growth, fingerprint image restoration, and image segmentation. However, when we discretize the AC equation with a conventional finite difference scheme, the directional bias in error terms introduces anisotropy into the numerical results, affecting interface dynamics. To address this issue, we use two- and three-dimensional isotropic finite difference schemes to solve the AC equation. Stability of the proposed algorithm is verified by deriving the time step constraints in both 2D and 3D domains. To demonstrate the sharp estimation of the stability constraints, we conducted several numerical experiments and found the maximum principle is guaranteed under the analyzed time-step constraint.

• General Summary

• Abstract

A fractional-order mathematical model of lung cancer is used to describe the dynamics of tumor growth and the interactions between cancer cells and immune cells. To obtain approximate solutions and better understand the behavior of the state functions, a pseudo-operational collocation scheme employing shifted Jacobi polynomials as basis functions is introduced. Initially, the existence and uniqueness of solutions to the model are established using the Leray-Schauder fixed-point theorem. Error bounds for the residual functions are estimated within a Jacobi-weighted $L^2$-space. To enhance the accuracy and reliability of the results, two distinct strategies are implemented: sensitivity analysis and feedback control. The feedback control of the proposed pseudo-operational spectral method is performed using the method of Lagrange multipliers, marking its first application in this context. Spectral solutions are derived by applying the pseudo-operational scheme to both the original model and the model with control functions. Improved performance and outputs are anticipated following the application of the feedback control strategy. Finally, comprehensive biological interpretations of the results are provided, offering insights into the practical implications of the model.

• General Summary

• Abstract

This paper is devoted to the study of a sub-diffusion equation involving a Clarke subdifferential boundary condition. It describes transport of particles governed by the anomalous diffusion in media with boundary semipermeability. The weak formulation of the model problem results in a time fractional parabolic hemivariational inequality. We first construct an abstract hemivariational evolutionary inclusion and prove its unique solvability using a time-discretization approximation, known as the Rothe method. In addition, a numerical approach based on a finite difference scheme in time and finite dimensional approximation in space is proposed and analyzed for the abstract problem. These results are then applied to establish the convergence of the numerical solution of the model problem. Under appropriate regularity assumptions, an optimal order error estimate for the linear finite element method is derived. Some numerical examples are provided to support the theoretical results.

• General Summary