We explore the applications of random matrix theory (RMT) in the training of deep neural networks (DNNs), focusing on layer pruning that reduces the number of DNN parameters (weights). Our numerical results show that this pruning leads to a drastic reduction of parameters while not reducing the accuracy of DNNs and convolutional neural network (CNNs). Moreover, pruning the fully connected DNNs actually increases the accuracy and decreases the variance for random initializations. Our numerics indicate that this enhancement in accuracy is due to the simplification of the loss landscape. We next provide rigorous mathematical underpinning of these numerical results by proving the RMT-based Pruning Theorem. Our results offer valuable insights into the practical application of RMT for the creation of more efficient and accurate deep-learning models.

This paper presents a mathematical analysis of ODE-Net, a continuum model of deep neural networks (DNNs). In recent years, machine learning researchers have introduced ideas of replacing the deep structure of DNNs with ODEs as a continuum limit. These studies regard the “learning” of ODE-Net as the minimization of a “loss” constrained by a parametric ODE. Although the existence of a minimizer for this minimization problem needs to be assumed, only a few studies have investigated the existence analytically in detail. In the present paper, the existence of a minimizer is discussed based on a formulation of ODE-Net as a measure-theoretic mean-field optimal control problem. The existence result is proved when a neural network describing a vector field of ODE-Net is linear with respect to learnable parameters. The proof employs the measure-theoretic formulation combined with the direct method of calculus of variations. Secondly, an idealized minimization problem is proposed to remove the above linearity assumption. Such a problem is inspired by a kinetic regularization associated with the Benamou-Brenier formula and universal approximation theorems for neural networks.

Accurate and automatic segmentation of the pancreas from abdominal computed tomography (CT) scans is crucial for diagnosing and treating pancreatic diseases. However, the pancreas is a tiny target abdominal organ with high anatomical variability and low tissue contrast in CT scans, making segmentation tasks challenging. To address this challenge, we propose a multilevel attention feature extraction network to segment the pancreas in abdominal CT images. Specifically, a multi-field attention convolution module (MFAC) and a connection feature fusion module (CFF) are added to the encoding and decoding structure to improve the extraction of pancreatic features. To further enhance the segmentation network’s extraction of pancreatic edge features, we propose a decoding feature recall module (DFC), which can be migrated to other encoding and decoding structures and pruned to capture pancreatic edge information better. We compared the performance of our method with that of the most advanced method on the NIH pancreatic segmentation dataset and the challenging pancreatic cancer CT image dataset collected by the Zhujiang Hospital of Southern Medical University. The experimental results show that the DSC of our method on NIH dataset and pancreatic cancer dataset is 84.69% and 78.18% respectively, which is superior to the existing methods.