TY - JOUR
T1 - Embedding Inequalities for Barron-Type Spaces
AU - Wu , Lei
JO - Journal of Machine Learning
VL - 4
SP - 259
EP - 270
PY - 2023
DA - 2023/12
SN - 2
DO - http://doi.org/10.4208/jml.230530
UR - https://global-sci.org/intro/article_detail/jml/22307.html
KW - Barron space, Two-layer neural network, High-dimensional approximation, Embedding theorem, Fourier transform.
AB - <p style="text-align: justify;">An important problem in machine learning theory is to understand the approximation and generalization properties of two-layer neural networks in high dimensions. To this end, researchers have introduced the Barron space $\mathcal{B}_s(Ω)$ and the spectral Barron space $\mathcal{F}_s(Ω),$ where the index $s ∈ [0, ∞)$ indicates the smoothness of functions within these spaces and $Ω ⊂ \mathbb{R}^d$ denotes the input domain. However, the precise relationship between the two types of Barron spaces remains unclear. In this paper, we establish a continuous embedding between them as implied by the following inequality: For any $\delta∈ (0, 1),s ∈ \mathbb{N}^+$ and $f : Ω\mapsto \mathbb{R},$ it holds that $$\delta||f||_{\mathcal{F}_{s−\delta}(Ω)}\lesssim_s||f||_{\mathcal{B}_s(Ω)}\lesssim_s||f||_{\mathcal{F}_{s+1}(Ω).}$$ Importantly, the constants do not depend on the input dimension $d,$ suggesting that the embedding is effective in high dimensions. Moreover, we also show that the lower and upper bound are both tight.</p>