full
search.png

Search

close.png

Cancel

Volume 2, Issue 1
A Brief Survey on the Approximation Theory for Sequence Modelling Featured Review

Haotian Jiang, Qianxiao Li, Zhong Li & Shida Wang

DOI: 10.4208/jml.221221

J. Mach. Learn. , 2 (2023), pp. 1-30.

Published online: 2023-03

Category: Theory

[An open-access article; the PDF is free to any online user.]

Summary Full PDF 3433 49832

Cited by

google scholar semantic scholar
Export citation
  • Abstract

We survey current developments in the approximation theory of sequence modelling in machine learning. Particular emphasis is placed on classifying existing results for various model architectures through the lens of classical approximation paradigms, and the insights one can gain from these results. We also outline some future research directions towards building a theory of sequence modelling.

  • General Summary

Sequence modelling has a wide variety of applications in natural language processing, finance, control engineering and other fields. However, theoretical understanding of the various machine learning models used are incomplete. In this paper, we survey both classical and recent results on the approximation theory for sequence modelling using machine learning, including recurrent neural networks, temporal convolutional networks, encoder-decoder and attention-based models. The goal of this survey is to summarise the current understanding of approximation theory for learning sequential relationships, and to outline some future research directions.

  • Keywords

Approximation theory, Sequence modelling, Machine learning, Deep learning, Dynamics.

  • AMS Subject Headings

  • Copyright

COPYRIGHT: © Global Science Press

  • Email address
  • BibTex
  • RIS
  • TXT
@Article{JML-2-1, author = {Jiang , HaotianLi , QianxiaoLi , Zhong and Wang , Shida}, title = {A Brief Survey on the Approximation Theory for Sequence Modelling}, journal = {Journal of Machine Learning}, year = {2023}, volume = {2}, number = {1}, pages = {1--30}, abstract = {

We survey current developments in the approximation theory of sequence modelling in machine learning. Particular emphasis is placed on classifying existing results for various model architectures through the lens of classical approximation paradigms, and the insights one can gain from these results. We also outline some future research directions towards building a theory of sequence modelling.

}, issn = {2790-2048}, doi = {https://doi.org/10.4208/jml.221221}, url = {http://global-sci.org/intro/article_detail/jml/21511.html} }
TY - JOUR T1 - A Brief Survey on the Approximation Theory for Sequence Modelling AU - Jiang , Haotian AU - Li , Qianxiao AU - Li , Zhong AU - Wang , Shida JO - Journal of Machine Learning VL - 1 SP - 1 EP - 30 PY - 2023 DA - 2023/03 SN - 2 DO - http://doi.org/10.4208/jml.221221 UR - https://global-sci.org/intro/article_detail/jml/21511.html KW - Approximation theory, Sequence modelling, Machine learning, Deep learning, Dynamics. AB -

We survey current developments in the approximation theory of sequence modelling in machine learning. Particular emphasis is placed on classifying existing results for various model architectures through the lens of classical approximation paradigms, and the insights one can gain from these results. We also outline some future research directions towards building a theory of sequence modelling.

Jiang , HaotianLi , QianxiaoLi , Zhong and Wang , Shida. (2023). A Brief Survey on the Approximation Theory for Sequence Modelling. Journal of Machine Learning. 2 (1). 1-30. doi:10.4208/jml.221221
Copy to clipboard
BibteX RIS TXT
The citation has been copied to your clipboard