TY - JOUR
T1 - Variant of Greedy Randomized Gauss-Seidel Method for Ridge Regression
AU - Duan , Li-Xiao
AU - Zhang , Guo-Feng
JO - Numerical Mathematics: Theory, Methods and Applications
VL - 3
SP - 714
EP - 737
PY - 2021
DA - 2021/06
SN - 14
DO - http://doi.org/10.4208/nmtma.OA-2020-0095
UR - https://global-sci.org/intro/article_detail/nmtma/19195.html
KW - Randomized algorithms, ridge regression, greedy randomized Gauss-Seidel, randomized Kaczmarz, iterative method.
AB - <p style="text-align: justify;">The variants of randomized Kaczmarz and randomized Gauss-Seidel algorithms are two effective stochastic iterative methods for solving ridge regression
problems. For solving ordinary least squares regression problems, the greedy randomized Gauss-Seidel (GRGS) algorithm always performs better than the randomized Gauss-Seidel algorithm (RGS) when the system is overdetermined. In this paper, inspired by the greedy modification technique of the GRGS algorithm, we extend
the variant of the randomized Gauss-Seidel algorithm, obtaining a variant of greedy
randomized Gauss-Seidel (VGRGS) algorithm for solving ridge regression problems.
In addition, we propose a relaxed VGRGS algorithm and the corresponding convergence theorem is established. Numerical experiments show that our algorithms
outperform the VRK-type and the VRGS algorithms when $m &gt; n$.</p>