TY - JOUR
T1 - A Stochastic Approximation Frame Algorithm with Adaptive Directions
AU - Zi Xu & Yu-Hong Dai
JO - Numerical Mathematics: Theory, Methods and Applications
VL - 4
SP - 460
EP - 474
PY - 2008
DA - 2008/01
SN - 1
DO - http://doi.org/
UR - https://global-sci.org/intro/article_detail/nmtma/9965.html
KW - Stochastic approximation, conjugate gradient, adaptive directions.
AB - <p style="text-align: justify;">Stochastic approximation problem is to find some roots or extremum of a nonlinear function for which only noisy measurements of the function are available. The
classical algorithm for stochastic approximation problem is the Robbins-Monro (RM)
algorithm, which uses the noisy evaluation of the negative gradient direction as the
iterative direction. In order to accelerate the RM algorithm, this paper gives a frame
algorithm using adaptive iterative directions. At each iteration, the new algorithm goes
towards either the noisy evaluation of the negative gradient direction or some other
directions under some switch criteria. Two feasible choices of the criteria are proposed and two corresponding frame algorithms are formed. Different choices of the
directions under the same given switch criterion in the frame can also form different
algorithms. We also proposed the simultanous perturbation difference forms for the
two frame algorithms. The almost surely convergence of the new algorithms are all
established. The numerical experiments show that the new algorithms are promising.</p>