# Download Stochastic Approximation and Recursive Algorithms and by Kushner H.J., Yin G. PDF

By Kushner H.J., Yin G.

Best mathematics books

Extra resources for Stochastic Approximation and Recursive Algorithms and Applications Applications of Mathematics ; 35

Example text

In one form or another such methods have been in experimental or practical use since the earliest work in stochastic approximation. Proofs of convergence and the rate of convergence were given in [135], for the case where the direction was selected at random on the surface of the unit sphere, with the conclusion that there was little advantage over the classical method. The work of Spall [212, 213, 226, 227, 228, 229], where the random directions were chosen in a diﬀerent way, showed advantages for such high dimensional problems and encouraged a reconsideration of the random directions method.

There is a sequence of M dimensional input vectors, the one at time n being φn = (φn,1 , . . , φn,M ), with associated actual network outputs vn (that can be observed) and desired outputs yn (that are not generally observable in applications, at least after the training period). If the network is well designed, then vn is a “good approximation” to yn . There are weights αij such that the input at time n to “neuron” j in the ﬁrst layer is M u1j (α, φn ) = αij φn,i , j = 1, . . , K. i=1 The output of this neuron is p1 (u1j (α, φn )), where p1 (·) is a real-valued antisymmetric nondecreasing and continuously diﬀerentiable function of a real variable, which is positive for positive arguments and is often chosen to be the sigmoid function.

Each player has a choice of actions, knows its own payoﬀs, but not those of the other player. The player tries to learn its optimal strategy via repeated games and observations of the opponents strategies. Such problems have arisen in the economics literature. The results illustrate new ways in which complex recursive stochastic algorithms arise, as well as some of the phenomena to be expected. 1 An Animal Learning Model This example concerns the purported learning behavior of an animal (say, a lizard) as it tries to maximize its reward per unit time in its food gathering activities.