Method and system for training a recurrent network

Abstract

A method for training a recurrent network represented by x(k+1)=f(W x(k)), where W is a weight matrix, x is the output of the network, and K is a time index includes (a) determining the weight matrix at a first time increment, (b) incrementing the time increment associated with a received data point, and (c) determining a change in the weight matrix at the incremented time interval according to the formula: Δ⁢ ⁢W⁡(K)=Δ⁢ ⁢W⁡(K-1)+η⁢ ⁢γ⁡(K)⁢xT⁡(K-1)⁢ ⁢V-1⁡(K-1)-B⁡(K-1)⁢ ⁢V-1⁡(K-1)⁢ ⁢x⁢ ⁢(K-1)⁡[V-1⁡(K-1)⁢ ⁢x⁢ ⁢(K-1)]T1+xT⁡(K-1)⁢ ⁢V-1⁡(K-1)⁢ ⁢x⁢ ⁢(K-1)