Zusammenfassung
Echo State Networks (ESNs) are a class of single-layer recurrent neural
networks with randomly generated internal weights, and a single layer of
tuneable outer weights, which are usually trained by regularised linear least
squares regression. Remarkably, ESNs still enjoy the universal approximation
property despite the training procedure being entirely linear. In this paper,
we prove that an ESN trained on a sequence of scalar observations from an
ergodic dynamical system (with invariant measure \mu) using Tikhonov least
squares will approximate future observations of the dynamical system in the
L2(\mu) norm. We call this the ESN Training Theorem. We demonstrate the
theory numerically by training an ESN using Tikhonov least squares on a
sequence of scalar observations of the Lorenz system, and compare the invariant
measure of these observations with the invariant measure of the future
predictions of the autonomous ESN.
Nutzer