Evolving Recurrent Neural Network Architectures by
Genetic Programming
Technical Report, CSC-96009. Faculty of Engineering, Glasgow G12 8QQ, Scotland, (1996)Abstract
We propose a novel design paradigm for recurrent
neural networks. This employs a two-stage Genetic
Programming / Simulated Annealing hybrid algorithm to
produce a neural network which satisfies a set of
design constraints. The Genetic Programming part of the
algorithm is used to evolve the general topology of the
network, along with specifications for the neuronal
transfer functions, while the Simulated Annealing
component of the algorithm adapts the network's
connection weights in response to a set of training
data. Our approach offers important advantages over
existing methods for automated network design. Firstly,
it allows us to develop recurrent network connections;
secondly, we are able to employ neurons with arbitrary
transfer functions, and thirdly, the approach yields an
efficient and easy to implement on-line training
algorithm. The procedures involved in using the GP/SA
hybrid algorithm are illustrated by using it to design
a neural network for adaptive filtering in a signal
processing application.