Comparative study of recurrent neural network architectures on learning temporal sequences

A recurrent neural networks with context units that can handle temporal sequences is proposed. In this paper, we show an architecture whose performance is better than the architectures proposed by Jordan and Elman respectively using error backpropagation learning algorithms. Three learning experiments were carried out. In the first experiment, we used the recurrent neural networks to simulate a finite state machine. In the second experiment, we use the recurrent networks to handle a combination retrieving problem. In the third experiment, we train the neural networks to recognize the periodicity in temporal sequence data. The results of three experiments showed that our system had a better performance.