An Adaptive Back-Propagation Learning Method: A Preliminary Study for Incremental Neural Networks

Almost all traditional learning methods of artificial neural networks focused on manipulating stationary training sets. In fact, this assumption is anti-biological and too strong to hinder many interesting applications. In this paper, we apply the concept of minimizing weight sensitivity cost and training square-error functions using gradient descent optimization techniques, and obtain a new supervised back-propagation learning algorithm on a biased 2-layered perceptron. In addition to illustrate the conflict locality of an inserted training instance with respect to previous training data, we point out this adaptive learning method can get a network with a measurable generalization ability. This work can also be extended to an incremental network which no training instances are needed to be remembered.