Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

Modified recursive least squares (RLS) algorithm for neural networks using piecewise linear function

Modified recursive least squares (RLS) algorithm for neural networks using piecewise linear function

For access to this article, please select a purchase option:

Buy article PDF
£12.50
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Name:*
Email:*
Your details
Name:*
Email:*
Department:*
Why are you recommending this title?
Select reason:
 
 
 
 
 
IEE Proceedings - Circuits, Devices and Systems — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

The recursive least squares (RLS) learning algorithm for multilayer feedforward neural networks uses a sigmoid nonlinearity at node outputs. It is shown that by using a piecewise linear function at node outputs, the algorithm becomes faster. The modified algorithm improves computational efficiency and by preserving matrix symmetry it is possible to avoid explosive divergence, which is normally seen in the conventional RLS algorithm due to the finite precision effects. Also the use of this piecewise linear function avoids the approximation, which is otherwise necessary in the derivation of the conventional algorithm with sigmoid nonlinearity. Simulation results on the XOR problem, 4–2–4 encoder and function approximation problem indicate that the modified algorithm reduces the occurrence of local minima and improves the convergence speed compared to the conventional RLS algorithm. A nonlinear system identification and control problem is considered to demonstrate the application of the algorithm to complex problems.

References

    1. 1)
      • C.S. Leung , K.W. Wong , J. Sum , L.W. Chan . Online training and pruning for RLS algorithm. Electron. Lett. , 2152 - 2153
    2. 2)
      • L. Ljung . (1999) System identification: theory for the user.
    3. 3)
      • S. Haykin . (1996) Adaptive filter theory.
    4. 4)
    5. 5)
    6. 6)
    7. 7)
    8. 8)
      • S. Haykin . (1994) Neural networks, a comprehensive foundation.
    9. 9)
      • J. Soberg , L. Lung . Over-training, regularisation and searching for a minimum, with application to neural networks. Int. J. Control , 1391 - 1407
    10. 10)
    11. 11)
    12. 12)
      • S. Shah , F. Palmieri , M. Datum . Optimal filtering algorithm for fast learning in feedforward neural networks. Neural Netw. , 5 , 779 - 788
    13. 13)
      • Reidmiller, M.: ‘Advanced supervised learning in multi-layer perceptrons - from backpropagation to adaptive learning algorithms.’ Institut für Logik, Komplexität und Deduktionssyteme, University of Karlsruhe, W-76128 Karlsruhe, FRG (available on Internet).
    14. 14)
    15. 15)
    16. 16)
    17. 17)
      • Fahlman, S.E.: `An empirical study of learning speed in back-propagation networks', CMU-CS-88-162, Technical Report, September 1988, available on Internet.
    18. 18)
      • J. Bilski , L. Rutkowski . A fast training algorithm for neural networks. IEEE Trans. Circuits Syst. , 6 , 749 - 753
    19. 19)
    20. 20)
    21. 21)
    22. 22)
      • L. Ljung , T. Soderstrom . (1983) Theory and practice of recursive identification.
    23. 23)
      • S. Chen , S.A. Billings , C.F.N. Cowan , P.M. Grant . Practical identification of NARMAX models using radial basis functions. Int. J. Control , 1327 - 1350
    24. 24)
      • S. Kollias , D. Anastassiou . An adaptive least squares algorithm for the efficient training of artificial neural networks. IEEE Trans. Circuits Syst. , 8 , 1092 - 1101
    25. 25)
      • Gokhale, A.P., Nawghare, P.M.: `Performance improvement of backpropagation algorithm by piece-wise linear approximation of non-linear function', Proc. Int. Symp. ISIC' 95, Sept. 1995, Singapore, p. 92–96.
http://iet.metastore.ingenta.com/content/journals/10.1049/ip-cds_20040614
Loading

Related content

content/journals/10.1049/ip-cds_20040614
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address