Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free Online approximate optimal control for affine non-linear systems with unknown internal dynamics using adaptive dynamic programming

© The Institution of Engineering and Technology
Received 19/02/2014, Accepted 06/06/2014, Revised 12/05/2014, Published 12/08/2014

In this study, a novel online adaptive dynamic programming (ADP)-based algorithm is developed for solving the optimal control problem of affine non-linear continuous-time systems with unknown internal dynamics. The present algorithm employs an observer–critic architecture to approximate the Hamilton–Jacobi–Bellman equation. Two neural networks (NNs) are used in this architecture: an NN state observer is constructed to estimate the unknown system dynamics and a critic NN is designed to derive the optimal control instead of typical action–critic dual networks employed in traditional ADP algorithms. Based on the developed architecture, the observer NN and the critic NN are tuned simultaneously. Meanwhile, unlike existing tuning laws for the critic, the newly developed critic update rule not only ensures convergence of the critic to the optimal control but also guarantees stability of the closed-loop system. No initial stabilising control is required, and by using recorded and instantaneous data simultaneously for the adaptation of the critic, the restrictive persistence of excitation condition is relaxed. In addition, Lyapunov direct method is utilised to demonstrate the uniform ultimate boundedness of the weights of the observer NN and the critic NN. Finally, an example is provided to verify the effectiveness of the present approach.

Inspec keywords: optimal control; nonlinear control systems; dynamic programming; continuous time systems; neurocontrollers; observers; Lyapunov methods; stability; closed loop systems

Other keywords: unknown internal dynamics; online approximate optimal control; critic update rule; Hamilton-Jacobi-Bellman equation; neural networks; NN state observer; closed-loop system; excitation condition; instantaneous data; observer-critic architecture; adaptive dynamic programming; affine nonlinear continuous-time systems; stability; recorded data; ADP-based algorithm; Lyapunov direct method; uniform ultimate boundedness; action-critic dual networks

Subjects: Stability in control theory; Neurocontrol; Optimal control; Nonlinear control systems; Optimisation techniques; Simulation, modelling and identification

References

    1. 1)
      • 32. Haykin, S.: ‘Neural networks and learning machines’ (Prentice-Hall, 2008, 3rd edn.).
    2. 2)
    3. 3)
    4. 4)
      • 40. Lewis, F.L., Jagannathan, S., Yesildirek, A.: ‘Neural network control of robot manipulators and nonlinear systems’ (Taylor & Francis, 1999).
    5. 5)
    6. 6)
    7. 7)
    8. 8)
    9. 9)
      • 19. Sutton, R.S., Barto, A.G.: ‘Reinforcement learning–an introduction’ (MIT Press, 1998).
    10. 10)
      • 14. Powell, W.B.: ‘Approximate dynamic programming: solving the curses of dimensionality’ (Wiley, 2011, 2nd edn.).
    11. 11)
      • 41. Gampbell, S.L., Meger, C.D.: ‘Generalized inverses of linear transformations’ (Dover Publications, 1991).
    12. 12)
      • 2. Lewis, F.L., Vrabie, D., Syrmos, V.L.: ‘Optimal control’ (John Wiley & Sons, 2012).
    13. 13)
    14. 14)
      • 44. Chowdhary, G.V.: ‘Concurrent learning for convergence in adaptive control without persistency of excitation’. PhD thesis, Georgia Institute of Technology, 2010.
    15. 15)
      • 33. Khalil, H.K.: ‘Nonlinear systems’ (Prentice-Hall, 2001, 3rd edn.).
    16. 16)
    17. 17)
    18. 18)
      • 7. Werbos, P.J.: ‘Beyond regression: new tools for prediction and analysis in the behavioral sciences’. PhD thesis, Harvard University, 1974.
    19. 19)
    20. 20)
      • 6. Bellman, R.E.: ‘Dynamic programming’ (Princeton University Press, 1957).
    21. 21)
      • 29. Dierks, T., Jagannathan, S.: ‘Optimal control of affine nonlinear continuous-time systems’. Am. Control Conf., Baltimore, MD, USA, June–July 2010, pp. 1568–1573.
    22. 22)
    23. 23)
      • 31. Nodland, D., Zargarzadeh, H., Jagannathan, S.: ‘Neural network-based optimal adaptive output feedback control of a helicopter UAV’, IEEE Trans. Neural Netw. Learn. Syst., 2013, 24, (7), pp. 1061–1073.
    24. 24)
    25. 25)
      • 1. Bryson, A.E., Ho, Y.C.: ‘Applied optimal control: optimization’, Estimation and Control (Taylor & Francis, 1975).
    26. 26)
    27. 27)
    28. 28)
      • 22. Wu, H.N., Luo, B.: ‘Neural network based online simultaneous policy update algorithm for solving the HJI equation in nonlinear H∞ Control’, IEEE Trans. Neural Netw. Learn. Syst., 2012, 23, (12), pp. 1884–1895.
    29. 29)
    30. 30)
    31. 31)
    32. 32)
    33. 33)
    34. 34)
    35. 35)
    36. 36)
    37. 37)
      • 46. Chowdhary, G.V.: ‘A singular value maximizing data recording algorithm for concurrent learning’. American Control Conf., San Francisco, CA, USA, 2011, pp. 3547–3552.
    38. 38)
      • 39. Rudin, W.: ‘Principles of mathematical analysis’ (McGraw-Hill’, Inc., 1976, 3rd edn.).
    39. 39)
    40. 40)
      • 42. Horn, R.A., Johnson, C.R.: ‘Matrix analysis’ (Cambridge University Press, 2012, 2nd edn.).
    41. 41)
      • 37. Yu, W.: ‘Recent advances in intelligent control systems’ (Springer-Verlag, 2009).
    42. 42)
      • 23. Ni, Z., He, H., Wu, J.: ‘Adaptive learning in tracking control based on the dual critic network design’, IEEE Trans. Neural Netw. Learn. Syst., 2013, 24, (6), pp. 913–928.
    43. 43)
    44. 44)
      • 8. Werbos, P.J.: ‘Approximate dynamic programming for real-time control and neural modeling’, in White, D.A., Sofge, D.A. (Eds.): ‘Handbook of intelligent control: neural, fuzzy, and adaptive approaches’ (Van Nostrand Reinhold, 1992).
    45. 45)
    46. 46)
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cta.2014.0186
Loading

Related content

content/journals/10.1049/iet-cta.2014.0186
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address