access icon free Distributed adaptive optimal regulation of uncertain large-scale interconnected systems using hybrid Q-learning approach

A novel hybrid Q-learning algorithm is introduced for the design of a linear adaptive optimal regulator for a large-scale interconnected system with event-sampled inputs and state vector. Here, the time-driven Q-learning along with proposed iterative parameter learning updates are utilised within the event-sampled instants to both improve efficiency of the optimal regulator and obtain a more generalised online Q-learning framework. The network-induced losses due to the presence of a communication network among the subsystems are considered along with the uncertain system dynamics. Stochastic model-free Q-learning and dynamic programming are utilised in the hybrid learning mode for the optimal regulator design. The asymptotic convergence of the system state vector and boundedness of the parameter vector is demonstrated using Lyapunov analysis. Further, when the regression vector of the Q-function estimator satisfies the persistency of excitation condition, the Q-function parameters converge to the expected target values. The analytical design is evaluated using numerical examples via simulation. The net result is the design of a data-driven event-sampled adaptive optimal regulator for an uncertain large-scale interconnected system.

Inspec keywords: distributed control; optimal control; adaptive control; dynamic programming; asymptotic stability; uncertain systems; Lyapunov methods; learning systems; control system synthesis; linear systems; convergence; interconnected systems

Other keywords: Q-function estimator; time-driven Q-learning; iterative parameter learning; linear adaptive optimal regulator; regression vector; optimal regulator design; system state vector; hybrid Q-learning approach; distributed adaptive optimal regulation; asymptotic convergence; dynamic programming; network-induced loss; Lyapunov analysis; stochastic model-free Q-learning; parameter vector; uncertain large-scale interconnected system

Subjects: Stability in control theory; Optimal control; Linear control systems; Control system analysis and synthesis methods; Multivariable control systems; Optimisation techniques; Self-adjusting control systems

References

    1. 1)
    2. 2)
    3. 3)
    4. 4)
    5. 5)
    6. 6)
    7. 7)
    8. 8)
    9. 9)
      • 8. Jamshidi, M.: ‘Large-scale systems: modeling, control, and fuzzy logic’ (Prentice-Hall, Inc., 1996).
    10. 10)
    11. 11)
      • 6. Sahoo, A., Jagannathan, S.: ‘Event-triggered optimal regulation of uncertain linear discrete-time systems by using Q-learning scheme’, Proc. IEEE 53rd Annual Conf. on Decision and Control, Los Angeles, CA, December 2014, pp. 12331238.
    12. 12)
      • 27. Xiangnan, Z., Zhen, Ni, Haibo, He., , et al: ‘Event-triggered reinforcement learning approach for unknown nonlinear continuous-time system’. Proc. Int. Joint Conf. on Neural Networks, Beijing, July 2014, pp. 36773684.
    13. 13)
    14. 14)
    15. 15)
    16. 16)
      • 30. Horn, R.A., Johnson, C.R.: ‘Matrix analysis’ (Cambridge University Press, 2012).
    17. 17)
      • 2. Bradtke, S.J., Ydstie, B.E., Barto, A.G.: ‘Adaptive linear quadratic control using policy iteration’. Proc. American Control Conf., July 1994, pp. 34753479.
    18. 18)
    19. 19)
    20. 20)
    21. 21)
      • 23. Meng, X., Chen, T.: ‘Event-driven communication for sampled-data control systems’. Proc. American Control Conf., Washington, DC, June 2013, pp. 30023007.
    22. 22)
    23. 23)
    24. 24)
    25. 25)
      • 29. Goodwin, G.C., Sin, K.S.: ‘Adaptive filtering prediction and control’ (Courier Corporation, 2014).
    26. 26)
      • 1. Lewis, F.L., Syrmos, V.L.: ‘Optimal control’ (John Wiley & Sons, 1995).
    27. 27)
    28. 28)
      • 13. Narendra, K.S., Mukhopadhyay, S.: ‘To communicate or not to communicate: A decision-theoretic approach to decentralized adaptive control’. Proc. American Control Conf., Baltimore, MD, July 2010, pp. 63696376.
    29. 29)
      • 22. Guinaldo, M., Lehmann, D., Sanchez, J., , et al: ‘Distributed event-triggered control with network delays and packet losses’, Proc. IEEE 51st Annual Conf. on Decision and Control, Maui, HI, July 2012, pp. 16.
    30. 30)
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cta.2015.0943
Loading

Related content

content/journals/10.1049/iet-cta.2015.0943
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading