Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

Event-driven semi-Markov switching state-space control processes

Event-driven semi-Markov switching state-space control processes

For access to this article, please select a purchase option:

Buy article PDF
£12.50
(plus tax if applicable)
Add to cart
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)
Add to cart

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Name:*
Email:*
Your details
Name:*
Email:*
Department:*
Why are you recommending this title?
Select reason:
 
 
 
 
 
IET Control Theory & Applications — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

  • Author(s): Q. Jiang 1, 2 ; H.-S. Xi 2 ; B.-Q. Yin 2
    • View affiliations
    • Affiliations: 1: School of Electrical Engineering and Automation, Hefei University of Technology, Hefei, People’s Republic of China
      2: Department of Automation, University of Science and Technology of China, Hefei, People’s Republic of China
  • Source: Volume 6, Issue 12, 16 August 2012, p. 1861 – 1869
    DOI:  10.1049/iet-cta.2011.0174 , Print ISSN 1751-8644, Online ISSN 1751-8652
© The Institution of Engineering and Technology
Published

Motivated by the optimisation of network communication systems, this paper presents a hierarchical analytical model for event-driven switching control of stochastic dynamic systems. First, the model called semi-Markov switching state-space control processes is introduced. The semi-Markov kernel and equivalent infinitesimal generator are constructed to characterise the hierarchical dynamics, and the sensitivity formula for performance difference under average criterion is derived based on potential theory. Then, by exploiting the structure of dynamic hierarchy and the features of event-driven policy, an online adaptive optimisation algorithm that combines potentials estimation and policy iteration is proposed. The convergence of this algorithm is also proved. Finally, as an illustrative example, the dynamic service composition problem in service overlay networks is formulated and addressed. Simulation results demonstrate the effectiveness of the presented approach.

Inspec keywords: Markov processes; optimisation; stochastic systems; iterative methods

Other keywords: performance difference; online adaptive optimisation algorithm; dynamic service composition problem; semiMarkov switching state-space control processes; semiMarkov kernel; event-driven switching control; policy iteration; stochastic dynamic system; service overlay network; dynamic hierarchy; equivalent infinitesimal generator; potential theory; hierarchical dynamics; network communication system optimisation; hierarchical analytical model

Subjects: Time-varying control systems; Markov processes; Interpolation and function approximation (numerical analysis); Optimisation techniques

References

    1. 1)
    2. 2)
      • M.L. Puterman . (1994) Markov decision processes: discrete stochastic dynamic programming.
    3. 3)
    4. 4)
    5. 5)
    6. 6)
    7. 7)
    8. 8)
      • X.-R. Cao . (2007) Stochastic learning and optimization: a sensitivity-based approach.
    9. 9)
    10. 10)
      • Ren, Z.-Y., Krogh, B.H.: `State aggregate in Markov decision processes', Proc. 41st IEEE Conf. Decision and Control, 2002, Las Vegas, NV, USA, p. 3819–3824.
    11. 11)
    12. 12)
    13. 13)
    14. 14)
      • T.G. Dietterich . Hierarchical reinforcement learning with the MAXQ value function decomposition. J. Artif. Intell. Res. , 227 - 303
    15. 15)
      • Bannazadeh, H., Leon-Garcia, A.: `Allocating service to applications using Markov decision processes', Proc. IEEE Int. Conf. Service-Oriented Computing and Applications, 2007, p. 141–146.
    16. 16)
    17. 17)
    18. 18)
    19. 19)
      • Parr, R.: `Hierarchical control and learning for Markov decision processes', 1998, Phd, University of California, Berkeley, CA, USA.
    20. 20)
    21. 21)
      • D.P. Bertsekas . (1995) Dynamic programming and optimal control.
    22. 22)
    23. 23)
    24. 24)
      • Ren, Z.-Y., Krogh, B.H.: `Switching control in multi-mode Markov decision processes', Proc. 40th IEEE Conf. on Decision and Control, 2001, Orlando, FL, USA, p. 2095–2101.
    25. 25)
    26. 26)
    27. 27)
    28. 28)
    29. 29)
    30. 30)
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cta.2011.0174
Loading

Related content

content/journals/10.1049/iet-cta.2011.0174
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address