Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

Q-adjusted annealing for Q-learning of bid selection in market-based multisource power systems

Q-adjusted annealing for Q-learning of bid selection in market-based multisource power systems

For access to this article, please select a purchase option:

Buy article PDF
£12.50
(plus tax if applicable)
Add to cart
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)
Add to cart

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Name:*
Email:*
Your details
Name:*
Email:*
Department:*
Why are you recommending this title?
Select reason:
 
 
 
 
 
IEE Proceedings - Generation, Transmission and Distribution — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

© The Institution of Engineering and Technology
Published

The electric power industry is confronted with a major restructuring in which operation scheduling is decided based on a competitive market. In this new arrangement, bidding strategy becomes an important issue. Participants of the deregulated energy marketplace may be able to compete better by choosing a suitable bidding strategy for trading electricity. In such a deregulated market, coalition formation with other participants may change the diffusion of profit. Therefore the problem changes to the selection of the best participant for coalition and joint bidding strategies. Different classic methods for decision making in the uncertain environment of the market can be applied to select a suitable strategy. Most of these methods, such as game theory that insures reaching the optimal solution for all market participants, require a lot of information about other market players and the market. However, in a real marketplace only a little information such as the spot price is available for all participants. A modified reinforcement-learning approach based on Q-adjusted annealing has been applied to determine the optimal strategy for a power supplier in an electricity market with multiple sources. A modified IEEE 30-bus system has been considered and the simulation results are shown to be the same as with standard game theory. The main advantage of the proposed method is that no information about other participants is required. Investigation shows that if all participants use this method they will stay in Nash equilibrium.

Inspec keywords: IEEE standards; simulated annealing; decision making; learning (artificial intelligence); power markets; power generation scheduling

Other keywords: competitive market; electricity market; IEEE 30-bus system; Nash equilibrium; deregulated energy market; operation scheduling; Q-adjusted annealing; electricity trading; electric power industry; market-based multisource power system; reinforcement-learning; power supplier; Q-learning

Subjects: Power system management, operation and economics; Optimisation techniques

References

    1. 1)
      • F. Schweppe , M. Caramanis , R. Tabors , R. Bohn . (1988) Spot pricing of electricity.
    2. 2)
      • G. Xiong , T. Hashiyama , S. Okuma . A Q-learning-based supplier bidding strategy in electricity auction market. Trans. Institute of Electrical Engineers of Japan
    3. 3)
      • E. Kutschinski , T. Uthmann , D. Polani . Learning competitive pricing strategies by multiagent reinforcement learning. J. Economic Dynam. Control , 2207 - 2218
    4. 4)
      • Naghibi-Sistani, M.B., Akbarzadeh-T, M.R., Rajabi-Mashhadi, H.: `Reinforcement learning for pricing electricity in deregulated power pools', Proc. 11th Iranian Conf. on Electrical Engineering ICEE2003, 2003, Shiraz, Iran, 1, p. 606–613.
    5. 5)
      • S.S. Keerthi , B. Ravindran . A tutorial survey of reinforcement learning. Machine Learning
    6. 6)
      • C. Wang , S.M. Shahidehpour . A decomposition approach to nonlinear multiarea generation scheduling with tie-line constraints using expert systems. IEEE Trans. Power Syst. , 4 , 1409 - 1417
    7. 7)
      • Minsky, M.L.: `Theory of neural-analog reinforcement systems and application to the brain-model problem', 1954, PhD dissertation, Princeton University, Princeton, NJ, USA.
    8. 8)
      • Alomoush, M.: `Auctionable fixed transmission rights for congestion management', 3 May 2000, PhD dissertation, Illinois Institute of Technology, Department of ECE.
    9. 9)
      • M. Guo , Y. Liu , J. Malec . A new q-learning algorithm based on the metropolis criterion. IEEE Trans. Syst., Man, Cybern.—Part B: Cybernetics , 5
    10. 10)
      • Abdul-Rahman, K.H., Shahidehpour, S.M., Aganagic, M., Mokhtari, S.: `A practical resource scheduling with OPF constraints', Proc. of 1995 IEEE Power Industry Computer Applications Conf., May 1995, Salt Lake City, p. 92–97.
    11. 11)
      • S. Sen , M. Sekaran , G. Weiss , S. Sen . (1995) Multiagent co-ordination with learning classifier systems, Adaptation and Learning in Multiagent Systems.
    12. 12)
      • Kutschinski, E.: `Simulation of a distributed market network with autonomous Java agents', 2000, MSc, Institut fur Informatik, Universitat Mainz.
    13. 13)
    14. 14)
      • M.B. Naghibi-Sistani , M.R. Akbarzadeh-T , M.H. Javidi-DB , H. Rajabi-Mashhadi . Application of Q-learning with temperature variation for bidding strategies in market based power systems. J. Energy Converg. Manag.
    15. 15)
      • C.J.C.H. Watkins , P. Dayan . Q-learning. Machine Learning , 3 , 279 - 292
    16. 16)
      • Gross, G., Finlay, D.J.: `Optimal bidding strategies in competitive electricity markets', Proc. of the 12th Power Systems Computation Conf., 1996, Dresden, p. 815–822.
    17. 17)
      • H. Rudnick , R. Varela , W. Hogan . Evaluation of alternatives for power system coordination and pooling in a competitive environment. IEEE Trans. Power Syst. , 330 - 331
    18. 18)
    19. 19)
      • S. Dekrajangpetch , G.B. Sheble , A.J. Conejo . Auction implementation problem using lagrangian relaxation. IEEE Trans. Power Syst. , 1 , 82 - 88
    20. 20)
      • P. Moris . (1994) Introduction to game theory.
    21. 21)
      • L.P. Kaelbling , M.L Littman , A.W. Moore . Reinforcement learning: a survey. J. Artificial Intell. Res. , 237 - 285
    22. 22)
      • Stefan, P.: `Combined use of reinforcement learning and simulated annealing: algorithms and applications', 2003, PhD dissertation, University of Miskolc, Department of Mechanical Engineering, Miskolc-Budapest.
    23. 23)
    24. 24)
      • Watkins, C.J.C.H.: `Learning from delayed reward', 1989, PhD dissertation, King's College, Cambridge, UK.
    25. 25)
      • Arai, S., Miyazaki, K., Kobayashi, S.: `Generating co-operative behaviour by multiagent reinforcement learning', Proc. 6th European Workshop on Learning Robots, 1997, p. 111–120.
    26. 26)
      • C. Wang , S.M. Shahidehpour . Power generation scheduling for multiarea hydrothermal systems with tie-line constraints, cascaded reservoir and uncertain data. IEEE Trans. Power Syst. , 3 , 1333 - 1340
    27. 27)
http://iet.metastore.ingenta.com/content/journals/10.1049/ip-gtd_20050285
Loading

Related content

content/journals/10.1049/ip-gtd_20050285
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address