TY - JOUR

T1 - Using a modified EREV—roth algorithm in an agent-based electricity market Model

AU - Gaivoronskaia, E. A.

AU - Tsyplakov, A. A.

PY - 2018/1/1

Y1 - 2018/1/1

N2 - One of the important tools for the analysis and prediction of operation of electricity markets are agent-based models, which simulate the behavior of decentralized agents (for example, producers and buyers), each with its own objectives and means. In these models learning of agents submitting price bids to a wholesale market plays an important role. In the process of repeated interaction an agent adapts to the environment and to the behavior of other agents, learns to predict the results of its own actions. The paper presents a modification of the classical Erev—Roth reinforcement learning algorithm which takes into account the distance between alternatives. The proposed modified algorithm is used to represent agents’ learning in an agent-based model of the Russian wholesale electricity market (Siberian pricing zone) within the bounds of the day-ahead market. It is shown that it has some significant advantages as compared to the original algorithm. In particular, the algorithm is naturally interpretable, is robust to the choice of discretization step, is invariant to a shift in payoffs scale. On the whole, the algorithm is more flexible than the original one. When the modified algorithm is used, one observes good coherence between the dynamics of model price and the observable dynamics of the price in the market.

AB - One of the important tools for the analysis and prediction of operation of electricity markets are agent-based models, which simulate the behavior of decentralized agents (for example, producers and buyers), each with its own objectives and means. In these models learning of agents submitting price bids to a wholesale market plays an important role. In the process of repeated interaction an agent adapts to the environment and to the behavior of other agents, learns to predict the results of its own actions. The paper presents a modification of the classical Erev—Roth reinforcement learning algorithm which takes into account the distance between alternatives. The proposed modified algorithm is used to represent agents’ learning in an agent-based model of the Russian wholesale electricity market (Siberian pricing zone) within the bounds of the day-ahead market. It is shown that it has some significant advantages as compared to the original algorithm. In particular, the algorithm is naturally interpretable, is robust to the choice of discretization step, is invariant to a shift in payoffs scale. On the whole, the algorithm is more flexible than the original one. When the modified algorithm is used, one observes good coherence between the dynamics of model price and the observable dynamics of the price in the market.

KW - Agent-based models

KW - Day-ahead market

KW - Erev—Roth algorithm

KW - Learning algorithms

KW - Wholesale electricity market

KW - agent-based models

KW - wholesale electricity market

KW - day-ahead market

KW - learning algorithms

KW - Erev-Roth algorithm

KW - SIMULATION

KW - POWER

KW - EFFICIENCY

KW - ENGLAND

KW - GAMES

UR - http://www.scopus.com/inward/record.url?scp=85061937366&partnerID=8YFLogxK

UR - https://www.elibrary.ru/item.asp?id=35647147

U2 - 10.31737/2221-2264-2018-39-3-3

DO - 10.31737/2221-2264-2018-39-3-3

M3 - Article

AN - SCOPUS:85061937366

VL - 39

SP - 55

EP - 83

JO - Журнал Новой экономической ассоциации

JF - Журнал Новой экономической ассоциации

SN - 2221-2264

IS - 3

ER -