Overview
ALBidS – Adaptive Learning strategic Bidding System
Introduction
The Multi-Agent Simulator for Competitive Electricity Markets - MASCEM is a modelling and simulation tool that has been developed with the purpose of studying complex restructured electricity markets operation. MASCEM models the complex dynamic market players, including their interactions and medium/long-term gathering of data and experience, to support players' decisions according to their very own characteristics and objectives. MASCEM most important features are presented in Figure 1.
Figure 1 – MASCEM key features.
MASCEM is implemented on the top of OAA, using OAA AgentLib library, and Java Virtual Machine 1.6.0. The OAA's Interagent Communication Language (ICL) is the interface and communication language shared by all agents, no matter which machine they are running on or which programming language they are programmed in, which allows integrating a variety of software modules.
Communication and cooperation between agents are brokered by one or more facilitators, which are responsible for matching requests, from users and agents, with descriptions of the capabilities of other agents.
OAA is not a framework specifically devoted to develop simulations; some extensions were made to make it suitable to deal with the energy markets that MASCEM currently supports, namely to introduce the time evolution mechanism of the simulation.
MASCEM's goal is to be able to simulate as many market models and players types as possible so it can reproduce in a realistic way the operation of real electricity markets. This enables it to be used as a simulation and decision-support tool for short/medium term purposes but also as a tool to support long-term decisions, such as the ones taken by regulators.
Multi-Agent Model
There are several entities involved in the negotiations in the scope of electricity markets; MASCEM's multi-agent model represents all the involved entities and their relationships. MASCEM multi-agent model includes: a market facilitator agent, seller agents, buyer agents, virtual power player (VPP) agents, VPP facilitator agents, a market operator agent and a system operator agent. Figure 2 presents a general overview of MASCEM's main entities interaction.
Figure 2 – MASCEM agent architecture.
The market operator agent is only present in pool or hybrid markets simulations. It receives the bids from buyer and seller agents, validates and analyses them, and determines the market price, and the accepted and refused bids.
The system operator agent is always present. It ensures that all conditions are met within the system, and it is also responsible for the system security. After being informed of all negotiations to be held, examines the technical feasibility from the power system point of view and solves congestion problems that may arise. In fact, this agent makes a connection with a power system simulator, through which the power flow analysis is performed.
The market facilitator agent coordinates and ensures the proper operation of the market, regulating all the existing negotiations. It knows all the market players, as they are registered in advance to the facilitator, specifying their role and services.
Buyer and seller agents are the key elements of the market. Consumers and distribution companies are represented by buyer agents. The electricity producers are represented by seller agents. Seller agents compete with each other to maximize their profits. On the other hand, seller agents may also cooperate with buyers trying to establish agreements that meet the objectives of both parties. The number of buyers and sellers, and their intrinsic and strategic characteristics are defined by the user for each scenario.
Figure 3 displays a screenshot of a buyer agent running, showing its bids, sold and unsold power, and also the requests it is receiving.
Figure 3 – Buyer agent's output.
The top part of Figure 3 shows the graphical representation of this agent's results in the Pool. It includes the amount of energy that it bought in each period, and the comparison between its bid price and the market price.
The bottom part presents the requests that this agent is receiving at each time, and some information about this agent's actions. In this case period 20 has ended and the agent is getting ready to start the negotiations of period 21. Three sets of information can be seen:
- First it received a notification from the market operator indicating that the time for the present period of negotiations ended. So, the agent performs the necessary arrangements to be ready for the next period;
- The second line presents a summary of the results that this agent obtained in the last period, including the information presented in the graph;
- Finally, the third line indicates the bid price and amount of power that this agent will negotiate in this market in the next period. These values are the output of the bidding strategy this agent is using for the next negotiation period.
Due to a significant increase of small independent producers negotiating in the market, comes the need to make alliances between them, in order to be able to compete on equal footing with the big producers. The VPP agents represent these alliances. They manage the information of their aggregates and are viewed from the market as seller agents. Each VPP is modelled as an independent multi-agent system, maintaining high performance and allowing agents to be installed on separate machines. To achieve this independence, individual VPP facilitators have been created to manage the communications between each VPP and its members independently from the rest of the simulation.
Virtual Power Players
Due to environmental and fossil fuels shortage concerns, renewable energy resources are being more used. The advantages of using renewable are clear from the environment point of view. From the technical and economical point of view there are problems that must be overcome to take advantage of an intensive use of renewables, which are mostly originated by distributed generation. An aggregating strategy can enable owners of renewable generation to gain technical and commercial advantages, making profit of the specific advantages of a mix of several generation technologies and overcoming serious disadvantages of some technologies. The aggregation of distributed generation plants gives place to the new concept of Virtual Power Player (VPP). VPPs integration into electricity markets is a very challenging domain that has been motivating MASCEM evolution.
VPPs are multi-technology and multi-site heterogeneous entities, being relationships among aggregated producers and among VPPs and the remaining electricity market agents a key factor for their success. Agent coalitions are especially important to address VPPs as these can be seen as a coalition of agents that represent the aggregated players.
Coalition formation is the coming together of a number of distinct, autonomous agents that agree to coordinate and cooperate, acting as a coherent grouping, in the performance of a specific task. Such coalitions can improve the performance of the individual agents and/or the system as a whole. It is an important form of interaction in multi-agent systems. The coalition formation process comprises several phases: coalition structure generation, optimization of the value of the coalition and payoff distribution.
Regarding the coalition formation process, for VPP modelling, the three main activities of coalition structure generation, optimization of the value of the coalition and payoff distribution should be considered under a scenario where agents operate in a dynamic and time dependent environment. This entails significant changes on MASCEM core model and communications infrastructure.
To sell energy in the market VPP must forecast the generation of aggregated producers and "save" some power capacity to ensure a reserve to compensate a generation oscillation of producers with natural resources technologies dependent.
The VPP can use different market strategies, considering specific aspects such as producers established contracts and range of generation forecast. The prediction errors increase with the distance between the forecasting and the forecast times. The standard errors are given as a percent of the installed capacity, since this is what the utilities are most interested in (installed capacity is easy to measure); sometimes they are given as the mean production or in absolute numbers.
MASCEM's modelling of VPPs enlarged the scope of negotiation procedures in this simulator, allowing the study of different types of negotiation outside the usual electricity markets' regulatory models.
Negotiation in MASCEM
MASCEM includes several negotiation mechanisms usually found in electricity markets, being able to simulate several types of markets, namely: Pool Markets, Bilateral Contracts, Balancing Markets and Forward Markets. Figure 4 presents the negotiation sequence for one day simulation in MASCEM.
Figure 4 – Negotiations timing for day n.
Based on the previously obtained results, buyer and seller agents review their strategies for the future. The strategic behaviour of each agent defines its desired price and amount of power to be negotiated in each market.
Time-dependent strategies and behaviour-dependent strategies are part of each agent, and define the price to negotiate in the next day according to the results obtained previously. There are four types of time-dependent strategies: Determined (prices remain constant throughout the period of negotiation); Anxious (minor changes to the price are made after little trading time); Moderated (small changes to the price are made in an intermediate stage of negotiation period); Gluttonous (the price is significantly changed, but only in late trading).
On the other hand, the behaviour-dependent strategies are: Composed Goal Directed (when an agent has two consecutive goals, in which the definition of the second objective depends on the fulfilment of the first); Adapted Derivative Following (the results of price changes made in previous trading periods are analyzed. If the agent finds that the change in the price of its proposals brought benefits, it maintains the same type of change for the next period. Otherwise, the change in price will go in the opposite direction); Market Price Following (this strategy bases the agent price fluctuations on the fluctuations of the market price).
Concerning the VPPs' operation, negotiations take place in some additional timings, namely in coalitions' formation and management. This type of negotiations provides players with the capabilities of achieving the most advantageous coalition contracts, both for the aggregator (VPP) and for the members (sellers and buyers). These negotiations take into account the players' characteristics, objectives, and goals, and allows them to get alternative deals to those they could get by negotiating exclusively on the market.
The different types of negotiation approached in MASCEM, the different types of markets implemented, and the distinct interactions between the participating entities in different situations, create the fundamental need for the use of machine learning techniques in this simulator.
ALBidS – Adaptive Learning strategic Bidding System
ALBidS is implemented as a multiagent system, in which there is one agent performing each distinct algorithm, detaining the exclusive knowledge of its execution. This way the system can be executing all the algorithms in parallel, preventing the system's performance degradation, in the possible amount. As each strategy agent gets its answer, it sends it to the Main Agent, which is responsible for choosing the most appropriate answer among all that it receives, depending on each context. Figure 5 presents the global structure of the ALBidS system, with the Main Agent as its central entity.
Figure 5 – ALBidS global structure.
Contexts are an important factor in what concerns the adaptation of the approaches to be chosen as the final action to be performed in the market by the supported player. A mechanism to analyse and define different market negotiating contexts provides the means for the chosen actions to be adapted and chosen depending of the different circumstances that are encountered at each moment.
The ALBidS system integrates several very distinct approaches in what concerns market negotiations, and prices forecast. The included techniques include: Neural Networks, data mining techniques, statistical approaches, machine learning algorithms, Game Theory for scenario analysis, competitor players' actions prediction, and approaches based on strategies used by other simulators for market analysis and costs forecasts.
In order to support some of these approaches, a competitor player's profile definition mechanism has been implemented, with the point of creating adequate player profiles to be used by the ALBidS strategies which require such profiles for their execution.
Additionally, a mechanism with the purpose of managing the balance between the efficiency and the effectiveness of the system has been developed so that the ALBidS system can be able to adapt to different simulation circumstances. This mechanism provides the means for the system to adapt its execution time depending on the purpose of the simulation, i.e., if the expected results from ALBidS are as best as it is able to achieve, or, on the other hand, if the main requirement is for the system to be executed rapidly, since the purpose of the considered simulation is to analyse issues other than player's optimal performance in the electricity market. The Efficiency/Effectiveness Management mechanism manipulates the strategies both externally and internally. From the system's perspective this mechanism contributes by deciding which tools are used at each moment for each circumstance; depending on their observed performance in terms of efficiency and effectiveness. This way this mechanism can choose to exclude certain strategies when they are not fulfilling the ALBidS' requirements for the case in matter. The strategies chosen to be executed are also manipulated internally, so that they can adapt their individual results quality/execution time balance to the needs of the current simulation.
Figure 6 – ALBidS integration with MASCEM.