Collaborative Research: CPS: Medium: Empowering Prosumers in Electricity Markets Through Market Design and Learning
Grant
Overview
Affiliation
View All
Overview
abstract
The availability of vast amounts of operational and end-user data in cyber-physical systems implies that paradigm improvements in monitoring and control can be attained via learning by many artificial intelligence agents despite them possessing vastly different abilities. Engaging this heterogeneous agent base in the context of the smart grid requires the use of hierarchical markets, wherein end-users participate in downstream markets collectively through aggregators, who in turn are coordinated by an upstream market. The goal of this project is to conduct a systematic study of such market-mediated learning and control. This project aims at much deeper levels of participation from end-users contributing electricity generation such as rooftop solar, shedding load via demand response, and providing storage capabilities such as electric vehicle batteries, to transform into reliable distributed energy resources (DER) at the level of wholesale markets. A methodological theme is multi-agent reinforcement learning (MARL) by agents that control physical systems via actions at different levels of the hierarchy. Underlying the whole project are well-founded physical models of the transmission and distribution grids, which provide structure to the problem domain and concrete use cases. This project facilitates a deeper level of decarbonization in the electricity sector, and contributes to climate change solutions by engineering a flat, interactive grid architecture that allows significant DERs to provide electricity services to both local and regional grids. Engagement with a grid-level market operator enables the project to address a problem space of immediate relevance to the current electricity grid. The project also includes the development of educational materials on data-analytics and energy systems. Intrinsic to the program are efforts at outreach to involve high-school students via demonstrations and lectures based on the technology developed. The goal of this project is a systematic and principled study of methods for hierarchical market-mediated learning and control, with the electric grid being the primary application domain. Multi-agent reinforcement learning (MARL) runs as a common methodological theme through the project, with strategic agents with varying information structures and concepts of rationality that control physical systems via actions at different levels of the hierarchy. The approach is different from studies on generic MARL algorithms in that attention is focused on well-founded physical models of the transmission and distribution grids, as well as the workings of the power system. The project is organized into three interdependent thrusts, namely, (i) Learning to bid as aggregators in wholesale markets, which studies dynamics of aggregators that provide supply offers and demand bids at the upstream market (wholesale level), while procuring these services from downstream DERs (retail level), (ii) Learning to incentivize retail users to contribute their resources, under which bounded rational agents learn to respond to a population-level distribution of other agents and incentives provided, and (iii) Evaluation and experimentation over a full-scale system emulator by integrating it with reinforcement learning tools. This project provides an architecture for DERs to provide electricity services to both local and regional grids, and hence contributes to developing solutions to climate change. Engagement with an independent system operator enables a focus on grid-specific issues, ensuring the applicability of the solutions to real-world problems. The impact is enhanced by specific minority inclusion activities, courses on computing tailored to broaden participation in the context of data-analytics and energy systems, and outreach to high-school students using demonstrations and lectures based on the project results. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
