Multi-agent optimization and learning methods for sustainable, smart and resilient power distribution systems and microgrids

Thumbnail Image
Zhang, Qianzhi
Major Professor
Wang, Zhaoyu
McCalley, James
Ajjarapu, Venkataramana
Dobson, Ian
Hu, Chao
Committee Member
Journal Title
Journal ISSN
Volume Title
Research Projects
Organizational Units
Journal Issue
Is Version Of
Electrical and Computer Engineering
The new computation and communication technologies in smart grid can be applied to boost economic efficiency, guarantee reliable operation and enhance system resilient of active distribution systems. The planning and operation models of modern power systems are becoming increasingly more complex, as the increasing size of grid and number of controllable devices. Some particular areas, such as volt/var control and power/energy management, that have received significant attention over the last decade. For example, co-optimization of smart inverters with fast-dispatch and conventional voltage regulation devices with slow-dispatch for voltage regulation, or coordination of networked microgrids (MGs) with costumer privacy maintenance and safety guarantee. Above problems can be directly solved by centralized solvers, which naturally requires collection of all the necessary data, reliable communication to a control center,solution to a large-scale centralized optimization problem, and communication of control signals back to all the active elements. Multi-agent (distributed) control, on the other hand, only needs to collect partial local data, solve much smaller optimization problems by decomposing a large-scale problem, and exchange the information with neighboring regions at the local controller level. Thus, distributed optimization could effectively handle complexities that cannot be managed centrally. Recently, the comparison between model-based optimization methods and model-free machine learning methods also has been paid attention by power and energy system area. Model-based optimization methods have to solve large-scale optimization problems with numerous nonlinear constraints that incur high computational costs and hinder real-time decision making. Furthermore, model-based methods are unable to adapt to the continuously evolving system conditions, as they need to re-solve the problem at each time step. To address the limitations of model-based methods, model-free reinforcement learning (RL) techniques have been used to solve the optimal power management problem through repeated interactions between a control agent and its environment. This approach eliminates the need to solve a large-scale optimization problem at each time point and enables the control agent to provide adaptive response to time-varying system states. In summary, my doctoral thesis focus on exploring and developing advanced distributed optimization algorithms and multi-agent reinforcement learning-based methods to solve various decision-making operational problems in power distribution systems and autonomous networked MGs under different operation conditions, including the following three parts: (1) Distributed optimization for volt/var control in distribution systems with high penetration of renewable generation: Inspired by recent advances in distributed optimization algorithms and multi-agent control framework, we have done the following works: (i) To alleviate the computational burden in large scale distribution networks and maintain customer data privacy, we propose a distributed optimization model to coordinate the fast dispatch of smart inverters with the slow dispatch of conventional voltage regulation devices for implementing conservation voltage reduction in unbalanced three-phase distribution systems. (ii) To further speed up the solution algorithm, we propose an online feedback-based linear approximation method. The instantaneous power and voltage measurements are used as system feedback in each iteration of the distributed algorithm to linearize the nonlinear terms of power flow and ZIP load models. (iii) To model a more practical distribution system with integration of medium-voltage primary networks and low-voltage secondary networks, we further modify the distributed algorithm by mapping the primary and secondary networks into a leader-follower distributed control framework. (2) Multi-agent safe RL for power management of networked MGs: The coordination of multiple MGs can offer various benefits, such as enhancing power system resilience and reliability. By taking advantage of model-free and online features of RL, we develop several RL-based methods to solve the power management problem of networked MGs: (i) The distribution system operators may only have limited knowledge of the detailed models behind the point of common coupling. This impedes conventional optimization for the constrained MG power management problem. We propose a bi-level cooperative framework using a RL-based method for a distribution system consisting of multiple networked privately-owned MGs. A non-profit cooperative RL agent maximizes the total revenues at a higher level by setting a retail power price signal. By considering the model-free nature of our RL-based method, the data privacy of MGs and the data confidentiality of customers are maintained. At a lower level, each MG agent receives the price signals from the higher level and solves a constrained mixed-integer nonlinear programming to dispatch their local generation and energy storage system units. (ii) We propose a supervised multi-agent safe RL method for optimal power management of networked MGs. Accordingly, the training process employs the gradient information of operational constraints to ensure that the optimal control policy functions generate safe and feasible decisions. Furthermore, we have developed a distributed consensus-based optimization approach to train the policy functions of agents, while maintaining MGs privacy and data ownership boundaries. The purpose of this work is to leverage the advantages of both model-free and model-based methods for scalable real-time decision-making while also maintaining a user-defined level of safety by considering constraints in the training process. (3) Pre-event preparation and post-event service restoration for resilience enhancement of distribution systems: Extreme weather events are common causes for power supply interpretations and power outages in distribution systems. To enhance the resilience of distribution systems, we have done some works about the pre-event preparation model and post-event system restoration: (i) Before extreme weather events, preparation and allocation of multiple flexible resources can mitigate the damage effects of extreme weather events on distribution systems. We propose a two-stage stochastic mixed-integer linear programming to optimize the preparation and resource allocation process for upcoming extreme weather events. The first stage in the optimization problem selects the resources and their locations. The second stage considers the constraints of the distribution system operation problem and repair crew scheduling problem. (ii) After extreme events, the damaged distribution system can be sectionalized into several isolated MGs to restore critical loads and tripped non-black start distributed generations by black start distributed generations. However, the high penetration of inverter-based distributed generations reduces the system inertia, which results in low-inertia issues and significant frequency fluctuation during the restoration process. To address this challenge, we propose a two-level simulation-assisted sequential service restoration model for unbalanced distribution systems and inverter-dominated MGs, which includes a mixed-integer linear programming-based optimization model and a transient simulation model. The proposed service restoration model explicitly incorporates the frequency response into constraints by interfacing with the transient simulation of inverter-dominated MGs.
Subject Categories