Deep Reinforcement Learning for Grid-interactive Energy-Efficient Buildings

Lead Performer: National Renewable Energy Laboratory – Golden, CO
Partners:
-- University of Colorado at Boulder – Boulder, CO
-- QCoefficient – Chicago, IL
-- Heila Technologies -- Somerville, MA
DOE Total Funding: $1,500,000
Project Term: July 1, 2019 – June 30, 2022
Funding Type: BENEFIT 2018 Funding Opportunity Announcement

Project Objective

Commercial buildings must become grid-interactive intelligent elements that can flexibly participate in grid-level operations to enhance resiliency of our national grid, while addressing the needs of occupants and economic objectives. This is particularly important in view of the ongoing transformation of the power distribution grid to an active network with volatile distributed energy resources. Recent advances in building operation, enabled by the model predictive control (MPC) framework, have focused on owner-centric objectives such as minimizing operating cost, energy use, or occupant discomfort. MPC algorithms typically require a knowledge of a building model and suffer from sensitivity to model inaccuracies, therefore necessitating extensive engineering time and effort for model development and tuning. Moreover, they are not easily replicable to many buildings and might suffer from scalability and stability issues.

Therefore, a better approach is needed to intelligent building control and operation that achieves both algorithmic scalability and deployment repeatability to finally unlock the potential our building stock offers, rather than require building-by-building customization. This project will overcome the shortcomings of the state-of-the-art by leveraging the emerging deep reinforcement learning (RL) paradigm. It will extend previous work on deep RL as well as previous efforts in the literature on RL control systems for buildings and will develop scalable multi-objective RL algorithms that will simultaneously address both building-centric and grid- serving objectives. All this will be achieved without the need of detailed and accurate building models by virtue of the RL methodology and will be replicable to heterogeneous buildings. The algorithms will be tested in simulation and compared to the MPC algorithms that are based on detailed and accurate models.

Project Impact

This project will develop an innovative machine learning control system to accomplish the following goals: learn the spatiotemporal patterns of building occupancy and building response to applied control actions; operate with high energy efficiency; maximize occupant comfort when and where occupied; participate in grid operations to increase grid stability and resiliency; and increase survivability by supplying critical assets during natural disasters and severe weather events.

Contacts

DOE Technology Manager: Erika Gupta
Lead Performer: Andrey Bernstein, National Renewable Energy Laboratory

Related Publications