Simulation-Based Policy Improvement for Energy Management in Commercial Office Buildings

Many commercial office buildings have become micro smart grids with on-site power generations, storage devices, and uncertain demands. Due to the pervasive nonlinearity and randomness of such a multi-energy system, simulation is usually the only faithful way to accurately describe the system dynamics and for performance evaluation. However, simulation is usually time-consuming and each sample path provides only noisy observations. Thus finding the optimal energy management policy is nontrivial. In this paper, a joint schedule problem is considered to schedule solar power, wind power, combined cooling, heating, and power generation, high temperature chiller, liquid desiccant fresh air unit, battery, and power grid in order to satisfy the electricity load, sensible heat load, and latent heat load in buildings with the minimal expected cost. We make two major contributions in this paper. First, three simulation-based policy improvement (SBPI) methods are developed to improve from given base policies. Second, the performance of these methods are systematically analyzed through numerical experiments. We show that when there are sufficient computing budget, the SBPI methods improve the given base policies. Different methods are recommended for problems with different computing budget. Sensitivity analysis of the policy and the value of accurate information are also discussed.