Q-learning Algorithm for Energy Management in Solar Powered Embedded Monitoring Systems

Environmental changes have become a considerable issue over the past years. To be able to continuously monitor such variations in the environment, it is convenient to employ so-called Environmental Monitoring Systems (EMS) that can be deployed in remote places of interest, and that are capable of measuring multiple ambient characteristics. However, supplying EMS with power in a long term to prevent blackouts is challenging. This paper introduces an energy management method based on Reinforcement Learning algorithm, particularly Q-learning. The EMS described in this study needs to meet a set of strict requirements, e.g. low power consumption, high reliability, self-sustainability with respect to power supply, backup of data acquired through the measurements in case of an unexpected failure and many others. The energy is harvested using solar panels and stored in supercapacitors. In addition, the implementation of a complex algorithm is not suitable for such a system, considering the energy constraints. The solution to the above mentioned challenges is described in this study. The findings were implemented in a physical EMS device, and subsequently tested in field so as to acquire real-life data.