In this article, we investigate the problem of real-time optimal control for end-users with integrated renewable energy generation and energy storage. The proposed a novel and comprehensive model consists of the electric vehicle (EV), renewable energy, the residential combined heat and power (resCHP) system and energy storage systems (e.g., battery and water tank). By jointly considering the operational constraints of energy storage, the stochastic behavior of renewable energy production and demand, and the volatility of electricity prices, the energy management problem is transformed into a stochastic optimization problem that minimizes the end-user's the long term time-averaged cost. First, to eliminate the effect of storage control coupling over time, the capacity constraints (physical constraints) of the stochastic optimization problem are relaxed to time-averaged constraints. Secondly, we solve this problem by designing an real-time online algorithm based on the improved Lyapunov optimization. The proposed algorithm has a significant feature that does not require any statistical information about the stochastic system inputs (e.g., the uncertain power and heat demand, the renewable energy production, and the time-varying electricity prices, and so on). Furthermore, the performance analysis of the proposed algorithm shows that it not only ensures that all control decisions are executed within the limited storage capacity, but also provides a near-optimal solution for the optimization problem. Finally, simulation results demonstrate that the proposed algorithm can reduce 74.4% of the total cost compared with the benchmark approach. (c) 2021 Elsevier Ltd. All rights reserved.