Energy Management System for a Hybrid Electric Vehicle Using Reinforcement Learning

This study develops an energy management system (EMS) for an engine/motor hybrid electric vehicle (HEV) using reinforcement learning (RL) approach. Firstly, the vehicle is modeled on the basis of second-order dynamics, and featured by five major segments: A battery, a spark ignition engine, a lithium battery, transmission and vehicle dynamics, and a driver model. Subsequently, a four-mode rule-based (RB) control strategy is designed to allocate the power distribution of dual power sources of HEV. In addition, to improve the energy efficiency for the HEV, the RL method is further utilized to find the optimal power-split ratio according to the built model. Meanwhile, a state reduction mechanism is developed to decrease the state number and thereby reducing the computational complexity of the RL method. During the RL search, equivalent fuel consumption of the engine and motor is used as the fitness function. Suitable energy allocation between the gasoline engine and the battery pack can be determined by the Q-learning algorithm. To compare the energy control performances of the RB and RL, an equivalent consumption minimization strategy is evaluated as the best case. The simulation results including equivalent fuel consumption and CO2 emission verify that the proposed RL-based EMS searches the optimal solution more efficiently than the RB control for the engine/motor HEV.