Hierarchical traffic light-aware routing via fuzzy reinforcement learning in software-defined vehicular networks

Lack of a fully vehicular topology view and restricted vehicles' movement to streets with the time-varying traffic light conditions have caused drastic gaps in the traditional vehicular routing protocols. Using software-defined networks (SDN), this paper proposes HIFS, a Hierarchical Intersection-based routing strategy that incorporates Fuzzy SARSA reinforcement learning to fill these gaps. At the first level of our HIFS scheme, a utility-based intersections selection policy is presented using fuzzy logic that jointly considers delay estimation, curve distance, and predicted of moving vehicles towards intersections. Then, a fuzzy logic-based path selection policy is proposed to choose the paths with highest flexibility against the intermittent connectivity and increased traffic loads. Residual bandwidth, Euclidean distance, angular orientation, and congestion are considered inputs of the fuzzy logic system. Meanwhile, traffic light states and nodes' information are used to tune the output fuzzy membership functions via reinforcement learning algorithm. The efficiency of our scheme in controlling ambiguity and uncertainty of the vehicular environment is confirmed through simulations in various vehicle densities and different traffic lights duration. Simulation results of average gains obtained for both scenarios show that our HIFS scheme increases the packet delivery ratio on average by 48.75%, 54.63%, and 8.78%, increases the throughput by 48.66%, 53.79%, and 8.61%, reduces end-to-end delay by 33.35%, 46.14%, and 15.38%, reduces the path length by 25.25%, 36.47%, and 15.32%, and reduces normalized routing overhead by 37.09%, 49.79%, and 20.17%, compared to MISR, ITAR-FQ, and GLS methods, respectively.