Multi-agent Reinforcement Learning for Traffic Signal Control

Optimal control of traffic lights at junctions or traffic signal control (TSC) is essential for reducing the average delay experienced by the road users amidst the rapid increase in the usage of vehicles. In this paper, we formulate the TSC problem as a discounted cost Markov decision process (MDP) and apply multi-agent reinforcement learning (MARL) algorithms to obtain dynamic TSC policies. We model each traffic signal junction as an independent agent. An agent decides the signal duration of its phases in a round-robin (RR) manner using multi-agent Q-learning with either is an element of-greedy or UCB 3] based exploration strategies. It updates its Q-factors based on the cost feedback signal received from its neighbouring agents. This feedback signal can be easily constructed and is shown to be effective in minimizing the average delay of the vehicles in the network. We show through simulations over VISSIM that our algorithms perform significantly better than both the standard fixed signal timing (FST) algorithm and the saturation balancing (SAT) algorithm 15] over two real road networks.

QoS Performance Analysis of Routing Protocols in Vehicular Ad-hoc Networks

Vehicular Ad-hoc Networks (VANET), is a type of wireless ad-hoc network that aims to provide communication among vehicles. A key characteristic of VANETs is the very high mobility of nodes that result in a frequently changing topology along with the frequent breakage and linkage of the paths among the nodes involved. These characteristics make the Quality of Service (QoS) requirements in VANET a challenging issue. In this paper we characterize the performance available to applications in infrastructureless VANETs in terms of path holding time, path breakage probability and per session throughput as a function of various vehicle densities on road, data traffic rate and number of connections formed among vehicles by making use of table-driven and on-demand routing algorithms. Several QoS constraints in the applications of infrastructureless VANETs are observed in the results obtained.

Investigation of spatial repeatability using a tire force measuring mat

A portable mat for measuring the dynamic tire forces of commercial vehicles is described. The mat is 56 m long and 13 mm thick and has 141 capacitative strip sensors spaced at 0.4-m intervals. The accuracy of the mat for measuring dynamic tire forces generated by heavy commercial vehicles is assessed using an instrumented vehicle. The spatial repeatability of dynamic wheel loads generated by 14 uninstrumented articulated vehicles is investigated, and it is concluded that approximately half of the vehicles tested are likely to contribute to a repeatable pattern of road loading.

Research Progress On Thermal Protection Materials And Structures Of Hypersonic Vehicles

Hypersonic vehicles represent future trends of military equipments and play an important role in future war. Thermal protection materials and structures, which relate to the safety of hypersonic vehicles, are one of the most key techniques in design and manufacture of hypersonic vehicles. Among these materials and structures, such as metallic temperature protection structure, the temperature ceramics and carbon/carbon composites are usually adopted in design. The recent progresses of research and application of ultra-high temperature materials in preparation, oxidation resistance, mechanical and physical characterization are summarized.