Multi-agent Reinforcement Learning for Traffic Signal Control

Optimal control of traffic lights at junctions or traffic signal control (TSC) is essential for reducing the average delay experienced by the road users amidst the rapid increase in the usage of vehicles. In this paper, we formulate the TSC problem as a discounted cost Markov decision process (MDP) and apply multi-agent reinforcement learning (MARL) algorithms to obtain dynamic TSC policies. We model each traffic signal junction as an independent agent. An agent decides the signal duration of its phases in a round-robin (RR) manner using multi-agent Q-learning with either is an element of-greedy or UCB 3] based exploration strategies. It updates its Q-factors based on the cost feedback signal received from its neighbouring agents. This feedback signal can be easily constructed and is shown to be effective in minimizing the average delay of the vehicles in the network. We show through simulations over VISSIM that our algorithms perform significantly better than both the standard fixed signal timing (FST) algorithm and the saturation balancing (SAT) algorithm 15] over two real road networks.

Optical-Phonon-Limited High-Field Transport in Layered Materials

An optical-phonon-limited velocity model has been employed to investigate high-field transport in a selection of layered 2-D materials for both, low-power logic switches with scaled supply voltages, and high-power, high-frequency transistors. Drain currents, effective electron velocities, and intrinsic cutoff frequencies as a function of carrier density have been predicted, thus providing a benchmark for the optical-phonon-limited high-field performance limits of these materials. The optical-phonon-limited carrier velocities for a selection of multi-layers of transition metal dichalcogenides and black phosphorus are found to be modest compared to their n-channel silicon counterparts, questioning the utility of biasing these devices in the source-injection dominated regime. h-BN, at the other end of the spectrum, is shown to be a very promising material for high-frequency, high-power devices, subject to the experimental realization of high carrier densities, primarily due to its large optical-phonon energy. Experimentally extracted saturation velocities from few-layer MoS2 devices show reasonable qualitative and quantitative agreement with the predicted values. The temperature dependence of the measured v(sat) is discussed and compared with the theoretically predicted dependence over a range of temperatures.