Multi-agent Reinforcement Learning for Traffic Signal Control

Optimal control of traffic lights at junctions or traffic signal control (TSC) is essential for reducing the average delay experienced by the road users amidst the rapid increase in the usage of vehicles. In this paper, we formulate the TSC problem as a discounted cost Markov decision process (MDP) and apply multi-agent reinforcement learning (MARL) algorithms to obtain dynamic TSC policies. We model each traffic signal junction as an independent agent. An agent decides the signal duration of its phases in a round-robin (RR) manner using multi-agent Q-learning with either is an element of-greedy or UCB 3] based exploration strategies. It updates its Q-factors based on the cost feedback signal received from its neighbouring agents. This feedback signal can be easily constructed and is shown to be effective in minimizing the average delay of the vehicles in the network. We show through simulations over VISSIM that our algorithms perform significantly better than both the standard fixed signal timing (FST) algorithm and the saturation balancing (SAT) algorithm 15] over two real road networks.

Probabilistic Prediction based Scheduling for Delay Sensitive Traffic in Internet of Things

This paper proposes a probabilistic prediction based approach for providing Quality of Service (QoS) to delay sensitive traffic for Internet of Things (IoT). A joint packet scheduling and dynamic bandwidth allocation scheme is proposed to provide service differentiation and preferential treatment to delay sensitive traffic. The scheduler focuses on reducing the waiting time of high priority delay sensitive services in the queue and simultaneously keeping the waiting time of other services within tolerable limits. The scheme uses the difference in probability of average queue length of high priority packets at previous cycle and current cycle to determine the probability of average weight required in the current cycle. This offers optimized bandwidth allocation to all the services by avoiding distribution of excess resources for high priority services and yet guaranteeing the services for it. The performance of the algorithm is investigated using MPEG-4 traffic traces under different system loading. The results show the improved performance with respect to waiting time for scheduling high priority packets and simultaneously keeping tolerable limits for waiting time and packet loss for other services. Crown Copyright (C) 2015 Published by Elsevier B.V.

Primates, Provisioning and Plants: Impacts of Human Cultural Behaviours on Primate Ecological Functions

Human provisioning of wildlife with food is a widespread global practice that occurs in multiple socio-cultural circumstances. Provisioning may indirectly alter ecosystem functioning through changes in the eco-ethology of animals, but few studies have quantified this aspect. Provisioning of primates by humans is known to impact their activity budgets, diets and ranging patterns. Primates are also keystone species in tropical forests through their role as seed dispersers; yet there is no information on how provisioning might affect primate ecological functions. The rhesus macaque is a major human-commensal species but is also an important seed disperser in the wild. In this study, we investigated the potential impacts of provisioning on the role of rhesus macaques as seed dispersers in the Buxa Tiger Reserve, India. We studied a troop of macaques which were provisioned for a part of the year and were dependent on natural resources for the rest. We observed feeding behaviour, seed handling techniques and ranging patterns of the macaques and monitored availability of wild fruits. Irrespective of fruit availability, frugivory and seed dispersal activities decreased when the macaques were provisioned. Provisioned macaques also had shortened daily ranges implying shorter dispersal distances. Finally, during provisioning periods, seeds were deposited on tarmac roads that were unconducive for germination. Provisioning promotes human-primate conflict, as commensal primates are often involved in aggressive encounters with humans over resources, leading to negative consequences for both parties involved. Preventing or curbing provisioning is not an easy task as feeding wild animals is a socio-cultural tradition across much of South and South-East Asia, including India. We recommend the initiation of literacy programmes that educate lay citizens about the ill-effects of provisioning and strongly caution them against the practice.

RD-TDMA: A Randomized and Distributed TDMA Scheduling for Correlated Contention in WSNs

In wireless sensor networks (WSNs), contention occurs when two or more nodes in a proximity simultaneously try to access the channel. The contention causes collisions, which are very likely to occur when traffic is correlated. The excessive collision not only affects the reliability and the QoS of the application, but also the lifetime of the network. It is well-known that random access mechanisms do not efficiently handle correlated-contention, and therefore, suffer from high collision rate. Most of the existing TDMA scheduling techniques try to find an optimal or a sub-optimal schedule. Usually, the situation of correlated-contention persists only for a short duration, and therefore, it is not worthwhile to take a long time to generate an optimal or a sub-optimal schedule. We propose a randomized distributed TDMA scheduling (RD-TDMA) algorithm to quickly generate a feasible schedule (not necessarily optimal) to handle correlated-contention in WSNs. In RD-TDMA, a node in the network negotiates a slot with its neighbors using the message exchange mechanism. The proposed protocol has been simulated using the Castalia simulator to evaluate its runtime performance. Simulation results show that the RD-TDMA algorithm considerably reduces the time required to schedule.

A survey on compressed domain video analysis techniques

Image and video analysis requires rich features that can characterize various aspects of visual information. These rich features are typically extracted from the pixel values of the images and videos, which require huge amount of computation and seldom useful for real-time analysis. On the contrary, the compressed domain analysis offers relevant information pertaining to the visual content in the form of transform coefficients, motion vectors, quantization steps, coded block patterns with minimal computational burden. The quantum of work done in compressed domain is relatively much less compared to pixel domain. This paper aims to survey various video analysis efforts published during the last decade across the spectrum of video compression standards. In this survey, we have included only the analysis part, excluding the processing aspect of compressed domain. This analysis spans through various computer vision applications such as moving object segmentation, human action recognition, indexing, retrieval, face detection, video classification and object tracking in compressed videos.

A Perspective on Computer Assisted Assessment Techniques for Short Free-Text Answers

Computer Assisted Assessment (CAA) has been existing for several years now. While some forms of CAA do not require sophisticated text understanding (e.g., multiple choice questions), there are also student answers that consist of free text and require analysis of text in the answer. Research towards the latter till date has concentrated on two main sub-tasks: (i) grading of essays, which is done mainly by checking the style, correctness of grammar, and coherence of the essay and (ii) assessment of short free-text answers. In this paper, we present a structured view of relevant research in automated assessment techniques for short free-text answers. We review papers spanning the last 15 years of research with emphasis on recent papers. Our main objectives are two folds. First we present the survey in a structured way by segregating information on dataset, problem formulation, techniques, and evaluation measures. Second we present a discussion on some of the potential future directions in this domain which we hope would be helpful for researchers.

Comparison of Matching Pursuit Algorithm with Other Signal Processing Techniques for Computation of the Time-Frequency Power Spectrum of Brain Signals

Signals recorded from the brain often show rhythmic patterns at different frequencies, which are tightly coupled to the external stimuli as well as the internal state of the subject. In addition, these signals have very transient structures related to spiking or sudden onset of a stimulus, which have durations not exceeding tens of milliseconds. Further, brain signals are highly nonstationary because both behavioral state and external stimuli can change on a short time scale. It is therefore essential to study brain signals using techniques that can represent both rhythmic and transient components of the signal, something not always possible using standard signal processing techniques such as short time fourier transform, multitaper method, wavelet transform, or Hilbert transform. In this review, we describe a multiscale decomposition technique based on an over-complete dictionary called matching pursuit (MP), and show that it is able to capture both a sharp stimulus-onset transient and a sustained gamma rhythm in local field potential recorded from the primary visual cortex. We compare the performance of MP with other techniques and discuss its advantages and limitations. Data and codes for generating all time-frequency power spectra are provided.

Nano-hardness Techniques and Its Applications in Mechanical Property Measurements for Surface Engineering

结合纳米硬度技术测量各类薄膜和块体材料表层的纳米压痕硬度、弹性模量、断裂韧性、膜厚、微结构的弯曲变形,采用纳米划痕硬度技术测量各类薄膜和块体材料的粗糙度、临界附着力、摩擦系数、划痕横剖面.纳米硬度计是检测材料表层微米乃至几十纳米力学性能的先进仪器,可广泛应用于表面工程中的质量检测.

Simple lattice Boltzmann model for traffic flows

A lattice Boltzmann model with 5-bit lattice for traffic flows is proposed. Using the Chapman-Enskog expansion and multi-scale technique, we obtain the higher-order moments of equilibrium distribution function. A simple traffic light problem is simulated by using the present lattice Boltzmann model, and the result agrees well with analytical solution.