Adaptive Sleep-Wake Control using Reinforcement Learning in Sensor Networks

The aim in this paper is to allocate the `sleep time' of the individual sensors in an intrusion detection application so that the energy consumption from the sensors is reduced, while keeping the tracking error to a minimum. We propose two novel reinforcement learning (RL) based algorithms that attempt to minimize a certain long-run average cost objective. Both our algorithms incorporate feature-based representations to handle the curse of dimensionality associated with the underlying partially-observable Markov decision process (POMDP). Further, the feature selection scheme used in our algorithms intelligently manages the energy cost and tracking cost factors, which in turn assists the search for the optimal sleeping policy. We also extend these algorithms to a setting where the intruder's mobility model is not known by incorporating a stochastic iterative scheme for estimating the mobility model. The simulation results on a synthetic 2-d network setting are encouraging.

QoS Performance Analysis of Routing Protocols in Vehicular Ad-hoc Networks

Vehicular Ad-hoc Networks (VANET), is a type of wireless ad-hoc network that aims to provide communication among vehicles. A key characteristic of VANETs is the very high mobility of nodes that result in a frequently changing topology along with the frequent breakage and linkage of the paths among the nodes involved. These characteristics make the Quality of Service (QoS) requirements in VANET a challenging issue. In this paper we characterize the performance available to applications in infrastructureless VANETs in terms of path holding time, path breakage probability and per session throughput as a function of various vehicle densities on road, data traffic rate and number of connections formed among vehicles by making use of table-driven and on-demand routing algorithms. Several QoS constraints in the applications of infrastructureless VANETs are observed in the results obtained.