Optimal sleep-wake scheduling for quickest intrusion detection using sensor networks

We consider the problem of quickest detection of an intrusion using a sensor network, keeping only a minimal number of sensors active. By using a minimal number of sensor devices, we ensure that the energy expenditure for sensing, computation and communication is minimized (and the lifetime of the network is maximized). We model the intrusion detection (or change detection) problem as a Markov decision process (MDP). Based on the theory of MDP, we develop the following closed loop sleep/wake scheduling algorithms: (1) optimal control of Mk+1, the number of sensors in the wake state in time slot k + 1, (2) optimal control of qk+1, the probability of a sensor in the wake state in time slot k + 1, and an open loop sleep/wake scheduling algorithm which (3) computes q, the optimal probability of a sensor in the wake state (which does not vary with time), based on the sensor observations obtained until time slot k. Our results show that an optimum closed loop control on Mk+1 significantly decreases the cost compared to keeping any number of sensors active all the time. Also, among the three algorithms described, we observe that the total cost is minimum for the optimum control on Mk+1 and is maximum for the optimum open loop control on q.

Optimal Sleep-Wake Policies for an Energy Harvesting Sensor Node

We study a sensor node with an energy harvesting source. In any slot,the sensor node is in one of two modes: Wake or Sleep. The generated energy is stored in a buffer. The sensor node senses a random field and generates a packet when it is awake. These packets are stored in a queue and transmitted in the wake mode using the energy available in the energy buffer. We obtain energy management policies which minimize a linear combination of the mean queue length and the mean data loss rate. Then, we obtain two easily implementable suboptimal policies and compare their performance to that of the optimal policy. Next, we extend the Throughput Optimal policy developed in our previous work to sensors with two modes. Via this policy, we can increase the through put substantially and stabilize the data queue by allowing the node to sleep in some slots and to drop some generated packets. This policy requires minimal statistical knowledge of the system. We also modify this policy to decrease the switching costs.

Tunable Locally-Optimal Geographical Forwarding in Wireless Sensor Networks with Sleep-Wake Cycling Nodes

We consider a wireless sensor network whose main function is to detect certain infrequent alarm events, and to forward alarm packets to a base station, using geographical forwarding. The nodes know their locations, and they sleep-wake cycle, waking up periodically but not synchronously. In this situation, when a node has a packet to forward to the sink, there is a trade-off between how long this node waits for a suitable neighbor to wake up and the progress the packet makes towards the sink once it is forwarded to this neighbor. Hence, in choosing a relay node, we consider the problem of minimizing average delay subject to a constraint on the average progress. By constraint relaxation, we formulate this next hop relay selection problem as a Markov decision process (MDP). The exact optimal solution (BF (Best Forward)) can be found, but is computationally intensive. Next, we consider a mathematically simplified model for which the optimal policy (SF (Simplified Forward)) turns out to be a simple one-step-look-ahead rule. Simulations show that SF is very close in performance to BF, even for reasonably small node density. We then study the end-to-end performance of SF in comparison with two extremal policies: Max Forward (MF) and First Forward (FF), and an end-to-end delay minimising policy proposed by Kim et al. 1]. We find that, with appropriate choice of one hop average progress constraint, SF can be tuned to provide a favorable trade-off between end-to-end packet delay and the number of hops in the forwarding path.

Optimal Sleep-Wake Scheduling for Quickest Intrusion Detection using wireless sensor Networks

We consider the problem of quickest detection of an intrusion using a sensor network, keeping only a minimal number of sensors active. By using a minimal number of sensor devices,we ensure that the energy expenditure for sensing, computation and communication is minimized (and the lifetime of the network is maximized). We model the intrusion detection (or change detection) problem as a Markov decision process (MDP). Based on the theory of MDP, we develop the following closed loop sleep/wake scheduling algorithms: 1) optimal control of Mk+1, the number of sensors in the wake state in time slot k + 1, 2) optimal control of qk+1, the probability of a sensor in the wake state in time slot k + 1, and an open loop sleep/wake scheduling algorithm which 3) computes q, the optimal probability of a sensor in the wake state (which does not vary with time),based on the sensor observations obtained until time slot k.Our results show that an optimum closed loop control onMk+1 significantly decreases the cost compared to keeping any number of sensors active all the time. Also, among the three algorithms described, we observe that the total cost is minimum for the optimum control on Mk+1 and is maximum for the optimum open loop control on q.

Relay Selection for Geographical Forwarding in Sleep-Wake Cycling Wireless Sensor Networks

Our work is motivated by geographical forwarding of sporadic alarm packets to a base station in a wireless sensor network (WSN), where the nodes are sleep-wake cycling periodically and asynchronously. We seek to develop local forwarding algorithms that can be tuned so as to tradeoff the end-to-end delay against a total cost, such as the hop count or total energy. Our approach is to solve, at each forwarding node enroute to the sink, the local forwarding problem of minimizing one-hop waiting delay subject to a lower bound constraint on a suitable reward offered by the next-hop relay; the constraint serves to tune the tradeoff. The reward metric used for the local problem is based on the end-to-end total cost objective (for instance, when the total cost is hop count, we choose to use the progress toward sink made by a relay as the reward). The forwarding node, to begin with, is uncertain about the number of relays, their wake-up times, and the reward values, but knows the probability distributions of these quantities. At each relay wake-up instant, when a relay reveals its reward value, the forwarding node's problem is to forward the packet or to wait for further relays to wake-up. In terms of the operations research literature, our work can be considered as a variant of the asset selling problem. We formulate our local forwarding problem as a partially observable Markov decision process (POMDP) and obtain inner and outer bounds for the optimal policy. Motivated by the computational complexity involved in the policies derived out of these bounds, we formulate an alternate simplified model, the optimal policy for which is a simple threshold rule. We provide simulation results to compare the performance of the inner and outer bound policies against the simple policy, and also against the optimal policy when the source knows the exact number of relays. Observing the good performance and the ease of implementation of the simple policy, we apply it to our motivating problem, i.e., local geographical routing of sporadic alarm packets in a large WSN. We compare the end-to-end performance (i.e., average total delay and average total cost) obtained by the simple policy, when used for local geographical forwarding, against that obtained by the globally optimal forwarding algorithm proposed by Kim et al. 1].

A study on fear memory retrieval and REM sleep in maternal separation and isolation stressed rats

As rapid brain development occurs during the neonatal period, environmental manipulation during this period may have a significant impact on sleep and memory functions. Moreover, rapid eye movement (REM) sleep plays an important role in integrating new information with the previously stored emotional experience. Hence, the impact of early maternal separation and isolation stress (MS) during the stress hyporesponsive period (SHRP) on fear memory retention and sleep in rats were studied. The neonatal rats were subjected to maternal separation and isolation stress during postnatal days 5-7 (6 h daily/3 d). Polysomnographic recordings and differential fear conditioning was carried out in two different sets of rats aged 2 months. The neuronal replay during REM sleep was analyzed using different parameters. MS rats showed increased time in REM stage and total sleep period also increased. MS rats showed fear generalization with increased fear memory retention than normal control (NC). The detailed analysis of the local field potentials across different time periods of REM sleep showed increased theta oscillations in the hippocampus, amygdala and cortical circuits. Our findings suggest that stress during SHRP has sensitized the hippocampus amygdala cortical loops which could be due to increased release of corticosterone that generally occurs during REM sleep. These rats when subjected to fear conditioning exhibit increased fear memory and increased, fear generalization. The development of helplessness, anxiety and sleep changes in human patients, thus, could be related to the reduced thermal, tactile and social stimulation during SHRP on brain plasticity and fear memory functions. (C) 2014 Elsevier B.V. All rights reserved.

Two timescale convergent Q-learning for sleep-scheduling in wireless sensor networks

In this paper, we consider an intrusion detection application for Wireless Sensor Networks. We study the problem of scheduling the sleep times of the individual sensors, where the objective is to maximize the network lifetime while keeping the tracking error to a minimum. We formulate this problem as a partially-observable Markov decision process (POMDP) with continuous stateaction spaces, in a manner similar to Fuemmeler and Veeravalli (IEEE Trans Signal Process 56(5), 2091-2101, 2008). However, unlike their formulation, we consider infinite horizon discounted and average cost objectives as performance criteria. For each criterion, we propose a convergent on-policy Q-learning algorithm that operates on two timescales, while employing function approximation. Feature-based representations and function approximation is necessary to handle the curse of dimensionality associated with the underlying POMDP. Our proposed algorithm incorporates a policy gradient update using a one-simulation simultaneous perturbation stochastic approximation estimate on the faster timescale, while the Q-value parameter (arising from a linear function approximation architecture for the Q-values) is updated in an on-policy temporal difference algorithm-like fashion on the slower timescale. The feature selection scheme employed in each of our algorithms manages the energy and tracking components in a manner that assists the search for the optimal sleep-scheduling policy. For the sake of comparison, in both discounted and average settings, we also develop a function approximation analogue of the Q-learning algorithm. This algorithm, unlike the two-timescale variant, does not possess theoretical convergence guarantees. Finally, we also adapt our algorithms to include a stochastic iterative estimation scheme for the intruder's mobility model and this is useful in settings where the latter is not known. Our simulation results on a synthetic 2-dimensional network setting suggest that our algorithms result in better tracking accuracy at the cost of only a few additional sensors, in comparison to a recent prior work.

Adaptive Sleep-Wake Control using Reinforcement Learning in Sensor Networks

The aim in this paper is to allocate the `sleep time' of the individual sensors in an intrusion detection application so that the energy consumption from the sensors is reduced, while keeping the tracking error to a minimum. We propose two novel reinforcement learning (RL) based algorithms that attempt to minimize a certain long-run average cost objective. Both our algorithms incorporate feature-based representations to handle the curse of dimensionality associated with the underlying partially-observable Markov decision process (POMDP). Further, the feature selection scheme used in our algorithms intelligently manages the energy cost and tracking cost factors, which in turn assists the search for the optimal sleeping policy. We also extend these algorithms to a setting where the intruder's mobility model is not known by incorporating a stochastic iterative scheme for estimating the mobility model. The simulation results on a synthetic 2-d network setting are encouraging.

Relay Selection with Channel Probing in Sleep-Wake Cycling Wireless Sensor Networks

In geographical forwarding of packets in a large wireless sensor network (WSN) with sleep-wake cycling nodes, we are interested in the local decision problem faced by a node that has ``custody'' of a packet and has to choose one among a set of next-hop relay nodes to forward the packet toward the sink. Each relay is associated with a ``reward'' that summarizes the benefit of forwarding the packet through that relay. We seek a solution to this local problem, the idea being that such a solution, if adopted by every node, could provide a reasonable heuristic for the end-to-end forwarding problem. Toward this end, we propose a local relay selection problem consisting of a forwarding node and a collection of relay nodes, with the relays waking up sequentially at random times. At each relay wake-up instant, the forwarder can choose to probe a relay to learn its reward value, based on which the forwarder can then decide whether to stop (and forward its packet to the chosen relay) or to continue to wait for further relays to wake up. The forwarder's objective is to select a relay so as to minimize a combination of waiting delay, reward, and probing cost. The local decision problem can be considered as a variant of the asset selling problem studied in the operations research literature. We formulate the local problem as a Markov decision process (MDP) and characterize the solution in terms of stopping sets and probing sets. We provide results illustrating the structure of the stopping sets, namely, the (lower bound) threshold and the stage independence properties. Regarding the probing sets, we make an interesting conjecture that these sets are characterized by upper bounds. Through simulation experiments, we provide valuable insights into the performance of the optimal local forwarding and its use as an end-to-end forwarding heuristic.

A sleep-inducing peptide from the venom of the Indian cone snail Conus araneosus

The marine snail Conus araneosus has unusual significance due to its confined distribution to coastal regions of southeast India and Sri Lanka. Due to its relative scarceness, this species has been poorly studied. In this work, we characterized the venom of C. araneosus to identify new venom peptides. We identified 14 novel compounds. We determined amino acid sequences from chemically-modified and unmodified crude venom using liquid chromatography-electrospray ionization mass spectrometry and matrix assisted laser desorption ionization time-of-flight mass spectrometry. Ten sequences showed six Cys residues arranged in a pattern that is most commonly associated with the M-superfamily of conotoxins. Four other sequences had four Cys residues in a pattern that is most commonly associated with the T-superfamily of conotoxins. The post-translationally modified residue (pyroglutamate) was determined at the N-terminus of two sequences, ar3h and ar3i respectively. In addition, two sequences, ar3g and ar3h were C-terminally amidated. At a dose of 2 nmol, peptide ar3j elicited sleep when injected intraperitoneally into mice. To our knowledge, this is the first report of a peptide from a molluscivorous cone snail with sleep-inducing effects in mice. The novel peptides characterized herein extend the repertoire of unique peptides derived from cone snails and may add value to the therapeutic promise of conotoxins. (C) 2015 Elsevier Ltd. All rights reserved.

Social Experience Is Sufficient to Modulate Sleep Need of Drosophila without Increasing Wakefulness

Organisms quickly learn about their surroundings and display synaptic plasticity which is thought to be critical for their survival. For example, fruit flies Drosophila melanogaster exposed to highly enriched social environment are found to show increased synaptic connections and a corresponding increase in sleep. Here we asked if social environment comprising a pair of same-sex individuals could enhance sleep in the participating individuals. To study this, we maintained individuals of D. melanogaster in same-sex pairs for a period of 1 to 4 days, and after separation, monitored sleep of the previously socialized and solitary individuals under similar conditions. Males maintained in pairs for 3 or more days were found to sleep significantly more during daytime and showed a tendency to fall asleep sooner as compared to solitary controls (both measures together are henceforth referred to as ``sleep-enhancement''). This sleep phenotype is not strain-specific as it is observed in males from three different ``wild type'' strains of D. melanogaster. Previous studies on social interaction mediated sleep-enhancement presumed `waking experience' during the interaction to be the primary underlying cause; however, we found sleep-enhancement to occur without any significant increase in wakefulness. Furthermore, while sleep-enhancement due to group-wise social interaction requires Pigment Dispersing Factor (PDF) positive neurons; PDF positive and CRYPTOCHROME (CRY) positive circadian clock neurons and the core circadian clock genes are not required for sleep-enhancement to occur when males interact in pairs. Pair-wise social interaction mediated sleep-enhancement requires dopamine and olfactory signaling, while visual and gustatory signaling systems seem to be dispensable. These results suggest that socialization alone (without any change in wakefulness) is sufficient to cause sleep-enhancement in fruit fly D. melanogaster males, and that its neuronal control is context-specific.