Two timescale convergent Q-learning for sleep-scheduling in wireless sensor networks

In this paper, we consider an intrusion detection application for Wireless Sensor Networks. We study the problem of scheduling the sleep times of the individual sensors, where the objective is to maximize the network lifetime while keeping the tracking error to a minimum. We formulate this problem as a partially-observable Markov decision process (POMDP) with continuous stateaction spaces, in a manner similar to Fuemmeler and Veeravalli (IEEE Trans Signal Process 56(5), 2091-2101, 2008). However, unlike their formulation, we consider infinite horizon discounted and average cost objectives as performance criteria. For each criterion, we propose a convergent on-policy Q-learning algorithm that operates on two timescales, while employing function approximation. Feature-based representations and function approximation is necessary to handle the curse of dimensionality associated with the underlying POMDP. Our proposed algorithm incorporates a policy gradient update using a one-simulation simultaneous perturbation stochastic approximation estimate on the faster timescale, while the Q-value parameter (arising from a linear function approximation architecture for the Q-values) is updated in an on-policy temporal difference algorithm-like fashion on the slower timescale. The feature selection scheme employed in each of our algorithms manages the energy and tracking components in a manner that assists the search for the optimal sleep-scheduling policy. For the sake of comparison, in both discounted and average settings, we also develop a function approximation analogue of the Q-learning algorithm. This algorithm, unlike the two-timescale variant, does not possess theoretical convergence guarantees. Finally, we also adapt our algorithms to include a stochastic iterative estimation scheme for the intruder's mobility model and this is useful in settings where the latter is not known. Our simulation results on a synthetic 2-dimensional network setting suggest that our algorithms result in better tracking accuracy at the cost of only a few additional sensors, in comparison to a recent prior work.

A non-resonant mass sensor to eliminate the ``missing mass'' effect during mass measurement of biological materials

Resonant sensors and crystal oscillators for mass detection need to be excited at very high natural frequencies (MHz). Use of such systems to measure mass of biological materials affects the accuracy of mass measurement due to their viscous and/or viscoelastic properties. The measurement limitation of such sensor system is the difficulty in accounting for the ``missing mass'' of the biological specimen in question. A sensor system has been developed in this work, to be operated in the stiffness controlled region at very low frequencies as compared to its fundamental natural frequency. The resulting reduction in the sensitivity due to non-resonant mode of operation of this sensor is compensated by the high resolution of the sensor. The mass of different aged drosophila melanogaster (fruit fly) is measured. The difference in its mass measurement during resonant mode of operation is also presented. That, viscosity effects do not affect the working of this non-resonant mass sensor is clearly established by direct comparison. (C) 2014 AIP Publishing LLC.

Highly Sensitive Carbon Nanotubes Coated Etched Fiber Bragg Grating Sensor for Humidity Sensing

The sensing of relative humidity (RH) at room temperature has potential applications in several areas ranging from biomedical to horticulture, paper, and textile industries. In this paper, a highly sensitive humidity sensor based on carbon nanotubes (CNTs) coated on the surface of an etched fiber Bragg grating (EFBG) sensor has been demonstrated, for detecting RH over a wide range of 20%-90% at room temperature. When water molecules interact with the CNT coated EFBG, the effective refractive index of the fiber core changes, resulting in a shift in the Bragg wavelength. It has been possible to achieve a high sensitivity of similar to 31 pm/% RH, which is the highest compared with many of the existing FBG-based humidity sensors. The limit of detection in the CNT coated EFBG has been found to be similar to 0.03 RH. The experimental data shows a linear response of Bragg wavelength shift with increase in humidity. This novel method of incorporating CNTs on to the FBG sensor for humidity sensing has not been reported before.

Sparse Representation Based Acoustic Rangefinders: From Sensor Platforms to Mobile Devices

Acoustic rangerfinders are a promising technology for accurate proximity detection, a critical requirement for many emerging mobile computing applications. While state-of-the-art systems deliver robust ranging performance, the computational intensiveness of their detection mechanism expedites the energy depletion of the associated devices that are typically powered by batteries. The contribution of this article is fourfold. First, it outlines the common factors that are important for ranging. Second, it presents a review of acoustic rangers and identifies their potential problems. Third, it explores the design of an information processing framework based on sparse representation that could potentially address existing challenges, especially for mobile devices. Finally, it presents mu-BeepBeep: a low energy acoustic ranging service for mobile devices, and empirically evaluates its benefits.

Energy-efficient, delay-constrained, QoS-aware broadcast for cooperative wireless sensor networks

The problem of delay-constrained, energy-efficient broadcast in cooperative wireless networks is NP-complete. While centralised setting allows some heuristic solutions, designing heuristics in distributed implementation poses significant challenges. This is more so in wireless sensor networks (WSNs) where nodes are deployed randomly and topology changes dynamically due to node failure/join and environment conditions. This paper demonstrates that careful design of network infrastructure can achieve guaranteed delay bounds and energy-efficiency, and even meet quality of service requirements during broadcast. The paper makes three prime contributions. First, we present an optimal lower bound on energy consumption for broadcast that is tighter than what has been previously proposed. Next, iSteiner, a lightweight, distributed and deterministic algorithm for creation of network infrastructure is discussed. iPercolate is the algorithm that exploits this structure to cooperatively broadcast information with guaranteed delivery and delay bounds, while allowing real-time traffic to pass undisturbed.

Fabrication and Experimentation of a Cantilever Beam Based Piezoelectric Actuator and Sensor for Vibration Energy Harvesting

In the immediate surroundings of our daily life, we can find a lot of places where the energy in the form of vibration is being wasted. Therefore, we have enormous opportunities to utilize the same. Piezoelectric character of matter enables us to convert this mechanical vibration energy into electrical energy which can be stored and used to power other device, instead of being wasted. This work is done to realize both actuator and sensor in a cantilever beam based on piezoelectricity. The sensor part is called vibration energy harvester. The numerical analyses were performed for the cantilever beam using the commercial package ANSYS and MATLAB. The cantilever beam is realized by taking a plate and fixing its one end between two massive plates. Two PZT patches were glued to the beam on its two faces. Experiments were performed using data acquisition system (DAQ) and LABVIEW software for actuating and sensing the vibration of the cantilever beam.

Colorimetric detection of melamine based on the size effect of AuNPs

A simple colorimetric detection of melamine was studied using 15 nm (AuNPs-I), 30 nm (AuNPs-II), and 40 nm (AuNPs-III) citrate-capped gold nanoparticles (AuNPs). The AuNPs aggregated in aqueous solution in the presence of melamine, showing a visual color change from red to blue. This color change led to a shift in the absorption peak from 527 nm, 526 nm, and 525 nm to 638 nm, 626 nm, and 680 nm for AuNPs-I, AuNPs-II, and AuNPs-III, respectively. For all the three AuNPs, linearity was observed between the melamine concentration in aqueous solution and the absorbance ratios, A(638/527), A(626/525), and A(680/526), respectively. The limit of detection (LOD) for melamine for the AuNPs-II was found to be 2.37 x 10(-8) M (correlation coefficient R-2 = 0.9745), which showed better sensitivity as compared to the LOD of the AuNPs-I and AuNPs-III, which were 3.3 x 10(-8) M and 8.9 x 10(-8) M, respectively. The synthesis of AuNPs-II also involved a lower HAuCl4 concentration compared with the other two types of AuNPs, which may reduce the process cost. The AuNPs-II was selected to analyze melamine in pre-treated milk samples, and the recovery percentage was in the range of 91-106%. Thus, the efficient detection of melamine was possible using AuNPs-II for the on-site detection without the aid of expensive instruments.

Validation of polyvinylidene fluoride nasal sensor to assess nasal obstruction in comparison with subjective technique

Objective: The aim of this study is to validate the applicability of the PolyVinyliDene Fluoride (PVDF) nasal sensor to assess the nasal airflow, in healthy subjects and patients with nasal obstruction and to correlate the results with the score of Visual Analogue Scale (VAS). Methods: PVDF nasal sensor and VAS measurements were carried out in 50 subjects (25-healthy subjects and 25 patients). The VAS score of nasal obstruction and peak-to-peak amplitude (Vp-p) of nasal cycle measured by PVDF nasal sensors were analyzed for right nostril (RN) and left nostril (LN) in both the groups. Spearman's rho correlation was calculated. The relationship between PVDF nasal sensor measurements and severity of nasal obstruction (VAS score) were assessed by ANOVA. Results: In healthy group, the measurement of nasal airflow by PVDF nasal sensor for RN and LN were found to be 51.14 +/- 5.87% and 48.85 +/- 5.87%, respectively. In patient group, PVDF nasal sensor indicated lesser nasal airflow in the blocked nostrils (RN: 23.33 +/- 10.54% and LN: 32.24 +/- 11.54%). Moderate correlation was observed in healthy group (r = 0.710, p < 0.001 for RN and r = 0.651, p < 0.001 for LN), and moderate to strong correlation in patient group (r = 0.751, p < 0.01 for RN and r = 0.885, p < 0.0001 for LN). Conclusion: PVDF nasal sensor method is a newly developed technique for measuring the nasal airflow. Moderate to strong correlation was observed between PVDF nasal sensor data and VAS scores for nasal obstruction. In our present study, PVDF nasal sensor technique successfully differentiated between healthy subjects and patients with nasal obstruction. Additionally, it can also assess severity of nasal obstruction in comparison with VAS. Thus, we propose that the PVDF nasal sensor technique could be used as a new diagnostic method to evaluate nasal obstruction in routine clinical practice. (C) 2015 Elsevier Inc. All rights reserved.

Adaptive Sleep-Wake Control using Reinforcement Learning in Sensor Networks

The aim in this paper is to allocate the `sleep time' of the individual sensors in an intrusion detection application so that the energy consumption from the sensors is reduced, while keeping the tracking error to a minimum. We propose two novel reinforcement learning (RL) based algorithms that attempt to minimize a certain long-run average cost objective. Both our algorithms incorporate feature-based representations to handle the curse of dimensionality associated with the underlying partially-observable Markov decision process (POMDP). Further, the feature selection scheme used in our algorithms intelligently manages the energy cost and tracking cost factors, which in turn assists the search for the optimal sleeping policy. We also extend these algorithms to a setting where the intruder's mobility model is not known by incorporating a stochastic iterative scheme for estimating the mobility model. The simulation results on a synthetic 2-d network setting are encouraging.

An Optoelectronic Profiling and Ranging Sensor for Monitoring of Perimeters

Electronic monitoring of perimeters plays vital roles in homeland security, management of traffic and of humanwildlife conflict. This paper reports the design and development of an optical beam-interruption-based ranging and profiling sensor for monitoring perimeters. The developed sensor system can determine the distance of the object from the sensing units and its temporal height profile as the object crosses the system. Together, these quantities can also be used to classify the object and to determine its speed. The sensor is designed, fabricated, and evaluated. The design enables compact construction, high sensitivity, and low measurement crosstalk. The evaluation demonstrates accuracy better than 98.5% in the determination of height and over 94% in determination of the distance of an object from the sensing units. Finally, a strategy is proposed to classify the objects based on the obtained height profiles. The strategy is demonstrated to correctly classify the objects despite differences in their speed and the location at which they cross the system.

A rhodamine-based `turn-on' Al3+ ion-selective reporter and the resultant complex as a secondary sensor for F- ion are applicable to living cell staining

A newly designed fluorescent aluminum(III) complex (L'-Al; 2) of a structurally characterized non-fluorescent rhodamine Schiff base (L) has been isolated in pure form and characterized using spectroscopic and physico-chemical methods with theoretical density functional theory (DFT) support. On addition of Al(III) ions to a solution of L in HEPES buffer (1 mM, pH 7.4; EtOH-water, 1 : 3 v/v) at 25 degrees C, the systematic increase in chelation-enhanced fluorescence (CHEF) enables the detection of Al(III) ions as low as 60 nM with high selectivity, unaffected by the presence of competitive ions. Interestingly, the Al(III) complex (L'-Al; 2) is specifically able to detect fluoride ions by quenching the fluorescence in the presence of large amounts of other anions in the HEPES buffer (1 mM, pH 7.4) at 25 degrees C. On the basis of our experimental and theoretical findings, the addition of Al3+ ions to a solution of L helps to generate a new fluorescence peak at 590 nm, due to the selective binding of Al3+ ions with L in a 1 : 1 ratio with a binding constant (K) of 8.13 x 10(4) M-1. The Schiff base L shows no cytotoxic effect, and it can therefore be employed for determining the intracellular concentration of Al3+ and F-ions by 2 in living cells using fluorescence microscopy.

Graphene based multifunctional flame sensor

Recently, graphene has attracted much attention due to its unique electrical and thermal properties along with its high surface area, and hence presents an ideal sensing material. We report a novel configuration of a graphene based flame sensor by exploiting the response of few layer graphene to a flame along two different directions, where flame detection results from a difference in heat transfer mechanisms. A complete sensor module was developed with a signal conditioning circuit that compensates for any drift in the baseline of the sensor, along with a flame detection algorithm implemented in a microcontroller to detect the flame. A pre-defined threshold for either of the sensors is tunable, which can be varied based on the nature of the flame, hence presenting a system that can be used for detection of any kind of flame. This finding also presents a scalable method that opens avenues to modify complicated sensing schemes.

Energy Sharing for Multiple Sensor Nodes With Finite Buffers

We consider the problem of finding optimal energy sharing policies that maximize the network performance of a system comprising of multiple sensor nodes and a single energy harvesting (EH) source. Sensor nodes periodically sense the random field and generate data, which is stored in the corresponding data queues. The EH source harnesses energy from ambient energy sources and the generated energy is stored in an energy buffer. Sensor nodes receive energy for data transmission from the EH source. The EH source has to efficiently share the stored energy among the nodes to minimize the long-run average delay in data transmission. We formulate the problem of energy sharing between the nodes in the framework of average cost infinite-horizon Markov decision processes (MDPs). We develop efficient energy sharing algorithms, namely Q-learning algorithm with exploration mechanisms based on the epsilon-greedy method as well as upper confidence bound (UCB). We extend these algorithms by incorporating state and action space aggregation to tackle state-action space explosion in the MDP. We also develop a cross entropy based method that incorporates policy parameterization to find near optimal energy sharing policies. Through simulations, we show that our algorithms yield energy sharing policies that outperform the heuristic greedy method.

Spray: A Multi-Modal Localization System for Stationary Sensor Network Deployment

We present a localization system that targets rapid deployment of stationary wireless sensor networks (WSN). The system uses a particle filter to fuse measurements from multiple localization modalities, such as RF ranging, neighbor information or maps, to obtain position estimations with higher accuracy than that of the individual modalities. The system isolates different modalities into separate components which can be included or excluded independently to tailor the system to a specific scenario. We show that position estimations can be improved with our system by combining multiple modalities. We evaluate the performance of the system in both an indoor and outdoor environment using combinations of five different modalities. Using two anchor nodes as reference points and combining all five modalities, we obtain RMS (Root Mean Square) estimation errors of approximately 2.5m in both cases, while using the components individually results in errors within the range of 3.5 and 9 m.

Relay Selection with Channel Probing in Sleep-Wake Cycling Wireless Sensor Networks

In geographical forwarding of packets in a large wireless sensor network (WSN) with sleep-wake cycling nodes, we are interested in the local decision problem faced by a node that has ``custody'' of a packet and has to choose one among a set of next-hop relay nodes to forward the packet toward the sink. Each relay is associated with a ``reward'' that summarizes the benefit of forwarding the packet through that relay. We seek a solution to this local problem, the idea being that such a solution, if adopted by every node, could provide a reasonable heuristic for the end-to-end forwarding problem. Toward this end, we propose a local relay selection problem consisting of a forwarding node and a collection of relay nodes, with the relays waking up sequentially at random times. At each relay wake-up instant, the forwarder can choose to probe a relay to learn its reward value, based on which the forwarder can then decide whether to stop (and forward its packet to the chosen relay) or to continue to wait for further relays to wake up. The forwarder's objective is to select a relay so as to minimize a combination of waiting delay, reward, and probing cost. The local decision problem can be considered as a variant of the asset selling problem studied in the operations research literature. We formulate the local problem as a Markov decision process (MDP) and characterize the solution in terms of stopping sets and probing sets. We provide results illustrating the structure of the stopping sets, namely, the (lower bound) threshold and the stage independence properties. Regarding the probing sets, we make an interesting conjecture that these sets are characterized by upper bounds. Through simulation experiments, we provide valuable insights into the performance of the optimal local forwarding and its use as an end-to-end forwarding heuristic.