Two timescale convergent Q-learning for sleep-scheduling in wireless sensor networks

In this paper, we consider an intrusion detection application for Wireless Sensor Networks. We study the problem of scheduling the sleep times of the individual sensors, where the objective is to maximize the network lifetime while keeping the tracking error to a minimum. We formulate this problem as a partially-observable Markov decision process (POMDP) with continuous stateaction spaces, in a manner similar to Fuemmeler and Veeravalli (IEEE Trans Signal Process 56(5), 2091-2101, 2008). However, unlike their formulation, we consider infinite horizon discounted and average cost objectives as performance criteria. For each criterion, we propose a convergent on-policy Q-learning algorithm that operates on two timescales, while employing function approximation. Feature-based representations and function approximation is necessary to handle the curse of dimensionality associated with the underlying POMDP. Our proposed algorithm incorporates a policy gradient update using a one-simulation simultaneous perturbation stochastic approximation estimate on the faster timescale, while the Q-value parameter (arising from a linear function approximation architecture for the Q-values) is updated in an on-policy temporal difference algorithm-like fashion on the slower timescale. The feature selection scheme employed in each of our algorithms manages the energy and tracking components in a manner that assists the search for the optimal sleep-scheduling policy. For the sake of comparison, in both discounted and average settings, we also develop a function approximation analogue of the Q-learning algorithm. This algorithm, unlike the two-timescale variant, does not possess theoretical convergence guarantees. Finally, we also adapt our algorithms to include a stochastic iterative estimation scheme for the intruder's mobility model and this is useful in settings where the latter is not known. Our simulation results on a synthetic 2-dimensional network setting suggest that our algorithms result in better tracking accuracy at the cost of only a few additional sensors, in comparison to a recent prior work.

Energy-efficient, delay-constrained, QoS-aware broadcast for cooperative wireless sensor networks

The problem of delay-constrained, energy-efficient broadcast in cooperative wireless networks is NP-complete. While centralised setting allows some heuristic solutions, designing heuristics in distributed implementation poses significant challenges. This is more so in wireless sensor networks (WSNs) where nodes are deployed randomly and topology changes dynamically due to node failure/join and environment conditions. This paper demonstrates that careful design of network infrastructure can achieve guaranteed delay bounds and energy-efficiency, and even meet quality of service requirements during broadcast. The paper makes three prime contributions. First, we present an optimal lower bound on energy consumption for broadcast that is tighter than what has been previously proposed. Next, iSteiner, a lightweight, distributed and deterministic algorithm for creation of network infrastructure is discussed. iPercolate is the algorithm that exploits this structure to cooperatively broadcast information with guaranteed delivery and delay bounds, while allowing real-time traffic to pass undisturbed.

An Optoelectronic Profiling and Ranging Sensor for Monitoring of Perimeters

Electronic monitoring of perimeters plays vital roles in homeland security, management of traffic and of humanwildlife conflict. This paper reports the design and development of an optical beam-interruption-based ranging and profiling sensor for monitoring perimeters. The developed sensor system can determine the distance of the object from the sensing units and its temporal height profile as the object crosses the system. Together, these quantities can also be used to classify the object and to determine its speed. The sensor is designed, fabricated, and evaluated. The design enables compact construction, high sensitivity, and low measurement crosstalk. The evaluation demonstrates accuracy better than 98.5% in the determination of height and over 94% in determination of the distance of an object from the sensing units. Finally, a strategy is proposed to classify the objects based on the obtained height profiles. The strategy is demonstrated to correctly classify the objects despite differences in their speed and the location at which they cross the system.

Energy Sharing for Multiple Sensor Nodes With Finite Buffers

We consider the problem of finding optimal energy sharing policies that maximize the network performance of a system comprising of multiple sensor nodes and a single energy harvesting (EH) source. Sensor nodes periodically sense the random field and generate data, which is stored in the corresponding data queues. The EH source harnesses energy from ambient energy sources and the generated energy is stored in an energy buffer. Sensor nodes receive energy for data transmission from the EH source. The EH source has to efficiently share the stored energy among the nodes to minimize the long-run average delay in data transmission. We formulate the problem of energy sharing between the nodes in the framework of average cost infinite-horizon Markov decision processes (MDPs). We develop efficient energy sharing algorithms, namely Q-learning algorithm with exploration mechanisms based on the epsilon-greedy method as well as upper confidence bound (UCB). We extend these algorithms by incorporating state and action space aggregation to tackle state-action space explosion in the MDP. We also develop a cross entropy based method that incorporates policy parameterization to find near optimal energy sharing policies. Through simulations, we show that our algorithms yield energy sharing policies that outperform the heuristic greedy method.

Relay Selection with Channel Probing in Sleep-Wake Cycling Wireless Sensor Networks

In geographical forwarding of packets in a large wireless sensor network (WSN) with sleep-wake cycling nodes, we are interested in the local decision problem faced by a node that has ``custody'' of a packet and has to choose one among a set of next-hop relay nodes to forward the packet toward the sink. Each relay is associated with a ``reward'' that summarizes the benefit of forwarding the packet through that relay. We seek a solution to this local problem, the idea being that such a solution, if adopted by every node, could provide a reasonable heuristic for the end-to-end forwarding problem. Toward this end, we propose a local relay selection problem consisting of a forwarding node and a collection of relay nodes, with the relays waking up sequentially at random times. At each relay wake-up instant, the forwarder can choose to probe a relay to learn its reward value, based on which the forwarder can then decide whether to stop (and forward its packet to the chosen relay) or to continue to wait for further relays to wake up. The forwarder's objective is to select a relay so as to minimize a combination of waiting delay, reward, and probing cost. The local decision problem can be considered as a variant of the asset selling problem studied in the operations research literature. We formulate the local problem as a Markov decision process (MDP) and characterize the solution in terms of stopping sets and probing sets. We provide results illustrating the structure of the stopping sets, namely, the (lower bound) threshold and the stage independence properties. Regarding the probing sets, we make an interesting conjecture that these sets are characterized by upper bounds. Through simulation experiments, we provide valuable insights into the performance of the optimal local forwarding and its use as an end-to-end forwarding heuristic.

Algorithm for Autonomous Reorganization of Mobile Wireless Camera Sensor Networks to Improve Coverage

In this paper, sensing coverage by wireless camera-embedded sensor networks (WCSNs), a class of directional sensors is studied. The proposed work facilitates the autonomous tuning of orientation parameters and displacement of camera-sensor nodes in the bounded field of interest (FoI), where the network coverage in terms of every point in the FoI is important. The proposed work is first of its kind to study the problem of maximizing coverage of randomly deployed mobile WCSNs which exploits their mobility. We propose an algorithm uncovered region exploration algorithm (UREA-CS) that can be executed in centralized and distributed modes. Further, the work is extended for two special scenarios: 1) to suit autonomous combing operations after initial random WCSN deployments and 2) to improve the network coverage with occlusions in the FoI. The extensive simulation results show that the performance of UREA-CS is consistent, robust, and versatile to achieve maximum coverage, both in centralized and distributed modes. The centralized and distributed modes are further analyzed with respect to the computational and communicational overheads.

QoS Constrained Optimal Sink and Relay Placement in Planned Wireless Sensor Networks

We are given a set of sensors at given locations, a set of potential locations for placing base stations (BSs, or sinks), and another set of potential locations for placing wireless relay nodes. There is a cost for placing a BS and a cost for placing a relay. The problem we consider is to select a set of BS locations, a set of relay locations, and an association of sensor nodes with the selected BS locations, so that the number of hops in the path from each sensor to its BS is bounded by h(max), and among all such feasible networks, the cost of the selected network is the minimum. The hop count bound suffices to ensure a certain probability of the data being delivered to the BS within a given maximum delay under a light traffic model. We observe that the problem is NP-Hard, and is hard to even approximate within a constant factor. For this problem, we propose a polynomial time approximation algorithm (SmartSelect) based on a relay placement algorithm proposed in our earlier work, along with a modification of the greedy algorithm for weighted set cover. We have analyzed the worst case approximation guarantee for this algorithm. We have also proposed a polynomial time heuristic to improve upon the solution provided by SmartSelect. Our numerical results demonstrate that the algorithms provide good quality solutions using very little computation time in various randomly generated network scenarios.

Ultrasonic Guided Wave Sensor Network Optimization for Monitoring Rivet Line

Guided waves using piezo-electric wafer active sensors (PWAS) is one of the useful techniques of damage detection. Sensor network optimization with minimal network hardware footprint and maximal area of coverage remains a challenging problem. PWAS sensors are placed at discrete locations in order to inspect damages in plates and the idea has the potential to be extended to assembled structures. Various actuator-sensor configurations are possible within the network in order to identify and locate damages. In this paper we present a correlation based approach to monitor cracks emanating from rivet line using a simulated guided wave signal whose sensor is operating in pulse echo mode. Discussions regarding the identification of phase change due to reflections from the crack are also discussed in this paper.

Relay node placement for wireless sensor networks deployed in tunnels

Node placement plays a significant role in the effective and successful deployment of Wireless Sensor Networks (WSNs), i.e., meeting design goals such as cost effectiveness, coverage, connectivity, lifetime and data latency. In this paper, we propose a new strategy to assist in the placement of Relay Nodes (RNs) for a WSN monitoring underground tunnel infrastructure. By applying for the first time an accurate empirical mean path loss propagation model along with a well fitted fading distribution model specifically defined for the tunnel environment, we address the RN placement problem with guaranteed levels of radio link performance. The simulation results show that the choice of appropriate path loss model and fading distribution model for a typical environment is vital in the determination of the number and the positions of RNs. Furthermore, we adapt a two-tier clustering multi-hop framework in which the first tier of the RN placement is modelled as the minimum set cover problem, and the second tier placement is solved using the search-and-find algorithm. The implementation of the proposed scheme is evaluated by simulation, and it lays the foundations for further work in WSN planning for underground tunnel applications. © 2010 IEEE.

Wave Sensor Technologies, St Petersburg, Florida, March 7-9, 2007: workshop proceedings

The Alliance for Coastal Technologies (ACT) convened a workshop on "Wave Sensor Technologies" in St. Petersburg, Florida on March 7-9, 2007, hosted by the University of South Florida (USF) College of Marine Science, an ACT partner institution. The primary objectives of this workshop were to: 1) define the present state of wave measurement technologies, 2) identify the major impediments to their advancement, and 3) make strategic recommendations for future development and on the necessary steps to integrate wave measurement sensors into operational coastal ocean observing systems. The participants were from various sectors, including research scientists, technology developers and industry providers, and technology users, such as operational coastal managers and coastal decision makers. Waves consistently are ranked as a critical variable for numerous coastal issues, from maritime transportation to beach erosion to habitat restoration. For the purposes of this workshop, the participants focused on measuring "wind waves" (i.e., waves on the water surface, generated by the wind, restored by gravity and existing between approximately 3 and 30-second periods), although it was recognized that a wide range of both forced and free waves exist on and in the oceans. Also, whereas the workshop put emphasis on the nearshore coastal component of wave measurements, the participants also stressed the importance of open ocean surface waves measurement. Wave sensor technologies that are presently available for both environments include bottom-mounted pressure gauges, surface following buoys, wave staffs, acoustic Doppler current profilers, and shore-based remote sensing radar instruments. One of the recurring themes of workshop discussions was the dichotomous nature of wave data users. The two separate groups, open ocean wave data users and the nearshore/coastal wave data users, have different requirements. Generally, the user requirements increase both in spatial/temporal resolution and precision as one moves closer to shore. Most ocean going mariners are adequately satisfied with measurements of wave period and height and a wave general direction. However, most coastal and nearshore users require at least the first five Fourier parameters ("First 5"): wave energy and the first four directional Fourier coefficients. Furthermore, wave research scientists would like sensors capable of providing measurements beyond the first four Fourier coefficients. It was debated whether or not high precision wave observations in one location can take the place of a less precise measurement at a different location. This could be accomplished by advancing wave models and using wave models to extend data to nearby areas. However, the consensus was that models are no substitution for in situ wave data.[PDF contains 26 pages]

Ground water-surface water interactions sensor technology, Savannah, Georgia, March 7-9, 2005: workshop proceedings

The Alliance for Coastal Technologies (ACT) held a Workshop on Sensor Technology for Assessing Groundwater-Surface Water Interactions in the Coastal Zone on March 7 to 9,2005 in Savannah, GA. The main goal of the workshop was to summarize the general parameters, which have been found to be useful in assessing groundwater-surface water (GW-SW) interactions in the coastal zone. The workshop participants (Appendix I) were specifically charged with identifying the types of sensor systems, if any, that have been used to obtain time-series data and to make known which parameters may be the most amenable to the development/application of sensor technology. The group consisted of researchers, industry representatives, and environmental managers. Four general recommendations were made: 1. Educate coastal managers and agencies on the importance of GW-SW interactions, keeping in mind that regulatory agencies are driven by a different set of rules than researchers: the focus is on understanding the significance of the problem and providing solutions. ACT could facilitate this process in two ways. First, given that the research literature on this subject is fairly diffuse, ACT could provide links from its web site to fact sheets or other literature. Second, ACT could organize a focused meeting for managers and/or agency groups. Encourage development of primary tools for quantifying flow. The most promising technology in this respect is flow meters designed for flux chambers, mainly because they should be simple to use and can be made relatively inexpensively. However, it should be kept in mind that they provide only point measurements and several would need to be deployed as a network in order to obtain reliable flow estimates. For evaluating system wide GW-SW interactions, tools that integrate the signal over large areas would be required. Suggestions include a user-friendly hydrogeologic models, keeping in mind that freshwater flow is not the entire story, or continuous radon monitors. Though the latter would be slightly more difficult to use in terms of background knowledge, such an instrument would be low power and easy to operate and maintain. ACT could facilitate this recommendation by identifying funding opportunities on its web site and/or performing evaluations of existing technologies that could be summarized on the web site. (pdf contains 18 pages)

Selective data gathering in community sensor networks

Smartphones and other powerful sensor-equipped consumer devices make it possible to sense the physical world at an unprecedented scale. Nearly 2 million Android and iOS devices are activated every day, each carrying numerous sensors and a high-speed internet connection. Whereas traditional sensor networks have typically deployed a fixed number of devices to sense a particular phenomena, community networks can grow as additional participants choose to install apps and join the network. In principle, this allows networks of thousands or millions of sensors to be created quickly and at low cost. However, making reliable inferences about the world using so many community sensors involves several challenges, including scalability, data quality, mobility, and user privacy.

This thesis focuses on how learning at both the sensor- and network-level can provide scalable techniques for data collection and event detection. First, this thesis considers the abstract problem of distributed algorithms for data collection, and proposes a distributed, online approach to selecting which set of sensors should be queried. In addition to providing theoretical guarantees for submodular objective functions, the approach is also compatible with local rules or heuristics for detecting and transmitting potentially valuable observations. Next, the thesis presents a decentralized algorithm for spatial event detection, and describes its use detecting strong earthquakes within the Caltech Community Seismic Network. Despite the fact that strong earthquakes are rare and complex events, and that community sensors can be very noisy, our decentralized anomaly detection approach obtains theoretical guarantees for event detection performance while simultaneously limiting the rate of false alarms.

On a fiber grating sensor system with the capacity of cross-sensitivity discrimination

A novel fiber Bragg grating (FBG) sensor system based on an interrogating technique by two parallel matched gratings was designed and theoretically discussed. With an interrogation grating playing the role of temperature compensation grating simultaneously, the wavelength drifts induced by temperature and strain were discriminated. Additionally, the expressions of temperature and strain were deduced for our solution, and dual-value problem and cross sensitivity were solved synchronously through data processing. The influence of the FBG's parameters on the dynamic range and precision was discussed. Besides, the change of environment temperature cannot influence the dynamic range of the sensor system through temperature tuning. The system proposed in this paper will be of great significance to accelerate the real engineering applications of FBG sensing techniques. (c) 2007 Elsevier GmbH. All rights reserved.

A high accuracy image sensor for sinusoidal phase-modulating interferometry

We proposed a high accuracy image sensor technique for sinusoidal phase-modulating interferometer in the field of the surface profile measurements. It solved the problem of the CCD's pixel offset of the same column under two adjacent rows, eliminated the spectral leakage, and reduced the influence of external interference to the measurement accuracy. We measured the surface profile of a glass plate, and its repeatability precision was less than 8 nm and its relative error was 1.15 %. The results show that it can be used to measure surface profile with high accuracy and strong anti-interference ability. (C) 2007 Elsevier GmbH. All rights reserved.

Dealing with the Effects of Sensor Displacement in Wearable Activity Recognition

Most wearable activity recognition systems assume a predefined sensor deployment that remains unchanged during runtime. However, this assumption does not reflect real-life conditions. During the normal use of such systems, users may place the sensors in a position different from the predefined sensor placement. Also, sensors may move from their original location to a different one, due to a loose attachment. Activity recognition systems trained on activity patterns characteristic of a given sensor deployment may likely fail due to sensor displacements. In this work, we innovatively explore the effects of sensor displacement induced by both the intentional misplacement of sensors and self-placement by the user. The effects of sensor displacement are analyzed for standard activity recognition techniques, as well as for an alternate robust sensor fusion method proposed in a previous work. While classical recognition models show little tolerance to sensor displacement, the proposed method is proven to have notable capabilities to assimilate the changes introduced in the sensor position due to self-placement and provides considerable improvements for large misplacements.