Two timescale convergent Q-learning for sleep-scheduling in wireless sensor networks

In this paper, we consider an intrusion detection application for Wireless Sensor Networks. We study the problem of scheduling the sleep times of the individual sensors, where the objective is to maximize the network lifetime while keeping the tracking error to a minimum. We formulate this problem as a partially-observable Markov decision process (POMDP) with continuous stateaction spaces, in a manner similar to Fuemmeler and Veeravalli (IEEE Trans Signal Process 56(5), 2091-2101, 2008). However, unlike their formulation, we consider infinite horizon discounted and average cost objectives as performance criteria. For each criterion, we propose a convergent on-policy Q-learning algorithm that operates on two timescales, while employing function approximation. Feature-based representations and function approximation is necessary to handle the curse of dimensionality associated with the underlying POMDP. Our proposed algorithm incorporates a policy gradient update using a one-simulation simultaneous perturbation stochastic approximation estimate on the faster timescale, while the Q-value parameter (arising from a linear function approximation architecture for the Q-values) is updated in an on-policy temporal difference algorithm-like fashion on the slower timescale. The feature selection scheme employed in each of our algorithms manages the energy and tracking components in a manner that assists the search for the optimal sleep-scheduling policy. For the sake of comparison, in both discounted and average settings, we also develop a function approximation analogue of the Q-learning algorithm. This algorithm, unlike the two-timescale variant, does not possess theoretical convergence guarantees. Finally, we also adapt our algorithms to include a stochastic iterative estimation scheme for the intruder's mobility model and this is useful in settings where the latter is not known. Our simulation results on a synthetic 2-dimensional network setting suggest that our algorithms result in better tracking accuracy at the cost of only a few additional sensors, in comparison to a recent prior work.

QoS Aware Trust Metric based Framework for Wireless Sensor Networks

Wireless Sensor Networks have gained popularity due to their real time applications and low-cost nature. These networks provide solutions to scenarios that are critical, complicated and sensitive like military fields, habitat monitoring, and disaster management. The nodes in wireless sensor networks are highly resource constrained. Routing protocols are designed to make efficient utilization of the available resources in communicating a message from source to destination. In addition to the resource management, the trustworthiness of neighboring nodes or forwarding nodes and the energy level of the nodes to keep the network alive for longer duration is to be considered. This paper proposes a QoS Aware Trust Metric based Framework for Wireless Sensor Networks. The proposed framework safeguards a wireless sensor network from intruders by considering the trustworthiness of the forwarder node at every stage of multi-hop routing. Increases network lifetime by considering the energy level of the node, prevents the adversary from tracing the route from source to destination by providing path variation. The framework is built on NS2 Simulator. Experimental results show that the framework provides energy balance through establishment of trustworthy paths from the source to the destination. (C) 2015 The Authors. Published by Elsevier B.V.

QoS Constrained Optimal Sink and Relay Placement in Planned Wireless Sensor Networks

We are given a set of sensors at given locations, a set of potential locations for placing base stations (BSs, or sinks), and another set of potential locations for placing wireless relay nodes. There is a cost for placing a BS and a cost for placing a relay. The problem we consider is to select a set of BS locations, a set of relay locations, and an association of sensor nodes with the selected BS locations, so that the number of hops in the path from each sensor to its BS is bounded by h(max), and among all such feasible networks, the cost of the selected network is the minimum. The hop count bound suffices to ensure a certain probability of the data being delivered to the BS within a given maximum delay under a light traffic model. We observe that the problem is NP-Hard, and is hard to even approximate within a constant factor. For this problem, we propose a polynomial time approximation algorithm (SmartSelect) based on a relay placement algorithm proposed in our earlier work, along with a modification of the greedy algorithm for weighted set cover. We have analyzed the worst case approximation guarantee for this algorithm. We have also proposed a polynomial time heuristic to improve upon the solution provided by SmartSelect. Our numerical results demonstrate that the algorithms provide good quality solutions using very little computation time in various randomly generated network scenarios.

Relay node placement for wireless sensor networks deployed in tunnels

Node placement plays a significant role in the effective and successful deployment of Wireless Sensor Networks (WSNs), i.e., meeting design goals such as cost effectiveness, coverage, connectivity, lifetime and data latency. In this paper, we propose a new strategy to assist in the placement of Relay Nodes (RNs) for a WSN monitoring underground tunnel infrastructure. By applying for the first time an accurate empirical mean path loss propagation model along with a well fitted fading distribution model specifically defined for the tunnel environment, we address the RN placement problem with guaranteed levels of radio link performance. The simulation results show that the choice of appropriate path loss model and fading distribution model for a typical environment is vital in the determination of the number and the positions of RNs. Furthermore, we adapt a two-tier clustering multi-hop framework in which the first tier of the RN placement is modelled as the minimum set cover problem, and the second tier placement is solved using the search-and-find algorithm. The implementation of the proposed scheme is evaluated by simulation, and it lays the foundations for further work in WSN planning for underground tunnel applications. © 2010 IEEE.

A direct-conversion mixer with DC-offset cancellation for IEEE 802.11a WLAN receiver

A DC-offset cancellation scheme in the 5GHz direct-conversion receiver compliant with IEEE 802.11a wireless LAN standard is described in this paper. It uses the analog feedback loop to eliminate the DC-offset at the output of the double-balanced mixer. The mixer has a simulation voltage conversion gain of IMB at 5.2GHz, noise figure of 9.67dB, IIP3 of 7.6dBm. The solution provides 39.1dB reduction according to the leakage value at LO and mixer load resistors, the additional noise figure added to mixer is less than 0.9dB, the added power dissipation is 0.1mW and was fabricated in 60GHz 0.35 mu m SiGe BiCMOS technology.

M^2RC: Multiplicative-increase/additive-decrease Multipath Routing Control for Wireless Sensor Networks

Routing protocols in wireless sensor networks (WSN) face two main challenges: first, the challenging environments in which WSNs are deployed negatively affect the quality of the routing process. Therefore, routing protocols for WSNs should recognize and react to node failures and packet losses. Second, sensor nodes are battery-powered, which makes power a scarce resource. Routing protocols should optimize power consumption to prolong the lifetime of the WSN. In this paper, we present a new adaptive routing protocol for WSNs, we call it M^2RC. M^2RC has two phases: mesh establishment phase and data forwarding phase. In the first phase, M^2RC establishes the routing state to enable multipath data forwarding. In the second phase, M^2RC forwards data packets from the source to the sink. Targeting hop-by-hop reliability, an M^2RC forwarding node waits for an acknowledgement (ACK) that its packets were correctly received at the next neighbor. Based on this feedback, an M^2RC node applies multiplicative-increase/additive-decrease (MIAD) to control the number of neighbors targeted by its packet broadcast. We simulated M^2RC in the ns-2 simulator and compared it to GRAB, Max-power, and Min-power routing schemes. Our simulations show that M^2RC achieves the highest throughput with at least 10-30% less consumed power per delivered report in scenarios where a certain number of nodes unexpectedly fail.

Practical assessment of hardware limitations on power aware wireless sensor networks - An anti-windup approach

This work considers the effect of hardware constraints that typically arise in practical power-aware wireless sensor network systems. A rigorous methodology is presented that quantifies the effect of output power limit and quantization constraints on bit error rate performance. The approach uses a novel, intuitively appealing means of addressing the output power constraint, wherein the attendant saturation block is mapped from the output of the plant to its input and compensation is then achieved using a robust anti-windup scheme. A priori levels of system performance are attained using a quantitative feedback theory approach on the initial, linear stage of the design paradigm. This hybrid design is assessed experimentally using a fully compliant 802.15.4 testbed where mobility is introduced through the use of autonomous robots. A benchmark comparison between the new approach and a number of existing strategies is also presented.

The s-mote: a versatile heterogeneous multi-radio platform for wireless sensor networks applications

This paper presents a novel architecture and its implementation for a versatile, miniaturised mote which can communicate concurrently using a variety of combinations of ISM bands, has increased processing capability, and interoperability with mainstream GSM technology. All these features are integrated in a small form factor platform. The platform can have many configurations which could satisfy a variety of applications’ constraints. To the best of our knowledge, it is the first integrated platform of this type reported in the literature. The proposed platform opens the way for enhanced levels of Quality of Service (QoS), with respect to reliability, availability and latency, in addition to facilitating interoperability and power reduction compared to existing platforms. The small form factor also allows potential of integration with other mobile platforms including smart phones.

Towards persistent structural health monitoring through sustainable wireless sensor networks

This paper documents the design, implementation and characterisation of a wireless sensor node (GENESI Node v1.0), applicable to long-term structural health monitoring. Presented is a three layer abstraction of the hardware platform; consisting of a Sensor Layer, a Main Layer and a Power Layer. Extended operational lifetime is one of the primary design goals, necessitating the inclusion of supplemental energy sources, energy awareness, and the implementation of optimal components (microcontroller(s), RF transceiver, etc.) to achieve lowest-possible power consumption, whilst ensuring that the functional requirements of the intended application area are satisfied. A novel Smart Power Unit has been developed; including intelligence, ambient available energy harvesting (EH), storage, electrochemical fuel cell integration, and recharging capability, which acts as the Power Layer for the node. The functional node has been prototyped, demonstrated and characterised in a variety of operational modes. It is demonstrable via simulation that, under normal operating conditions within a structural health monitoring application, the node may operate perpetually.

Development of miniaturized wireless sensor nodes suitable for building energy management and modelling

Buildings consume 40% of Ireland's total annual energy translating to 3.5 billion (2004). The EPBD directive (effective January 2003) places an onus on all member states to rate the energy performance of all buildings in excess of 50m2. Energy and environmental performance management systems for residential buildings do not exist and consist of an ad-hoc integration of wired building management systems and Monitoring & Targeting systems for non-residential buildings. These systems are unsophisticated and do not easily lend themselves to cost effective retrofit or integration with other enterprise management systems. It is commonly agreed that a 15-40% reduction of building energy consumption is achievable by efficiently operating buildings when compared with typical practice. Existing research has identified that the level of information available to Building Managers with existing Building Management Systems and Environmental Monitoring Systems (BMS/EMS) is insufficient to perform the required performance based building assessment. The cost of installing additional sensors and meters is extremely high, primarily due to the estimated cost of wiring and the needed labour. From this perspective wireless sensor technology provides the capability to provide reliable sensor data at the required temporal and spatial granularity associated with building energy management. In this paper, a wireless sensor network mote hardware design and implementation is presented for a building energy management application. Appropriate sensors were selected and interfaced with the developed system based on user requirements to meet both the building monitoring and metering requirements. Beside the sensing capability, actuation and interfacing to external meters/sensors are provided to perform different management control and data recording tasks associated with minimisation of energy consumption in the built environment and the development of appropriate Building information models(BIM)to enable the design and development of energy efficient spaces.

GENESI: Wireless sensor networks for structural monitoring

The GENESI project has the ambitious goal of bringing WSN technology to the level where it can provide the core of the next generation of systems for structural health monitoring that are long lasting, pervasive and totally distributed and autonomous. This goal requires embracing engineering and scientific challenges never successfully tackled before. Sensor nodes will be redesigned to overcome their current limitations, especially concerning energy storage and provisioning (we need devices with virtually infinite lifetime) and resilience to faults and interferences (for reliability and robustness). New software and protocols will be defined to fully take advantage of the new hardware, providing new paradigms for cross-layer interaction at all layers of the protocol stack and satisfying the requirements of a new concept of Quality of Service (QoS) that is application-driven, truly reflecting the end user perspective and expectations. The GENESI project will develop long lasting sensor nodes by combining cutting edge technologies for energy generation from the environment (energy harvesting) and green energy supply (small form factor fuel cells); GENESI will define models for energy harvesting, energy conservation in super-capacitors and supplemental energy availability through fuel cells, in addition to the design of new algorithms and protocols for dynamic allocation of sensing and communication tasks to the sensors. The project team will design communication protocols for large scale heterogeneous wireless sensor/actuator networks with energy-harvesting capabilities and define distributed mechanisms for context assessment and situation awareness. This paper presents an analysis of the GENESI system requirements in order to achieve the ambitious goals of the project. Extending from the requirements presented, the emergent system specification is discussed with respect to the selection and integration of relevant system components.The resulting integrated system will be evaluated and characterised to ensure that it is capable of satisfying the functional requirements of the project

Energy harvesting system design and optimization for wireless sensor networks

Wireless sensor networks (WSN) are becoming widely adopted for many applications including complicated tasks like building energy management. However, one major concern for WSN technologies is the short lifetime and high maintenance cost due to the limited battery energy. One of the solutions is to scavenge ambient energy, which is then rectified to power the WSN. The objective of this thesis was to investigate the feasibility of an ultra-low energy consumption power management system suitable for harvesting sub-mW photovoltaic and thermoelectric energy to power WSNs. To achieve this goal, energy harvesting system architectures have been analyzed. Detailed analysis of energy storage units (ESU) have led to an innovative ESU solution for the target applications. Battery-less, long-lifetime ESU and its associated power management circuitry, including fast-charge circuit, self-start circuit, output voltage regulation circuit and hybrid ESU, using a combination of super-capacitor and thin film battery, were developed to achieve continuous operation of energy harvester. Low start-up voltage DC/DC converters have been developed for 1mW level thermoelectric energy harvesting. The novel method of altering thermoelectric generator (TEG) configuration in order to match impedance has been verified in this work. Novel maximum power point tracking (MPPT) circuits, exploring the fractional open circuit voltage method, were particularly developed to suit the sub-1mW photovoltaic energy harvesting applications. The MPPT energy model has been developed and verified against both SPICE simulation and implemented prototypes. Both indoor light and thermoelectric energy harvesting methods proposed in this thesis have been implemented into prototype devices. The improved indoor light energy harvester prototype demonstrates 81% MPPT conversion efficiency with 0.5mW input power. This important improvement makes light energy harvesting from small energy sources (i.e. credit card size solar panel in 500lux indoor lighting conditions) a feasible approach. The 50mm × 54mm thermoelectric energy harvester prototype generates 0.95mW when placed on a 60oC heat source with 28% conversion efficiency. Both prototypes can be used to continuously power WSN for building energy management applications in typical office building environment. In addition to the hardware development, a comprehensive system energy model has been developed. This system energy model not only can be used to predict the available and consumed energy based on real-world ambient conditions, but also can be employed to optimize the system design and configuration. This energy model has been verified by indoor photovoltaic energy harvesting system prototypes in long-term deployed experiments.

On efficient data integrity and data origin authentication for wireless sensor networks utilising block cipher design techniques

Analysis of the generic attacks and countermeasures for block cipher based message authentication code algorithms (MAC) in sensor applications is undertaken; the conclusions are used in the design of two new MAC constructs Quicker Block Chaining MAC1 (QBC-MAC1) and Quicker Block Chaining MAC2 (QBC-MAC2). Using software simulation we show that our new constructs point to improvements in usage of CPU instruction clock cycle and energy requirement when benchmarked against the de facto Cipher Block Chaining MAC (CBC-MAC) based construct used in the TinySec security protocol for wireless sensor networks.