Two timescale convergent Q-learning for sleep-scheduling in wireless sensor networks

In this paper, we consider an intrusion detection application for Wireless Sensor Networks. We study the problem of scheduling the sleep times of the individual sensors, where the objective is to maximize the network lifetime while keeping the tracking error to a minimum. We formulate this problem as a partially-observable Markov decision process (POMDP) with continuous stateaction spaces, in a manner similar to Fuemmeler and Veeravalli (IEEE Trans Signal Process 56(5), 2091-2101, 2008). However, unlike their formulation, we consider infinite horizon discounted and average cost objectives as performance criteria. For each criterion, we propose a convergent on-policy Q-learning algorithm that operates on two timescales, while employing function approximation. Feature-based representations and function approximation is necessary to handle the curse of dimensionality associated with the underlying POMDP. Our proposed algorithm incorporates a policy gradient update using a one-simulation simultaneous perturbation stochastic approximation estimate on the faster timescale, while the Q-value parameter (arising from a linear function approximation architecture for the Q-values) is updated in an on-policy temporal difference algorithm-like fashion on the slower timescale. The feature selection scheme employed in each of our algorithms manages the energy and tracking components in a manner that assists the search for the optimal sleep-scheduling policy. For the sake of comparison, in both discounted and average settings, we also develop a function approximation analogue of the Q-learning algorithm. This algorithm, unlike the two-timescale variant, does not possess theoretical convergence guarantees. Finally, we also adapt our algorithms to include a stochastic iterative estimation scheme for the intruder's mobility model and this is useful in settings where the latter is not known. Our simulation results on a synthetic 2-dimensional network setting suggest that our algorithms result in better tracking accuracy at the cost of only a few additional sensors, in comparison to a recent prior work.

An Eigen Approach to Transmit Beamforming in Wireless Networks With Single Antenna Receivers

In this paper, we propose an eigen framework for transmit beamforming for single-hop and dual-hop network models with single antenna receivers. In cases where number of receivers is not more than three, the proposed Eigen approach is vastly superior in terms of ease of implementation and computational complexity compared with the existing convex-relaxation-based approaches. The essential premise is that the precoding problems can be posed as equivalent optimization problems of searching for an optimal vector in the joint numerical range of Hermitian matrices. We show that the latter problem has two convex approximations: the first one is a semi-definite program that yields a lower bound on the solution, and the second one is a linear matrix inequality that yields an upper bound on the solution. We study the performance of the proposed and existing techniques using numerical simulations.

Analytical evaluation of harmonic distortion factor corresponding to generalised advanced bus-clamping pulse width modulation

A few advanced bus-clamping pulse width modulation (ABCPWM) methods have been proposed recently for a three-phase inverter. With these methods, each phase is clamped, switched at nominal frequency, and switched at twice the nominal frequency in different regions of the fundamental cycle. This study proposes a generalised ABCPWM scheme, encompassing the few ABCPWM schemes that have been proposed and many more ABCPWM schemes that have not been reported yet. Furthermore, analytical closed-form expression is derived for the harmonic distortion factor corresponding to the generalised ABCPWM. This factor is independent of load parameters. The analytical expression derived here brings out the dependence of root-mean-square (RMS) current ripple on modulation index, and can be used to evaluate the RMS current ripple corresponding to any ABCPWM scheme. The analytical closed-form expression is validated experimentally in terms of measured weighted total harmonic distortion (THD) in line voltage (V-WTHD) and measured THD in line current (I-THD) on a 6 kW induction motor drive.

Universal correction to higher spin entanglement entropy

We consider conformal field theories in 1 + 1 dimensions with W-algebra symmetries, deformed by a chemical potential mu for the spin-three current. We show that the order mu(2) correction to the Renyi and entanglement entropies of a single interval in the deformed theory, on the infinite spatial line and at finite temperature, is universal. The correction is completely determined by the operator product expansion of two spin-three currents, and by the expectation values of the stress tensor, its descendants and its composites, evaluated on the n-sheeted Riemann surface branched along the interval. This explains the recently found agreement of the order mu(2) correction across distinct free field CFTs and higher spin black hole solutions holographically dual to CFTs with W symmetry.

Energy-efficient, delay-constrained, QoS-aware broadcast for cooperative wireless sensor networks

The problem of delay-constrained, energy-efficient broadcast in cooperative wireless networks is NP-complete. While centralised setting allows some heuristic solutions, designing heuristics in distributed implementation poses significant challenges. This is more so in wireless sensor networks (WSNs) where nodes are deployed randomly and topology changes dynamically due to node failure/join and environment conditions. This paper demonstrates that careful design of network infrastructure can achieve guaranteed delay bounds and energy-efficiency, and even meet quality of service requirements during broadcast. The paper makes three prime contributions. First, we present an optimal lower bound on energy consumption for broadcast that is tighter than what has been previously proposed. Next, iSteiner, a lightweight, distributed and deterministic algorithm for creation of network infrastructure is discussed. iPercolate is the algorithm that exploits this structure to cooperatively broadcast information with guaranteed delivery and delay bounds, while allowing real-time traffic to pass undisturbed.

Discrete-Rate Adaptation and Selection in Energy Harvesting Wireless Systems

In a system with energy harvesting (EH) nodes, the design focus shifts from minimizing energy consumption by infrequently transmitting less information to making the best use of available energy to efficiently deliver data while adhering to the fundamental energy neutrality constraint. We address the problem of maximizing the throughput of a system consisting of rate-adaptive EH nodes that transmit to a destination. Unlike related literature, we focus on the practically important discrete-rate adaptation model. First, for a single EH node, we propose a discrete-rate adaptation rule and prove its optimality for a general class of stationary and ergodic EH and fading processes. We then study a general system with multiple EH nodes in which one is opportunistically selected to transmit. We first derive a novel and throughput-optimal joint selection and rate adaptation rule (TOJSRA) when the nodes are subject to a weaker average power constraint. We then propose a novel rule for a multi-EH node system that is based on TOJSRA, and we prove its optimality for stationary and ergodic EH and fading processes. We also model the various energy overheads of the EH nodes and characterize their effect on the adaptation policy and the system throughput.

Structural and functional analysis of two universal stress proteins YdaA and YnaF from Salmonella typhimurium: possible roles in microbial stress tolerance

In many organisms ``Universal Stress Proteins'' CUSPS) are induced in response to a variety of environmental stresses. Here we report the structures of two USPs, YnaF and YdaA from Salmonella typhimurium determined at 1.8 angstrom and 2.4 angstrom resolutions, respectively. YnaF consists of a single USP domain and forms a tetrameric organization stabilized by interactions mediated through chloride ions. YdaA is a larger protein consisting of two tandem USP domains. Two protomers of YdaA associate to form a structure similar to the YnaF tetramer. YdaA showed ATPase activity and an ATP binding motif G-2X-G-9X-G(S/T/N) was found in its C-terminal domain. The residues corresponding to this motif were not conserved in YnaF although YnaF could bind ATP. However, unlike YdaA, YnaF did not hydrolyse ATP in vitro. Disruption of interactions mediated through chloride ions by selected mutations converted YnaF into an ATPase. Residues that might be important for ATP hydrolysis could be identified by comparing the active sites of native and mutant structures. Only the C-terminal domain of YdaA appears to be involved in ATP hydrolysis. The structurally similar N-terminal domain was found to bind a zinc ion near the segment equivalent to the phosphate binding loop of the C-terminal domain. Mass spectrometric analysis showed that YdaA might bind a ligand of approximate molecular weight 800 daltons. Structural comparisons suggest that the ligand, probably related to an intermediate in lipid A biosynthesis, might bind at a site close to the zinc ion. Therefore, the N-terminal domain of YdaA binds zinc and might play a role in lipid metabolism. Thus, USPs appear to perform several distinct functions such as ATP hydrolysis, altering membrane properties and chloride sensing. (C) 2015 Elsevier Inc. All rights reserved.

Low Complexity Power and Scheduling Policies for Wireless Networks to Provide Quality of Service

We consider optimal power allocation policies for a single server, multiuser system. The power is consumed in transmission of data only. The transmission channel may experience multipath fading. We obtain very efficient, low computational complexity algorithms which minimize power and ensure stability of the data queues. We also obtain policies when the users may have mean delay constraints. If the power required is a linear function of rate then we exploit linearity and obtain linear programs with low complexity.

Low-frequency dc bus ripple cancellation in single phase pulse-width modulation inverters

This study presents a topology for a single-phase pulse-width modulation (PWM) converter which achieves low-frequency ripple reduction in the dc bus even when there are grid frequency variations. A hybrid filter is introduced to absorb the low-frequency current ripple in the dc bus. The control strategy for the proposed filter does not require the measurement of the dc bus ripple current. The design criteria for selecting the filter components are also presented in this study. The effectiveness of the proposed circuit has been tested and validated experimentally. A smaller dc-link capacitor is sufficient to keep the low-frequency bus ripple to an acceptable range in the proposed topology.

Optimal power allocation for protective jamming in wireless networks: A flow based model

We address the problem of passive eavesdroppers in multi-hop wireless networks using the technique of friendly jamming. The network is assumed to employ Decode and Forward (DF) relaying. Assuming the availability of perfect channel state information (CSI) of legitimate nodes and eavesdroppers, we consider a scheduling and power allocation (PA) problem for a multiple-source multiple-sink scenario so that eavesdroppers are jammed, and source-destination throughput targets are met while minimizing the overall transmitted power. We propose activation sets (AS-es) for scheduling, and formulate an optimization problem for PA. Several methods for finding AS-es are discussed and compared. We present an approximate linear program for the original nonlinear, non-convex PA optimization problem, and argue that under certain conditions, both the formulations produce identical results. In the absence of eavesdroppers' CSI, we utilize the notion of Vulnerability Region (VR), and formulate an optimization problem with the objective of minimizing the VR. Our results show that the proposed solution can achieve power-efficient operation while defeating eavesdroppers and achieving desired source-destination throughputs simultaneously. (C) 2015 Elsevier B.V. All rights reserved.

A Universal Stress Protein (USP) in Mycobacteria Binds cAMP

Mycobacteria are endowed with rich and diverse machinery for the synthesis, utilization, and degradation of cAMP. The actions of cyclic nucleotides are generally mediated by binding of cAMP to conserved and well characterized cyclic nucleotide binding domains or structurally distinct cGMP-specific and -regulated cyclic nucleotide phosphodiesterase, adenylyl cyclase, and E. coli transcription factor FhlA (GAF) domain-containing proteins. Proteins with cyclic nucleotide binding and GAF domains can be identified in the genome of mycobacterial species, and some of them have been characterized. Here, we show that a significant fraction of intracellular cAMP is bound to protein in mycobacterial species, and by using affinity chromatography techniques, we identify specific universal stress proteins (USP) as abundantly expressed cAMP-binding proteins in slow growing as well as fast growing mycobacteria. We have characterized the biochemical and thermodynamic parameters for binding of cAMP, and we show that these USPs bind cAMP with a higher affinity than ATP, an established ligand for other USPs. We determined the structure of the USP MSMEG_3811 bound to cAMP, and we confirmed through structure-guided mutagenesis, the residues important for cAMP binding. This family of USPs is conserved in all mycobacteria, and we suggest that they serve as ``sinks'' for cAMP, making this second messenger available for downstream effectors as and when ATP levels are altered in the cell.

Wireless Aircraft Fuel Quantity Indication System

A wireless fuel quantity indication system (FQIS) has been developed using an RFID-enabled sensing platform. The system comprises a fully passive tag, modified reader protocol, capacitive fuel probe, and auxiliary antenna for additional energy harvesting. Results of fluid testing show sensitivity to changes in fluid height of less than 0.25in. An RF-DC harvesting circuit was developed, which delivers up to 5dBm of input power through a remote radio frequency (RF) source. Testing was conducted in a loaded reverberation chamber to emulate the fuel tank environment. Results demonstrate feasibility of the remote source to power the sensor with less than 1W of maximum transmit power and under 100ms dwell time (100mW average power) into the tank. This indicates adequate coverage for large transport aircraft at safe operating levels with a sample rate of up to 1 sample/s.

Optimal Power Allocation Policies for Low Powered Wireless Communication Systems

We consider optimal average power allocation policies in a wireless channel in the presence of individual delay constraints on the transmitted packets. Power is consumed in transmission of data only. We consider the case when the power used in transmission is a linear function of the data transmitted. The transmission channel may experience multipath fading. We have developed a computationally efficient online algorithm, when there is same hard delay constraint for all packets. Later on, we generalize it to the case when there are multiple real time streams with different hard deadline constraints. Our algorithm uses linear programming and has very low complexity.

Relay Selection with Channel Probing in Sleep-Wake Cycling Wireless Sensor Networks

In geographical forwarding of packets in a large wireless sensor network (WSN) with sleep-wake cycling nodes, we are interested in the local decision problem faced by a node that has ``custody'' of a packet and has to choose one among a set of next-hop relay nodes to forward the packet toward the sink. Each relay is associated with a ``reward'' that summarizes the benefit of forwarding the packet through that relay. We seek a solution to this local problem, the idea being that such a solution, if adopted by every node, could provide a reasonable heuristic for the end-to-end forwarding problem. Toward this end, we propose a local relay selection problem consisting of a forwarding node and a collection of relay nodes, with the relays waking up sequentially at random times. At each relay wake-up instant, the forwarder can choose to probe a relay to learn its reward value, based on which the forwarder can then decide whether to stop (and forward its packet to the chosen relay) or to continue to wait for further relays to wake up. The forwarder's objective is to select a relay so as to minimize a combination of waiting delay, reward, and probing cost. The local decision problem can be considered as a variant of the asset selling problem studied in the operations research literature. We formulate the local problem as a Markov decision process (MDP) and characterize the solution in terms of stopping sets and probing sets. We provide results illustrating the structure of the stopping sets, namely, the (lower bound) threshold and the stage independence properties. Regarding the probing sets, we make an interesting conjecture that these sets are characterized by upper bounds. Through simulation experiments, we provide valuable insights into the performance of the optimal local forwarding and its use as an end-to-end forwarding heuristic.

Algorithm for Autonomous Reorganization of Mobile Wireless Camera Sensor Networks to Improve Coverage

In this paper, sensing coverage by wireless camera-embedded sensor networks (WCSNs), a class of directional sensors is studied. The proposed work facilitates the autonomous tuning of orientation parameters and displacement of camera-sensor nodes in the bounded field of interest (FoI), where the network coverage in terms of every point in the FoI is important. The proposed work is first of its kind to study the problem of maximizing coverage of randomly deployed mobile WCSNs which exploits their mobility. We propose an algorithm uncovered region exploration algorithm (UREA-CS) that can be executed in centralized and distributed modes. Further, the work is extended for two special scenarios: 1) to suit autonomous combing operations after initial random WCSN deployments and 2) to improve the network coverage with occlusions in the FoI. The extensive simulation results show that the performance of UREA-CS is consistent, robust, and versatile to achieve maximum coverage, both in centralized and distributed modes. The centralized and distributed modes are further analyzed with respect to the computational and communicational overheads.