Two timescale convergent Q-learning for sleep-scheduling in wireless sensor networks

In this paper, we consider an intrusion detection application for Wireless Sensor Networks. We study the problem of scheduling the sleep times of the individual sensors, where the objective is to maximize the network lifetime while keeping the tracking error to a minimum. We formulate this problem as a partially-observable Markov decision process (POMDP) with continuous stateaction spaces, in a manner similar to Fuemmeler and Veeravalli (IEEE Trans Signal Process 56(5), 2091-2101, 2008). However, unlike their formulation, we consider infinite horizon discounted and average cost objectives as performance criteria. For each criterion, we propose a convergent on-policy Q-learning algorithm that operates on two timescales, while employing function approximation. Feature-based representations and function approximation is necessary to handle the curse of dimensionality associated with the underlying POMDP. Our proposed algorithm incorporates a policy gradient update using a one-simulation simultaneous perturbation stochastic approximation estimate on the faster timescale, while the Q-value parameter (arising from a linear function approximation architecture for the Q-values) is updated in an on-policy temporal difference algorithm-like fashion on the slower timescale. The feature selection scheme employed in each of our algorithms manages the energy and tracking components in a manner that assists the search for the optimal sleep-scheduling policy. For the sake of comparison, in both discounted and average settings, we also develop a function approximation analogue of the Q-learning algorithm. This algorithm, unlike the two-timescale variant, does not possess theoretical convergence guarantees. Finally, we also adapt our algorithms to include a stochastic iterative estimation scheme for the intruder's mobility model and this is useful in settings where the latter is not known. Our simulation results on a synthetic 2-dimensional network setting suggest that our algorithms result in better tracking accuracy at the cost of only a few additional sensors, in comparison to a recent prior work.

Linear Transceiver Design for an Amplify-and-Forward Relay Based on the MBER Criterion

A design methodology based on the Minimum Bit Error Ratio (MBER) framework is proposed for a non-regenerative Multiple-Input Multiple-Output (MIMO) relay-aided system to determine various linear parameters. We consider both the Relay-Destination (RD) as well as the Source-Relay-Destination (SRD) link design based on this MBER framework, including the pre-coder, the Amplify-and-Forward (AF) matrix and the equalizer matrix of our system. It has been shown in the previous literature that MBER based communication systems are capable of reducing the Bit-Error-Ratio (BER) compared to their Linear Minimum Mean Square Error (LMMSE) based counterparts. We design a novel relay-aided system using various signal constellations, ranging from QPSK to the general M-QAM and M-PSK constellations. Finally, we propose its sub-optimal versions for reducing the computational complexity imposed. Our simulation results demonstrate that the proposed scheme indeed achieves a significant BER reduction over the existing LMMSE scheme.

AN OPTIMUM SHRINKAGE ESTIMATOR BASED ON MINIMUM-PROBABILITY-OF-ERROR CRITERION AND APPLICATION TO SIGNAL DENOISING

We address the problem of designing an optimal pointwise shrinkage estimator in the transform domain, based on the minimum probability of error (MPE) criterion. We assume an additive model for the noise corrupting the clean signal. The proposed formulation is general in the sense that it can handle various noise distributions. We consider various noise distributions (Gaussian, Student's-t, and Laplacian) and compare the denoising performance of the estimator obtained with the mean-squared error (MSE)-based estimators. The MSE optimization is carried out using an unbiased estimator of the MSE, namely Stein's Unbiased Risk Estimate (SURE). Experimental results show that the MPE estimator outperforms the SURE estimator in terms of SNR of the denoised output, for low (0 -10 dB) and medium values (10 - 20 dB) of the input SNR.

Discrete-Rate Adaptation and Selection in Energy Harvesting Wireless Systems

In a system with energy harvesting (EH) nodes, the design focus shifts from minimizing energy consumption by infrequently transmitting less information to making the best use of available energy to efficiently deliver data while adhering to the fundamental energy neutrality constraint. We address the problem of maximizing the throughput of a system consisting of rate-adaptive EH nodes that transmit to a destination. Unlike related literature, we focus on the practically important discrete-rate adaptation model. First, for a single EH node, we propose a discrete-rate adaptation rule and prove its optimality for a general class of stationary and ergodic EH and fading processes. We then study a general system with multiple EH nodes in which one is opportunistically selected to transmit. We first derive a novel and throughput-optimal joint selection and rate adaptation rule (TOJSRA) when the nodes are subject to a weaker average power constraint. We then propose a novel rule for a multi-EH node system that is based on TOJSRA, and we prove its optimality for stationary and ergodic EH and fading processes. We also model the various energy overheads of the EH nodes and characterize their effect on the adaptation policy and the system throughput.

Shape and size directed self-selection in organic cage formation

The selective formation of a single isomer of a 3+2] self-assembled organic cage from a reaction mixture of an unsymmetrical aldehyde and a flexible amine is discussed. The experimental and theoretical findings suggest that in such a process, the geometric features of the aldehyde play a key role.

Quick, Decentralized, Energy-Efficient One-Shot Max Function Computation Using Timer-Based Selection

In several wireless sensor networks, it is of interest to determine the maximum of the sensor readings and identify the sensor responsible for it. We propose a novel, decentralized, scalable, energy-efficient, timer-based, one-shot max function computation (TMC) algorithm. In it, the sensor nodes do not transmit their readings in a centrally pre-defined sequence. Instead, the nodes are grouped into clusters, and computation occurs over two contention stages. First, the nodes in each cluster contend with each other using the timer scheme to transmit their reading to their cluster-heads. Thereafter, the cluster-heads use the timer scheme to transmit the highest sensor reading in their cluster to the fusion node. One new challenge is that the use of the timer scheme leads to collisions, which can make the algorithm fail. We optimize the algorithm to minimize the average time required to determine the maximum subject to a constraint on the probability that it fails to find the maximum. TMC significantly lowers average function computation time, average number of transmissions, and average energy consumption compared to approaches proposed in the literature.

Timer-Based Distributed Node Selection Scheme Exploiting Power Control and Capture

Opportunistic selection in multi-node wireless systems improves system performance by selecting the ``best'' node and by using it for data transmission. In these systems, each node has a real-valued local metric, which is a measure of its ability to improve system performance. Our goal is to identify the best node, which has the largest metric. We propose, analyze, and optimize a new distributed, yet simple, node selection scheme that combines the timer scheme with power control. In it, each node sets a timer and transmit power level as a function of its metric. The power control is designed such that the best node is captured even if. other nodes simultaneously transmit with it. We develop several structural properties about the optimal metric-to-timer-and-power mapping, which maximizes the probability of selecting the best node. These significantly reduce the computational complexity of finding the optimal mapping and yield valuable insights about it. We show that the proposed scheme is scalable and significantly outperforms the conventional timer scheme. We investigate the effect of. and the number of receive power levels. Furthermore, we find that the practical peak power constraint has a negligible impact on the performance of the scheme.

Direct Link-Aware Optimal Relay Selection and a Low Feedback Variant for Underlay CR

Cooperative relaying combined with selection has been extensively studied in the literature to improve the performance of interference-constrained secondary users in underlay cognitive radio (CR). We present a novel symbol error probability (SEP)-optimal amplify-and-forward relay selection rule for an average interference-constrained underlay CR system. A fundamental principle, which is unique to average interference-constrained underlay CR, that the proposed rule brings out is that the choice of the optimal relay is affected not just by the source-to-relay, relay-to-destination, and relay-to-primary receiver links, which are local to the relay, but also by the direct source-to-destination (SD) link, even though it is not local to any relay. We also propose a simpler, practically amenable variant of the optimal rule called the 1-bit rule, which requires just one bit of feedback about the SD link gain to the relays, and incurs a marginal performance loss relative to the optimal rule. We analyze its SEP and develop an insightful asymptotic SEP analysis. The proposed rules markedly outperform several ad hoc SD link-unaware rules proposed in the literature. They also generalize the interference-unconstrained and SD link-unaware optimal rules considered in the literature.

Relay Selection with Channel Probing in Sleep-Wake Cycling Wireless Sensor Networks

In geographical forwarding of packets in a large wireless sensor network (WSN) with sleep-wake cycling nodes, we are interested in the local decision problem faced by a node that has ``custody'' of a packet and has to choose one among a set of next-hop relay nodes to forward the packet toward the sink. Each relay is associated with a ``reward'' that summarizes the benefit of forwarding the packet through that relay. We seek a solution to this local problem, the idea being that such a solution, if adopted by every node, could provide a reasonable heuristic for the end-to-end forwarding problem. Toward this end, we propose a local relay selection problem consisting of a forwarding node and a collection of relay nodes, with the relays waking up sequentially at random times. At each relay wake-up instant, the forwarder can choose to probe a relay to learn its reward value, based on which the forwarder can then decide whether to stop (and forward its packet to the chosen relay) or to continue to wait for further relays to wake up. The forwarder's objective is to select a relay so as to minimize a combination of waiting delay, reward, and probing cost. The local decision problem can be considered as a variant of the asset selling problem studied in the operations research literature. We formulate the local problem as a Markov decision process (MDP) and characterize the solution in terms of stopping sets and probing sets. We provide results illustrating the structure of the stopping sets, namely, the (lower bound) threshold and the stage independence properties. Regarding the probing sets, we make an interesting conjecture that these sets are characterized by upper bounds. Through simulation experiments, we provide valuable insights into the performance of the optimal local forwarding and its use as an end-to-end forwarding heuristic.

Lead selection and characterization of antitubercular compounds using the Nested Chemical Library

Discovering new drugs to treat tuberculosis more efficiently and to overcome multidrug resistance is a world health priority. To find novel antitubercular agents several approaches have been used in various institutions worldwide, including target-based approaches against several validated mycobacterial enzymes and phenotypic screens. We screened more than 17,000 compounds from Vichem's Nested Chemical Library(TM) using an integrated strategy involving whole cell-based assays with Corynebacterium glutamicum and Mycobacterium tuberculosis, and target-based assays with protein kinases PknA, PknB and PknG as well as other targets such as PimA and bacterial topoisomerases simultaneously. With the help of the target-based approach we have found very potent hits inhibiting the selected target enzymes, but good minimal inhibitory concentrations (MIC) against M. tuberculosis were not achieved. Focussing on the whole cell-based approach several potent hits were found which displayed minimal inhibitory concentrations (MIC) against M. tuberculosis below 10 mu M and were non-mutagenic, non-cytotoxic and the targets of some of the hits were also identified. The most active hits represented various scaffolds. Medicinal chemistry-based lead optimization was performed applying various strategies and, as a consequence, a series of novel potent compounds were synthesized. These efforts resulted in some effective potential antitubercular lead compounds which were confirmed in phenotypic assays. (C) 2015 Elsevier Ltd. All rights reserved.

Quantifying the Transmit Diversity Order of Euclidean Distance Based Antenna Selection in Spatial Modulation

In this letter, we quantify the transmit diversity order of the SM system operating in a closed-loop scenario. Specifically, the SM system relying on Euclidean distance based antenna subset selection (EDAS) is considered and the achievable diversity gain is evaluated. Furthermore, the resultant trade-off between the achievable diversity gain and switching gain is studied. Simulation results confirm our theoretical results. Specifically, at a symbol error rate of about 10(-4) the signal-to-noise ratio gain achieved by EDAS is about 7 dB in case of 16-QAM and about 5 dB in case of 64-QAM.

Novel Relay Selection Rules for Average Interference-Constrained Cognitive AF Relay Networks

Cooperative relaying combined with selection exploits spatial diversity to significantly improve the performance of interference-constrained secondary users in an underlay cognitive radio (CR) network. However, unlike conventional relaying, the state of the links between the relay and the primary receiver affects the choice of the relay. Further, while the optimal amplify-and-forward (AF) relay selection rule for underlay CR is well understood for the peak interference-constraint, this is not so for the less conservative average interference constraint. For the latter, we present three novel AF relay selection (RS) rules, namely, symbol error probability (SEP)-optimal, inverse-of-affine (IOA), and linear rules. We analyze the SEPs of the IOA and linear rules and also develop a novel, accurate approximation technique for analyzing the performance of AF relays. Extensive numerical results show that all the three rules outperform several RS rules proposed in the literature and generalize the conventional AF RS rule.

Combinatorial selection of molecular conformations and supramolecular synthons in quercetin cocrystal landscapes: a route to ternary solids

The crystallization of 28 binary and ternary cocrystals of quercetin with dibasic coformers is analyzed in terms of a combinatorial selection from a solution of preferred molecular conformations and supramolecular synthons. The crystal structures are characterized by distinctive O-H center dot center dot center dot N and O-H center dot center dot center dot O based synthons and are classified as nonporous, porous and helical. Variability in molecular conformation and synthon structure led to an increase in the energetic and structural space around the crystallization event. This space is the crystal structure landscape of the compound and is explored by fine-tuning the experimental conditions of crystallization. In the landscape context, we develop a strategy for the isolation of ternary cocrystals with the use of auxiliary template molecules to reduce the molecular and supramolecular `confusion' that is inherent in a molecule like quercetin. The absence of concomitant polymorphism in this study highlights the selectivity in conformation and synthon choice from the virtual combinatorial library in solution.

Sampling Period Selection of Control Application Connected to FlexRay Bus

Flexray is a high speed communication protocol designed for distributive control in automotive control applications. Control performance not only depends on the control algorithm but also on the scheduling constraints in communication. A balance between the control performance and communication constraints must required for the choice of the sampling rates of the control loops in a node. In this paper, an optimum sampling period of control loops to minimize the cost function, satisfying the scheduling constraints is obtained. An algorithm to obtain the delay in service of each task in a node of the control loop in the hyper period has been also developed. (C) 2015 The Authors. Published by Elsevier B.V.

Optimum Parameter Selection in Sparse Reconstruction of Frequency-Domain Optical-Coherence Tomography Signals

For a multilayered specimen, the back-scattered signal in frequency-domain optical-coherence tomography (FDOCT) is expressible as a sum of cosines, each corresponding to a change of refractive index in the specimen. Each of the cosines represent a peak in the reconstructed tomogram. We consider a truncated cosine series representation of the signal, with the constraint that the coefficients in the basis expansion be sparse. An l(2) (sum of squared errors) data error is considered with an l(1) (summation of absolute values) constraint on the coefficients. The optimization problem is solved using Weiszfeld's iteratively reweighted least squares (IRLS) algorithm. On real FDOCT data, improved results are obtained over the standard reconstruction technique with lower levels of background measurement noise and artifacts due to a strong l(1) penalty. The previous sparse tomogram reconstruction techniques in the literature proposed collecting sparse samples, necessitating a change in the data capturing process conventionally used in FDOCT. The IRLS-based method proposed in this paper does not suffer from this drawback.