Two timescale convergent Q-learning for sleep-scheduling in wireless sensor networks

In this paper, we consider an intrusion detection application for Wireless Sensor Networks. We study the problem of scheduling the sleep times of the individual sensors, where the objective is to maximize the network lifetime while keeping the tracking error to a minimum. We formulate this problem as a partially-observable Markov decision process (POMDP) with continuous stateaction spaces, in a manner similar to Fuemmeler and Veeravalli (IEEE Trans Signal Process 56(5), 2091-2101, 2008). However, unlike their formulation, we consider infinite horizon discounted and average cost objectives as performance criteria. For each criterion, we propose a convergent on-policy Q-learning algorithm that operates on two timescales, while employing function approximation. Feature-based representations and function approximation is necessary to handle the curse of dimensionality associated with the underlying POMDP. Our proposed algorithm incorporates a policy gradient update using a one-simulation simultaneous perturbation stochastic approximation estimate on the faster timescale, while the Q-value parameter (arising from a linear function approximation architecture for the Q-values) is updated in an on-policy temporal difference algorithm-like fashion on the slower timescale. The feature selection scheme employed in each of our algorithms manages the energy and tracking components in a manner that assists the search for the optimal sleep-scheduling policy. For the sake of comparison, in both discounted and average settings, we also develop a function approximation analogue of the Q-learning algorithm. This algorithm, unlike the two-timescale variant, does not possess theoretical convergence guarantees. Finally, we also adapt our algorithms to include a stochastic iterative estimation scheme for the intruder's mobility model and this is useful in settings where the latter is not known. Our simulation results on a synthetic 2-dimensional network setting suggest that our algorithms result in better tracking accuracy at the cost of only a few additional sensors, in comparison to a recent prior work.

Beyond classical dynamic structural plasticity using mesh-free modelling techniques

The problem of modelling the transient response of an elastic-perfectly-plastic cantilever beam, carrying an impulsively loaded tip mass, is,often referred to as the Parkes cantilever problem 25]; The permanent deformation of a cantilever struck transversely at its tip, Proc. R. Soc. A., 288, pp. 462). This paradigm for classical modelling of projectile impact on structures is re-visited and updated using the mesh-free method, smoothed particle hydrodynamics (SPH). The purpose of this study is to investigate further the behaviour of cantilever beams subjected to projectile impact at its tip, by considering especially physically real effects such as plastic shearing close to the projectile, shear deformation, and the variation of the shear strain along the length and across the thickness of the beam. Finally, going beyond macroscopic structural plasticity, a strategy to incorporate physical discontinuity (due to crack formation) in SPH discretization is discussed and explored in the context of tip-severance of the cantilever beam. Consequently, the proposed scheme illustrates the potency for a more refined treatment of penetration mechanics, paramount in the exploration of structural response under ballistic loading. The objective is to contribute to formulating a computational modelling framework within which transient dynamic plasticity and even penetration/failure phenomena for a range of materials, structures and impact conditions can be explored ab initio, this being essential for arriving at suitable tools for the design of armour systems. (C) 2014 Elsevier Ltd. All rights reserved.

An Eigen Approach to Transmit Beamforming in Wireless Networks With Single Antenna Receivers

In this paper, we propose an eigen framework for transmit beamforming for single-hop and dual-hop network models with single antenna receivers. In cases where number of receivers is not more than three, the proposed Eigen approach is vastly superior in terms of ease of implementation and computational complexity compared with the existing convex-relaxation-based approaches. The essential premise is that the precoding problems can be posed as equivalent optimization problems of searching for an optimal vector in the joint numerical range of Hermitian matrices. We show that the latter problem has two convex approximations: the first one is a semi-definite program that yields a lower bound on the solution, and the second one is a linear matrix inequality that yields an upper bound on the solution. We study the performance of the proposed and existing techniques using numerical simulations.

Optimal error estimate using mesh equidistribution technique for singularly perturbed system of reaction-diffusion boundary-value problems

In this article, we study the problem of determining an appropriate grading of meshes for a system of coupled singularly perturbed reaction-diffusion problems having diffusion parameters with different magnitudes. The central difference scheme is used to discretize the problem on adaptively generated mesh where the mesh equation is derived using an equidistribution principle. An a priori monitor function is obtained from the error estimate. A suitable a posteriori analogue of this monitor function is also derived for the mesh construction which will lead to an optimal second-order parameter uniform convergence. We present the results of numerical experiments for linear and semilinear reaction-diffusion systems to support the effectiveness of our preferred monitor function obtained from theoretical analysis. (C) 2014 Elsevier Inc. All rights reserved.

Efficient Adaptive Mesh Refinement for MoM-based Package-Board 3D Full-wave Extraction

The accurate solution of 3D full-wave Method of Moments (MoM) on an arbitrary mesh of a package-board structure does not guarantee accuracy, since the discretizations may not be fine enough to capture rapid spatial changes in the solution variable. At the same time, uniform over-meshing on the entire structure generates large number of solution variables and therefore requires an unnecessarily large matrix solution. In this work, a suitable refinement criterion for MoM based electromagnetic package-board extraction is proposed and the advantages of the adaptive strategy are demonstrated from both accuracy and speed perspectives.

Adaptive mesh generation for singularly perturbed fourth-order ordinary differential equations

In this paper, we consider a singularly perturbed boundary-value problem for fourth-order ordinary differential equation (ODE) whose highest-order derivative is multiplied by a small perturbation parameter. To solve this ODE, we transform the differential equation into a coupled system of two singularly perturbed ODEs. The classical central difference scheme is used to discretize the system of ODEs on a nonuniform mesh which is generated by equidistribution of a positive monitor function. We have shown that the proposed technique provides first-order accuracy independent of the perturbation parameter. Numerical experiments are provided to validate the theoretical results.

Energy-efficient, delay-constrained, QoS-aware broadcast for cooperative wireless sensor networks

The problem of delay-constrained, energy-efficient broadcast in cooperative wireless networks is NP-complete. While centralised setting allows some heuristic solutions, designing heuristics in distributed implementation poses significant challenges. This is more so in wireless sensor networks (WSNs) where nodes are deployed randomly and topology changes dynamically due to node failure/join and environment conditions. This paper demonstrates that careful design of network infrastructure can achieve guaranteed delay bounds and energy-efficiency, and even meet quality of service requirements during broadcast. The paper makes three prime contributions. First, we present an optimal lower bound on energy consumption for broadcast that is tighter than what has been previously proposed. Next, iSteiner, a lightweight, distributed and deterministic algorithm for creation of network infrastructure is discussed. iPercolate is the algorithm that exploits this structure to cooperatively broadcast information with guaranteed delivery and delay bounds, while allowing real-time traffic to pass undisturbed.

Performance Analysis of Space-Shift Keying in Decode-and-Forward Multihop MIMO Networks

In this paper, space-shift keying (SSK) is considered for multihop multiple-input-multiple-output (MIMO) networks. In SSK, only one among n(s) = 2(m) available transmit antennas, chosen on the basis of m information bits, is activated during transmission. We consider two different systems of multihop co-operation, where each node has multiple antennas and employs SSK. In system I, a multihop diversity relaying scheme is considered. In system II, a multihop multibranch relaying scheme is considered. In both systems, we adopt decode-and-forward (DF) relaying, where each relay forwards the signal only when it correctly decodes. We analyze the end-to-end bit error rate (BER) and diversity order of both the systems with SSK. For binary SSK (n(s) = 2), our analytical BER expression is exact, and our numerical results show that the BERs evaluated through the analytical expression overlap with those obtained through Monte Carlo simulations. For nonbinary SSK (n(s) > 2), we derive an approximate BER expression, where the analytically evaluated BER results closely follow the simulated BER results. We show the comparison of the BERs of SSK and conventional phase-shift keying (PSK) and also show the instances where SSK outperforms PSK. We also present the diversity analyses for SSK in systems I and II, which predict the achievable diversity orders as a function of system parameters.

Percolation on networks with antagonistic and dependent interactions

Drawing inspiration from real world interacting systems, we study a system consisting of two networks that exhibit antagonistic and dependent interactions. By antagonistic and dependent interactions we mean that a proportion of functional nodes in a network cause failure of nodes in the other, while failure of nodes in the other results in failure of links in the first. In contrast to interdependent networks, which can exhibit first-order phase transitions, we find that the phase transitions in such networks are continuous. Our analysis shows that, compared to an isolated network, the system is more robust against random attacks. Surprisingly, we observe a region in the parameter space where the giant connected components of both networks start oscillating. Furthermore, we find that for Erdos-Renyi and scale-free networks the system oscillates only when the dependence and antagonism between the two networks are very high. We believe that this study can further our understanding of real world interacting systems.

A Smooth Discretization Bridging Finite Element and Mesh-free Methods Using Polynomial Reproducing Simplex Splines

This work sets forth a `hybrid' discretization scheme utilizing bivariate simplex splines as kernels in a polynomial reproducing scheme constructed over a conventional Finite Element Method (FEM)-like domain discretization based on Delaunay triangulation. Careful construction of the simplex spline knotset ensures the success of the polynomial reproduction procedure at all points in the domain of interest, a significant advancement over its precursor, the DMS-FEM. The shape functions in the proposed method inherit the global continuity (Cp-1) and local supports of the simplex splines of degree p. In the proposed scheme, the triangles comprising the domain discretization also serve as background cells for numerical integration which here are near-aligned to the supports of the shape functions (and their intersections), thus considerably ameliorating an oft-cited source of inaccuracy in the numerical integration of mesh-free (MF) schemes. Numerical experiments show the proposed method requires lower order quadrature rules for accurate evaluation of integrals in the Galerkin weak form. Numerical demonstrations of optimal convergence rates for a few test cases are given and the method is also implemented to compute crack-tip fields in a gradient-enhanced elasticity model.

Adaptive Sleep-Wake Control using Reinforcement Learning in Sensor Networks

The aim in this paper is to allocate the `sleep time' of the individual sensors in an intrusion detection application so that the energy consumption from the sensors is reduced, while keeping the tracking error to a minimum. We propose two novel reinforcement learning (RL) based algorithms that attempt to minimize a certain long-run average cost objective. Both our algorithms incorporate feature-based representations to handle the curse of dimensionality associated with the underlying partially-observable Markov decision process (POMDP). Further, the feature selection scheme used in our algorithms intelligently manages the energy cost and tracking cost factors, which in turn assists the search for the optimal sleeping policy. We also extend these algorithms to a setting where the intruder's mobility model is not known by incorporating a stochastic iterative scheme for estimating the mobility model. The simulation results on a synthetic 2-d network setting are encouraging.

Adaptive Mesh Refinement for Fast Convergence of EFIE-Based 3-D Extraction

3-D full-wave method of moments (MoM) based electromagnetic analysis is a popular means toward accurate solution of Maxwell's equations. The time and memory bottlenecks associated with such a solution have been addressed over the last two decades by linear complexity fast solver algorithms. However, the accurate solution of 3-D full-wave MoM on an arbitrary mesh of a package-board structure does not guarantee accuracy, since the discretization may not be fine enough to capture spatial changes in the solution variable. At the same time, uniform over-meshing on the entire structure generates a large number of solution variables and therefore requires an unnecessarily large matrix solution. In this paper, different refinement criteria are studied in an adaptive mesh refinement platform. Consequently, the most suitable conductor mesh refinement criterion for MoM-based electromagnetic package-board extraction is identified and the advantages of this adaptive strategy are demonstrated from both accuracy and speed perspectives. The results are also compared with those of the recently reported integral equation-based h-refinement strategy. Finally, a new methodology to expedite each adaptive refinement pass is proposed.

A stabilization approach for mesh-free simulations of systems developing shocks or extreme strain localizations

A new stabilization scheme, based on a stochastic representation of the discretized field variables, is proposed with a view to reduce or even eliminate unphysical oscillations in the mesh-free numerical simulations of systems developing shocks or exhibiting localized bands of extreme deformation in the response. The origin of the stabilization scheme may be traced to nonlinear stochastic filtering and, consistent with a class of such filters, gain-based additive correction terms are applied to the simulated solution of the system, herein achieved through the element-free Galerkin method, in order to impose a set of constraints that help arresting the spurious oscillations. The method is numerically illustrated through its Applications to inviscid Burgers' equations, wherein shocks may develop as a result of intersections of the characteristics, and to a gradient plasticity model whose response is often characterized by a developing shear band as the external load is gradually increased. The potential of the method in stabilized yet accurate numerical simulations of such systems involving extreme gradient variations in the response is thus brought forth. (C) 2014 Elsevier Ltd. All rights reserved.

Low Complexity Power and Scheduling Policies for Wireless Networks to Provide Quality of Service

We consider optimal power allocation policies for a single server, multiuser system. The power is consumed in transmission of data only. The transmission channel may experience multipath fading. We obtain very efficient, low computational complexity algorithms which minimize power and ensure stability of the data queues. We also obtain policies when the users may have mean delay constraints. If the power required is a linear function of rate then we exploit linearity and obtain linear programs with low complexity.

Secure Transmission in Cooperative Networks with Weak Eavesdroppers

In this letter, we propose a scheme to improve the secrecy rate of cooperative networks using Analog Network Coding (ANC). ANC mixes the signals in the air; the desired signal is then separated out, from the mixed signals, at the legitimate receiver using techniques like self interference subtraction and signal nulling, thereby achieving better secrecy rates. Assuming global channel state information, memoryless adversaries and the decode-and-forward strategy, we seek to maximize the average secrecy rate between the source and the destination, subject to an overall power budget. Then, exploiting the structure of the optimization problem, we compute its optimal solution. Finally, we use numerical evaluations to compare our scheme with the conventional approaches.