Feature Search in the Grassmanian in Online Reinforcement Learning

We consider the problem of finding the best features for value function approximation in reinforcement learning and develop an online algorithm to optimize the mean square Bellman error objective. For any given feature value, our algorithm performs gradient search in the parameter space via a residual gradient scheme and, on a slower timescale, also performs gradient search in the Grassman manifold of features. We present a proof of convergence of our algorithm. We show empirical results using our algorithm as well as a similar algorithm that uses temporal difference learning in place of the residual gradient scheme for the faster timescale updates.

Transmit Power Control Policies for Energy Harvesting Sensors With Retransmissions

This paper addresses the problem of finding outage-optimal power control policies for wireless energy harvesting sensor (EHS) nodes with automatic repeat request (ARQ)-based packet transmissions. The power control policy of the EHS specifies the transmission power for each packet transmission attempt, based on all the information available at the EHS. In particular, the acknowledgement (ACK) or negative acknowledgement (NACK) messages received provide the EHS with partial information about the channel state. We solve the problem of finding an optimal power control policy by casting it as a partially observable Markov decision process (POMDP). We study the structure of the optimal power policy in two ways. First, for the special case of binary power levels at the EHS, we show that the optimal policy for the underlying Markov decision process (MDP) when the channel state is observable is a threshold policy in the battery state. Second, we benchmark the performance of the EHS by rigorously analyzing the outage probability of a general fixed-power transmission scheme, where the EHS uses a predetermined power level at each slot within the frame. Monte Carlo simulation results illustrate the performance of the POMDP approach and verify the accuracy of the analysis. They also show that the POMDP solutions can significantly outperform conventional ad hoc approaches.

Wireless Network-Coded Bidirectional Relaying Using Latin Squares for M-PSK Modulation

The design of modulation schemes for the physical layer network-coded two-way relaying scenario is considered with a protocol which employs two phases: multiple access (MA) phase and broadcast (BC) phase. It was observed by Koike-Akino et al. that adaptively changing the network coding map used at the relay according to the channel conditions greatly reduces the impact of MA interference which occurs at the relay during the MA phase and all these network coding maps should satisfy a requirement called the exclusive law. We show that every network coding map that satisfies the exclusive law is representable by a Latin Square and conversely, that this relationship can be used to get the network coding maps satisfying the exclusive law. The channel fade states for which the minimum distance of the effective constellation at the relay become zero are referred to as the singular fade states. For M - PSK modulation (M any power of 2), it is shown that there are (M-2/4 - M/2 + 1) M singular fade states. Also, it is shown that the constraints which the network coding maps should satisfy so that the harmful effects of the singular fade states are removed, can be viewed equivalently as partially filled Latin Squares (PFLS). The problem of finding all the required maps is reduced to finding a small set of maps for M - PSK constellations (any power of 2), obtained by the completion of PFLS. Even though the completability of M x M PFLS using M symbols is an open problem, specific cases where such a completion is always possible are identified and explicit construction procedures are provided. Having obtained the network coding maps, the set of all possible channel realizations (the complex plane) is quantized into a finite number of regions, with a specific network coding map chosen in a particular region. It is shown that the complex plane can be partitioned into two regions: a region in which any network coding map which satisfies the exclusive law gives the same best performance and a region in which the choice of the network coding map affects the performance. The quantization thus obtained analytically, leads to the same as the one obtained using computer search for M = 4-PSK signal set by Koike-Akino et al., when specialized for Simulation results show that the proposed scheme performs better than the conventional exclusive-OR (XOR) network coding and in some cases outperforms the scheme proposed by Koike-Akino et al.

Design and Evaluation of a Robust Optical Beam-Interruption-Based Vehicle Classifier System

This paper presents the design and development of a novel optical vehicle classifier system, which is based on interruption of laser beams, that is suitable for use in places with poor transportation infrastructure. The system can estimate the speed, axle count, wheelbase, tire diameter, and the lane of motion of a vehicle. The design of the system eliminates the need for careful optical alignment, whereas the proposed estimation strategies render the estimates insensitive to angular mounting errors and to unevenness of the road. Strategies to estimate vehicular parameters are described along with the optimization of the geometry of the system to minimize estimation errors due to quantization. The system is subsequently fabricated, and the proposed features of the system are experimentally demonstrated. The relative errors in the estimation of velocity and tire diameter are shown to be within 0.5% and to change by less than 17% for angular mounting errors up to 30 degrees. In the field, the classifier demonstrates accuracy better than 97.5% and 94%, respectively, in the estimation of the wheelbase and lane of motion and can classify vehicles with an average accuracy of over 89.5%.

Proposal for Graphene-Boron Nitride Heterobilayer-Based Tunnel FET

We investigate the gate-controlled direct band-to-band tunneling (BTBT) current in a graphene-boron nitride (G-BN) heterobilayer channel-based tunnel field effect transistor. We first study the imaginary band structure of hexagonal and Bernal-stacked heterobilayers by density functional theory, which is then used to evaluate the gate-controlled current under the Wentzel-Kramers-Brillouin approximation. It is shown that the direct BTBT is probable for a certain interlayer spacing of the G-BN which depends on the stacking orders.

A Novel Monte-Carlo-Sampling-Based Receiver for Large-Scale Uplink Multiuser MIMO Systems

In this paper, we propose low-complexity algorithms based on Monte Carlo sampling for signal detection and channel estimation on the uplink in large-scale multiuser multiple-input-multiple-output (MIMO) systems with tens to hundreds of antennas at the base station (BS) and a similar number of uplink users. A BS receiver that employs a novel mixed sampling technique (which makes a probabilistic choice between Gibbs sampling and random uniform sampling in each coordinate update) for detection and a Gibbs-sampling-based method for channel estimation is proposed. The algorithm proposed for detection alleviates the stalling problem encountered at high signal-to-noise ratios (SNRs) in conventional Gibbs-sampling-based detection and achieves near-optimal performance in large systems with M-ary quadrature amplitude modulation (M-QAM). A novel ingredient in the detection algorithm that is responsible for achieving near-optimal performance at low complexity is the joint use of a mixed Gibbs sampling (MGS) strategy coupled with a multiple restart (MR) strategy with an efficient restart criterion. Near-optimal detection performance is demonstrated for a large number of BS antennas and users (e. g., 64 and 128 BS antennas and users). The proposed Gibbs-sampling-based channel estimation algorithm refines an initial estimate of the channel obtained during the pilot phase through iterations with the proposed MGS-based detection during the data phase. In time-division duplex systems where channel reciprocity holds, these channel estimates can be used for multiuser MIMO precoding on the downlink. The proposed receiver is shown to achieve good performance and scale well for large dimensions.

An 8-to-1 bit 1-MS/s SAR ADC With VGA and Integrated Data Compression for Neural Recording

Low power consumption per channel and data rate minimization are two key challenges which need to be addressed in future generations of neural recording systems (NRS). Power consumption can be reduced by avoiding unnecessary processing whereas data rate is greatly decreased by sending spike time-stamps along with spike features as opposed to raw digitized data. Dynamic range in NRS can vary with time due to change in electrode-neuron distance or background noise, which demands adaptability. An analog-to-digital converter (ADC) is one of the most important blocks in a NRS. This paper presents an 8-bit SAR ADC in 0.13-mu m CMOS technology along with input and reference buffer. A novel energy efficient digital-to-analog converter switching scheme is proposed, which consumes 37% less energy than the present state-of-the-art. The use of a ping-pong input sampling scheme is emphasized for multichannel input to alleviate the bandwidth requirement of the input buffer. To reduce the data rate, the A/D process is only enabled through the in-built background noise rejection logic to ensure that the noise is not processed. The ADC resolution can be adjusted from 8 to 1 bit in 1-bit step based on the input dynamic range. The ADC consumes 8.8 mu W from 1 V supply at 1 MS/s speed. It achieves effective number of bits of 7.7 bits and FoM of 42.3 fJ/conversion-step.

Asynchronous Measurement of Transient Phase-Shift Resulting From RF Receiver State-Change

In a wireless receiver, a down-converted RF signal undergoes a transient phase shift, when the gain state is changed to adjust for varying conditions in transmission and propagation. A method is developed, in which such phase shifts are detected asynchronously, and their undesirable effects on the bit error rate are corrected. The method was developed for and used in, the system-level characterization and calibration of a 65-nm CMOS UHF receiver. The phase-shifts associated with specific gain-state transitions were measured within a test framework, and used in the baseband signal processing blocks to compensate for errors, whenever the receiver anticipated a gain-state transition.

Solution of Time Dependent Joule Heat Equation for a Graphene Sheet Under Thomson Effect

We address a physics-based solution of joule heating phenomenon in a single-layer graphene (SLG) sheet under the presence of Thomson effect. We demonstrate that the temperature in an isotopically pure (containing only C-12) SLG sheet attains its saturation level quicker than when doped with its isotopes (C-13). From the solution of the joule heating equation, we find that the thermal time constant of the SLG sheet is in the order of tenths of a nanosecond for SLG dimensions of a few micrometers. These results have been formulated using the electron interactions with the inplane and flexural phonons to demonstrate a field-dependent Landauer transmission coefficient. We further develop an analytical model of the SLG specific heat using the quadratic (out of plane) phonon band structure over the room temperature. Additionally, we show that a cooling effect in the SLG sheet can be substantially enhanced with the addition of C-13. The methodologies as discussed in this paper can be put forward to analyze the graphene heat spreader theory.

Estimation of Series Capacitance for a Three-Phase Transformer Winding From Its Measured Frequency Response

Recently, authors published a method to indirectly measure series capacitance (C-s) of a single, isolated, uniformly wound transformer winding, from its measured frequency response. The next step was to implement it on an actual three-phase transformer. This task is not as straightforward as it might appear at first glance, since the measured frequency response on a three-phase transformer is influenced by nontested windings and their terminal connections, core, tank, etc. To extract the correct value of C-s from this composite frequency response, the formulation has to be reworked to first identify all significant influences and then include their effects. Initially, the modified method and experimental results on a three-phase transformer (4 MVA, 33 kV/433 V) are presented along with results on the winding considered in isolation (for cross validation). Later, the method is directly implemented on another three-phase unit (3.5 MVA, 13.8 kV/765 V) to show repeatability.

On Computing Amplitude, Phase, and Frequency Modulations Using a Vector Interpretation of the Analytic Signal

The amplitude-modulation (AM) and phase-modulation (PM) of an amplitude-modulated frequency-modulated (AM-FM) signal are defined as the modulus and phase angle, respectively, of the analytic signal (AS). The FM is defined as the derivative of the PM. However, this standard definition results in a PM with jump discontinuities in cases when the AM index exceeds unity, resulting in an FM that contains impulses. We propose a new approach to define smooth AM, PM, and FM for the AS, where the PM is computed as the solution to an optimization problem based on a vector interpretation of the AS. Our approach is directly linked to the fractional Hilbert transform (FrHT) and leads to an eigenvalue problem. The resulting PM and AM are shown to be smooth, and in particular, the AM turns out to be bipolar. We show an equivalence of the eigenvalue formulation to the square of the AS, and arrive at a simple method to compute the smooth PM. Some examples on synthesized and real signals are provided to validate the theoretical calculations.

Temporal Envelope Fit of Transient Audio Signals

We address the problem of temporal envelope modeling for transient audio signals. We propose the Gamma distribution function (GDF) as a suitable candidate for modeling the envelope keeping in view some of its interesting properties such as asymmetry, causality, near-optimal time-bandwidth product, controllability of rise and decay, etc. The problem of finding the parameters of the GDF becomes a nonlinear regression problem. We overcome the hurdle by using a logarithmic envelope fit, which reduces the problem to one of linear regression. The logarithmic transformation also has the feature of dynamic range compression. Since temporal envelopes of audio signals are not uniformly distributed, in order to compute the amplitude, we investigate the importance of various loss functions for regression. Based on synthesized data experiments, wherein we have a ground truth, and real-world signals, we observe that the least-squares technique gives reasonably accurate amplitude estimates compared with other loss functions.

Terminal Impact Angle Constrained Guidance Laws Using Variable Structure Systems Theory

In this brief, variable structure systems theory based guidance laws, to intercept maneuvering targets at a desired impact angle, are presented. Choosing the missile's lateral acceleration (latax) to enforce sliding mode, which is the principal operating mode of variable structure systems, on a switching surface defined by the line-of-sight angle leads to a guidance law that allows the achievement of the desired terminal impact angle. As will be shown, this law does not ensure interception for all states of the missile and the target during the engagement. Hence, additional switching surfaces are designed and a switching logic is developed that allows the latax to switch between enforcing sliding mode on one of these surfaces so that the target can be intercepted at the desired impact angle. The guidance laws are designed using nonlinear engagement dynamics for the general case of a maneuvering target.

Noncoherent Integration for Signal Detection: Analysis Under Model Uncertainties

The performance of postdetection integration (PDI) techniques for the detection of Global Navigation Satellite Systems (GNSS) signals in the presence of uncertainties in frequency offsets, noise variance, and unknown data-bits is studied. It is shown that the conventional PDI techniques are generally not robust to uncertainty in the data-bits and/or the noise variance. Two new modified PDI techniques are proposed, and they are shown to be robust to these uncertainties. The receiver operating characteristics (ROC) and sample complexity performance of the PDI techniques in the presence of model uncertainties are analytically derived. It is shown that the proposed methods significantly outperform existing methods, and hence they could become increasingly important as the GNSS receivers attempt to push the envelope on the minimum signal-to-noise ratio (SNR) for reliable detection.

A Medium-Voltage Inverter-Fed IM Drive Using Multilevel 12-Sided Polygonal Vectors, With Nearly Constant Switching Frequency Current Hysteresis Controller

In this paper, a current error space vector (CESV)-based hysteresis current controller for a multilevel 12-sided voltage space vector-based inverter-fed induction motor (IM) drive is proposed. The proposed controller gives a nearly constant switching frequency operation throughout different speeds in the linear modulation region. It achieves the elimination of 6n +/- 1, n = odd harmonics from the phase voltages and currents in the entire modulation range, with an increase in the linear modulation range. It also exhibits fast dynamic behavior under different transient conditions and has a simple controller implementation. Nearly constant switching frequency is obtained by matching the steady-state CESV boundaries of the proposed controller with that of a constant switching frequency SVPWM-based drive. In the proposed controller, the CESV reference boundaries are computed online, using the switching dwell time and voltage error vector of each applied vector. These quantities are calculated from estimated sampled reference phase voltages. Vector change is decided by projecting the actual current error along the computed hysteresis space vector boundary of the presently applied vector. The estimated reference phase voltages are found from the stator current error ripple and the parameters of the IM.