The Borkar-Meyn theorem for asynchronous stochastic approximations

In this paper, we give a generalization of a result by Borkar and Meyn (2000) 1], on the stability and convergence of synchronous-update stochastic approximation algorithms, to the case of asynchronous stochastic approximations with delays. We then describe an interesting application of the result to asynchronous distributed temporal difference (TD) learning with function approximation and delays. (C) 2011 Elsevier B.V. All rights reserved.

A Communication-Theoretic Framework for 2-DMR Channel Modeling: Performance Evaluation of Coding and Signal Processing Methods

We develop a communication theoretic framework for modeling 2-D magnetic recording channels. Using the model, we define the signal-to-noise ratio (SNR) for the channel considering several physical parameters, such as the channel bit density, code rate, bit aspect ratio, and noise parameters. We analyze the problem of optimizing the bit aspect ratio for maximizing SNR. The read channel architecture comprises a novel 2-D joint self-iterating equalizer and detection system with noise prediction capability. We evaluate the system performance based on our channel model through simulations. The coded performance with the 2-D equalizer detector indicates similar to 5.5 dB of SNR gain over uncoded data.

Three-dimensional profiling using a still shot

A method to reliably extract object profiles even with surface discontinuities that leads to 2n pi phase jumps is proposed. The proposed method uses an amplitude-modulated Ronchi grating, which allows one to extract phase and unwrap the same with a single image. Ronchi equivalent image can be derived from modified grating image, which aids in extracting wrapped phase using Fourier transform profilometry. The amplitude of the modified grating aids in phase unwrapping. As we only need a projector that projects an amplitude-modulated grating, the proposed method allows one to extract three-dimensional profile without using full video projectors. This article also deals with noise reduction algorithms for fringe projection techniques. (C) 2014 Society of Photo-Optical Instrumentation Engineers (SPIE)

Timing Recovery Algorithms and Architectures for 2-D Magnetic Recording Systems

We investigate the problem of timing recovery for 2-D magnetic recording (TDMR) channels. We develop a timing error model for TDMR channel considering the phase and frequency offsets with noise. We propose a 2-D data-aided phase-locked loop (PLL) architecture for tracking variations in the position and movement of the read head in the down-track and cross-track directions and analyze the convergence of the algorithm under non-separable timing errors. We further develop a 2-D interpolation-based timing recovery scheme that works in conjunction with the 2-D PLL. We quantify the efficiency of our proposed algorithms by simulations over a 2-D magnetic recording channel with timing errors.

A Team of Continuous-Action Learning Automata for Noise-Tolerant Learning of Half-Spaces

Learning automata are adaptive decision making devices that are found useful in a variety of machine learning and pattern recognition applications. Although most learning automata methods deal with the case of finitely many actions for the automaton, there are also models of continuous-action-set learning automata (CALA). A team of such CALA can be useful in stochastic optimization problems where one has access only to noise-corrupted values of the objective function. In this paper, we present a novel formulation for noise-tolerant learning of linear classifiers using a CALA team. We consider the general case of nonuniform noise, where the probability that the class label of an example is wrong may be a function of the feature vector of the example. The objective is to learn the underlying separating hyperplane given only such noisy examples. We present an algorithm employing a team of CALA and prove, under some conditions on the class conditional densities, that the algorithm achieves noise-tolerant learning as long as the probability of wrong label for any example is less than 0.5. We also present some empirical results to illustrate the effectiveness of the algorithm.

l(1)-K-SVD: A robust dictionary learning algorithm with simultaneous update

We develop a new dictionary learning algorithm called the l(1)-K-svp, by minimizing the l(1) distortion on the data term. The proposed formulation corresponds to maximum a posteriori estimation assuming a Laplacian prior on the coefficient matrix and additive noise, and is, in general, robust to non-Gaussian noise. The l(1) distortion is minimized by employing the iteratively reweighted least-squares algorithm. The dictionary atoms and the corresponding sparse coefficients are simultaneously estimated in the dictionary update step. Experimental results show that l(1)-K-SVD results in noise-robustness, faster convergence, and higher atom recovery rate than the method of optimal directions, K-SVD, and the robust dictionary learning algorithm (RDL), in Gaussian as well as non-Gaussian noise. For a fixed value of sparsity, number of dictionary atoms, and data dimension, l(1)-K-SVD outperforms K-SVD and RDL on small training sets. We also consider the generalized l(p), 0 < p < 1, data metric to tackle heavy-tailed/impulsive noise. In an image denoising application, l(1)-K-SVD was found to result in higher peak signal-to-noise ratio (PSNR) over K-SVD for Laplacian noise. The structural similarity index increases by 0.1 for low input PSNR, which is significant and demonstrates the efficacy of the proposed method. (C) 2015 Elsevier B.V. All rights reserved.

Bi-criteria single machine scheduling problem with a learning effect: Aneja-Nair method to obtain the set of optimal sequences

In this paper, we consider the bi-criteria single machine scheduling problem of n jobs with a learning effect. The two objectives considered are the total completion time (TC) and total absolute differences in completion times (TADC). The objective is to find a sequence that performs well with respect to both the objectives: the total completion time and the total absolute differences in completion times. In an earlier study, a method of solving bi-criteria transportation problem is presented. In this paper, we use the methodology of solvin bi-criteria transportation problem, to our bi-criteria single machine scheduling problem with a learning effect, and obtain the set of optimal sequences,. Numerical examples are presented for illustrating the applicability and ease of understanding.

Relaxation Labeling with Learning Automata

Relaxation labeling processes are a class of mechanisms that solve the problem of assigning labels to objects in a manner that is consistent with respect to some domain-specific constraints. We reformulate this using the model of a team of learning automata interacting with an environment or a high-level critic that gives noisy responses as to the consistency of a tentative labeling selected by the automata. This results in an iterative linear algorithm that is itself probabilistic. Using an explicit definition of consistency we give a complete analysis of this probabilistic relaxation process using weak convergence results for stochastic algorithms. Our model can accommodate a range of uncertainties in the compatibility functions. We prove a local convergence result and show that the point of convergence depends both on the initial labeling and the constraints. The algorithm is implementable in a highly parallel fashion.

A Minimax Filter for Systems with Large Plant Uncertainties Using Measurements Corrupted by Colored Noise

A minimax filter is derived to estimate the state of a system, using observations corrupted by colored noise, when large uncertainties in the plant dynamics and process noise are presen.

Tracking Random Walk of Individual Domain Walls in Cylindrical Nanomagnets with Resistance Noise

The stochasticity of domain-wall (DW) motion in magnetic nanowires has been probed by measuring slow fluctuations, or noise, in electrical resistance at small magnetic fields. By controlled injection of DWs into isolated cylindrical nanowires of nickel, we have been able to track the motion of the DWs between the electrical leads by discrete steps in the resistance. Closer inspection of the time dependence of noise reveals a diffusive random walk of the DWs with a universal kinetic exponent. Our experiments outline a method with which electrical resistance is able to detect the kinetic state of the DWs inside the nanowires, which can be useful in DW-based memory designs.

An Estimator Algorithm For Learning Automata With Changing Number Of Actions

In many problems of decision making under uncertainty the system has to acquire knowledge of its environment and learn the optimal decision through its experience. Such problems may also involve the system having to arrive at the globally optimal decision, when at each instant only a subset of the entire set of possible alternatives is available. These problems can be successfully modelled and analysed by learning automata. In this paper an estimator learning algorithm, which maintains estimates of the reward characteristics of the random environment, is presented for an automaton with changing number of actions. A learning automaton using the new scheme is shown to be e-optimal. The simulation results demonstrate the fast convergence properties of the new algorithm. The results of this study can be extended to the design of other types of estimator algorithms with good convergence properties.

Learning dynamic prices in multiseller electronic retail markets with price sensitive customers, stochastic demands, and inventory replenishments

In this paper, we use reinforcement learning (RL) as a tool to study price dynamics in an electronic retail market consisting of two competing sellers, and price sensitive and lead time sensitive customers. Sellers, offering identical products, compete on price to satisfy stochastically arriving demands (customers), and follow standard inventory control and replenishment policies to manage their inventories. In such a generalized setting, RL techniques have not previously been applied. We consider two representative cases: 1) no information case, were none of the sellers has any information about customer queue levels, inventory levels, or prices at the competitors; and 2) partial information case, where every seller has information about the customer queue levels and inventory levels of the competitors. Sellers employ automated pricing agents, or pricebots, which use RL-based pricing algorithms to reset the prices at random intervals based on factors such as number of back orders, inventory levels, and replenishment lead times, with the objective of maximizing discounted cumulative profit. In the no information case, we show that a seller who uses Q-learning outperforms a seller who uses derivative following (DF). In the partial information case, we model the problem as a Markovian game and use actor-critic based RL to learn dynamic prices. We believe our approach to solving these problems is a new and promising way of setting dynamic prices in multiseller environments with stochastic demands, price sensitive customers, and inventory replenishments.

Resistance fluctuations, phase randomness and noise in one-dimensional conductors with a long-range correlated potential

When the size (L) of a one-dimensional metallic conductor is less than the correlation length λ-1 of the Gaussian random potential, one expects transport properties to show ballistic behaviour. Using an invariant imbedding method, we study the exact distribution of the resistance, of the phase θ of the reflection amplitude of an incident electron of wave number k0, and of dθ/dk0, for λL ll 1. The resistance is non-self-averaging and the n-th resistance moment varies periodically as (1 - cos 2k0L)n. The charge fluctuation noise, determined by the distribution of dθ/dk0, is constant at low frequencies.

Analytical prediction of break-out noise from a reactive rectangular plenum with four flexible walls

This paper describes an analytical calculation of break-out noise from a rectangular plenum with four flexible walls by incorporating three-dimensional effects along with the acoustical and structural wave coupling phenomena. The breakout noise from rectangular plenums is important and the coupling between acoustic waves within the plenum and structural waves in the flexible plenum walls plays a critical role in prediction of the transverse transmission loss. The first step in breakout noise prediction is to calculate the inside plenum pressure field and the normal flexible plenum wall vibration by using an impedance-mobility approach, which results in a compact matrix formulation. In the impedance-mobility compact matrix (IMCM) approach, it is presumed that the coupled response can be described in terms of finite sets of the uncoupled acoustic subsystem and the structural subsystem. The flexible walls of the plenum are modeled as an unfolded plate to calculate natural frequencies and mode shapes of the uncoupled structural subsystem. The second step is to calculate the radiated sound power from the flexible walls using Kirchhoff-Helmholtz (KH) integral formulation. Analytical results are validated with finite element and boundary element (FEM-BEM) numerical models. (C) 2010 Acoustical Society of America. DOI: 10.1121/1.3463801]