Diffusion Gradient Temporal Difference for Cooperative Reinforcement Learning with Linear Function Approximation

We introduce a diffusion-based algorithm in which multiple agents cooperate to predict a common and global statevalue function by sharing local estimates and local gradient information among neighbors. Our algorithm is a fully distributed implementation of the gradient temporal difference with linear function approximation, to make it applicable to multiagent settings. Simulations illustrate the benefit of cooperation in learning, as made possible by the proposed algorithm.

A unified framework for linear function approximation of value functions in stochastic control

This paper contributes with a unified formulation that merges previ- ous analysis on the prediction of the performance ( value function ) of certain sequence of actions ( policy ) when an agent operates a Markov decision process with large state-space. When the states are represented by features and the value function is linearly approxi- mated, our analysis reveals a new relationship between two common cost functions used to obtain the optimal approximation. In addition, this analysis allows us to propose an efficient adaptive algorithm that provides an unbiased linear estimate. The performance of the pro- posed algorithm is illustrated by simulation, showing competitive results when compared with the state-of-the-art solutions.

Optimal Piecewise Linear Function Approximation for GPU-based Applications

Many computer vision and human-computer interaction applications developed in recent years need evaluating complex and continuous mathematical functions as an essential step toward proper operation. However, rigorous evaluation of this kind of functions often implies a very high computational cost, unacceptable in real-time applications. To alleviate this problem, functions are commonly approximated by simpler piecewise-polynomial representations. Following this idea, we propose a novel, efficient, and practical technique to evaluate complex and continuous functions using a nearly optimal design of two types of piecewise linear approximations in the case of a large budget of evaluation subintervals. To this end, we develop a thorough error analysis that yields asymptotically tight bounds to accurately quantify the approximation performance of both representations. It provides an improvement upon previous error estimates and allows the user to control the trade-off between the approximation error and the number of evaluation subintervals. To guarantee real-time operation, the method is suitable for, but not limited to, an efficient implementation in modern Graphics Processing Units (GPUs), where it outperforms previous alternative approaches by exploiting the fixed-function interpolation routines present in their texture units. The proposed technique is a perfect match for any application requiring the evaluation of continuous functions, we have measured in detail its quality and efficiency on several functions, and, in particular, the Gaussian function because it is extensively used in many areas of computer vision and cybernetics, and it is expensive to evaluate.

On the maximum and minimum values of a linear function of the radial co-ordinates of a point with respect to simplicissimum in space of n dimensions.

Cover title.

A comparisation of methods for the optimisation of a non-linear function subject to non-linear constraints

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT

Stability and convergence of an implicit numerical method for the non-linear fractional reaction-subdiffusion process

In this paper, we consider the following non-linear fractional reaction–subdiffusion process (NFR-SubDP): Formula where f(u, x, t) is a linear function of u, the function g(u, x, t) satisfies the Lipschitz condition and 0Dt1–{gamma} is the Riemann–Liouville time fractional partial derivative of order 1 – {gamma}. We propose a new computationally efficient numerical technique to simulate the process. Firstly, the NFR-SubDP is decoupled, which is equivalent to solving a non-linear fractional reaction–subdiffusion equation (NFR-SubDE). Secondly, we propose an implicit numerical method to approximate the NFR-SubDE. Thirdly, the stability and convergence of the method are discussed using a new energy method. Finally, some numerical examples are presented to show the application of the present technique. This method and supporting theoretical results can also be applied to fractional integrodifferential equations.

A non-linear analogue-to-digital converter for linearization of transducer input-output relationships

Analogue and digital techniques for linearization of non-linear input-output relationship of transducers are briefly reviewed. The condition required for linearizing a non-linear function y = f(x) using a non-linear analogue-to-digital converter, is explained. A simple technique to construct a non-linear digital-to-analogue converter, based on ' segments of equal digital interval ' is described. The technique was used to build an N-DAC which can be employed in a successive approximation or counter-ramp type ADC to linearize the non-linear transfer function of a thermistor-resistor combination. The possibility of achieving an order of magnitude higher accuracy in the measurement of temperature is shown.

Application of ultraspherical polynomials to forced oscillations of a third-order non-linear system

In this paper the method of ultraspherical polynomial approximation is applied to study the steady-state response in forced oscillations of a third-order non-linear system. The non-linear function is expanded in ultraspherical polynomials and the expansion is restricted to the linear term. The equation for the response curve is obtained by using the linearized equation and the results are presented graphically. The agreement between the approximate solution and the analog computer solution is satisfactory. The problem of stability is not dealt with in this paper.

A weighted mean square method of linearization in non-linear oscillations

The paper deals with a linearization technique in non-linear oscillations for systems which are governed by second-order non-linear ordinary differential equations. The method is based on approximation of the non-linear function by a linear function such that the error is least in the weighted mean square sense. The method has been applied to cubic, sine, hyperbolic sine, and odd polynomial types of non-linearities and the results obtained are more accurate than those given by existing linearization methods.

Optimal non-linear reinforcement schemes for stochastic automata

Two optimal non-linear reinforcement schemes—the Reward-Inaction and the Penalty-Inaction—for the two-state automaton functioning in a stationary random environment are considered. Very simple conditions of symmetry of the non-linear function figuring in the reinforcement scheme are shown to be necessary and sufficient for optimality. General expressions for the variance and rate of learning are derived. These schemes are compared with the already existing optimal linear schemes in the light of average variance and average rate of learning.

Rossby Waves With Linear Topography In Barotropic Fluids

Rossby waves are the most important waves in the atmosphere and ocean, and are parts of a large-scale system in fluid. The theory and observation show that, they satisfy quasi-geostrophic and quasi-static equilibrium approximations. In this paper, solitary Rossby waves induced by linear topography in barotropic fluids with a shear flow are studied. In order to simplify the problem, the topography is taken as a linear function of latitude variable y, then employing a weakly nonlinear method and a perturbation method, a KdV (Korteweg-de Vries) equation describing evolution of the amplitude of solitary Rossby waves induced by linear topography is derived. The results show that the variation of linear topography can induce the solitary Rossby waves in barotropic fluids with a shear flow, and extend the classical geophysical theory of fluid dynamics.

Rossby waves with linear topography in barotropic fluids

Rossby waves are the most important waves in the atmosphere and ocean, and are parts of a large-scale system in fluid. The theory and observation show that, they satisfy quasi-geostrophic and quasi-static equilibrium approximations. In this paper, solitary Rossby waves induced by linear topography in barotropic fluids with a shear flow are studied. In order to simplify the problem, the topography is taken as a linear function of latitude variable y, then employing a weakly nonlinear method and a perturbation method, a KdV (Korteweg-de Vries) equation describing evolution of the amplitude of solitary Rossby waves induced by linear topography is derived. The results show that the variation of linear topography can induce the solitary Rossby waves in barotropic fluids with a shear flow, and extend the classical geophysical theory of fluid dynamics.

Approximating a class of combinatorial problems with rational objective function

In the late seventies, Megiddo proposed a way to use an algorithm for the problem of minimizing a linear function a(0) + a(1)x(1) + ... + a(n)x(n) subject to certain constraints to solve the problem of minimizing a rational function of the form (a(0) + a(1)x(1) + ... + a(n)x(n))/(b(0) + b(1)x(1) + ... + b(n)x(n)) subject to the same set of constraints, assuming that the denominator is always positive. Using a rather strong assumption, Hashizume et al. extended Megiddo`s result to include approximation algorithms. Their assumption essentially asks for the existence of good approximation algorithms for optimization problems with possibly negative coefficients in the (linear) objective function, which is rather unusual for most combinatorial problems. In this paper, we present an alternative extension of Megiddo`s result for approximations that avoids this issue and applies to a large class of optimization problems. Specifically, we show that, if there is an alpha-approximation for the problem of minimizing a nonnegative linear function subject to constraints satisfying a certain increasing property then there is an alpha-approximation (1 1/alpha-approximation) for the problem of minimizing (maximizing) a nonnegative rational function subject to the same constraints. Our framework applies to covering problems and network design problems, among others.

Area-Efficient Linear Regression Architecture for Real-Time Signal Processing on FPGAs

Linear regression is a technique widely used in digital signal processing. It consists on finding the linear function that better fits a given set of samples. This paper proposes different hardware architectures for the implementation of the linear regression method on FPGAs, specially targeting area restrictive systems. It saves area at the cost of constraining the lengths of the input signal to some fixed values. We have implemented the proposed scheme in an Automatic Modulation Classifier, meeting the hard real-time constraints this kind of systems have.

Linear and nonlinear pathways of spectral information transmission in the cochlear nucleus

At the level of the cochlear nucleus (CN), the auditory pathway divides into several parallel circuits, each of which provides a different representation of the acoustic signal. Here, the representation of the power spectrum of an acoustic signal is analyzed for two CN principal cells—chopper neurons of the ventral CN and type IV neurons of the dorsal CN. The analysis is based on a weighting function model that relates the discharge rate of a neuron to first- and second-order transformations of the power spectrum. In chopper neurons, the transformation of spectral level into rate is a linear (i.e., first-order) or nearly linear function. This transformation is a predominantly excitatory process involving multiple frequency components, centered in a narrow frequency range about best frequency, that usually are processed independently of each other. In contrast, type IV neurons encode spectral information linearly only near threshold. At higher stimulus levels, these neurons are strongly inhibited by spectral notches, a behavior that cannot be explained by level transformations of first- or second-order. Type IV weighting functions reveal complex excitatory and inhibitory interactions that involve frequency components spanning a wider range than that seen in choppers. These findings suggest that chopper and type IV neurons form parallel pathways of spectral information transmission that are governed by two different mechanisms. Although choppers use a predominantly linear mechanism to transmit tonotopic representations of spectra, type IV neurons use highly nonlinear processes to signal the presence of wide-band spectral features.