A generalized data windowing scheme for adaptive conjugate gradient algorithms

The performance of the modified adaptive conjugate gradient (CG) algorithms based on the iterative CG method for adaptive filtering is highly related to the ways of estimating the correlation matrix and the cross-correlation vector. The existing approaches of implementing the CG algorithms using the data windows of exponential form or sliding form result in either loss of convergence or increase in misadjustment. This paper presents and analyzes a new approach to the implementation of the CG algorithms for adaptive filtering by using a generalized data windowing scheme. For the new modified CG algorithms, we show that the convergence speed is accelerated, the misadjustment and tracking capability comparable to those of the recursive least squares (RLS) algorithm are achieved. Computer simulations demonstrated in the framework of linear system modeling problem show the improvements of the new modifications.

Diffusion Gradient Temporal Difference for Cooperative Reinforcement Learning with Linear Function Approximation

We introduce a diffusion-based algorithm in which multiple agents cooperate to predict a common and global statevalue function by sharing local estimates and local gradient information among neighbors. Our algorithm is a fully distributed implementation of the gradient temporal difference with linear function approximation, to make it applicable to multiagent settings. Simulations illustrate the benefit of cooperation in learning, as made possible by the proposed algorithm.

Generalized mean field approximation for parallel dynamics of the Ising model

The dynamics of the non-equilibrium Ising model with parallel updates is investigated using a generalized mean field approximation that incorporates multiple two-site correlations at any two time steps, which can be obtained recursively. The proposed method shows significant improvement in predicting local system properties compared to other mean field approximation techniques, particularly in systems with symmetric interactions. Results are also evaluated against those obtained from Monte Carlo simulations. The method is also employed to obtain parameter values for the kinetic inverse Ising modeling problem, where couplings and local field values of a fully connected spin system are inferred from data. © 2014 IOP Publishing Ltd and SISSA Medialab srl.

On the Approximation of the Generalized Cut Function of Degree p+1 By Smooth Sigmoid Functions

We introduce a modification of the familiar cut function by replacing the linear part in its definition by a polynomial of degree p + 1 obtaining thus a sigmoid function called generalized cut function of degree p + 1 (GCFP). We then study the uniform approximation of the (GCFP) by smooth sigmoid functions such as the logistic and the shifted logistic functions. The limiting case of the interval-valued Heaviside step function is also discussed which imposes the use of Hausdorff metric. Numerical examples are presented using CAS MATHEMATICA.

Stability and convergence of a new explicit finite-difference approximation for the variable-order nonlinear fractional diffusion equation

In this paper, we consider the variable-order nonlinear fractional diffusion equation View the MathML source where xRα(x,t) is a generalized Riesz fractional derivative of variable order View the MathML source and the nonlinear reaction term f(u,x,t) satisfies the Lipschitz condition |f(u1,x,t)-f(u2,x,t)|less-than-or-equals, slantL|u1-u2|. A new explicit finite-difference approximation is introduced. The convergence and stability of this approximation are proved. Finally, some numerical examples are provided to show that this method is computationally efficient. The proposed method and techniques are applicable to other variable-order nonlinear fractional differential equations.

A Fuzzy Approximation Scheme for Sequential Learning in Pattern Recognition

An adaptive learning scheme, based on a fuzzy approximation to the gradient descent method for training a pattern classifier using unlabeled samples, is described. The objective function defined for the fuzzy ISODATA clustering procedure is used as the loss function for computing the gradient. Learning is based on simultaneous fuzzy decisionmaking and estimation. It uses conditional fuzzy measures on unlabeled samples. An exponential membership function is assumed for each class, and the parameters constituting these membership functions are estimated, using the gradient, in a recursive fashion. The induced possibility of occurrence of each class is useful for estimation and is computed using 1) the membership of the new sample in that class and 2) the previously computed average possibility of occurrence of the same class. An inductive entropy measure is defined in terms of induced possibility distribution to measure the extent of learning. The method is illustrated with relevant examples.

Generalized pencils of rays in statistical wave optics

The recently introduced generalized pencil of Sudarshan which gives an exact ray picture of wave optics is analysed in some situations of interest to wave optics. A relationship between ray dispersion and statistical inhomogeneity of the field is obtained. A paraxial approximation which preserves the rectilinear propagation character of the generalized pencils is presented. Under this approximation the pencils can be computed directly from the field conditions on a plane, without the necessity to compute the cross-spectral density function in the entire space as an intermediate quantity. The paraxial results are illustrated with examples. The pencils are shown to exhibit an interesting scaling behaviour in the far-zone. This scaling leads to a natural generalization of the Fraunhofer range criterion and of the classical van Cittert-Zernike theorem to planar sources of arbitrary state of coherence. The recently derived results of radiometry with partially coherent sources are shown to be simple consequences of this scaling.

Beyond fractional derivatives: local approximation of other convolution integrals

Dynamic systems involving convolution integrals with decaying kernels, of which fractionally damped systems form a special case, are non-local in time and hence infinite dimensional. Straightforward numerical solution of such systems up to time t needs O(t(2)) computations owing to the repeated evaluation of integrals over intervals that grow like t. Finite-dimensional and local approximations are thus desirable. We present here an approximation method which first rewrites the evolution equation as a coupled in finite-dimensional system with no convolution, and then uses Galerkin approximation with finite elements to obtain linear, finite-dimensional, constant coefficient approximations for the convolution. This paper is a broad generalization, based on a new insight, of our prior work with fractional order derivatives (Singh & Chatterjee 2006 Nonlinear Dyn. 45, 183-206). In particular, the decaying kernels we can address are now generalized to the Laplace transforms of known functions; of these, the power law kernel of fractional order differentiation is a special case. The approximation can be refined easily. The local nature of the approximation allows numerical solution up to time t with O(t) computations. Examples with several different kernels show excellent performance. A key feature of our approach is that the dynamic system in which the convolution integral appears is itself approximated using another system, as distinct from numerically approximating just the solution for the given initial values; this allows non-standard uses of the approximation, e. g. in stability analyses.

Coherent quantum Boltzmann equations from cQPA

We reformulate and extend our recently introduced quantum kinetic theory for interacting fermion and scalar fields. Our formalism is based on the coherent quasiparticle approximation (cQPA) where nonlocal coherence information is encoded in new spectral solutions at off-shell momenta. We derive explicit forms for the cQPA propagators in the homogeneous background and show that the collision integrals involving the new coherence propagators need to be resummed to all orders in gradient expansion. We perform this resummation and derive generalized momentum space Feynman rules including coherent propagators and modified vertex rules for a Yukawa interaction. As a result we are able to set up self-consistent quantum Boltzmann equations for both fermion and scalar fields. We present several examples of diagrammatic calculations and numerical applications including a simple toy model for coherent baryogenesis.

Stochastic Approximation Algorithms for Constrained Optimization via Simulation

We develop four algorithms for simulation-based optimization under multiple inequality constraints. Both the cost and the constraint functions are considered to be long-run averages of certain state-dependent single-stage functions. We pose the problem in the simulation optimization framework by using the Lagrange multiplier method. Two of our algorithms estimate only the gradient of the Lagrangian, while the other two estimate both the gradient and the Hessian of it. In the process, we also develop various new estimators for the gradient and Hessian. All our algorithms use two simulations each. Two of these algorithms are based on the smoothed functional (SF) technique, while the other two are based on the simultaneous perturbation stochastic approximation (SPSA) method. We prove the convergence of our algorithms and show numerical experiments on a setting involving an open Jackson network. The Newton-based SF algorithm is seen to show the best overall performance.

An actor-critic algorithm with function approximation for discounted cost constrained Markov decision processes

We develop in this article the first actor-critic reinforcement learning algorithm with function approximation for a problem of control under multiple inequality constraints. We consider the infinite horizon discounted cost framework in which both the objective and the constraint functions are suitable expected policy-dependent discounted sums of certain sample path functions. We apply the Lagrange multiplier method to handle the inequality constraints. Our algorithm makes use of multi-timescale stochastic approximation and incorporates a temporal difference (TD) critic and an actor that makes a gradient search in the space of policy parameters using efficient simultaneous perturbation stochastic approximation (SPSA) gradient estimates. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal policy. (C) 2010 Elsevier B.V. All rights reserved.

Incremental natural-gradient actor-critic algorithms

We present four new reinforcement learning algorithms based on actor-critic and natural-gradient ideas, and provide their convergence proofs. Actor-critic rein- forcement learning methods are online approximations to policy iteration in which the value-function parameters are estimated using temporal difference learning and the policy parameters are updated by stochastic gradient descent. Methods based on policy gradients in this way are of special interest because of their com- patibility with function approximation methods, which are needed to handle large or infinite state spaces. The use of temporal difference learning in this way is of interest because in many applications it dramatically reduces the variance of the gradient estimates. The use of the natural gradient is of interest because it can produce better conditioned parameterizations and has been shown to further re- duce variance in some cases. Our results extend prior two-timescale convergence results for actor-critic methods by Konda and Tsitsiklis by using temporal differ- ence learning in the actor and by incorporating natural gradients, and they extend prior empirical studies of natural actor-critic methods by Peters, Vijayakumar and Schaal by providing the first convergence proofs and the first fully incremental algorithms.

Generalized model predictive static programming and its application to 3D impact angle constrained guidance of air-to-surface missiles

A new `generalized model predictive static programming (G-MPSP)' technique is presented in this paper in the continuous time framework for rapidly solving a class of finite-horizon nonlinear optimal control problems with hard terminal constraints. A key feature of the technique is backward propagation of a small-dimensional weight matrix dynamics, using which the control history gets updated. This feature, as well as the fact that it leads to a static optimization problem, are the reasons for its high computational efficiency. It has been shown that under Euler integration, it is equivalent to the existing model predictive static programming technique, which operates on a discrete-time approximation of the problem. Performance of the proposed technique is demonstrated by solving a challenging three-dimensional impact angle constrained missile guidance problem. The problem demands that the missile must meet constraints on both azimuth and elevation angles in addition to achieving near zero miss distance, while minimizing the lateral acceleration demand throughout its flight path. Both stationary and maneuvering ground targets are considered in the simulation studies. Effectiveness of the proposed guidance has been verified by considering first order autopilot lag as well as various target maneuvers.