Smoothed Functional Algorithms for Stochastic Optimization Using q-Gaussian Distributions

Smoothed functional (SF) schemes for gradient estimation are known to be efficient in stochastic optimization algorithms, especially when the objective is to improve the performance of a stochastic system However, the performance of these methods depends on several parameters, such as the choice of a suitable smoothing kernel. Different kernels have been studied in the literature, which include Gaussian, Cauchy, and uniform distributions, among others. This article studies a new class of kernels based on the q-Gaussian distribution, which has gained popularity in statistical physics over the last decade. Though the importance of this family of distributions is attributed to its ability to generalize the Gaussian distribution, we observe that this class encompasses almost all existing smoothing kernels. This motivates us to study SF schemes for gradient estimation using the q-Gaussian distribution. Using the derived gradient estimates, we propose two-timescale algorithms for optimization of a stochastic objective function in a constrained setting with a projected gradient search approach. We prove the convergence of our algorithms to the set of stationary points of an associated ODE. We also demonstrate their performance numerically through simulations on a queuing model.

Newton-based stochastic optimization using q-Gaussian smoothed functional algorithms

We present the first q-Gaussian smoothed functional (SF) estimator of the Hessian and the first Newton-based stochastic optimization algorithm that estimates both the Hessian and the gradient of the objective function using q-Gaussian perturbations. Our algorithm requires only two system simulations (regardless of the parameter dimension) and estimates both the gradient and the Hessian at each update epoch using these. We also present a proof of convergence of the proposed algorithm. In a related recent work (Ghoshdastidar, Dukkipati, & Bhatnagar, 2014), we presented gradient SF algorithms based on the q-Gaussian perturbations. Our work extends prior work on SF algorithms by generalizing the class of perturbation distributions as most distributions reported in the literature for which SF algorithms are known to work turn out to be special cases of the q-Gaussian distribution. Besides studying the convergence properties of our algorithm analytically, we also show the results of numerical simulations on a model of a queuing network, that illustrate the significance of the proposed method. In particular, we observe that our algorithm performs better in most cases, over a wide range of q-values, in comparison to Newton SF algorithms with the Gaussian and Cauchy perturbations, as well as the gradient q-Gaussian SF algorithms. (C) 2014 Elsevier Ltd. All rights reserved.

Iterated stochastic filters with additive updates for dynamic system identification: Annealing-type iterations and the filter bank

A nonlinear stochastic filtering scheme based on a Gaussian sum representation of the filtering density and an annealing-type iterative update, which is additive and uses an artificial diffusion parameter, is proposed. The additive nature of the update relieves the problem of weight collapse often encountered with filters employing weighted particle based empirical approximation to the filtering density. The proposed Monte Carlo filter bank conforms in structure to the parent nonlinear filtering (Kushner-Stratonovich) equation and possesses excellent mixing properties enabling adequate exploration of the phase space of the state vector. The performance of the filter bank, presently assessed against a few carefully chosen numerical examples, provide ample evidence of its remarkable performance in terms of filter convergence and estimation accuracy vis-a-vis most other competing filters especially in higher dimensional dynamic system identification problems including cases that may demand estimating relatively minor variations in the parameter values from their reference states. (C) 2014 Elsevier Ltd. All rights reserved.

Descending from infinity: Convergence of tailed distributions

We investigate the relaxation of long-tailed distributions under stochastic dynamics that do not support such tails. Linear relaxation is found to be a borderline case in which long tails are exponentially suppressed in time but not eliminated. Relaxation stronger than linear suppresses long tails immediately, but may lead to strong transient peaks in the probability distribution. We also find that a delta-function initial distribution under stronger than linear decay displays not one but two different regimes of diffusive spreading.

Necessary and sufficient conditions for optimality in constrained general sum stochastic games

In this paper we first derive a necessary and sufficient condition for a stationary strategy to be the Nash equilibrium of discounted constrained stochastic game under certain assumptions. In this process we also develop a nonlinear (non-convex) optimization problem for a discounted constrained stochastic game. We use the linear best response functions of every player and complementary slackness theorem for linear programs to derive both the optimization problem and the equivalent condition. We then extend this result to average reward constrained stochastic games. Finally, we present a heuristic algorithm motivated by our necessary and sufficient conditions for a discounted cost constrained stochastic game. We numerically observe the convergence of this algorithm to Nash equilibrium. (C) 2015 Elsevier B.V. All rights reserved.

A Stochastic Approximation Algorithm for Quantile Estimation

In this paper, we present two new stochastic approximation algorithms for the problem of quantile estimation. The algorithms uses the characterization of the quantile provided in terms of an optimization problem in 1]. The algorithms take the shape of a stochastic gradient descent which minimizes the optimization problem. Asymptotic convergence of the algorithms to the true quantile is proven using the ODE method. The theoretical results are also supplemented through empirical evidence. The algorithms are shown to provide significant improvement in terms of memory requirement and accuracy.

Stochastic differential games with multiple modes

We have studied two person stochastic differential games with multiple modes. For the zero-sum game we have established the existence of optimal strategies for both players. For the nonzero-sum case we have proved the existence of a Nash equilibrium.

Fluctuations in protein synthesis from a single RNA template: Stochastic kinetics of ribosomes

Proteins are polymerized by cyclic machines called ribosomes, which use their messenger RNA (mRNA) track also as the corresponding template, and the process is called translation. We explore, in depth and detail, the stochastic nature of the translation. We compute various distributions associated with the translation process; one of them-namely, the dwell time distribution-has been measured in recent single-ribosome experiments. The form of the distribution, which fits best with our simulation data, is consistent with that extracted from the experimental data. For our computations, we use a model that captures both the mechanochemistry of each individual ribosome and their steric interactions. We also demonstrate the effects of the sequence inhomogeneities of real genes on the fluctuations and noise in translation. Finally, inspired by recent advances in the experimental techniques of manipulating single ribosomes, we make theoretical predictions on the force-velocity relation for individual ribosomes. In principle, all our predictions can be tested by carrying out in vitro experiments.

Intermittent Dynamics, Stochastic Resonance and Dynamical Heterogeneity in Supercooled Liquid Water

Here we find through computer simulations and theoretical analysis that the low temperature thermodynamic anomalies of liquid water arises from the intermittent fluctuation between its high density and low density forms, consisting largely of 5-coordinated and 4-coordinated water molecules, respectively. The fluctuations exhibit strong dynamic heterogeneity (defined by the four point time correlation function), accompanied by a divergence like growth of the dynamic correlation length, of the type encountered in fragile supercooled liquids. The intermittency has been explained by invoking a two state model often employed to understand stochastic resonance, with the relevant periodic perturbation provided here by the fluctuation of the total volume of the system.

Response And Stability Of A Stochastic Beam-Column Using Stochastic Fem

A beam-column resting on continuous Winkler foundation and discrete elastic supports is considered. The beam-column is of variable cross-section and the variation of sectional properties along the axis of the beam-column is deterministic. Young's modulus, mass per unit length and distributed axial loadings of the beam-column have a stochastic distribution. The foundation stiffness coefficient of the Winkler model, the stiffnesses of discrete elastic supports, stiffnesses of end springs and the end thrust, are all considered as random parameters. The material property fluctuations and distributed axial loadings are considered to constitute independent, one-dimension uni-variate homogeneous real stochastic fields in space. The foundation stiffness coefficient, stiffnesses of the discrete elastic supports, stiffnesses of end springs and the end thrust are considered to constitute independent random variables. Static response, free vibration and stability behaviour of the beam-column are studied. Hamilton's principle is used to formulate the problem using stochastic FEM. Sensitivity vectors of the response and stability parameters are evaluated. Using these statistics of free vibration frequencies, mode shapes, buckling parameters, etc., are evaluated. A numerical example is given.

Performance modelling of a fault-tolerant real-time multiprocessor using stochastic Petri nets

The fault-tolerant multiprocessor (ftmp) is a bus-based multiprocessor architecture with real-time and fault- tolerance features and is used in critical aerospace applications. A preliminary performance evaluation is of crucial importance in the design of such systems. In this paper, we review stochastic Petri nets (spn) and developspn-based performance models forftmp. These performance models enable efficient computation of important performance measures such as processing power, bus contention, bus utilization, and waiting times.

An E-Convergence Method For The Evaluation Of Critical Parameters

A simple cconversence technique is applied to obtain accurate estimates of critical temperatures and critical it\ponmts of a few two- and threpdiniensional king models. When applied to the virial series for hard spheres and hard discs, this method predicts a divergence of the equation-of-state at the density of closest packing.

Non-stationary signal analysis by least mean fourth (LMF) adaptive algorithm

A modified least mean fourth (LMF) adaptive algorithm applicable to non-stationary signals is presented. The performance of the proposed algorithm is studied by simulation for non-stationarities in bandwidth, centre frequency and gain of a stochastic signal. These non-stationarities are in the form of linear, sinusoidal and jump variations of the parameters. The proposed LMF adaptation is found to have better parameter tracking capability than the LMS adaptation for the same speed of convergence.

A global convergence criterion for discrete-time multivariable adaptive systems

The paper deals with the basic problem of adjusting a matrix gain in a discrete-time linear multivariable system. The object is to obtain a global convergence criterion, i.e. conditions under which a specified error signal asymptotically approaches zero and other signals in the system remain bounded for arbitrary initial conditions and for any bounded input to the system. It is shown that for a class of up-dating algorithms for the adjustable gain matrix, global convergence is crucially dependent on a transfer matrix G(z) which has a simple block diagram interpretation. When w(z)G(z) is strictly discrete positive real for a scalar w(z) such that w-1(z) is strictly proper with poles and zeros within the unit circle, an augmented error scheme is suggested and is proved to result in global convergence. The solution avoids feeding back a quadratic term as recommended in other schemes for single-input single-output systems.

Invited paper Application of the least mean fourth (LMF) adaptive algorithm to signals associated with gaussian noise

This paper considers the applicability of the least mean fourth (LM F) power gradient adaptation criteria with 'advantage' for signals associated with gaussian noise, the associated noise power estimate not being known. The proposed method, as an adaptive spectral estimator, is found to provide superior performance than the least mean square (LMS) adaptation for the same (or even lower) speed of convergence for signals having sufficiently high signal-to-gaussian noise ratio. The results include comparison of the performance of the LMS-tapped delay line, LMF-tapped delay line, LMS-lattice and LMF-lattice algorithms, with the Burg's block data method as reference. The signals, like sinusoids with noise and stochastic signals like EEG, are considered in this study.