A four-timescale algorithm for constrained stochastic optimization of RED

The overall performance of random early detection (RED) routers in the Internet is determined by the settings of their associated parameters. The non-availability of a functional relationship between the RED performance and its parameters makes it difficult to implement optimization techniques directly in order to optimize the RED parameters. In this paper, we formulate a generic optimization framework using a stochastically bounded delay metric to dynamically adapt the RED parameters. The constrained optimization problem thus formulated is solved using traditional nonlinear programming techniques. Here, we implement the barrier and penalty function approaches, respectively. We adopt a second-order nonlinear optimization framework and propose a novel four-timescale stochastic approximation algorithm to estimate the gradient and Hessian of the barrier and penalty objectives and update the RED parameters. A convergence analysis of the proposed algorithm is briefly sketched. We perform simulations to evaluate the performance of our algorithm with both barrier and penalty objectives and compare these with RED and a variant of it in the literature. We observe an improvement in performance using our proposed algorithm over RED, and the above variant of it.

Stochastic models of IEEE 802.11e wireless net works with multimedia applications

We provide a survey of some of our recent results ([9], [13], [4], [6], [7]) on the analytical performance modeling of IEEE 802.11 wireless local area networks (WLANs). We first present extensions of the decoupling approach of Bianchi ([1]) to the saturation analysis of IEEE 802.11e networks with multiple traffic classes. We have found that even when analysing WLANs with unsaturated nodes the following state dependent service model works well: when a certain set of nodes is nonempty, their channel attempt behaviour is obtained from the corresponding fixed point analysis of the saturated system. We will present our experiences in using this approximation to model multimedia traffic over an IEEE 802.11e network using the enhanced DCF channel access (EDCA) mechanism. We have found that we can model TCP controlled file transfers, VoIP packet telephony, and streaming video in the IEEE802.11e setting by this simple approximation.

A suprathreshold stochastic resonance based pre-processor for direction-of-arrival estimation

In this paper we propose a nonlinear preprocessor for enhancing the performance of processors used for direction-of-arrival (DOA) estimation in heavy-tailed non-Gaussian noise. The preprocessor based on the phenomenon of suprathreshold stochastic resonance (SSR), provides SNR gain. The preprocessed data is used for DOA estimation by the MUSIC algorithm. Simulation results are presented to show that the SSR preprocessor provides a significant improvement in the performance of MUSIC in heavy-tailed noise at low SNR.

Prediction of inplane damping from deterministic and stochastic models

This paper reviews computational reliability, computer algebra, stochastic stability and rotating frame turbulence (RFT) in the context of predicting the blade inplane mode stability, a mode which is at best weakly damped. Computational reliability can be built into routine Floquet analysis involving trim analysis and eigenanalysis, and a highly portable special purpose processor restricted to rotorcraft dynamics analysis is found to be more economical than a multipurpose processor. While the RFT effects are dominant in turbulence modeling, the finding that turbulence stabilizes the inplane mode is based on the assumption that turbulence is white noise.

Critical age-dependent branching Markov processes and their scaling limits

This paper studies:(i)the long-time behaviour of the empirical distribution of age and normalized position of an age-dependent critical branching Markov process conditioned on non-extinction;and (ii) the super-process limit of a sequence of age-dependent critical branching Brownian motions.

Learning dynamic prices in multiseller electronic retail markets with price sensitive customers, stochastic demands, and inventory replenishments

In this paper, we use reinforcement learning (RL) as a tool to study price dynamics in an electronic retail market consisting of two competing sellers, and price sensitive and lead time sensitive customers. Sellers, offering identical products, compete on price to satisfy stochastically arriving demands (customers), and follow standard inventory control and replenishment policies to manage their inventories. In such a generalized setting, RL techniques have not previously been applied. We consider two representative cases: 1) no information case, were none of the sellers has any information about customer queue levels, inventory levels, or prices at the competitors; and 2) partial information case, where every seller has information about the customer queue levels and inventory levels of the competitors. Sellers employ automated pricing agents, or pricebots, which use RL-based pricing algorithms to reset the prices at random intervals based on factors such as number of back orders, inventory levels, and replenishment lead times, with the objective of maximizing discounted cumulative profit. In the no information case, we show that a seller who uses Q-learning outperforms a seller who uses derivative following (DF). In the partial information case, we model the problem as a Markovian game and use actor-critic based RL to learn dynamic prices. We believe our approach to solving these problems is a new and promising way of setting dynamic prices in multiseller environments with stochastic demands, price sensitive customers, and inventory replenishments.

Stochastic Analysis of Yielding System

The probability distribution of the instantaneous incremental yield of an inelastic system is characterized in terms of a conditional probability and average rate of crossing. The detailed yield statistics of a single degree-of-freedom elasto-plastic system under a Gaussian white noise are obtained for both nonstationary and stationary response. The present analysis indicates that the yield damage is sensitive to viscous damping. The spectra of mean and mean square damage rate are presented.

Probability Distribution of the Eigenvalues of the Random String Equation

The probability distribution of the eigenvalues of a second-order stochastic boundary value problem is considered. The solution is characterized in terms of the zeros of an associated initial value problem. It is further shown that the probability distribution is related to the solution of a first-order nonlinear stochastic differential equation. Solutions of this equation based on the theory of Markov processes and also on the closure approximation are presented. A string with stochastic mass distribution is considered as an example for numerical work. The theoretical probability distribution functions are compared with digital simulation results. The comparison is found to be reasonably good.

Estimating Frequencies of Minority Nevirapine-Resistant Strains in Chronically HIV-1-Infected Individuals Naive to Nevirapine by Using Stochastic Simulations and a Mathematical Model

Nevirapine forms the mainstay of our efforts to curtail the pediatric AIDS epidemic through prevention of mother-to-child transmission of HIV-1. A key limitation, however, is the rapid selection of HIV-1 strains resistant to nevirapine following the administration of a single dose. This rapid selection of resistance suggests that nevirapine-resistant strains preexist in HIV-1 patients and may adversely affect outcomes of treatment. The frequencies of nevirapine-resistant strains in vivo, however, remain poorly estimated, possibly because they exist as a minority below current assay detection limits. Here, we employ stochastic simulations and a mathematical model to estimate the frequencies of strains carrying different combinations of the common nevirapine resistance mutations K103N, V106A, Y181C, Y188C, and G190A in chronically infected HIV-1 patients naive to nevirapine. We estimate the relative fitness of mutant strains from an independent analysis of previous competitive growth assays. We predict that single mutants are likely to preexist in patients at frequencies (similar to 0.01% to 0.001%) near or below current assay detection limits (>0.01%), emphasizing the need for more-sensitive assays. The existence of double mutants is subject to large stochastic variations. Triple and higher mutants are predicted not to exist. Our estimates are robust to variations in the recombination rate, cellular superinfection frequency, and the effective population size. Thus, with 10(7) to 10(8) infected cells in HIV-1 patients, even when undetected, nevirapine-resistant genomes may exist in substantial numbers and compromise efforts to prevent mother-to-child transmission of HIV-1, accelerate the failure of subsequent antiretroviral treatments, and facilitate the transmission of drug resistance.

Bayesian Analysis of a Superimposed Renewal Process

Stochastic behavior of an aero-engine failure/repair process has been analyzed from a Bayesian perspective. Number of failures/repairs in the component-sockets of this multi-component system are assumed to follow independent renewal processes with Weibull inter-arrival times. Based on the field failure/repair data of a large number of such engines and independent Gamma priors on the scale parameters and log-concave priors on the shape parameters, an exact method of sampling from the resulting posterior distributions of the parameters has been proposed. These generated parameter values are next utilised in obtaining the posteriors of the expected number of system repairs, system failure rate, and the conditional intensity function, which are computed using a recursive formula.

Characterization of unitary processes with independent and stationary increments

This is a continuation of the earlier work (Publ. Res. Inst. Math. Sci. 45 (2009) 745-785) to characterize unitary stationary independent increment Gaussian processes. The earlier assumption of uniform continuity is replaced by weak continuity and with technical assumptions on the domain of the generator, unitary equivalence of the process to the solution of an appropriate Hudson-Parthasarathy equation is proved.

Free vibration of a stochastic beam-column using stochastic fem

The free vibrational characteristics of a beam-column, which is having randomly varying Young's modulus and mass density and subjected to randomly distributed axial loading is analysed. The material property fluctuations and axial loadings are considered to constitute independent one-dimensional, uni-variate, homogeneous real, spatially distributed stochastic fields. Hamilton's principle is used to formulate the problem using stochastic FEM. Vibration frequencies and mode shapes are analysed for their statistical descriptions. A numerical example is shown.