One practical algorithm for both stochastic and adversarial bandits

We present an algorithm for multiarmed bandits that achieves almost optimal performance in both stochastic and adversarial regimes without prior knowledge about the nature of the environment. Our algorithm is based on augmentation of the EXP3 algorithm with a new control lever in the form of exploration parameters that are tailored individually for each arm. The algorithm simultaneously applies the “old” control lever, the learning rate, to control the regret in the adversarial regime and the new control lever to detect and exploit gaps between the arm losses. This secures problem-dependent “logarithmic” regret when gaps are present without compromising on the worst-case performance guarantee in the adversarial regime. We show that the algorithm can exploit both the usual expected gaps between the arm losses in the stochastic regime and deterministic gaps between the arm losses in the adversarial regime. The algorithm retains “logarithmic” regret guarantee in the stochastic regime even when some observations are contaminated by an adversary, as long as on average the contamination does not reduce the gap by more than a half. Our results for the stochastic regime are supported by experimental validation.

Estimating the parameters of stochastic volatility models using option price data

This article describes a maximum likelihood method for estimating the parameters of the standard square-root stochastic volatility model and a variant of the model that includes jumps in equity prices. The model is fitted to data on the S&P 500 Index and the prices of vanilla options written on the index, for the period 1990 to 2011. The method is able to estimate both the parameters of the physical measure (associated with the index) and the parameters of the risk-neutral measure (associated with the options), including the volatility and jump risk premia. The estimation is implemented using a particle filter whose efficacy is demonstrated under simulation. The computational load of this estimation method, which previously has been prohibitive, is managed by the effective use of parallel computing using graphics processing units (GPUs). The empirical results indicate that the parameters of the models are reliably estimated and consistent with values reported in previous work. In particular, both the volatility risk premium and the jump risk premium are found to be significant.

Stochastic differential games with multiple modes

We have studied two person stochastic differential games with multiple modes. For the zero-sum game we have established the existence of optimal strategies for both players. For the nonzero-sum case we have proved the existence of a Nash equilibrium.

Fluctuations in protein synthesis from a single RNA template: Stochastic kinetics of ribosomes

Proteins are polymerized by cyclic machines called ribosomes, which use their messenger RNA (mRNA) track also as the corresponding template, and the process is called translation. We explore, in depth and detail, the stochastic nature of the translation. We compute various distributions associated with the translation process; one of them-namely, the dwell time distribution-has been measured in recent single-ribosome experiments. The form of the distribution, which fits best with our simulation data, is consistent with that extracted from the experimental data. For our computations, we use a model that captures both the mechanochemistry of each individual ribosome and their steric interactions. We also demonstrate the effects of the sequence inhomogeneities of real genes on the fluctuations and noise in translation. Finally, inspired by recent advances in the experimental techniques of manipulating single ribosomes, we make theoretical predictions on the force-velocity relation for individual ribosomes. In principle, all our predictions can be tested by carrying out in vitro experiments.

Intermittent Dynamics, Stochastic Resonance and Dynamical Heterogeneity in Supercooled Liquid Water

Here we find through computer simulations and theoretical analysis that the low temperature thermodynamic anomalies of liquid water arises from the intermittent fluctuation between its high density and low density forms, consisting largely of 5-coordinated and 4-coordinated water molecules, respectively. The fluctuations exhibit strong dynamic heterogeneity (defined by the four point time correlation function), accompanied by a divergence like growth of the dynamic correlation length, of the type encountered in fragile supercooled liquids. The intermittency has been explained by invoking a two state model often employed to understand stochastic resonance, with the relevant periodic perturbation provided here by the fluctuation of the total volume of the system.

Response And Stability Of A Stochastic Beam-Column Using Stochastic Fem

A beam-column resting on continuous Winkler foundation and discrete elastic supports is considered. The beam-column is of variable cross-section and the variation of sectional properties along the axis of the beam-column is deterministic. Young's modulus, mass per unit length and distributed axial loadings of the beam-column have a stochastic distribution. The foundation stiffness coefficient of the Winkler model, the stiffnesses of discrete elastic supports, stiffnesses of end springs and the end thrust, are all considered as random parameters. The material property fluctuations and distributed axial loadings are considered to constitute independent, one-dimension uni-variate homogeneous real stochastic fields in space. The foundation stiffness coefficient, stiffnesses of the discrete elastic supports, stiffnesses of end springs and the end thrust are considered to constitute independent random variables. Static response, free vibration and stability behaviour of the beam-column are studied. Hamilton's principle is used to formulate the problem using stochastic FEM. Sensitivity vectors of the response and stability parameters are evaluated. Using these statistics of free vibration frequencies, mode shapes, buckling parameters, etc., are evaluated. A numerical example is given.

Performance modelling of a fault-tolerant real-time multiprocessor using stochastic Petri nets

The fault-tolerant multiprocessor (ftmp) is a bus-based multiprocessor architecture with real-time and fault- tolerance features and is used in critical aerospace applications. A preliminary performance evaluation is of crucial importance in the design of such systems. In this paper, we review stochastic Petri nets (spn) and developspn-based performance models forftmp. These performance models enable efficient computation of important performance measures such as processing power, bus contention, bus utilization, and waiting times.

Stochastic growth models for analyzing crustacean data

The contemporary methodology for growth models of organisms is based on continuous trajectories and thus it hinders us from modelling stepwise growth in crustacean populations. Growth models for fish are normally assumed to follow a continuous function, but a different type of model is needed for crustacean growth. Crustaceans must moult in order for them to grow. The growth of crustaceans is a discontinuous process due to the periodical shedding of the exoskeleton in moulting. The stepwise growth of crustaceans through the moulting process makes the growth estimation more complex. Stochastic approaches can be used to model discontinuous growth or what are commonly known as "jumps" (Figure 1). However, in stochastic growth model we need to ensure that the stochastic growth model results in only positive jumps. In view of this, we will introduce a subordinator that is a special case of a Levy process. A subordinator is a non-decreasing Levy process, that will assist in modelling crustacean growth for better understanding of the individual variability and stochasticity in moulting periods and increments. We develop the estimation methods for parameter estimation and illustrate them with the help of a dataset from laboratory experiments. The motivational dataset is from the ornate rock lobster, Panulirus ornatus, which can be found between Australia and Papua New Guinea. Due to the presence of sex effects on the growth (Munday et al., 2004), we estimate the growth parameters separately for each sex. Since all hard parts are shed too often, the exact age determination of a lobster can be challenging. However, the growth parameters for the aforementioned moult processes from tank data being able to estimate through: (i) inter-moult periods, and (ii) moult increment. We will attempt to derive a joint density, which is made up of two functions: one for moult increments and the other for time intervals between moults. We claim these functions are conditionally independent given pre-moult length and the inter-moult periods. The variables moult increments and inter-moult periods are said to be independent because of the Markov property or conditional probability. Hence, the parameters in each function can be estimated separately. Subsequently, we integrate both of the functions through a Monte Carlo method. We can therefore obtain a population mean for crustacean growth (e. g. red curve in Figure 1). [GRAPHICS]

Early stopping by using stochastic curtailment in a three-arm sequential trial

Summary. Interim analysis is important in a large clinical trial for ethical and cost considerations. Sometimes, an interim analysis needs to be performed at an earlier than planned time point. In that case, methods using stochastic curtailment are useful in examining the data for early stopping while controlling the inflation of type I and type II errors. We consider a three-arm randomized study of treatments to reduce perioperative blood loss following major surgery. Owing to slow accrual, an unplanned interim analysis was required by the study team to determine whether the study should be continued. We distinguish two different cases: when all treatments are under direct comparison and when one of the treatments is a control. We used simulations to study the operating characteristics of five different stochastic curtailment methods. We also considered the influence of timing of the interim analyses on the type I error and power of the test. We found that the type I error and power between the different methods can be quite different. The analysis for the perioperative blood loss trial was carried out at approximately a quarter of the planned sample size. We found that there is little evidence that the active treatments are better than a placebo and recommended closure of the trial.

Estimating equations for parameters in stochastic growth models from tag-recapture data

James (1991, Biometrics 47, 1519-1530) constructed unbiased estimating functions for estimating the two parameters in the von Bertalanffy growth curve from tag-recapture data. This paper provides unbiased estimating functions for a class of growth models that incorporate stochastic components and explanatory variables. a simulation study using seasonal growth models indicates that the proposed method works well while the least-squares methods that are commonly used in the literature may produce substantially biased estimates. The proposed model and method are also applied to real data from tagged rack lobsters to assess the possible seasonal effect on growth.

Bias reduction using stochastic approximation

The paper studies stochastic approximation as a technique for bias reduction. The proposed method does not require approximating the bias explicitly, nor does it rely on having independent identically distributed (i.i.d.) data. The method always removes the leading bias term, under very mild conditions, as long as auxiliary samples from distributions with given parameters are available. Expectation and variance of the bias-corrected estimate are given. Examples in sequential clinical trials (non-i.i.d. case), curved exponential models (i.i.d. case) and length-biased sampling (where the estimates are inconsistent) are used to illustrate the applications of the proposed method and its small sample properties.

Risk management strategies using seasonal climate forecasting in irrigated cotton production: a tale of stochastic dominance.

Decision-making in agriculture is carried out in an uncertain environment with farmers often seeking information to reduce risk. As a result of the extreme variability of rainfall and stream-flows in north-eastern Australia, water supplies for irrigated agriculture are a limiting factor and a source of risk. The present study examined the use of seasonal climate forecasting (SCF) when calculating planting areas for irrigated cotton in the northern Murray Darling Basin. Results show that minimising risk by adjusting plant areas in response to SCF can lead to significant gains in gross margin returns. However, how farmers respond to SCF is dependent on several other factors including irrigators’ attitude towards risk.

Stochastic demography and population dynamics in the red kangaroo Macropus rufus

1. Many organisms inhabit strongly fluctuating environments but their demography and population dynamics are often analysed using deterministic models and elasticity analysis, where elasticity is defined as the proportional change in population growth rate caused by a proportional change in a vital rate. Deterministic analyses may not necessarily be informative because large variation in a vital rate with a small deterministic elasticity may affect the population growth rate more than a small change in a less variable vital rate having high deterministic elasticity. 2. We analyse a stochastic environment model of the red kangaroo (Macropus rufus), a species inhabiting an environment characterized by unpredictable and highly variable rainfall, and calculate the elasticity of the stochastic growth rate with respect to the mean and variability in vital rates. 3. Juvenile survival is the most variable vital rate but a proportional change in the mean adult survival rate has a much stronger effect on the stochastic growth rate. 4. Even if changes in average rainfall have a larger impact on population growth rate, increased variability in rainfall may still be important also in long-lived species. The elasticity with respect to the standard deviation of rainfall is comparable to the mean elasticities of all vital rates but the survival in age class 3 because increased variation in rainfall affects both the mean and variability of vital rates. 5. Red kangaroos are harvested and, under the current rainfall pattern, an annual harvest fraction of c. 20% would yield a stochastic growth rate about unity. However, if average rainfall drops by more than c. 10%, any level of harvesting may be unsustainable, emphasizing the need for integrating climate change predictions in population management and increase our understanding of how environmental stochasticity translates into population growth rate.

Efficient implementations of a pseudodynamical stochastic filtering strategy for static elastography

A computationally efficient pseudodynamical filtering setup is established for elasticity imaging (i.e., reconstruction of shear modulus distribution) in soft-tissue organs given statically recorded and partially measured displacement data. Unlike a regularized quasi-Newton method (QNM) that needs inversion of ill-conditioned matrices, the authors explore pseudodynamic extended and ensemble Kalman filters (PD-EKF and PD-EnKF) that use a parsimonious representation of states and bypass explicit regularization by recursion over pseudotime. Numerical experiments with QNM and the two filters suggest that the PD-EnKF is the most robust performer as it exhibits no sensitivity to process noise covariance and yields good reconstruction even with small ensemble sizes.

Stochastic Modeling of an Aircraft Traversing a Runway Using Time Series Analysis

Time series, from a narrow point of view, is a sequence of observations on a stochastic process made at discrete and equally spaced time intervals. Its future behavior can be predicted by identifying, fitting, and confirming a mathematical model. In this paper, time series analysis is applied to problems concerning runwayinduced vibrations of an aircraft. A simple mathematical model based on this technique is fitted to obtain the impulse response coefficients of an aircraft system considered as a whole for a particular type of operation. Using this model, the output which is the aircraft response can be obtained with lesser computation time for any runway profile as the input.