Stochasticity and bistability in horizontal transfer control of a genomic island in Pseudomonas.

Genomic islands (GEI) comprise a recently recognized large family of potentially mobile DNA elements and play an important role in the rapid differentiation and adaptation of bacteria. Most importantly, GEIs have been implicated in the acquisition of virulence factors, antibiotic resistances or toxic compound metabolism. Despite detailed information on coding capacities of GEIs, little is known about the regulatory decisions in individual cells controlling GEI transfer. Here, we show how self-transfer of ICEclc, a GEI in Pseudomonas knackmussii B13 is controlled by a series of stochastic processes, the result of which is that only a few percent of cells in a population will excise ICEclc and launch transfer. Stochastic processes have been implicated before in producing bistable phenotypic transitions, such as sporulation and competence development, but never before in horizontal gene transfer (HGT). Bistability is instigated during stationary phase at the level of expression of an activator protein InrR that lays encoded on ICEclc, and then faithfully propagated to a bistable expression of the IntB13 integrase, the enzyme responsible for excision and integration of the ICEclc. Our results demonstrate how GEI of a very widespread family are likely to control their transfer rates. Furthermore, they help to explain why HGT is typically confined to few members within a population of cells. The finding that, despite apparent stochasticity, HGT rates can be modulated by external environmental conditions provides an explanation as to why selective conditions can promote DNA exchange.

Spontaneous cell polarization: Feedback control of Cdc42 GTPase breaks cellular symmetry.

Spontaneous polarization without spatial cues, or symmetry breaking, is a fundamental problem of spatial organization in biological systems. This question has been extensively studied using yeast models, which revealed the central role of the small GTPase switch Cdc42. Active Cdc42-GTP forms a coherent patch at the cell cortex, thought to result from amplification of a small initial stochastic inhomogeneity through positive feedback mechanisms, which induces cell polarization. Here, I review and discuss the mechanisms of Cdc42 activity self-amplification and dynamic turnover. A robust Cdc42 patch is formed through the combined effects of Cdc42 activity promoting its own activation and active Cdc42-GTP displaying reduced membrane detachment and lateral diffusion compared to inactive Cdc42-GDP. I argue the role of the actin cytoskeleton in symmetry breaking is not primarily to transport Cdc42 to the active site. Finally, negative feedback and competition mechanisms serve to control the number of polarization sites.

On the Convergence of Stochastic Iterative Dynamic Programming Algorithms

Recent developments in the area of reinforcement learning have yielded a number of new algorithms for the prediction and control of Markovian environments. These algorithms, including the TD(lambda) algorithm of Sutton (1988) and the Q-learning algorithm of Watkins (1989), can be motivated heuristically as approximations to dynamic programming (DP). In this paper we provide a rigorous proof of convergence of these DP-based learning algorithms by relating them to the powerful techniques of stochastic approximation theory via a new convergence theorem. The theorem establishes a general class of convergent algorithms to which both TD(lambda) and Q-learning belong.

Coordinating Inventory Control and Pricing Strategies with Random Demand and Fixed Ordering Cost: The Finite Horizon Case

We analyze a finite horizon, single product, periodic review model in which pricing and production/inventory decisions are made simultaneously. Demands in different periods are random variables that are independent of each other and their distributions depend on the product price. Pricing and ordering decisions are made at the beginning of each period and all shortages are backlogged. Ordering cost includes both a fixed cost and a variable cost proportional to the amount ordered. The objective is to find an inventory policy and a pricing strategy maximizing expected profit over the finite horizon. We show that when the demand model is additive, the profit-to-go functions are k-concave and hence an (s,S,p) policy is optimal. In such a policy, the period inventory is managed based on the classical (s,S) policy and price is determined based on the inventory position at the beginning of each period. For more general demand functions, i.e., multiplicative plus additive functions, we demonstrate that the profit-to-go function is not necessarily k-concave and an (s,S,p) policy is not necessarily optimal. We introduce a new concept, the symmetric k-concave functions and apply it to provide a characterization of the optimal policy.

Coordinating Inventory Control and Pricing Strategies with Random Demand and Fixed Ordering Cost: the Infinite Horizon Case

We analyze an infinite horizon, single product, periodic review model in which pricing and production/inventory decisions are made simultaneously. Demands in different periods are identically distributed random variables that are independent of each other and their distributions depend on the product price. Pricing and ordering decisions are made at the beginning of each period and all shortages are backlogged. Ordering cost includes both a fixed cost and a variable cost proportional to the amount ordered. The objective is to maximize expected discounted, or expected average profit over the infinite planning horizon. We show that a stationary (s,S,p) policy is optimal for both the discounted and average profit models with general demand functions. In such a policy, the period inventory is managed based on the classical (s,S) policy and price is determined based on the inventory position at the beginning of each period.

Improvement of classification accuracy for stochastic discrimination - Multi-class classification

In this paper, an improved stochastic discrimination (SD) is introduced to reduce the error rate of the standard SD in the context of multi-class classification problem. The learning procedure of the improved SD consists of two stages. In the first stage, a standard SD, but with shorter learning period is carried out to identify an important space where all the misclassified samples are located. In the second stage, the standard SD is modified by (i) restricting sampling in the important space; and (ii) introducing a new discriminant function for samples in the important space. It is shown by mathematical derivation that the new discriminant function has the same mean, but smaller variance than that of standard SD for samples in the important space. It is also analyzed that the smaller the variance of the discriminant function, the lower the error rate of the classifier. Consequently, the proposed improved SD improves standard SD by its capability of achieving higher classification accuracy. Illustrative examples axe provided to demonstrate the effectiveness of the proposed improved SD.

Relationship between åström control and the kalman linear regulator — Caines revisited

The relationship between minimum variance and minimum expected quadratic loss feedback controllers for linear univariate discrete-time stochastic systems is reviewed by taking the approach used by Caines. It is shown how the two methods can be regarded as providing identical control actions as long as a noise-free measurement state-space model is employed.

Resource allocation analysis of the stochastic diffusion search

The Stochastic Diffusion Search (SDS) was developed as a solution to the best-fit search problem. Thus, as a special case it is capable of solving the transform invariant pattern recognition problem. SDS is efficient and, although inherently probabilistic, produces very reliable solutions in widely ranging search conditions. However, to date a systematic formal investigation of its properties has not been carried out. This thesis addresses this problem. The thesis reports results pertaining to the global convergence of SDS as well as characterising its time complexity. However, the main emphasis of the work, reports on the resource allocation aspect of the Stochastic Diffusion Search operations. The thesis introduces a novel model of the algorithm, generalising an Ehrenfest Urn Model from statistical physics. This approach makes it possible to obtain a thorough characterisation of the response of the algorithm in terms of the parameters describing the search conditions in case of a unique best-fit pattern in the search space. This model is further generalised in order to account for different search conditions: two solutions in the search space and search for a unique solution in a noisy search space. Also an approximate solution in the case of two alternative solutions is proposed and compared with predictions of the extended Ehrenfest Urn model. The analysis performed enabled a quantitative characterisation of the Stochastic Diffusion Search in terms of exploration and exploitation of the search space. It appeared that SDS is biased towards the latter mode of operation. This novel perspective on the Stochastic Diffusion Search lead to an investigation of extensions of the standard SDS, which would strike a different balance between these two modes of search space processing. Thus, two novel algorithms were derived from the standard Stochastic Diffusion Search, ‘context-free’ and ‘context-sensitive’ SDS, and their properties were analysed with respect to resource allocation. It appeared that they shared some of the desired features of their predecessor but also possessed some properties not present in the classic SDS. The theory developed in the thesis was illustrated throughout with carefully chosen simulations of a best-fit search for a string pattern, a simple but representative domain, enabling careful control of search conditions.

Learning in neural networks and stochastic approximation methods with averaging

The problem of adjusting the weights (learning) in multilayer feedforward neural networks (NN) is known to be of a high importance when utilizing NN techniques in various practical applications. The learning procedure is to be performed as fast as possible and in a simple computational fashion, the two requirements which are usually not satisfied practically by the methods developed so far. Moreover, the presence of random inaccuracies are usually not taken into account. In view of these three issues, an alternative stochastic approximation approach discussed in the paper, seems to be very promising.

Stress field control of eruption dynamics at a rift volcano: Nyamuragira, D.R.Congo

Using the record of 30 flank eruptions over the last 110 years at Nyamuragira, we have tested the relationship between the eruption dynamics and the local stress field. There are two groups of eruptions based on their duration (< 80days >) that are also clustered in space and time. We find that the eruptions fed by dykes parallel to the East African Rift Valley have longer durations (and larger volumes) than those eruptions fed by dykes with other orientations. This is compatible with a model for compressible magma transported through an elastic-walled dyke in a differential stress field from an over-pressured reservoir (Woods et al., 2006). The observed pattern of eruptive fissures is consistent with a local stress field modified by a northwest-trending, right lateral slip fault that is part of the northern transfer zone of the Kivu Basin rift segment. We have also re-tested with new data the stochastic eruption models for Nyamuragira of Burt et al. (1994). The time-predictable, pressure-threshold model remains the best fit and is consistent with the typically observed declining rate of sulphur dioxide emission during the first few days of eruption with lava emission from a depressurising, closed, crustal reservoir. The 2.4-fold increase in long-term eruption rate that occurred after 1977 is confirmed in the new analysis. Since that change, the record has been dominated by short-duration eruptions fed by dykes perpendicular to the Rift. We suggest that the intrusion of a major dyke during the 1977 volcano-tectonic event at neighbouring Nyiragongo volcano inhibited subsequent dyke formation on the southern flanks of Nyamuragira and this may also have resulted in more dykes reaching the surface elsewhere. Thus that sudden change in output was a result of a changed stress field that forced more of the deep magma supply to the surface. Another volcano-tectonic event in 2002 may also have changed the magma output rate at Nyamuragira.

Optimal power management strategy for energy storage with stochastic loads

In this paper, a power management strategy (PMS) has been developed for the control of energy storage in a system subjected to loads of random duration. The PMS minimises the costs associated with the energy consumption of specific systems powered by a primary energy source and equipped with energy storage, under the assumption that the statistical distribution of load durations is known. By including the variability of the load in the cost function, it was possible to define the optimality criteria for the power flow of the storage. Numerical calculations have been performed obtaining the control strategies associated with the global minimum in energy costs, for a wide range of initial conditions of the system. The results of the calculations have been tested on a MATLAB/Simulink model of a rubber tyre gantry (RTG) crane equipped with a flywheel energy storage system (FESS) and subjected to a test cycle, which corresponds to the real operation of a crane in the Port of Felixstowe. The results of the model show increased energy savings and reduced peak power demand with respect to existing control strategies, indicating considerable potential savings for port operators in terms of energy and maintenance costs.

Stochastic Assessment Of Environmental Flows In Semiarid Environments

The regimen of environmental flows (EF) must be included as terms of environmental demand in the management of water resources. Even though there are numerous methods for the computation of EF, the criteria applied at different steps in the calculation process are quite subjective whereas the results are fixed values that must be meet by water planners. This study presents a friendly-user tool for the assessment of the probability of compliance of a certain EF scenario with the natural regimen in a semiarid area in southern Spain. 250 replications of a 25-yr period of different hydrological variables (rainfall, minimum and maximum flows, ...) were obtained at the study site from the combination of Monte Carlo technique and local hydrological relationships. Several assumptions are made such as the independence of annual rainfall from year to year and the variability of occurrence of the meteorological agents, mainly precipitation as the main source of uncertainty. Inputs to the tool are easily selected from a first menu and comprise measured rainfall data, EF values and the hydrological relationships for at least a 20-yr period. The outputs are the probabilities of compliance of the different components of the EF for the study period. From this, local optimization can be applied to establish EF components with a certain level of compliance in the study period. Different options for graphic output and analysis of results are included in terms of graphs and tables in several formats. This methodology turned out to be a useful tool for the implementation of an uncertainty analysis within the scope of environmental flows in water management and allowed the simulation of the impacts of several water resource development scenarios in the study site.

Stochastic dynamics of Musca domestica (Diptera: Muscidae) and Chrysomya putoria (Diptera: Calliphoridae)

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Correlation times in stochastic equations with delayed feedback and multiplicative noise

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)