Modelling within-host spatiotemporal dynamics of invasive bacterial disease

Mechanistic determinants of bacterial growth, death, and spread within mammalian hosts cannot be fully resolved studying a single bacterial population. They are also currently poorly understood. Here, we report on the application of sophisticated experimental approaches to map spatiotemporal population dynamics of bacteria during an infection. We analyzed heterogeneous traits of simultaneous infections with tagged Salmonella enterica populations (wild-type isogenic tagged strains [WITS]) in wild-type and gene-targeted mice. WITS are phenotypically identical but can be distinguished and enumerated by quantitative PCR, making it possible, using probabilistic models, to estimate bacterial death rate based on the disappearance of strains through time. This multidisciplinary approach allowed us to establish the timing, relative occurrence, and immune control of key infection parameters in a true host-pathogen combination. Our analyses support a model in which shortly after infection, concomitant death and rapid bacterial replication lead to the establishment of independent bacterial subpopulations in different organs, a process controlled by host antimicrobial mechanisms. Later, decreased microbial mortality leads to an exponential increase in the number of bacteria that spread locally, with subsequent mixing of bacteria between organs via bacteraemia and further stochastic selection. This approach provides us with an unprecedented outlook on the pathogenesis of S. enterica infections, illustrating the complex spatial and stochastic effects that drive an infectious disease. The application of the novel method that we present in appropriate and diverse host-pathogen combinations, together with modelling of the data that result, will facilitate a comprehensive view of the spatial and stochastic nature of within-host dynamics. © 2008 Grant et al.

The Evaluation of Fisheries Management: A Dynamic Stochastic Approach

In this article, we analyze how to evaluate fishery resource management under “ecological uncertainty”. In this context, an efficient policy consists of applying a different exploitation rule depending on the state of the resource and we could say that the stock is always in transition, jumping from one steady state to another. First, we propose a method for calibrating the growth path of the resource such that observed dynamics of resource and captures are matched. Second, we apply the calibration procedure proposed in two different fishing grounds: the European Anchovy (Division VIII) and the Southern Stock of Hake. Our results show that the role played by uncertainty is essential for the conclusions. For European Anchovy fishery (Division VIII) we find, in contrast with Del Valle et al. (2001), that this is not an overexploited fishing ground. However, we show that the Southern Stock of Hake is in a dangerous situation. In both cases our results are in accordance with ICES advice.

Role of feedback and dynamics in a gene regulatory network

Cells exhibit a diverse repertoire of dynamic behaviors. These dynamic functions are implemented by circuits of interacting biomolecules. Although these regulatory networks function deterministically by executing specific programs in response to extracellular signals, molecular interactions are inherently governed by stochastic fluctuations. This molecular noise can manifest as cell-to-cell phenotypic heterogeneity in a well-mixed environment. Single-cell variability may seem like a design flaw but the coexistence of diverse phenotypes in an isogenic population of cells can also serve a biological function by increasing the probability of survival of individual cells upon an abrupt change in environmental conditions. Decades of extensive molecular and biochemical characterization have revealed the connectivity and mechanisms that constitute regulatory networks. We are now confronted with the challenge of integrating this information to link the structure of these circuits to systems-level properties such as cellular decision making. To investigate cellular decision-making, we used the well studied galactose gene-regulatory network in \textit{Saccharomyces cerevisiae}. We analyzed the mechanism and dynamics of the coexistence of two stable on and off states for pathway activity. We demonstrate that this bimodality in the pathway activity originates from two positive feedback loops that trigger bistability in the network. By measuring the dynamics of single-cells in a mixed sugar environment, we observe that the bimodality in gene expression is a transient phenomenon. Our experiments indicate that early pathway activation in a cohort of cells prior to galactose metabolism can accelerate galactose consumption and provide a transient increase in growth rate. Together these results provide important insights into strategies implemented by cells that may have been evolutionary advantageous in competitive environments.

Efficient methods for stochastic optimal control

The Hamilton Jacobi Bellman (HJB) equation is central to stochastic optimal control (SOC) theory, yielding the optimal solution to general problems specified by known dynamics and a specified cost functional. Given the assumption of quadratic cost on the control input, it is well known that the HJB reduces to a particular partial differential equation (PDE). While powerful, this reduction is not commonly used as the PDE is of second order, is nonlinear, and examples exist where the problem may not have a solution in a classical sense. Furthermore, each state of the system appears as another dimension of the PDE, giving rise to the curse of dimensionality. Since the number of degrees of freedom required to solve the optimal control problem grows exponentially with dimension, the problem becomes intractable for systems with all but modest dimension.

In the last decade researchers have found that under certain, fairly non-restrictive structural assumptions, the HJB may be transformed into a linear PDE, with an interesting analogue in the discretized domain of Markov Decision Processes (MDP). The work presented in this thesis uses the linearity of this particular form of the HJB PDE to push the computational boundaries of stochastic optimal control.

This is done by crafting together previously disjoint lines of research in computation. The first of these is the use of Sum of Squares (SOS) techniques for synthesis of control policies. A candidate polynomial with variable coefficients is proposed as the solution to the stochastic optimal control problem. An SOS relaxation is then taken to the partial differential constraints, leading to a hierarchy of semidefinite relaxations with improving sub-optimality gap. The resulting approximate solutions are shown to be guaranteed over- and under-approximations for the optimal value function. It is shown that these results extend to arbitrary parabolic and elliptic PDEs, yielding a novel method for Uncertainty Quantification (UQ) of systems governed by partial differential constraints. Domain decomposition techniques are also made available, allowing for such problems to be solved via parallelization and low-order polynomials.

The optimization-based SOS technique is then contrasted with the Separated Representation (SR) approach from the applied mathematics community. The technique allows for systems of equations to be solved through a low-rank decomposition that results in algorithms that scale linearly with dimensionality. Its application in stochastic optimal control allows for previously uncomputable problems to be solved quickly, scaling to such complex systems as the Quadcopter and VTOL aircraft. This technique may be combined with the SOS approach, yielding not only a numerical technique, but also an analytical one that allows for entirely new classes of systems to be studied and for stability properties to be guaranteed.

The analysis of the linear HJB is completed by the study of its implications in application. It is shown that the HJB and a popular technique in robotics, the use of navigation functions, sit on opposite ends of a spectrum of optimization problems, upon which tradeoffs may be made in problem complexity. Analytical solutions to the HJB in these settings are available in simplified domains, yielding guidance towards optimality for approximation schemes. Finally, the use of HJB equations in temporal multi-task planning problems is investigated. It is demonstrated that such problems are reducible to a sequence of SOC problems linked via boundary conditions. The linearity of the PDE allows us to pre-compute control policy primitives and then compose them, at essentially zero cost, to satisfy a complex temporal logic specification.

Stochastic optimal control

H. J. Kushner has obtained the differential equation satisfied by the optimal feedback control law for a stochastic control system in which the plant dynamics and observations are perturbed by independent additive Gaussian white noise processes. However, the differentiation includes the first and second functional derivatives and, except for a restricted set of systems, is too complex to solve with present techniques.

This investigation studies the optimal control law for the open loop system and incorporates it in a sub-optimal feedback control law. This suboptimal control law's performance is at least as good as that of the optimal control function and satisfies a differential equation involving only the first functional derivative. The solution of this equation is equivalent to solving two two-point boundary valued integro-partial differential equations. An approximate solution has advantages over the conventional approximate solution of Kushner's equation.

As a result of this study, well known results of deterministic optimal control are deduced from the analysis of optimal open loop control.

Modelling Lipid Competition Dynamics in Heterogeneous Protocell Populations

Recent experimental work in the field of synthetic protocell biology has shown that prebiotic vesicles are able to 'steal' lipids from each other. This phenomenon is driven purely by asymmetries in the physical state or composition of the vesicle membranes, and, when lipid resource is limited, translates directly into competition amongst the vesicles. Such a scenario is interesting from an origins of life perspective because a rudimentary form of cell-level selection emerges. To sharpen intuition about possible mechanisms underlying this behaviour, experimental work must be complemented with theoretical modelling. The aim of this paper is to provide a coarse-grain mathematical model of protocell lipid competition. Our model is capable of reproducing, often quantitatively, results from core experimental papers that reported distinct types vesicle competition. Additionally, we make some predictions untested in the lab, and develop a general numerical method for quickly solving the equilibrium point of a model vesicle population.

Analysis of multivariate stochastic signals sampled by on-line particle analyzers: Application to the quantitative assessment of occupational exposure to NOAA in multisource industrial scenarios (MSIS)

In multisource industrial scenarios (MSIS) coexist NOAA generating activities with other productive sources of airborne particles, such as parallel processes of manufacturing or electrical and diesel machinery. A distinctive characteristic of MSIS is the spatially complex distribution of aerosol sources, as well as their potential differences in dynamics, due to the feasibility of multi-task configuration at a given time. Thus, the background signal is expected to challenge the aerosol analyzers at a probably wide range of concentrations and size distributions, depending of the multisource configuration at a given time. Monitoring and prediction by using statistical analysis of time series captured by on-line particle analyzers in industrial scenarios, have been proven to be feasible in predicting PNC evolution provided a given quality of net signals (difference between signal at source and background). However the analysis and modelling of non-consistent time series, influenced by low levels of SNR (Signal-Noise Ratio) could build a misleading basis for decision making. In this context, this work explores the use of stochastic models based on ARIMA methodology to monitor and predict exposure values (PNC). The study was carried out in a MSIS where an case study focused on the manufacture of perforated tablets of nano-TiO2 by cold pressing was performed

Triggering in a thermoacoustic system with stochastic noise

This paper explores the mechanism of triggering in a simple thermoacoustic system, the Rijke tube. It is demonstrated that additive stochastic perturbations can cause triggering before the linear stability limit of a thermoacoustic system. When triggering from low noise amplitudes, the system is seen to evolve to self-sustained oscillations via an unstable periodic solution of the governing equations. Practical stability is introduced as a measure of the stability of a linearly stable state when finite perturbations are present. The concept of a stochastic stability map is used to demonstrate the change in practical stability limits for a system with a subcritical bifurcation, once stochastic terms are included. The practical stability limits are found to be strongly dependent on the strength of noise.

Lymph Node Colonization Dynamics after Oral Salmonella Typhimurium Infection in Mice

An understanding of how pathogens colonize their hosts is crucial for the rational design of vaccines or therapy. While the molecular factors facilitating the invasion and systemic infection by pathogens are a central focus of research in microbiology, the population biological aspects of colonization are still poorly understood. Here, we investigated the early colonization dynamics of Salmonella enterica subspecies 1 serovar Typhimurium (S. Tm) in the streptomycin mouse model for diarrhea. We focused on the first step on the way to systemic infection - the colonization of the cecal lymph node (cLN) from the gut - and studied roles of inflammation, dendritic cells and innate immune effectors in the colonization process. To this end, we inoculated mice with mixtures of seven wild type isogenic tagged strains (WITS) of S. Tm. The experimental data were analyzed with a newly developed mathematical model describing the stochastic immigration, replication and clearance of bacteria in the cLN. We estimated that in the beginning of infection only 300 bacterial cells arrive in the cLN per day. We further found that inflammation decreases the net replication rate in the cLN by 23%. In ccr7-/- mice, in which dendritic cell movement is impaired, the bacterial migration rate was reduced 10-fold. In contrast, cybb-/- mice that cannot generate toxic reactive oxygen species displayed a 4-fold higher migration rate from gut to cLN than wild type mice. Thus, combining infections with mixed inocula of barcoded strains and mathematical analysis represents a powerful method for disentangling immigration into the cLN from replication in this compartment. The estimated parameters provide an important baseline to assess and predict the efficacy of interventions. © 2013 Kaiser et al.

Robustness and dissipation of mitogen-activated protein kinases signal transduction network: Underlying funneled landscape against stochastic fluctuations

We uncovered the underlying energy landscape of the mitogen-activated protein kinases signal transduction cellular network by exploring the statistical natures of the Brownian dynamical trajectories. We introduce a dimensionless quantity: The robustness ratio of energy gap versus local roughness to measure the global topography of the underlying landscape. A high robustness ratio implies funneled landscape. The landscape is quite robust against environmental fluctuations and variants of the intrinsic chemical reaction rates.

The Essential Dynamics Algorithm: Essential Results

This paper presents a novel algorithm for learning in a class of stochastic Markov decision processes (MDPs) with continuous state and action spaces that trades speed for accuracy. A transform of the stochastic MDP into a deterministic one is presented which captures the essence of the original dynamics, in a sense made precise. In this transformed MDP, the calculation of values is greatly simplified. The online algorithm estimates the model of the transformed MDP and simultaneously does policy search against it. Bounds on the error of this approximation are proven, and experimental results in a bicycle riding domain are presented. The algorithm learns near optimal policies in orders of magnitude fewer interactions with the stochastic MDP, using less domain knowledge. All code used in the experiments is available on the project's web site.

Holevo-Ordering and the Continuous-Time Limit for Open Floquet Dynamics

Gough, John, (2004) 'Holevo-Ordering and the Continuous-Time Limit for Open Floquet Dynamics', Letters in Mathematical Physcis 67(3) pp.207-221 RAE2008