Approximate Bayesian techniques for inference in stochastic dynamical systems

This thesis is concerned with approximate inference in dynamical systems, from a variational Bayesian perspective. When modelling real world dynamical systems, stochastic differential equations appear as a natural choice, mainly because of their ability to model the noise of the system by adding a variant of some stochastic process to the deterministic dynamics. Hence, inference in such processes has drawn much attention. Here two new extended frameworks are derived and presented that are based on basis function expansions and local polynomial approximations of a recently proposed variational Bayesian algorithm. It is shown that the new extensions converge to the original variational algorithm and can be used for state estimation (smoothing). However, the main focus is on estimating the (hyper-) parameters of these systems (i.e. drift parameters and diffusion coefficients). The new methods are numerically validated on a range of different systems which vary in dimensionality and non-linearity. These are the Ornstein-Uhlenbeck process, for which the exact likelihood can be computed analytically, the univariate and highly non-linear, stochastic double well and the multivariate chaotic stochastic Lorenz '63 (3-dimensional model). The algorithms are also applied to the 40 dimensional stochastic Lorenz '96 system. In this investigation these new approaches are compared with a variety of other well known methods such as the ensemble Kalman filter / smoother, a hybrid Monte Carlo sampler, the dual unscented Kalman filter (for jointly estimating the systems states and model parameters) and full weak-constraint 4D-Var. Empirical analysis of their asymptotic behaviour as a function of observation density or length of time window increases is provided.

Structure-function analysis of RAMP1 by Alanine Mutagenesis

Receptor activity modifying protein 1 (RAMP1) is an integral component of several receptors including the calcitonin gene-related peptide (CGRP) receptor. It forms a complex with the calcitonin receptor-like receptor (CLR) and is required for receptor trafficking and ligand binding. The N-terminus of RAMP1 comprises three helices. The current study investigated regions of RAMP1 important for CGRP or CLR interactions by alanine mutagenesis. Modeling suggested the second and third helices were important in protein-protein interactions. Most of the conserved residues in the N-terminus (M48, W56, Y66, P85, N66, H97, F101, D113, P114, P115), together with a further 13 residues spread throughout three helices of RAMP1, were mutated to alanine and coexpressed with CLR in Cos 7 cells. None of the mutations significantly reduced RAMP expression. Of the nine mutants from helix 1, only M48A had any effect, producing a modest reduction in trafficking of CLR to the cell surface. In helix 2 Y66A almost completely abolished CLR trafficking; L69A and T73A reduced the potency of CGRP to produce cAMP. In helix 3, H97A abolished CLR trafficking; P85A, N86A, and F101A had caused modest reductions in CLR trafficking and also reduced the potency of CGRP on cAMP production. F93A caused a modest reduction in CLR trafficking alone and L94A increased cAMP production. The data are consistent with a CLR recognition site particularly involving Y66 and H97, with lesser roles for adjacent residues in helix 3. L69 and T73 may contribute to a CGRP recognition site in helix 2 also involving nearby residues.

Patterns of phonological errors as a function of a phonological versus an articulatory locus of impairment

WWe present the case of two aphasic patients: one with fluent speech, MM, and one with dysfluent speech, DB. Both patients make similar proportions of phonological errors in speech production and the errors have similar characteristics. A closer analysis, however, shows a number of differences. DB's phonological errors involve, for the most part, simplifications of syllabic structure; they affect consonants more than vowels; and, among vowels, they show effects of sonority/complexity. This error pattern may reflect articulatory difficulties. MM's errors, instead, show little effect of syllable structure, affect vowels at least as much as consonants and, and affect all different vowels to a similar extent. This pattern is consistent with a more central impairment involving the selection of the right phoneme among competing alternatives. We propose that, at this level, vowel selection may be more difficult than consonant selection because vowels belong to a smaller set of repeatedly activated units.

Aberrant reactive oxygen and nitrogen species generation in rheumatoid arthritis (RA):causes and consequences for immune function, cell survival, and therapeutic intervention

The infiltration and persistence of hematopoietic immune cells within the rheumatoid arthritis (RA) joint results in elevated levels of pro-inflammatory cytokines, increased reactive oxygen (ROS) and -nitrogen (RNS) species generation, that feeds a continuous self-perpetuating cycle of inflammation and destruction. Meanwhile, the controlled production of ROS is required for signaling within the normal physiological reaction to perceived "foreign matter" and for effective apoptosis. This review focuses on the signaling pathways responsible for the induction of the normal immune response and the contribution of ROS to this process. Evidence for defects in the ability of immune cells in RA to regulate the generation of ROS and the consequence for their immune function and for RA progression is considered. As the hypercellularity of the rheumatoid joint and the associated persistence of hematopoietic cells within the rheumatoid joint are symptomatic of unresponsiveness to apoptotic stimuli, the role of apoptotic signaling proteins (specifically Bcl-2 family members and the tumor suppressor p53) as regulators of ROS generation and apoptosis are considered, evaluating evidence for their aberrant expression and function in RA. We postulate that ROS generation is required for effective therapeutic intervention.

Estimating parameters in stochastic systems:a variational Bayesian approach

This work is concerned with approximate inference in dynamical systems, from a variational Bayesian perspective. When modelling real world dynamical systems, stochastic differential equations appear as a natural choice, mainly because of their ability to model the noise of the system by adding a variation of some stochastic process to the deterministic dynamics. Hence, inference in such processes has drawn much attention. Here a new extended framework is derived that is based on a local polynomial approximation of a recently proposed variational Bayesian algorithm. The paper begins by showing that the new extension of this variational algorithm can be used for state estimation (smoothing) and converges to the original algorithm. However, the main focus is on estimating the (hyper-) parameters of these systems (i.e. drift parameters and diffusion coefficients). The new approach is validated on a range of different systems which vary in dimensionality and non-linearity. These are the Ornstein–Uhlenbeck process, the exact likelihood of which can be computed analytically, the univariate and highly non-linear, stochastic double well and the multivariate chaotic stochastic Lorenz ’63 (3D model). As a special case the algorithm is also applied to the 40 dimensional stochastic Lorenz ’96 system. In our investigation we compare this new approach with a variety of other well known methods, such as the hybrid Monte Carlo, dual unscented Kalman filter, full weak-constraint 4D-Var algorithm and analyse empirically their asymptotic behaviour as a function of observation density or length of time window increases. In particular we show that we are able to estimate parameters in both the drift (deterministic) and the diffusion (stochastic) part of the model evolution equations using our new methods.

Technoeconomic modelling of coal conversion processes for liquid fuel production

Since the oil crisis of 1973 considerable interest has been shown in the production of liquid fuels from alternative sources. In particular processes utilizing coal as the feedstock have received considerable interest. These processes can be divided into direct and indirect liquefaction and pyrolysis. This thesis describes the modelling of indirect coal liquefaction processes for the purpose of performing technical and economic assessment of the production of liquid fuels from coal and lignite, using a variety of gasification and synthesis gas liquefaction technologies. The technologies were modeled on a 'step model' basis where a step is defined as a combination of individual unit operations which together perform a significant function on the process streams, such as a methanol synthesis step or a gasification and physical gas cleaning step. Sample results of the modelling, covering a wide range of gasifiers, liquid synthesis processes and products are presented in this thesis. Due to the large number of combinations of gasifier, liquid synthesis processes, products and economic sensitivity cases, a complete set of results is impractical to present in a single publication. The main results show that methanol is the cheapest fuel to produce from coal followed by fuel alcohol, diesel from the Shell Middle Distillate Synthesis process,gasoline from Mobil Methanol to Gasoline (MTG) process, diesel from the Mobil Methanol Olefins Gasoline Diesel (MOGD) process and finally gasoline from the same process. Some variation in production costs of all the products was shown depending on type of gasifier chosen and feedstock.

Multi-function controllers for a hydraulic pump

This project has been undertaken for Hamworthy Hydraulics Limited. Its objective was to design and develop a controller package for a variable displacement, hydraulic pump for use mainly on mobile earth moving machinery. A survey was undertaken of control options used in practice and from this a design specification was formulated, the successful implementation of which would give Hamworthy an advantage over its competitors. Two different modes for the controller were envisaged. One consisted of using conventional hydro-mechanics and the other was based upon a microprocessor. To meet short term customer prototype requirements the first section of work was the realisation of the hydro-mechanical system. Mathematical models were made to evaluate controller stability and hence aid their design. The final package met the requirements of the specification and a single version could operate all sizes of variable displacement pumps in the Hamworthy range. The choice of controller options and combinations totalled twenty-four. The hydro-mechanical controller was complex and it was realised that a micro-processor system would allow all options to be implemented with just one design of hardware, thus greatly simplifying production. The final section of this project was to determine whether such a design was feasible. This entailed finding cheap, reliable transducers, using mathematical models to predict electro-hydraulic interface stability, testing such interfaces and finally incorporating a micro-processor in an interactive control loop. The study revealed that such a system was technically possible but it would cost 60% more than its hydro-mechanical counterpart. It was therefore concluded that, in the short term, for the markets considered, the hydro-mechanical design was the better solution. Regarding the micro-processor system the final conclusion was that, because the relative costs of the two systems are decreasing, the electro-hydraulic controller will gradually become more attractive and therefore Hamworthy should continue with its development.

Exploiting uncertainty in nonlinear stochastic control problem

This work introduces a novel inversion-based neurocontroller for solving control problems involving uncertain nonlinear systems which could also compensate for multi-valued systems. The approach uses recent developments in neural networks, especially in the context of modelling statistical distributions, which are applied to forward and inverse plant models. Provided that certain conditions are met, an estimate of the intrinsic uncertainty for the outputs of neural networks can be obtained using the statistical properties of networks. More generally, multicomponent distributions can be modelled by the mixture density network. Based on importance sampling from these distributions a novel robust inverse control approach is obtained. This importance sampling provides a structured and principled approach to constrain the complexity of the search space for the ideal control law. The developed methodology circumvents the dynamic programming problem by using the predicted neural network uncertainty to localise the possible control solutions to consider. Convergence of the output error for the proposed control method is verified by using a Lyapunov function. Several simulation examples are provided to demonstrate the efficiency of the developed control method. The manner in which such a method is extended to nonlinear multi-variable systems with different delays between the input-output pairs is considered and demonstrated through simulation examples.

An approach to the development of expert systems within production planning and control

The thesis presents an account of an attempt to utilize expert systems within the domain of production planning and control. The use of expert systems was proposed due to the problematical nature of a particular function within British Steel Strip Products' Operations Department: the function of Order Allocation, allocating customer orders to a production week and site. Approaches to tackling problems within production planning and control are reviewed, as are the general capabilities of expert systems. The conclusions drawn are that the domain of production planning and control contains both `soft' and `hard' problems, and that while expert systems appear to be a useful technology for this domain, this usefulness has by no means yet been demonstrated. Also, it is argued that the main stream methodology for developing expert systems is unsuited for the domain. A problem-driven approach is developed and used to tackle the Order Allocation function. The resulting system, UAAMS, contained two expert components. One of these, the scheduling procedure was not fully implemented due to inadequate software. The second expert component, the product routing procedure, was untroubled by such difficulties, though it was unusable on its own; thus a second system was developed. This system, MICRO-X10, duplicated the function of X10, a complex database query routine used daily by Order Allocation. A prototype version of MICRO-X10 proved too slow to be useful but allowed implementation and maintenance issues to be analysed. In conclusion, the usefulness of the problem-driven approach to expert systems development within production planning and control is demonstrated but restrictions imposed by current expert system software are highlighted in that the abilities of such software to cope with `hard' scheduling constructs and also the slow processing speeds of such software can restrict the current usefulness of expert systems within production planning and control.

Improving the function of islet and beta-cell grafts

Background: Human islet transplantation would offer a less invasive and more physiological alternative than whole pancreas transplantation and insulin injections respectively for the treatment of diabetes mellitus if islet graft survival can be improved. Initial recipient post-transplant insulin independence declines to <10% after 5 years. Factors contributing to graft failure include enzymatic disruption of the islet microenvironment during isolation, diabetogenic effects of immunosuppressants and metabolic stress resulting from slow revascularisation. Aims: To investigate the effect of co-culture in both static (SC) and rotational culture (RC) of BRINBDII beta-cells (Dl1) and human umbilical vein endothelial cells (HUVEC) on Dl1 insulin secretion; and the effect of a thiazolidinedione (TZD) on DII function and HUVEC proliferation. To assess the effect of culture media, SC, RC and a TZD on human islet morphology, insulin secretion and VEGF production. To initiate in vivo protocol development for assessment of revascularisation of human islet grafts. Methods: D11 cells were cultured +/-TZD and co-cultured with HUVEC +/-TZD in SC and RC. Dl1 insulin secretion was induced by static incubation with low glucose (1.67mM), high glucose (l6.7mM: and high glucose with 10mM theophylline (G+T) and determined by ELISA. HUVEC were cultured +/-TZD in SC and RC and proliferation was assessed by ATP luminescence assay and VEGF ELISA. D II and HUVEC morphology was determined by immunocytochemistry. Human islets were cultured in SC and RC in various media +/-TZD. Insulin secretion was determined as above and VEGF production by fluorescence immunocytochemistry (FI) and ELISA. Revascularisation of islet grafts was assessed by vascular corrosion cast and FI. Results: Dll cultures showed significantly increased insulin secretion in response to 16.7mM and G+T over basal; this was enhanced by RC and further improved by adding 10mM TZD. Untreated Dll/HUVEC co-cultures displayed significantly increased insulin secretion in response to 16.7mM and G+T over basal, again enhanced by RC and improved with 10mM TZD. 10mM TZD significantly increased HUVEC proliferation over control. Human islets maintained in medium 199 (mI99) in SC and RC exhibited comparable maintenance of morphology and insulin secretory profiles compared to islets maintained in RPMI, endothelial growth media and dedicated islet medium Miami# I. All cultures showed significantly increased insulin secretion in response to 16.7mM and G+T over basal; this was enhanced by RC and in certain instances further improved by adding 25mM TZD. TZD increased VEGF production and release as determined by ELISA. Post-implant vascular corrosion casts of mouse kidneys analysed by x-ray micro tomography indicates a possible TZD enhancement of microvessel growth via VEGF upregulation. Conclusions: D II /HUVEC co-culture in SC or RC does not alter the morphology of either cell type and supports D 11 function. TZD improves 0 I I and D I I/HUVEC SC and RC co-culture insulin secretion while increasing HUVEC proliferation. Human islet RC supports islet functional viability and structural integrity compared to SC while the addition of TZD occasionally further improves secretagogue induced insulin secretion. Expensive, 'dedicated' islet media showed no advantage over ml99 in terms of maintaining islet morphology or function. TZD upregulates VEGF in islets as shown by ELISA and suggested by x-ray micro tomography analysis of vascular corrosion casts. Maintenance of islets in RC and treatment with TZD prior to transplant may improve the functional viability and revascularisation rate of islet grafts.

A Bayesian perspective on stochastic neurocontrol

Control design for stochastic uncertain nonlinear systems is traditionally based on minimizing the expected value of a suitably chosen loss function. Moreover, most control methods usually assume the certainty equivalence principle to simplify the problem and make it computationally tractable. We offer an improved probabilistic framework which is not constrained by these previous assumptions, and provides a more natural framework for incorporating and dealing with uncertainty. The focus of this paper is on developing this framework to obtain an optimal control law strategy using a fully probabilistic approach for information extraction from process data, which does not require detailed knowledge of system dynamics. Moreover, the proposed control method framework allows handling the problem of input-dependent noise. A basic paradigm is proposed and the resulting algorithm is discussed. The proposed probabilistic control method is for the general nonlinear class of discrete-time systems. It is demonstrated theoretically on the affine class. A nonlinear simulation example is also provided to validate theoretical development.

Extracellular loops 1 and 3 and their associated transmembrane regions of the calcitonin receptor-like receptor are needed for CGRP receptor function

The first and third extracellular loops (ECL) of G protein-coupled receptors (GPCRs) have been implicated in ligand binding and receptor function. This study describes the results of an alanine/leucine scan of ECLs 1 and 3 and loop-associated transmembrane (TM) domains of the secretin-like GPCR calcitonin receptor-like receptor which associates with receptor activity modifying protein 1 to form the CGRP receptor. Leu195Ala, Val198Ala and Ala199Leu at the top of TM2 all reduced aCGRP-mediated cAMP production and internalization; Leu195Ala and Ala199Leu also reduced aCGRP binding. These residues form a hydrophobic cluster within an area defined as the "minor groove" of rhodopsin-like GPCRs. Within ECL1, Ala203Leu and Ala206Leu influenced the ability of aCGRP to stimulate adenylate cyclase. In TM3, His219Ala, Leu220Ala and Leu222Ala have influences on aCGRP binding and cAMP production; they are likely to indirectly influence the binding site for aCGRP as well as having an involvement in signal transduction. On the exofacial surfaces of TMs 6 and 7, a number of residues were identified that reduced cell surface receptor expression, most noticeably Leu351Ala and Glu357Ala in TM6. The residues may contribute to the RAMP1 binding interface. Ile360Ala impaired aCGRP-mediated cAMP production. Ile360 is predicted to be located close to ECL2 and may facilitate receptor activation. Identification of several crucial functional loci gives further insight into the activation mechanism of this complex receptor system and may aid rational drug design.

Receptor Activity Modifying Proteins (RAMPs) interact with the VPAC2 Receptor and CRF1 receptors and modulate their function

Background and Purpose Although it is established that the receptor activity modifying proteins (RAMPs) can interact with a number of GPCRs, little is known about the consequences of these interactions. Here the interaction of RAMPs with the glucagon-like peptide 1 receptor (GLP-1 receptor), the human vasoactive intestinal polypeptide/pituitary AC-Activating peptide 2 receptor (VPAC) and the type 1 corticotrophin releasing factor receptor (CRF) has been examined. Experimental Approach GPCRs were co-transfected with RAMPs in HEK 293S and CHO-K1 cells. Cell surface expression of RAMPs and GPCRs was examined by elisa. Where there was evidence for interactions, agonist-stimulated cAMP production, Ca mobilization and GTPγS binding to G, G, G and G were examined. The ability of CRF to stimulate adrenal corticotrophic hormone release in Ramp2 mice was assessed. Key Results The GLP-1 receptor failed to enhance the cell surface expression of any RAMP. VPAC enhanced the cell surface expression of all three RAMPs. CRF enhanced the cell surface expression of RAMP2; the cell surface expression of CRF was also increased. There was no effect on agonist-stimulated cAMP production. However, there was enhanced G-protein coupling in a receptor and agonist-dependent manner. The CRF: RAMP2 complex resulted in enhanced elevation of intracellular calcium to CRF and urocortin 1 but not sauvagine. In Ramp2 mice, there was a loss of responsiveness to CRF. Conclusions and Implications The VPAC and CRF receptors interact with RAMPs. This modulates G-protein coupling in an agonist-specific manner. For CRF, coupling to RAMP2 may be of physiological significance. © 2012 The Authors. British Journal of Pharmacology © 2012 The British Pharmacological Society.

A non-parametric data envelopment analysis approach for improving energy efficiency of grape production

Grape is one of the world's largest fruit crops with approximately 67.5 million tonnes produced each year and energy is an important element in modern grape productions as it heavily depends on fossil and other energy resources. Efficient use of these energies is a necessary step toward reducing environmental hazards, preventing destruction of natural resources and ensuring agricultural sustainability. Hence, identifying excessive use of energy as well as reducing energy resources is the main focus of this paper to optimize energy consumption in grape production.In this study we use a two-stage methodology to find the association of energy efficiency and performance explained by farmers' specific characteristics. In the first stage a non-parametric Data Envelopment Analysis is used to model efficiencies as an explicit function of human labor, machinery, chemicals, FYM (farmyard manure), diesel fuel, electricity and water for irrigation energies. In the second step, farm specific variables such as farmers' age, gender, level of education and agricultural experience are used in a Tobit regression framework to explain how these factors influence efficiency of grape farming.The result of the first stage shows substantial inefficiency between the grape producers in the studied area while the second stage shows that the main difference between efficient and inefficient farmers was in the use of chemicals, diesel fuel and water for irrigation. The use of chemicals such as insecticides, herbicides and fungicides were considerably less than inefficient ones. The results revealed that the more educated farmers are more energy efficient in comparison with their less educated counterparts. © 2013.

Hydrogen production by steam reforming of DME in a large scale CFB reactor. Part I:computational model and predictions

This study presents a computational fluid dynamic (CFD) study of Dimethyl Ether steam reforming (DME-SR) in a large scale Circulating Fluidized Bed (CFB) reactor. The CFD model is based on Eulerian-Eulerian dispersed flow and solved using commercial software (ANSYS FLUENT). The DME-SR reactions scheme and kinetics in the presence of a bifunctional catalyst of CuO/ZnO/Al2O3+ZSM-5 were incorporated in the model using in-house developed user-defined function. The model was validated by comparing the predictions with experimental data from the literature. The results revealed for the first time detailed CFB reactor hydrodynamics, gas residence time, temperature distribution and product gas composition at a selected operating condition of 300 °C and steam to DME mass ratio of 3 (molar ratio of 7.62). The spatial variation in the gas species concentrations suggests the existence of three distinct reaction zones but limited temperature variations. The DME conversion and hydrogen yield were found to be 87% and 59% respectively, resulting in a product gas consisting of 72 mol% hydrogen. In part II of this study, the model presented here will be used to optimize the reactor design and study the effect of operating conditions on the reactor performance and products.