Bayesian methods for estimation, optimization and experimental design

Mathematical models often contain parameters that need to be calibrated from measured data. The emergence of efficient Markov Chain Monte Carlo (MCMC) methods has made the Bayesian approach a standard tool in quantifying the uncertainty in the parameters. With MCMC, the parameter estimation problem can be solved in a fully statistical manner, and the whole distribution of the parameters can be explored, instead of obtaining point estimates and using, e.g., Gaussian approximations. In this thesis, MCMC methods are applied to parameter estimation problems in chemical reaction engineering, population ecology, and climate modeling. Motivated by the climate model experiments, the methods are developed further to make them more suitable for problems where the model is computationally intensive. After the parameters are estimated, one can start to use the model for various tasks. Two such tasks are studied in this thesis: optimal design of experiments, where the task is to design the next measurements so that the parameter uncertainty is minimized, and model-based optimization, where a model-based quantity, such as the product yield in a chemical reaction model, is optimized. In this thesis, novel ways to perform these tasks are developed, based on the output of MCMC parameter estimation. A separate topic is dynamical state estimation, where the task is to estimate the dynamically changing model state, instead of static parameters. For example, in numerical weather prediction, an estimate of the state of the atmosphere must constantly be updated based on the recently obtained measurements. In this thesis, a novel hybrid state estimation method is developed, which combines elements from deterministic and random sampling methods.

DENDROCLIMATOLOGICAL RECONSTRUCTION OF SPRING-SUMMER PRECIPITATION FOR FAZENDA RIO GRANDE, PR, WITH SAMPLES OF Araucaria angustifolia (Bertol.) Kuntze

ABSTRACT Samples of Araucaria angustifolia were collected at Fazenda Rio Grande, Paraná, Brazil (25°39'S 49 18'O) in January 2011. The 32 samples from 8 trees were subjected to treatments following dendrochronological techniques. The cores were measured and dated using optical and computational methodology, and then standardized to obtain a growth-ring time series, which considers the 1907-2009 time range and represents Fazenda Rio Grande. Tree-ring indices were analyzed and correlated to temperature and precipitation averages from the 1961-2009 range. This procedure aimed to study and understand the influence of the local climate on the plant growth and if this influence can be quantified. A. angustifolia trees produce visible annual growth rings, and their earlywood and latewood are clearly defined. The present study shows that A. angustifolia is sensitive to climate variables (e.g., low temperatures in wintertime tend to stop the growth rate). The correlation between tree rings and monthly precipitation series showed a common trend, making it possible to estimate the seasonal rainfall behavior for the entire 1907-2009 range.

Communicative and Anticipatory Decision-Making Supported by Bayesian Networks

The purpose of this research is to draw up a clear construction of an anticipatory communicative decision-making process and a successful implementation of a Bayesian application that can be used as an anticipatory communicative decision-making support system. This study is a decision-oriented and constructive research project, and it includes examples of simulated situations. As a basis for further methodological discussion about different approaches to management research, in this research, a decision-oriented approach is used, which is based on mathematics and logic, and it is intended to develop problem solving methods. The approach is theoretical and characteristic of normative management science research. Also, the approach of this study is constructive. An essential part of the constructive approach is to tie the problem to its solution with theoretical knowledge. Firstly, the basic definitions and behaviours of an anticipatory management and managerial communication are provided. These descriptions include discussions of the research environment and formed management processes. These issues define and explain the background to further research. Secondly, it is processed to managerial communication and anticipatory decision-making based on preparation, problem solution, and solution search, which are also related to risk management analysis. After that, a solution to the decision-making support application is formed, using four different Bayesian methods, as follows: the Bayesian network, the influence diagram, the qualitative probabilistic network, and the time critical dynamic network. The purpose of the discussion is not to discuss different theories but to explain the theories which are being implemented. Finally, an application of Bayesian networks to the research problem is presented. The usefulness of the prepared model in examining a problem and the represented results of research is shown. The theoretical contribution includes definitions and a model of anticipatory decision-making. The main theoretical contribution of this study has been to develop a process for anticipatory decision-making that includes management with communication, problem-solving, and the improvement of knowledge. The practical contribution includes a Bayesian Decision Support Model, which is based on Bayesian influenced diagrams. The main contributions of this research are two developed processes, one for anticipatory decision-making, and the other to produce a model of a Bayesian network for anticipatory decision-making. In summary, this research contributes to decision-making support by being one of the few publicly available academic descriptions of the anticipatory decision support system, by representing a Bayesian model that is grounded on firm theoretical discussion, by publishing algorithms suitable for decision-making support, and by defining the idea of anticipatory decision-making for a parallel version. Finally, according to the results of research, an analysis of anticipatory management for planned decision-making is presented, which is based on observation of environment, analysis of weak signals, and alternatives to creative problem solving and communication.

Monotonic trend and change points in southern Brazil precipitation

Scientific evidence on climate changes at global level has gained increasing interest in the scientific community in general. The impacts of climate change as well as anthropogenic actions may cause errors in hydro-agricultural projects existent in the watershed under study. This study aimed to identify the presence or absence of trend in total annual precipitation series of the watershed of the Mirim Lagoon, state of Rio Grande do Sul-RS / Brazil / Uruguay (Brazilian side) as well as to detect the period in which they occurred. For that, it was analyzed the precipitation data belonging to 14 weather stations. To detect the existence of monotonic trend and change points, it was used the nonparametric tests of Mann-Kendall and Mann-Whitney, the "t" test of Student for two samples of unpaired data (parametric), as well as the technique of progressive mean. The Weather Station 3152014 (Pelotas) presented changes in the trend in the series of annual precipitation in the period from 1953 to 2007. The methodologies that use subdivided series were more efficient in detecting change in trend when compared with the Mann-Kendall test, which uses the complete series (from 1921 to 2007).

Dry months in the agricultural region of Ribeirão Preto, state of São Paulo-Brazil: an study based on the extreme value theory

The application of the Extreme Value Theory (EVT) to model the probability of occurrence of extreme low Standardized Precipitation Index (SPI) values leads to an increase of the knowledge related to the occurrence of extreme dry months. This sort of analysis can be carried out by means of two approaches: the block maxima (BM; associated with the General Extreme Value distribution) and the peaks-over-threshold (POT; associated with the Generalized Pareto distribution). Each of these procedures has its own advantages and drawbacks. Thus, the main goal of this study is to compare the performance of BM and POT in characterizing the probability of occurrence of extreme dry SPI values obtained from the weather station of Ribeirão Preto-SP (1937-2012). According to the goodness-of-fit tests, both BM and POT can be used to assess the probability of occurrence of the aforementioned extreme dry SPI monthly values. However, the scalar measures of accuracy and the return level plots indicate that POT provides the best fit distribution. The study also indicated that the uncertainties in the parameters estimates of a probabilistic model should be taken into account when the probability associated with a severe/extreme dry event is under analysis.

Spatialization of climate, physical and socioeconomic factors that affect the dairy goat production in Brazil and their impact on animal breeding decisions

Brazil has high climate, soil and environmental diversity, as well as distinct socioeconomic and political realities, what results in differences among the political administrative regions of the country. The objective of this study was to determine spatial distribution of the physical, climatic and socioeconomic aspects that best characterize the production of dairy goats in Brazil. Production indices of milk per goat, goat production, milk production, as well as temperature range, mean temperature, precipitation, normalized difference vegetation index, relative humidity, altitude, agricultural farms; farms with native pasture, farms with good quality pasture, farms with water resources, farms that receive technical guidance, family farming properties, non-familiar farms and the human development index were evaluated. The multivariate analyses were carried out to spatialize climatic, physical and socioeconomic variables and so differenciate the Brazilian States and Regions. The highest yields of milk and goat production were observed in the Northeast. The Southeast Region had the second highest production of milk, followed by the South, Midwest and North. Multivariate analysis revealed distinctions between clusters of political-administrative regions of Brazil. The climatic variables were most important to discriminate between regions of Brazil. Therefore, it is necessary to implement animal breeding programs to meet the needs of each region.

Multidata inverse problems and Bayesian solution methods in astronomy

Statistical analyses of measurements that can be described by statistical models are of essence in astronomy and in scientific inquiry in general. The sensitivity of such analyses, modelling approaches, and the consequent predictions, is sometimes highly dependent on the exact techniques applied, and improvements therein can result in significantly better understanding of the observed system of interest. Particularly, optimising the sensitivity of statistical techniques in detecting the faint signatures of low-mass planets orbiting the nearby stars is, together with improvements in instrumentation, essential in estimating the properties of the population of such planets, and in the race to detect Earth-analogs, i.e. planets that could support liquid water and, perhaps, life on their surfaces. We review the developments in Bayesian statistical techniques applicable to detections planets orbiting nearby stars and astronomical data analysis problems in general. We also discuss these techniques and demonstrate their usefulness by using various examples and detailed descriptions of the respective mathematics involved. We demonstrate the practical aspects of Bayesian statistical techniques by describing several algorithms and numerical techniques, as well as theoretical constructions, in the estimation of model parameters and in hypothesis testing. We also apply these algorithms to Doppler measurements of nearby stars to show how they can be used in practice to obtain as much information from the noisy data as possible. Bayesian statistical techniques are powerful tools in analysing and interpreting noisy data and should be preferred in practice whenever computational limitations are not too restrictive.

Effect of additives, precursors and process conditions on particle morphology

The objective of the thesis was to develop methods to manufacture and control calcium carbonate crystal nucleation and growth in precipitation process. The work consists of experimental part and literature part that addresses theory of nucleation, crystallization and precipitation. In the experimental part calcium carbonate was precipitated using carbonization reaction. Precipitation was carried out in presence of known morphology controlling agents (anionic polymers and sodium silicate) and by using different operation conditions. Formed material was characterized using SEM images, and its thermal stability was assessed. This work demonstrates that carbon dioxide feeding rate and concentrations of calcium hydroxide and additives can be used to control size, shape and amount of precipitating calcium carbonate.

INFLUENCE OF PRECIPITATION AND SUGARCANE STRAW IN AMINOCYCLOPYRACHLOR AND INDAZIFLAM CONTROL EFFICIENCY

ABSTRACTThe raw sugarcane harvesting system has changed the dynamics of weed tillage for this crop, changing the predominant weed species and providing a barrier between the herbicide and the soil. Thus, this study has aimed to assess the influence of precipitation and sugarcane straw in the aminocyclopyrachlor and indaziflam herbicides control efficiency for the species Ipomoea trilobaand Euphorbia heterophylla. There were two trials, one for aminocyclopyrachlor and one for the indaziflam, both in the greenhouse at the campus of Faculdade Integrado in the Brazilian city of Campo Mourão, PR. Each experiment consisted of eight treatments with four replications. The treatments consisted of the combination of the presence of straw (10 t ha-1), capillary irrigation and rainfall simulation (20 mm). Assessments of control percentage of I.triloba and E.heterophylla were carried out, as well as the number of plants per pot. The aminocyclopyrachlor and indaziflam herbicides applied directly to the soil were efficient in controlling these species. The 20 mm rainfall simulation or daily irrigation on the straw are indispensable to promote the removal of aminocyclopyrachlor and indaziflam from the straw and provide satisfactory control of I.triloba and E.heterophylla.

Water relations of tree species growing on a rock outcrop in the "Parque Estadual de Itapuã", RS

Water relations of the tree species Myrsine umbellata Mart. ex A. DC., Dodonaea viscosa Jacq. and Erythroxylum argentinum O. E. Schulz, growing on a rock outcrop in the "Parque Estadual de Itapuã" (RS), were studied. Environmental (precipitation, temperature, soil water) and plant (water potential, vapor pressure deficit, stomatal conductance, transpiration, leaf specific hydraulic conductance, osmotic potential and cell wall elasticity) parameters were collected in five periods and pooled into two sets of data: wet and dry periods. Myrsine umbellata showed great stability of the plant parameters, including the maintenance of high pre-dawn (psiwpd) and mid-day (psiwmd) water potentials in the dry period (-0.48 and -1.12 MPa, respectively), suggesting the presence of a deep root system. Dodonaea viscosa and E. argentinum reached lower psiwpd (-1.41 and -1.97 MPa, respectively) and a greater degree of stomatal closure in the dry period, suggesting a shallower root system. Differential exposure to soil drought was also corroborated by differential drought effects on the whole-plant leaf specific hydraulic conductance (Gt). Correlation analysis pointed to weak correlations between psiwpd and g s. Erythroxylum argentinum was the only species to show osmotic adjustment in response to drought. It is suggested that M. umbellata has low tolerance to water deficits, adopting an avoidance behavior. The much lower values of psiw reached by D. viscosa and E. argentinum suggest a greater tolerance to drought by these species.

Adaptive Markov chain Monte Carlo and Bayesian filtering for state space models

This thesis is concerned with the state and parameter estimation in state space models. The estimation of states and parameters is an important task when mathematical modeling is applied to many different application areas such as the global positioning systems, target tracking, navigation, brain imaging, spread of infectious diseases, biological processes, telecommunications, audio signal processing, stochastic optimal control, machine learning, and physical systems. In Bayesian settings, the estimation of states or parameters amounts to computation of the posterior probability density function. Except for a very restricted number of models, it is impossible to compute this density function in a closed form. Hence, we need approximation methods. A state estimation problem involves estimating the states (latent variables) that are not directly observed in the output of the system. In this thesis, we use the Kalman filter, extended Kalman filter, Gauss–Hermite filters, and particle filters to estimate the states based on available measurements. Among these filters, particle filters are numerical methods for approximating the filtering distributions of non-linear non-Gaussian state space models via Monte Carlo. The performance of a particle filter heavily depends on the chosen importance distribution. For instance, inappropriate choice of the importance distribution can lead to the failure of convergence of the particle filter algorithm. In this thesis, we analyze the theoretical Lᵖ particle filter convergence with general importance distributions, where p ≥2 is an integer. A parameter estimation problem is considered with inferring the model parameters from measurements. For high-dimensional complex models, estimation of parameters can be done by Markov chain Monte Carlo (MCMC) methods. In its operation, the MCMC method requires the unnormalized posterior distribution of the parameters and a proposal distribution. In this thesis, we show how the posterior density function of the parameters of a state space model can be computed by filtering based methods, where the states are integrated out. This type of computation is then applied to estimate parameters of stochastic differential equations. Furthermore, we compute the partial derivatives of the log-posterior density function and use the hybrid Monte Carlo and scaled conjugate gradient methods to infer the parameters of stochastic differential equations. The computational efficiency of MCMC methods is highly depend on the chosen proposal distribution. A commonly used proposal distribution is Gaussian. In this kind of proposal, the covariance matrix must be well tuned. To tune it, adaptive MCMC methods can be used. In this thesis, we propose a new way of updating the covariance matrix using the variational Bayesian adaptive Kalman filter algorithm.

Semantic and fuzzy modelling for human behaviour recognition in smart spaces : a case study on ambient assisted living.

Human activity recognition in everyday environments is a critical, but challenging task in Ambient Intelligence applications to achieve proper Ambient Assisted Living, and key challenges still remain to be dealt with to realize robust methods. One of the major limitations of the Ambient Intelligence systems today is the lack of semantic models of those activities on the environment, so that the system can recognize the speci c activity being performed by the user(s) and act accordingly. In this context, this thesis addresses the general problem of knowledge representation in Smart Spaces. The main objective is to develop knowledge-based models, equipped with semantics to learn, infer and monitor human behaviours in Smart Spaces. Moreover, it is easy to recognize that some aspects of this problem have a high degree of uncertainty, and therefore, the developed models must be equipped with mechanisms to manage this type of information. A fuzzy ontology and a semantic hybrid system are presented to allow modelling and recognition of a set of complex real-life scenarios where vagueness and uncertainty are inherent to the human nature of the users that perform it. The handling of uncertain, incomplete and vague data (i.e., missing sensor readings and activity execution variations, since human behaviour is non-deterministic) is approached for the rst time through a fuzzy ontology validated on real-time settings within a hybrid data-driven and knowledgebased architecture. The semantics of activities, sub-activities and real-time object interaction are taken into consideration. The proposed framework consists of two main modules: the low-level sub-activity recognizer and the high-level activity recognizer. The rst module detects sub-activities (i.e., actions or basic activities) that take input data directly from a depth sensor (Kinect). The main contribution of this thesis tackles the second component of the hybrid system, which lays on top of the previous one, in a superior level of abstraction, and acquires the input data from the rst module's output, and executes ontological inference to provide users, activities and their in uence in the environment, with semantics. This component is thus knowledge-based, and a fuzzy ontology was designed to model the high-level activities. Since activity recognition requires context-awareness and the ability to discriminate among activities in di erent environments, the semantic framework allows for modelling common-sense knowledge in the form of a rule-based system that supports expressions close to natural language in the form of fuzzy linguistic labels. The framework advantages have been evaluated with a challenging and new public dataset, CAD-120, achieving an accuracy of 90.1% and 91.1% respectively for low and high-level activities. This entails an improvement over both, entirely data-driven approaches, and merely ontology-based approaches. As an added value, for the system to be su ciently simple and exible to be managed by non-expert users, and thus, facilitate the transfer of research to industry, a development framework composed by a programming toolbox, a hybrid crisp and fuzzy architecture, and graphical models to represent and con gure human behaviour in Smart Spaces, were developed in order to provide the framework with more usability in the nal application. As a result, human behaviour recognition can help assisting people with special needs such as in healthcare, independent elderly living, in remote rehabilitation monitoring, industrial process guideline control, and many other cases. This thesis shows use cases in these areas.

A Bayesian Approach for Forecasting Heat Load in a District Heating System

The growing population in cities increases the energy demand and affects the environment by increasing carbon emissions. Information and communications technology solutions which enable energy optimization are needed to address this growing energy demand in cities and to reduce carbon emissions. District heating systems optimize the energy production by reusing waste energy with combined heat and power plants. Forecasting the heat load demand in residential buildings assists in optimizing energy production and consumption in a district heating system. However, the presence of a large number of factors such as weather forecast, district heating operational parameters and user behavioural parameters, make heat load forecasting a challenging task. This thesis proposes a probabilistic machine learning model using a Naive Bayes classifier, to forecast the hourly heat load demand for three residential buildings in the city of Skellefteå, Sweden over a period of winter and spring seasons. The district heating data collected from the sensors equipped at the residential buildings in Skellefteå, is utilized to build the Bayesian network to forecast the heat load demand for horizons of 1, 2, 3, 6 and 24 hours. The proposed model is validated by using four cases to study the influence of various parameters on the heat load forecast by carrying out trace driven analysis in Weka and GeNIe. Results show that current heat load consumption and outdoor temperature forecast are the two parameters with most influence on the heat load forecast. The proposed model achieves average accuracies of 81.23 % and 76.74 % for a forecast horizon of 1 hour in the three buildings for winter and spring seasons respectively. The model also achieves an average accuracy of 77.97 % for three buildings across both seasons for the forecast horizon of 1 hour by utilizing only 10 % of the training data. The results indicate that even a simple model like Naive Bayes classifier can forecast the heat load demand by utilizing less training data.

Influence of multi-phase phenomena on semibatch crystallization processes of aqueous solutions

Crystal properties, product quality and particle size are determined by the operating conditions in the crystallization process. Thus, in order to obtain desired end-products, the crystallization process should be effectively controlled based on reliable kinetic information, which can be provided by powerful analytical tools such as Raman spectrometry and thermal analysis. The present research work studied various crystallization processes such as reactive crystallization, precipitation with anti-solvent and evaporation crystallization. The goal of the work was to understand more comprehensively the fundamentals, phenomena and utilizations of crystallization, and establish proper methods to control particle size distribution, especially for three phase gas-liquid-solid crystallization systems. As a part of the solid-liquid equilibrium studies in this work, prediction of KCl solubility in a MgCl2-KCl-H2O system was studied theoretically. Additionally, a solubility prediction model by Pitzer thermodynamic model was investigated based on solubility measurements of potassium dihydrogen phosphate with the presence of non-electronic organic substances in aqueous solutions. The prediction model helps to extend literature data and offers an easy and economical way to choose solvent for anti-solvent precipitation. Using experimental and modern analytical methods, precipitation kinetics and mass transfer in reactive crystallization of magnesium carbonate hydrates with magnesium hydroxide slurry and CO2 gas were systematically investigated. The obtained results gave deeper insight into gas-liquid-solid interactions and the mechanisms of this heterogeneous crystallization process. The research approach developed can provide theoretical guidance and act as a useful reference to promote development of gas-liquid reactive crystallization. Gas-liquid mass transfer of absorption in the presence of solid particles in a stirred tank was investigated in order to gain understanding of how different-sized particles interact with gas bubbles. Based on obtained volumetric mass transfer coefficient values, it was found that the influence of the presence of small particles on gas-liquid mass transfer cannot be ignored since there are interactions between bubbles and particles. Raman spectrometry was successfully applied for liquid and solids analysis in semi-batch anti-solvent precipitation and evaporation crystallization. Real-time information such as supersaturation, formation of precipitates and identification of crystal polymorphs could be obtained by Raman spectrometry. The solubility prediction models, monitoring methods for precipitation and empirical model for absorption developed in this study together with the methodologies used gives valuable information for aspects of industrial crystallization. Furthermore, Raman analysis was seen to be a potential controlling method for various crystallization processes.

Design of automatic control system for the precipitation of bromelain from the extract of pineapple wastes

In this work, bromelain was recovered from ground pineapple stem and rind by means of precipitation with alcohol at low temperature. Bromelain is the name of a group of powerful protein-digesting, or proteolytic, enzymes that are particularly useful for reducing muscle and tissue inflammation and as a digestive aid. Temperature control is crucial to avoid irreversible protein denaturation and consequently to improve the quality of the enzyme recovered. The process was carried out alternatively in two fed-batch pilot tanks: a glass tank and a stainless steel tank. Aliquots containing 100 mL of pineapple aqueous extract were fed into the tank. Inside the jacketed tank, the protein was exposed to unsteady operating conditions during the addition of the precipitating agent (ethanol 99.5%) because the dilution ratio "aqueous extract to ethanol" and heat transfer area changed. The coolant flow rate was manipulated through a variable speed pump. Fine tuned conventional and adaptive PID controllers were on-line implemented using a fieldbus digital control system. The processing performance efficiency was enhanced and so was the quality (enzyme activity) of the product.