Reinforcement Learning Based on the Bayesian Theorem for Electricity Markets Decision Support

This paper presents the applicability of a reinforcement learning algorithm based on the application of the Bayesian theorem of probability. The proposed reinforcement learning algorithm is an advantageous and indispensable tool for ALBidS (Adaptive Learning strategic Bidding System), a multi-agent system that has the purpose of providing decision support to electricity market negotiating players. ALBidS uses a set of different strategies for providing decision support to market players. These strategies are used accordingly to their probability of success for each different context. The approach proposed in this paper uses a Bayesian network for deciding the most probably successful action at each time, depending on past events. The performance of the proposed methodology is tested using electricity market simulations in MASCEM (Multi-Agent Simulator of Competitive Electricity Markets). MASCEM provides the means for simulating a real electricity market environment, based on real data from real electricity market operators.

Separating sequential equilibrium in a Bayesian international duopoly

We consider two Cournot firms, one located in the home country and the other in the foreign country, producing substitute goods for consumption in a third country. We suppose that neither the home government nor the foreign firm know the costs of the home firm, while the foreign firm cost is common knowledge. We determine the separating sequential equilibrium outputs.

Hierarchical Reinforcement Learning in Behavior and the Brain

Dissertation presented to obtain the Ph.D degree in Biology, Neuroscience

Spatio-temporal patterns of tuberculosis incidence in Ribeirão Preto, State of São Paulo, southeast Brazil, and their relationship with social vulnerability: a Bayesian analysis

INTRODUCTION: The purpose of this ecological study was to evaluate the urban spatial and temporal distribution of tuberculosis (TB) in Ribeirão Preto, State of São Paulo, southeast Brazil, between 2006 and 2009 and to evaluate its relationship with factors of social vulnerability such as income and education level. METHODS: We evaluated data from TBWeb, an electronic notification system for TB cases. Measures of social vulnerability were obtained from the SEADE Foundation, and information about the number of inhabitants, education and income of the households were obtained from Brazilian Institute of Geography and Statistics. Statistical analyses were conducted by a Bayesian regression model assuming a Poisson distribution for the observed new cases of TB in each area. A conditional autoregressive structure was used for the spatial covariance structure. RESULTS: The Bayesian model confirmed the spatial heterogeneity of TB distribution in Ribeirão Preto, identifying areas with elevated risk and the effects of social vulnerability on the disease. We demonstrated that the rate of TB was correlated with the measures of income, education and social vulnerability. However, we observed areas with low vulnerability and high education and income, but with high estimated TB rates. CONCLUSIONS: The study identified areas with different risks for TB, given that the public health system deals with the characteristics of each region individually and prioritizes those that present a higher propensity to risk of TB. Complex relationships may exist between TB incidence and a wide range of environmental and intrinsic factors, which need to be studied in future research.

A bayesian approach for NDT data fusion: The Saint Torcato Church case study

This paper presents a methodology based on the Bayesian data fusion techniques applied to non-destructive and destructive tests for the structural assessment of historical constructions. The aim of the methodology is to reduce the uncertainties of the parameter estimation. The Young's modulus of granite stones was chosen as an example for the present paper. The methodology considers several levels of uncertainty since the parameters of interest are considered random variables with random moments. A new concept of Trust Factor was introduced to affect the uncertainty related to each test results, translated by their standard deviation, depending on the higher or lower reliability of each test to predict a certain parameter.

Updating of the hierarchical rock mass rating (HRMR) system and a new subsystem developed for weathered granite formations

The RMR system is still very much applied in rock mechanics engineering context. It is based on the evaluation of six weights to obtain a final rating. To obtain the final rating a considerable amount of information is needed concerning the rock mass which can be difficult to obtain in some projects or project stages at least with accuracy. In 2007 an alternative classification scheme based on the RMR, the Hierarchical Rock Mass Rating (HRMR) was presented. The main feature of this system was the adaptation to the level of knowledge existent about the rock mass to obtain the classification of the rock mass since it followed a decision tree approach. However, the HRMR was only valid for hard rock granites with low fracturing degrees. In this work, the database was enlarged with approximately 40% more cases considering other types of granite rock masses including weathered granites and based on this increased database the system was updated. Granite formations existent in the north of Portugal including Porto city are predominantly granites. Some years ago a light rail infrastructure was built in the city of Porto and surrounding municipalities whi h involved considerable challenges due to the high heterogeneity levels of the granite formations and the difficulties involved in their geomechanical characterization. In this work it is intended to provide also a contribution to improve the characterization of these formations with special emphasis to the weathered horizons. A specific subsystem applicable to the weathered formations was developed. The results of the validation of these systems are presented and show acceptable performances in identifying the correct class using less information than with the RMR system.

Multiphasic, multistructured and hierarchical strategies for cartilage regeneration

Cartilage tissue is a complex nonlinear, viscoelastic, anisotropic, and multiphasic material with a very low coefficient of friction, which allows to withstand millions of cycles of joint loading over decades of wear. Upon damage, cartilage tissue has a low self-reparative capacity due to the lack of neural connections, vascularization, and a latent pool of stem/chondroprogenitor cells. Therefore, the healing of articular cartilage defects remains a significant clinical challenge, affecting millions of people worldwide. A plethora of biomaterials have been proposed to fabricate devices for cartilage regeneration, assuming a wide range of forms and structures, such as sponges, hydrogels, capsules, fibers, and microparticles. In common, the fabricated devices were designed taking in consideration that to fully achieve the regeneration of functional cartilage it is mandatory a well-orchestrated interplay of biomechanical properties, unique hierarchical structures, extracellular matrix (ECM), and bioactive factors. In fact, the main challenge in cartilage tissue engineering is to design an engineered device able to mimic the highly organized zonal architecture of articular cartilage, specifically its spatiomechanical properties and ECM composition, while inducing chondrogenesis, either by the proliferation of chondrocytes or by stimulating the chondrogenic differentiation  of stem/chondro-progenitor cells. In this chapter we present the recent advances in the development of innovative and complex biomaterials that fulfill the required structural key elements for cartilage regeneration. In particular, multiphasic, multiscale, multilayered, and hierarchical strategies composed by single or multiple biomaterials combined in a welldefined structure will be addressed. Those strategies include biomimetic scaffolds mimicking the structure of articular cartilage or engineered scaffolds as models of research to fully understand the biological mechanisms that influence the regeneration of cartilage tissue.

Nonlinear behavior of gelatin networks reveals a hierarchical structure

We investigate the strain hardening behavior of various gelatin networks-namely physical gelatin gel, chemically cross-linked gelatin gel, and a hybrid gel made of a combination of the former two-under large shear deformations using the pre-stress, strain ramp, and large amplitude oscillations shear protocols. Further, the internal structures of physical gelatin gels and chemically cross-linked gelatin gels were characterized by small angle neutron scattering (SANS) to enable their internal structures to be correlated with their nonlinear rheology. The Kratky plots of SANS data demonstrate the presence of small cross-linked aggregates within the chemically cross-linked network whereas, in the physical gelatin gels, a relatively homogeneous structure is observed. Through model fitting to the scattering data, we were able to obtain structural parameters, such as the correlation length (ξ), the cross-sectional polymer chain radius (Rc) and the fractal dimension (df) of the gel networks. The fractal dimension df obtained from the SANS data of the physical and chemically cross-linked gels is 1.31 and 1.53, respectively. These values are in excellent agreement with the ones obtained from a generalized nonlinear elastic theory that has been used to fit the stress-strain curves. The chemical cross-linking that generates coils and aggregates hinders the free stretching of the triple helix bundles in the physical gels.

Closed hybrid hierarchical reservoirs based on the deconstruction of the cellular native microenvironment

The stem cell niche organization and dynamics provide valuable cues for the development of mimetic environments that could have potential to stimulate the regenerative process. We propose the use of biodegradable biomaterials to produce closed miniaturised structures able to encapsulate different cell types or bioactive molecules. In particular, capsules are fabricated using the so-called layer-by-layer technology, where the consecutive (nano-sized) layers are well stabilized by electrostatic interactions or other weak forces. Using alginate-based spherical templates containing cells or other elements (e.g. proteins, magnetic nanoparticles, microparticles) it is possible to produce liquefied capsules that may entrap the entire cargo under mild conditions. The inclusion of liquefied micropcapsules may be used to produce hierarchical compartmentalised systems for the delivery of bioactive agents. The presence of solid microparticles inside such capsules offers adequate surface area for adherent cell attachment increasing the biological performance of these hierarchical systems, while maintain both permeability and injectability. We demonstrated that the encapsulation of distinct cell types (including mesenchymal stem cells and endothelial cells) enhances the osteogenic capability of this system, that could be useful in bone tissue engineering applications.

Superhydrophobic surfaces as a tool for the fabrication of hierarchical spherical polymeric carriers

Hierarchical polymeric carriers with high encapsulation efficiencies are fabricated via a biocompatible strategy developed using superhydrophobic (SH) surfaces. The carries are obtained by the incorporation of cell/BSA-loaded dextran-methacrylate (DEXT-MA) microparticles into alginate (ALG) macroscopic beads. Engineered devices like these are expected to boost the development of innovative and customizable systems for biomedical and biotechnological purposes.

Optimal designs for the prediction in hierarchical random coefficient regression models

Magdeburg, Univ., Fak. für Mathematik, Diss., 2015

Generalized factor models: a bayesian approach

There is recent interest in the generalization of classical factor models in which the idiosyncratic factors are assumed to be orthogonal and there are identification restrictions on cross-sectional and time dimensions. In this study, we describe and implement a Bayesian approach to generalized factor models. A flexible framework is developed to determine the variations attributed to common and idiosyncratic factors. We also propose a unique methodology to select the (generalized) factor model that best fits a given set of data. Applying the proposed methodology to the simulated data and the foreign exchange rate data, we provide a comparative analysis between the classical and generalized factor models. We find that when there is a shift from classical to generalized, there are significant changes in the estimates of the structures of the covariance and correlation matrices while there are less dramatic changes in the estimates of the factor loadings and the variation attributed to common factors.

Hydrogeophysical data integration at larger scales: Application of Bayesian sequential simulation for the characterization of heterogeneous alluvial aquifers

Knowledge of the spatial distribution of hydraulic conductivity (K) within an aquifer is critical for reliable predictions of solute transport and the development of effective groundwater management and/or remediation strategies. While core analyses and hydraulic logging can provide highly detailed information, such information is inherently localized around boreholes that tend to be sparsely distributed throughout the aquifer volume. Conversely, larger-scale hydraulic experiments like pumping and tracer tests provide relatively low-resolution estimates of K in the investigated subsurface region. As a result, traditional hydrogeological measurement techniques contain a gap in terms of spatial resolution and coverage, and they are often alone inadequate for characterizing heterogeneous aquifers. Geophysical methods have the potential to bridge this gap. The recent increased interest in the application of geophysical methods to hydrogeological problems is clearly evidenced by the formation and rapid growth of the domain of hydrogeophysics over the past decade (e.g., Rubin and Hubbard, 2005).