Hierarchical Mixtures of Experts and the EM Algorithm

We present a tree-structured architecture for supervised learning. The statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models (GLIM's). Learning is treated as a maximum likelihood problem; in particular, we present an Expectation-Maximization (EM) algorithm for adjusting the parameters of the architecture. We also develop an on-line learning algorithm in which the parameters are updated incrementally. Comparative simulation results are presented in the robot dynamics domain.

Using the EM Algorithm to Train Neural Networks: Misconceptions and a New Algorithm for Multiclass Classification

The expectation-maximization (EM) algorithm has been of considerable interest in recent years as the basis for various algorithms in application areas of neural networks such as pattern recognition. However, there exists some misconceptions concerning its application to neural networks. In this paper, we clarify these misconceptions and consider how the EM algorithm can be adopted to train multilayer perceptron (MLP) and mixture of experts (ME) networks in applications to multiclass classification. We identify some situations where the application of the EM algorithm to train MLP networks may be of limited value and discuss some ways of handling the difficulties. For ME networks, it is reported in the literature that networks trained by the EM algorithm using iteratively reweighted least squares (IRLS) algorithm in the inner loop of the M-step, often performed poorly in multiclass classification. However, we found that the convergence of the IRLS algorithm is stable and that the log likelihood is monotonic increasing when a learning rate smaller than one is adopted. Also, we propose the use of an expectation-conditional maximization (ECM) algorithm to train ME networks. Its performance is demonstrated to be superior to the IRLS algorithm on some simulated and real data sets.

An Incremental EM-based Learning Approach for On-Line Prediction of Hospital Resource Utilization

Objective: Inpatient length of stay (LOS) is an important measure of hospital activity, health care resource consumption, and patient acuity. This research work aims at developing an incremental expectation maximization (EM) based learning approach on mixture of experts (ME) system for on-line prediction of LOS. The use of a batchmode learning process in most existing artificial neural networks to predict LOS is unrealistic, as the data become available over time and their pattern change dynamically. In contrast, an on-line process is capable of providing an output whenever a new datum becomes available. This on-the-spot information is therefore more useful and practical for making decisions, especially when one deals with a tremendous amount of data. Methods and material: The proposed approach is illustrated using a real example of gastroenteritis LOS data. The data set was extracted from a retrospective cohort study on all infants born in 1995-1997 and their subsequent admissions for gastroenteritis. The total number of admissions in this data set was n = 692. Linked hospitalization records of the cohort were retrieved retrospectively to derive the outcome measure, patient demographics, and associated co-morbidities information. A comparative study of the incremental learning and the batch-mode learning algorithms is considered. The performances of the learning algorithms are compared based on the mean absolute difference (MAD) between the predictions and the actual LOS, and the proportion of predictions with MAD < 1 day (Prop(MAD < 1)). The significance of the comparison is assessed through a regression analysis. Results: The incremental learning algorithm provides better on-line prediction of LOS when the system has gained sufficient training from more examples (MAD = 1.77 days and Prop(MAD < 1) = 54.3%), compared to that using the batch-mode learning. The regression analysis indicates a significant decrease of MAD (p-value = 0.063) and a significant (p-value = 0.044) increase of Prop(MAD

Bayesian nonlinear regression models with scale mixtures of skew-normal distributions: Estimation and case influence diagnostics

The purpose of this paper is to develop a Bayesian analysis for nonlinear regression models under scale mixtures of skew-normal distributions. This novel class of models provides a useful generalization of the symmetrical nonlinear regression models since the error distributions cover both skewness and heavy-tailed distributions such as the skew-t, skew-slash and the skew-contaminated normal distributions. The main advantage of these class of distributions is that they have a nice hierarchical representation that allows the implementation of Markov chain Monte Carlo (MCMC) methods to simulate samples from the joint posterior distribution. In order to examine the robust aspects of this flexible class, against outlying and influential observations, we present a Bayesian case deletion influence diagnostics based on the Kullback-Leibler divergence. Further, some discussions on the model selection criteria are given. The newly developed procedures are illustrated considering two simulations study, and a real data previously analyzed under normal and skew-normal nonlinear regression models. (C) 2010 Elsevier B.V. All rights reserved.

Multivariate measurement error models based on scale mixtures of the skew-normal distribution

Scale mixtures of the skew-normal (SMSN) distribution is a class of asymmetric thick-tailed distributions that includes the skew-normal (SN) distribution as a special case. The main advantage of these classes of distributions is that they are easy to simulate and have a nice hierarchical representation facilitating easy implementation of the expectation-maximization algorithm for the maximum-likelihood estimation. In this paper, we assume an SMSN distribution for the unobserved value of the covariates and a symmetric scale mixtures of the normal distribution for the error term of the model. This provides a robust alternative to parameter estimation in multivariate measurement error models. Specific distributions examined include univariate and multivariate versions of the SN, skew-t, skew-slash and skew-contaminated normal distributions. The results and methods are applied to a real data set.

Use of Eutectic Mixtures for Preparation of Monolithic Carbons with CO2-Adsorption and Gas-Separation Capabilities

With global warming becoming one of the main problems our society is facing nowadays, there is an urgent demand to develop materials suitable for CO2 storage as well as for gas separation. Within this context, hierarchical porous structures are of great interest for in-flow applications because of the desirable combination of an extensive internal reactive surface along narrow nanopores with facile molecular transport through broad “highways” leading to and from these pores. Deep eutectic solvents (DESs) have been recently used in the synthesis of carbon monoliths exhibiting a bicontinuous porous structure composed of continuous macroporous channels and a continuous carbon network that contains a certain microporosity and provides considerable surface area. In this work, we have prepared two DESs for the preparation of two hierarchical carbon monoliths with different compositions (e.g., either nitrogen-doped or not) and structure. It is worth noting that DESs played a capital role in the synthesis of hierarchical carbon monoliths not only promoting the spinodal decomposition that governs the formation of the bicontinuous porous structure but also providing the precursors required to tailor the composition and the molecular sieve structure of the resulting carbons. We have studied the performance of these two carbons for CO2, N2, and CH4 adsorption in both monolithic and powdered form. We have also studied the selective adsorption of CO2 versus CH4 in equilibrium and dynamic conditions. We found that these materials combined a high CO2-sorption capacity besides an excellent CO2/N2 and CO2/CH4 selectivity and, interestingly, this performance was preserved when processed in both monolithic and powdered form.

Electrochemical Performance of Hierarchical Porous Carbon Materials Obtained from the Infiltration of Lignin into Zeolite Templates

Hierarchical porous carbon materials prepared by the direct carbonization of lignin/zeolite mixtures and the subsequent basic etching of the inorganic template have been electrochemically characterized in acidic media. These lignin-based templated carbons have interesting surface chemistry features, such as a variety of surface oxygen groups and also pyridone and pyridinic groups, which results in a high capacitance enhancement compared to petroleum-pitch-based carbons obtained by the same procedure. Furthermore, they are easily electro-oxidized in a sulfuric acid electrolyte under positive polarization to produce a large amount of surface oxygen groups that boosts the pseudocapacitance. The lignin-based templated carbons showed a specific capacitance as high as 250 F g−1 at 50 mA g−1, with a capacitance retention of 50 % and volumetric capacitance of 75 F cm−3 at current densities higher than 20 A g−1 thanks to their suitable porous texture. These results indicate the potential use of inexpensive biomass byproducts, such as lignin, as carbon precursors in the production of hierarchical carbon materials for electrodes in electrochemical capacitors.

Bayesian Learning with Dependency Structures via Latent Factors, Mixtures, and Copulas

Bayesian methods offer a flexible and convenient probabilistic learning framework to extract interpretable knowledge from complex and structured data. Such methods can characterize dependencies among multiple levels of hidden variables and share statistical strength across heterogeneous sources. In the first part of this dissertation, we develop two dependent variational inference methods for full posterior approximation in non-conjugate Bayesian models through hierarchical mixture- and copula-based variational proposals, respectively. The proposed methods move beyond the widely used factorized approximation to the posterior and provide generic applicability to a broad class of probabilistic models with minimal model-specific derivations. In the second part of this dissertation, we design probabilistic graphical models to accommodate multimodal data, describe dynamical behaviors and account for task heterogeneity. In particular, the sparse latent factor model is able to reveal common low-dimensional structures from high-dimensional data. We demonstrate the effectiveness of the proposed statistical learning methods on both synthetic and real-world data.

The Crystal-t Algorithm: A New Approach To Calculate The Sle Of Lipidic Mixtures Presenting Solid Solutions.

Lipidic mixtures present a particular phase change profile highly affected by their unique crystalline structure. However, classical solid-liquid equilibrium (SLE) thermodynamic modeling approaches, which assume the solid phase to be a pure component, sometimes fail in the correct description of the phase behavior. In addition, their inability increases with the complexity of the system. To overcome some of these problems, this study describes a new procedure to depict the SLE of fatty binary mixtures presenting solid solutions, namely the Crystal-T algorithm. Considering the non-ideality of both liquid and solid phases, this algorithm is aimed at the determination of the temperature in which the first and last crystal of the mixture melts. The evaluation is focused on experimental data measured and reported in this work for systems composed of triacylglycerols and fatty alcohols. The liquidus and solidus lines of the SLE phase diagrams were described by using excess Gibbs energy based equations, and the group contribution UNIFAC model for the calculation of the activity coefficients of both liquid and solid phases. Very low deviations of theoretical and experimental data evidenced the strength of the algorithm, contributing to the enlargement of the scope of the SLE modeling.

Unusual 1,6-diphenyl-1,3,5-hexatriene (DPH) spectrophotometric behavior in water/ethanol and water/DMSO mixtures

The absorption spectra of DPH at fixed concentration do not change with water content in organic solvents. It exhibits monomer bands, such as those obtained in ethanol. The absorption did not change for solutions up to 54 and 46% of water in ethanol and DMSO, respectively, for [DPH] = 5.0 × 10-6 mol L-1 at 30 °C. However, at the same experimental conditions, a gradual sharp decay of the DPH fluorescence is observed. It is proposed that water molecules below these water concentration limits act as quenchers of the excited states of DPH. Stern-Volmer quenching constants by intensities measurements are 7.4 × 10-2 (water/ethanol) and 2.6 × 10-2 L mol-1 (water/DMSO). DPH lifetime measurements in the absence and presence of water resulted in 7.1 × 10-2 L mol-1 in water/ethanol, which pointed out that the process is a dynamic quenching by water molecules. For experiments using DPH as probe, this process can affect data, leading to misunderstanding interpretation.

Localized modes of binary mixtures of Bose-Einstein condensates in nonlinear optical lattices

The properties of the localized states of a two-component Bose-Einstein condensate confined in a nonlinear periodic potential (nonlinear optical lattice) are investigated. We discuss the existence of different types of solitons and study their stability by means of analytical and numerical approaches. The symmetry properties of the localized states with respect to nonlinear optical lattices are also investigated. We show that nonlinear optical lattices allow the existence of bright soliton modes with equal symmetry in both components and bright localized modes of mixed symmetry type, as well as dark-bright bound states and bright modes on periodic backgrounds. In spite of the quasi-one-dimensional nature of the problem, the fundamental symmetric localized modes undergo a delocalizing transition when the strength of the nonlinear optical lattice is varied. This transition is associated with the existence of an unstable solution, which exhibits a shrinking (decaying) behavior for slightly overcritical (undercritical) variations in the number of atoms.

Statistical models of mixtures with a biaxial nematic phase

We consider a simple Maier-Saupe statistical model with the inclusion of disorder degrees of freedom to mimic the phase diagram of a mixture of rodlike and disklike molecules. A quenched distribution of shapes leads to a phase diagram with two uniaxial and a biaxial nematic structure. A thermalized distribution, however, which is more adequate to liquid mixtures, precludes the stability of this biaxial phase. We then use a two-temperature formalism, and assume a separation of relaxation times, to show that a partial degree of annealing is already sufficient to stabilize a biaxial nematic structure.

Thermo-solvatochromism in binary mixtures of water and ionic liquids: on the relative importance of solvophobic interactions

The thermo-solvatochromism of 2,6-dibromo-4-[(E)-2-(1-methylpyridinium-4-yl)ethenyl] phenolate, MePMBr(2), has been studied in mixtures of water, W, with ionic liquids, ILs, in the temperature range of 10 to 60 degrees C, where feasible. The objectives of the study were to test the applicability of a recently introduced solvation model, and to assess the relative importance of solute-solvent solvophobic interactions. The ILs were 1-allyl-3-alkylimidazolium chlorides, where the alkyl groups are methyl, 1-butyl, and 1-hexyl, respectively. The equilibrium constants for the interaction of W and the ILs were calculated from density data; they were found to be linearly dependent on N(C), the number of carbon atoms of the alkyl group; van't Hoff equation (log K versus 1/T) applied satisfactorily. Plots of the empirical solvent polarities, E(T) (MePMBr(2)) in kcal mol(-1), versus the mole fraction of water in the binary mixture, chi(w), showed non-linear, i.e., non-ideal behavior. The dependence of E(T) (MePMBr(2)) on chi(w), has been conveniently quantified in terms of solvation by W, IL, and the ""complex"" solvent IL-W. The non-ideal behavior is due to preferential solvation by the IL and, more efficiently, by IL-W. The deviation from linearity increases as a function of increasing N(C) of the IL, and is stronger than that observed for solvation of MePMBr(2) by aqueous 1-propanol, a solvent whose lipophilicity is 12.8 to 52.1 times larger than those of the ILs investigated. The dependence on N(C) is attributed to solute-solvent solvophobic interactions, whose relative contribution to solvation are presumably greater than that in mixtures of water and 1-propanol.

The Effect of the Mixtures of Photoinitiators in Polymerization Efficiencies

The photoinitiated polymerization of methyl methacrylate using the mixtures of camphorquinone (CQ) and acylphosphine oxides (monoacylphosphine oxide, MAPO or bisacylphosphine oxide,BAPO) was studied to determine the possible synergistic effects. The addition of the acylphosphines to CQ resulted in an increase of the polymerization rate compared with CQ alone. On the other hand, a significant decrease of the polymerization quantum yield is observed for the mixtures compared with the pure acylphosphines. Therefore, the increase in the polymerization efficiency of the two rnixtures studied, MAPO/CQ and BAPO/CQ (compared with CQ) can be traced to the larger light absorption range, rather than to the onset of new mechanisms. The presence of the coinitiator ethyl 4-dimethylaminobenzoate, EDB, always present in CQ formulations, has no effect at all on the rates of polymerization photoinitiated by the acylphosphine oxides. From the point of view of photopolymerization quantum yields, an antagonistic effect is observed because Of the energy transfer of the M more efficient initiator (MAPO or BAPO) to the less efficient one (CQ). (C) 2008 Wiley Periodicals, Inc. j Appl Polym Sci 112: 129-134, 2009

Functionality of the approach of hierarchical analysis in the full cost accounting in the IRP of a metropolitan airport

This work shows the application of the analytic hierarchy process (AHP) in the full cost accounting (FCA) within the integrated resource planning (IRP) process. For this purpose, a pioneer case was developed and different energy solutions of supply and demand for a metropolitan airport (Congonhas) were considered [Moreira, E.M., 2005. Modelamento energetico para o desenvolvimento limpo de aeroporto metropolitano baseado na filosofia do PIR-O caso da metropole de Sao Paulo. Dissertacao de mestrado, GEPEA/USP]. These solutions were compared and analyzed utilizing the software solution ""Decision Lens"" that implements the AHP. The final part of this work has a classification of resources that can be considered to be the initial target as energy resources, thus facilitating the restraints of the IRP of the airport and setting parameters aiming at sustainable development. (C) 2007 Elsevier Ltd. All rights reserved.