Bayesian training of mixture density networks

Mixture Density Networks (MDNs) are a well-established method for modelling the conditional probability density which is useful for complex multi-valued functions where regression methods (such as MLPs) fail. In this paper we extend earlier research of a regularisation method for a special case of MDNs to the general case using evidence based regularisation and we show how the Hessian of the MDN error function can be evaluated using R-propagation. The method is tested on two data sets and compared with early stopping.

Regularisation of mixture density networks

Mixture Density Networks are a principled method to model conditional probability density functions which are non-Gaussian. This is achieved by modelling the conditional distribution for each pattern with a Gaussian Mixture Model for which the parameters are generated by a neural network. This thesis presents a novel method to introduce regularisation in this context for the special case where the mean and variance of the spherical Gaussian Kernels in the mixtures are fixed to predetermined values. Guidelines for how these parameters can be initialised are given, and it is shown how to apply the evidence framework to mixture density networks to achieve regularisation. This also provides an objective stopping criteria that can replace the `early stopping' methods that have previously been used. If the neural network used is an RBF network with fixed centres this opens up new opportunities for improved initialisation of the network weights, which are exploited to start training relatively close to the optimum. The new method is demonstrated on two data sets. The first is a simple synthetic data set while the second is a real life data set, namely satellite scatterometer data used to infer the wind speed and wind direction near the ocean surface. For both data sets the regularisation method performs well in comparison with earlier published results. Ideas on how the constraint on the kernels may be relaxed to allow fully adaptable kernels are presented.

Regularisation of mixture density networks

Mixture Density Networks are a principled method to model conditional probability density functions which are non-Gaussian. This is achieved by modelling the conditional distribution for each pattern with a Gaussian Mixture Model for which the parameters are generated by a neural network. This thesis presents a novel method to introduce regularisation in this context for the special case where the mean and variance of the spherical Gaussian Kernels in the mixtures are fixed to predetermined values. Guidelines for how these parameters can be initialised are given, and it is shown how to apply the evidence framework to mixture density networks to achieve regularisation. This also provides an objective stopping criteria that can replace the `early stopping' methods that have previously been used. If the neural network used is an RBF network with fixed centres this opens up new opportunities for improved initialisation of the network weights, which are exploited to start training relatively close to the optimum. The new method is demonstrated on two data sets. The first is a simple synthetic data set while the second is a real life data set, namely satellite scatterometer data used to infer the wind speed and wind direction near the ocean surface. For both data sets the regularisation method performs well in comparison with earlier published results. Ideas on how the constraint on the kernels may be relaxed to allow fully adaptable kernels are presented.

Mixture density networks

Minimization of a sum-of-squares or cross-entropy error function leads to network outputs which approximate the conditional averages of the target data, conditioned on the input vector. For classifications problems, with a suitably chosen target coding scheme, these averages represent the posterior probabilities of class membership, and so can be regarded as optimal. For problems involving the prediction of continuous variables, however, the conditional averages provide only a very limited description of the properties of the target variables. This is particularly true for problems in which the mapping to be learned is multi-valued, as often arises in the solution of inverse problems, since the average of several correct target values is not necessarily itself a correct value. In order to obtain a complete description of the data, for the purposes of predicting the outputs corresponding to new input vectors, we must model the conditional probability distribution of the target data, again conditioned on the input vector. In this paper we introduce a new class of network models obtained by combining a conventional neural network with a mixture density model. The complete system is called a Mixture Density Network, and can in principle represent arbitrary conditional probability distributions in the same way that a conventional neural network can represent arbitrary functions. We demonstrate the effectiveness of Mixture Density Networks using both a toy problem and a problem involving robot inverse kinematics.

Flow patterns and performance of distillation trays

It is known that distillation tray efficiency depends on the liquid flow pattern, particularly for large diameter trays. Scale·up failures due to liquid channelling have occurred, and it is known that fitting flow control devices to trays sometirr.es improves tray efficiency. Several theoretical models which explain these observations have been published. Further progress in understanding is at present blocked by lack of experimental measurements of the pattern of liquid concentration over the tray. Flow pattern effects are expected to be significant only on commercial size trays of a large diameter and the lack of data is a result of the costs, risks and difficulty of making these measurements on full scale production columns. This work presents a new experiment which simulates distillation by water cooling. and provides a means of testing commercial size trays in the laboratory. Hot water is fed on to the tray and cooled by air forced through the perforations. The analogy between heat and mass transfer shows that the water temperature at any point is analogous to liquid concentration and the enthalpy of the air is analogous to vapour concentration. The effect of the liquid flow pattern on mass transfer is revealed by the temperature field on the tray. The experiment was implemented and evaluated in a column of 1.2 m. dia. The water temperatures were measured by thennocouples interfaced to an electronic computerised data logging system. The "best surface" through the experimental temperature measurements was obtained by the mathematical technique of B. splines, and presented in tenos of lines of constant temperature. The results revealed that in general liquid channelling is more imponant in the bubbly "mixed" regime than in the spray regime. However, it was observed that severe channelling also occurred for intense spray at incipient flood conditions. This is an unexpected result. A computer program was written to calculate point efficiency as well as tray efficiency, and the results were compared with distillation efficiencies for similar loadings. The theoretical model of Porter and Lockett for predicting distillation was modified to predict water cooling and the theoretical predictions were shown to be similar to the experimental temperature profiles. A comparison of the repeatability of the experiments with an errors analysis revealed that accurate tray efficiency measurements require temperature measurements to better than ± 0.1 °c which is achievable with conventional techniques. This was not achieved in this work, and resulted in considerable scatter in the efficiency results. Nevertheless it is concluded that the new experiment is a valuable tool for investigating the effect of the liquid flow pattern on tray mass transfer.

Measurement and correlation of the solubility of telmisartan (form A) in nine different solvents from 277.85 to 338.35 K

The solubility of telmisartan (form A) in nine organic solvents (chloroform, dichloromethane, ethanol, toluene, benzene, 2-propanol, ethyl acetate, methanol and acetone) was determined by a laser monitoring technique at temperatures from 277.85 to 338.35 K. The solubility of telmisartan (form A) in all of the nine solvents increased with temperature as did the rates at which the solubility increased except in chloroform and dichloromethane. The mole fraction solubility in chloroform is higher than that in dichloromethane, which are both one order of magnitude higher than those in the other seven solvents at the experimental temperatures. The solubility data were correlated with the modified Apelblat equation and λh equations. The results show that the λh equation is in better agreement with the experimental data than the Apelblat equation. The relative root mean square deviations (σ) of the λh equation are in the range from 0.004 to 0.45 %. The dissolution enthalpies, entropies and Gibbs energies of telmisartan in these solvents were estimated by the Van’t Hoff equation and the Gibbs equation. The melting point and the fusion enthalpy of telmisartan were determined by differential scanning calorimetry.

Influence of alkyl chain length on sulfated zirconia catalysed batch and continuous esterification of carboxylic acids by light alcohols

The impact of alkyl chain length on the esterification of C2–C16 organic acids with C1–C4 alcohols has been systematically investigated over bulk and SBA-15 supported sulfated zirconias (SZs). Rates of catalytic esterification for methanol with acetic acid are directly proportional to the sulfur content for both SZ and SZ/SBA-15, with the high dispersion of SZ achievable in conformal coatings over mesoporous SBA-15 confering significant rate-enhancements. Esterification over the most active 0.24 mmol gcat−1 bulk SZ and 0.29 mmol gcat−1 SZ/SBA-15 materials was inversely proportional to the alkyl chain length of alcohol and acid reactants; being most sensitive to changes from methanol to ethanol and acetic to hexanoic acids respectively. Kinetic analyses reveal that these alkyl chain dependencies are in excellent accord with the Taft relationship for polar and steric effects in aliphatic systems and the enthalpy of alcohol adsorption, implicating a Langmuir–Hinshelwood mechanism. The first continuous production of methyl propionate over a SZ fixed-bed is also demonstrated.

Encapsulation of phase change materials using rice-husk-char

This paper explored a new approach to prepare phase change microcapsules using carbon-based particles via Pickering emulsions for energy storage applications. Rice-husk-char, a by-product in biofuel production, containing 53.58 wt% of carbon was used as a model carbon-based material to encapsulate hexadecane. As a model phase change material, hexadecane was emulsified in aqueous suspensions of rice-husk-char nanoparticles. Water soluble polymers poly(diallyldimethyl-ammonium chloride) and poly(sodium styrene sulfonate) were used to fix the rice-husk-char nanoparticles on the emulsion droplets through layer-by-layer assembly to enhance the structural stability of the microcapsules. The microcapsules formed are composed of a thin shell encompassing a large core consisting of hexadecane. Thermal gravimetrical and differential scanning calorimeter analyses showed the phase change enthalpy of 80.9 kJ kg−1 or 120.0 MJ m−3. Design criteria of phase change microcapsules and preparation considerations were discussed in terms of desired applications. This work demonstrated possible utilisations of biomass-originated carbon-based material for thermal energy recovery and storage applications, which can be a new route of carbon capture and utilisation.

High-sensitivity differential scanning calorimetry for measurement of steroid entrapment in nebulised liposomes generated from proliposomes

A novel approach to the determination of steroid entrapment in the bilayers of aerosolised liposomes has been introduced using high-sensitivity differential scanning calorimetry (DSC). Proliposomes were dispersed in water within an air-jet nebuliser and the energy produced during atomisation was used to hydrate the proliposomes and generate liposome aerosols. Proliposomes that included the steroid beclometasone dipropionate (BDP) produced lower aerosol and lipid outputs than steroid-free proliposomes. Size analysis and transmission electron microscopy showed an evidence of liposome formation within the nebuliser, which was followed by deaggregation and size reduction of multilamellar liposomes on nebulisation to a two-stage impinger. For each formulation, no difference in thermal transitions was observed between delivered liposomes and those remaining in the nebuliser. However, steroid (5 mole%) lowered the onset temperature and the enthalpy of the pretransition, and produced a similar onset temperature and larger enthalpy of the main transition, with broadened pretransition and main transitions. This indicates that BDP was entrapped and exhibited an interaction with the liposome phospholipid membranes. Since the pretransition was depressed but not completely removed and no phase separation occurred, it is suggested that the bilayers of the multilamellar liposomes can entrap more than 5 mole% BDP. Overall, liposomes were generated from proliposomes and DSC investigations indicated that the steroid was entrapped in the bilayers of aerosolised multilamellar vesicles.

Isomer resolution by micelle-assisted diffusion-ordered spectroscopy

Diffusion-ordered NMR spectroscopy ("DOSY") is a useful tool for the identification of mixture components. In its basic form it relies on simple differences in hydrodynamic radius to distinguish between different species. This can be very effective where species have significantly different molecular sizes, but generally fails for isomeric species. The use of surfactant co-solutes can allow isomeric species to be distinguished by virtue of their different degrees of interaction with micelles or reversed micelles. The use of micelle-assisted DOSY to resolve the NMR spectra of isomers is illustrated for the case of the three dihydroxybenzenes (catechol, resorcinol, and hydroquinone) in aqueous solution containing sodium dodecyl sulfate micelles, and in chloroform solution containing AOT reversed micelles. © 2009 American Chemical Society.

Humans assume a mixture of diffuse and point-source lighting when viewing sinusoidal shading patterns

Observers perceive sinusoidal shading patterns as being due to sinusoidally corrugated surfaces, and perceive surface peaks to be offset from luminance maxima by between zero and 1/4 wavelength. This offset varies with grating orientation. Physically, the shading profile of a sinusoidal surface will be approximately sinusoidal, with the same spatial frequency as the surface, only when: (A) it is lit suitably obliquely by a point source, or (B) the light source is diffuse and hemispherical--the 'dark is deep' rule applies. For A, surface peaks will be offset by 1/4 wavelength from the luminance maxima; for B, this offset will be zero. As the sum of two same-frequency sinusoids with different phases is a sinusoid of intermediate phase, our results suggest that observers assume a mixture of two light sources whose relative strength varies with grating orientation. The perceived surface offsets imply that gratings close to horizontal are taken to be lit by a point source; those close to vertical by a diffuse source. [Supported by EPSRC grants to AJS and MAG].

Sun and sky:does human vision assume a mixture of point and diffuse illumination when interpreting shape-from-shading?

People readily perceive smooth luminance variations as being due to the shading produced by undulations of a 3-D surface (shape-from-shading). In doing so, the visual system must simultaneously estimate the shape of the surface and the nature of the illumination. Remarkably, shape-from-shading operates even when both these properties are unknown and neither can be estimated directly from the image. In such circumstances humans are thought to adopt a default illumination model. A widely held view is that the default illuminant is a point source located above the observer's head. However, some have argued instead that the default illuminant is a diffuse source. We now present evidence that humans may adopt a flexible illumination model that includes both diffuse and point source elements. Our model estimates a direction for the point source and then weights the contribution of this source according to a bias function. For most people the preferred illuminant direction is overhead with a strong diffuse component.

Thin film bearings with phase change of lubricant

The possible evaporation of lubricant in fluid film bearings has been investigated theoretically and by experiment using a radial flow hydrostatic bearing supplied with liquid refrigerant R114. Good correlation between measured and theoretical values was obtained using a bespoke computational fluid dynamic model in which the flow was assumed to be laminar and adiabatic. The effects of viscous dissipation and vapour generation within the fluid film are fully accounted for by applying a fourth order Runge-Kutta routine to satisfy the radial and filmwise transverse constraints of momentum, energy and mass conservation. The results indicate that the radial velocity profile remains parabolic while the flow remains in the liquid phase and that the radial rate of enthalpy generation is then constant across the film at a given radius. The results also show that evaporation will commence at a radial location determined by geometry and flow conditions and in fluid layers adjacent to the solid boundaries. Evaporation is shown to progress in the radial direction and the load carrying capacity of such a bearing is reduced significantly. Expressions for the viscosity of the liquid/vapour mixture found in the literature survey have not been tested against experimental data. A new formulation is proposed in which the suitable choice of a characteristic constant yields close representation to any of these expressions. Operating constraints imposed by the design of the experimental apparatus limited the extent of the surface over which evaporation could be obtained, and prevented clear identification of the most suitable relationship for the viscosity of the liquid/vapour mixture. The theoretical model was extended to examine the development of two phase flow in a rotating shaft face seal of uniform thickness. Previous theoretical analyses have been based on the assumption that the radial velocity profile of the flow is always parabolic, and that the tangential component of velocity varies linearly from the value at the rotating surface, to zero at the stationary surface. The computational fluid dynamic analysis shows that viscous shear and dissipation in the fluid adjacent to the rotating surface leads to developing evaporation with a consequent reduction in tangential shear forces. The tangential velocity profile is predicted to decay rapidly through the film, exhibiting a profile entirely different to that assumed by previous investigators. Progressive evaporation takes place close to the moving wall and does not occur completely at a single radial location, as has been claimed in earlier work.

Novel Ni-Mg-Al-Ca catalyst for enhanced hydrogen production for the pyrolysis-gasification of a biomass/plastic mixture

A Ni-Mg-Al-Ca catalyst was prepared by a co-precipitation method for hydrogen production from polymeric materials. The prepared catalyst was designed for both the steam cracking of hydrocarbons and for the in situ absorption of CO2 via enhancement of the water-gas shift reaction. The influence of Ca content in the catalyst and catalyst calcination temperature in relation to the pyrolysis-gasification of a wood sawdust/polypropylene mixture was investigated. The highest hydrogen yield of 39.6molH2/g Ni with H2/CO ratio of 1.90 was obtained in the presence of the Ca containing catalyst of molar ratio Ni:Mg:Al:Ca=1:1:1:4, calcined at 500°C. In addition, thermogravimetric and morphology analyses of the reacted catalysts revealed that Ca introduction into the Ni-Mg-Al catalyst prevented the deposition of filamentous carbon on the catalyst surface. Furthermore, all metals were well dispersed in the catalyst after the pyrolysis-gasification process with 20-30nm of NiO sized particles observed after the gasification without significant aggregation.