Reinforcement learning for parameter estimation in statistical spoken dialogue systems

Reinforcement techniques have been successfully used to maximise the expected cumulative reward of statistical dialogue systems. Typically, reinforcement learning is used to estimate the parameters of a dialogue policy which selects the system's responses based on the inferred dialogue state. However, the inference of the dialogue state itself depends on a dialogue model which describes the expected behaviour of a user when interacting with the system. Ideally the parameters of this dialogue model should be also optimised to maximise the expected cumulative reward. This article presents two novel reinforcement algorithms for learning the parameters of a dialogue model. First, the Natural Belief Critic algorithm is designed to optimise the model parameters while the policy is kept fixed. This algorithm is suitable, for example, in systems using a handcrafted policy, perhaps prescribed by other design considerations. Second, the Natural Actor and Belief Critic algorithm jointly optimises both the model and the policy parameters. The algorithms are evaluated on a statistical dialogue system modelled as a Partially Observable Markov Decision Process in a tourist information domain. The evaluation is performed with a user simulator and with real users. The experiments indicate that model parameters estimated to maximise the expected reward function provide improved performance compared to the baseline handcrafted parameters. © 2011 Elsevier Ltd. All rights reserved.

Robust multiplexed MPC for distributed multi-agent systems

This paper extends the authors' earlier work which adapted robust multiplexed MPC for application to distributed control of multi-agent systems with non-interacting dynamics and coupled constraint sets in the presence of persistent unknown, but bounded disturbances. Specifically, we propose exploiting the single agent update nature of the multiplexed approach, and fix the update sequence to enable input move-blocking and increased discretisation rates. This permits a higher rate of individual policy update to be achieved, whilst incurring no additional computational cost in the corresponding optimal control problems to be solved. A disturbance feedback policy is included between updates to facilitate finding feasible solutions. The new formulation inherits the property of rapid response to disturbances from multiplexing the control and numerical results show that fixing the update sequence does not incur any loss in performance. © 2011 IFAC.

Simultaneous reduction of particulate matter and NOx emissions using 4-way catalyzed filtration systems

The next generation of diesel emission control devices includes 4-way catalyzed filtration systems (4WCFS) consisting of both NOx and diesel particulate matter (DPM) control. A methodology was developed to simultaneously evaluate the NOx and DPM control performance of miniature 4WCFS made from acicular mullite, an advanced ceramic material (ACM), that were challenged with diesel exhaust. The impact of catalyst loading and substrate porosity on catalytic performance of the NOx trap was evaluated. Simultaneously with NOx measurements, the real-time solid particle filtration performance of catalyst-coated standard and high porosity filters was determined for steady-state and regenerative conditions. The use of high porosity ACM 4-way catalyzed filtration systems reduced NOx by 99% and solid and total particulate matter by 95% when averaged over 10 regeneration cycles. A "regeneration cycle" refers to an oxidizing ("lean") exhaust condition followed by a reducing ("rich") exhaust condition resulting in NOx storage and NOx reduction (i.e., trap "regeneration"), respectively. Standard porosity ACM 4-way catalyzed filtration systems reduced NOx by 60-75% and exhibited 99.9% filtration efficiency. The rich/lean cycling used to regenerate the filter had almost no impact on solid particle filtration efficiency but impacted NOx control. Cycling resulted in the formation of very low concentrations of semivolatile nucleation mode particles for some 4WCFS formulations. Overall, 4WCFS show promise for significantly reducing diesel emissions into the atmosphere in a single control device. © 2013 American Chemical Society.

An integrated 2-D model of a lithium ion battery: The effect of material parameters and morphology on storage particle stress

An integrated 2-D model of a lithium ion battery is developed to study the mechanical stress in storage particles as a function of material properties. A previously developed coupled stress-diffusion model for storage particles is implemented in 2-D and integrated into a complete battery system. The effect of morphology on the stress and lithium concentration is studied for the case of extraction of lithium in terms of previously developed non-dimensional parameters. These non-dimensional parameters include the material properties of the storage particles in the system, among other variables. We examine particles functioning in isolation as well as in closely-packed systems. Our results show that the particle distance from the separator, in combination with the material properties of the particle, is critical in predicting the stress generated within the particle. © 2012 Springer-Verlag.

Bayesian inference and learning in Gaussian process state-space models with Particle MCMC

State-space models are successfully used in many areas of science, engineering and economics to model time series and dynamical systems. We present a fully Bayesian approach to inference and learning (i.e. state estimation and system identification) in nonlinear nonparametric state-space models. We place a Gaussian process prior over the state transition dynamics, resulting in a flexible model able to capture complex dynamical phenomena. To enable efficient inference, we marginalize over the transition dynamics function and, instead, infer directly the joint smoothing distribution using specially tailored Particle Markov Chain Monte Carlo samplers. Once a sample from the smoothing distribution is computed, the state transition predictive distribution can be formulated analytically. Our approach preserves the full nonparametric expressivity of the model and can make use of sparse Gaussian processes to greatly reduce computational complexity.

Collective and single-particle excitations in coupled quantum wires in magnetic fields

The full spectra of magnetoplasmons and single-particle excitations are obtained of coupled one-dimensional electron gases in parallel semiconductor quantum wires with tunneling. We show the effects of the interwire Coulomb interaction and the tunneling, as well as the magnetic-field-induced localization on the elementary excitations in symmetric and asymmetric coulped quantum wire structures. The interacton and resonance between the plasmon and the intersubband single-particle excitations are found in magnetic fields.

Green's function of a two-dimensional interacting electron gas in a perpendicular magnetic field

It is rigorously proved that the Green's function of a uniform two-dimensional interacting electron gas in a perpendicular magnetic field is diagonal with respect to single-particle states in the Landau gauge. The implication of this theorem is briefly discussed.

Di-nuclear systems studied with the velocity filter SHIP

Various nuclear reactions like quasi-fission, fusion-fission or particle and cluster evaporation from excited compound nuclei were studied in heavy-ion reactions at the velocity filter SHIP of GSI. The velocity filter offers the possibility to detect all reaction products under zero degree relative to the beam direction. Together with the measurement of the product velocity distribution this allows for an identification of the underlying reaction mechanism. This article is focussed on reactions of Mg-25 and Ni-64 beams on Pb-206,Pb-207 targets at energies of 5.9 x A MeV and 8.7 x A MeV. Besides evaporation residues from Mg-25 + Pb-206 collisions we found evidence for rotation and quasi-fission of nuclear molecules formed in the entrance channel after the capture stage. The break-up of the systems showed a preferred clustering leading to isotopes in the region 84 <= Z <= 88 and 122 <= N <= 127 of the chart of nuclei.

Progress of XHV technology at particle accelerators

Recent vacuum system development with an XHV condition for the particle accelerators is briefly described. The progress of selecting and treatment of the materials used in XHV systems is introduced, and the choice of the main pump for an XHV system and some new pumping method are presented. Some leak detection experiences both for the superconducting and warm vacuum systems are recommended and the status of XHV measurement and the gauge calibration are introduced.

Effect of cavitations on brittle-ductile transition of particle toughened thermoplastics

In this study, we established a correlation between cavitations volume and the brittle-ductile transition (BDT) for particle toughened thermoplastics. The brittle-ductile transition temperature (T-BD) was calculated as a function of T* and interparticle distance (ED), respectively, where T* was a parameter related to the volume of cavitations. The results showed that the smaller the cavitations volume, the higher the brittle-ductile transition temperature. The calculations correlated well with the experimental data. With respect to rubber particle, the rigid particle was too hard to be voided during deformation, thereby the TED of the blend was much higher than that of rubber particle toughened thermoplastic. This was a main reason that rubber particle could toughen thermoplastics effectively, whereas rigid particle could not.

Preparation and application of a novel core-shell-particle-supported zirconocene catalyst

The use of functional groups bearing silica/poly(styrene-co-4-vinylpyridine) core-shell particles as a support for a zirconocene catalyst in ethylene polymerization was studied. Several factors affecting the behavior of the supported catalyst and the properties of the resulting polymer, such as time, temperature, Al/N (molar ratio), and Al/Zr (molar ratio), were examined. The conditions of the supported catalyst preparation were more important than those of the ethylene polymerization. The state of the supported catalyst itself played a decisive role in both the catalytic behavior of the supported catalyst and the properties of polyethylene (PE). IR and X-ray photoelectron spectroscopy were used to follow the formation of the supports. The formation of cationic active species is hypothesized, and the performance of the core-shell-particle-supported zirconocene catalyst is discussed as well. The bulk density of the PE formed was higher than that of the polymer obtained from homogeneous and polymer-supported Cp2ZrCl2/methylaluminoxane catalyst systems. (C) 2001 John Wiley & Sons, Inc.

Diminishing Returns of Engineering Effort in Telerobotic Systems

Wilson, M.S. and Neal, M.J., 'Diminishing Returns of Engineering Effort in Telerobotic Systems', IEEE Transactions on Systems, Man and Cybernetics - Part A:Systems and Humans, 2001, September, volume 31, number 5, pp 459-465, IEEE Robotics and Automation Society, ed. Dautenhahn,K., Special Issue on Socially Intelligent Agents - The Human in the Loop

Granularity and disaggregation in compositional modelling with applications to ecological systems

J. Keppens and Q. Shen. Granularity and disaggregation in compositional modelling with applications to ecological systems. Applied Intelligence, 25(3):269-292, 2006.

Correlations in low dimensional quantum systems

In this thesis I present the work done during my PhD in the area of low dimensional quantum gases. The chapters of this thesis are self contained and represent individual projects which have been peer reviewed and accepted for publication in respected international journals. Various systems are considered, the first of which is a two particle model which possesses an exact analytical solution. I investigate the non-classical correlations that exist between the particles as a function of the tunable properties of the system. In the second work I consider the coherences and out of equilibrium dynamics of a one-dimensional Tonks-Girardeau gas. I show how the coherence of the gas can be inferred from various properties of the reduced state and how this may be observed in experiments. I then present a model which can be used to probe a one-dimensional Fermi gas by performing a measurement on an impurity which interacts with the gas. I show how this system can be used to observe the so-called orthogonality catastrophe using modern interferometry techniques. In the next chapter I present a simple scheme to create superposition states of particles with special emphasis on the NOON state. I explore the effect of inter-particle interactions in the process and then characterise the usefulness of these states for interferometry. Finally I present my contribution to a project on long distance entanglement generation in ion chains. I show how carefully tuning the environment can create decoherence-free subspaces which allows one to create and preserve entanglement.