Bayesian Learning of Noisy Markov Decision Processes

We consider the inverse reinforcement learning problem, that is, the problem of learning from, and then predicting or mimicking a controller based on state/action data. We propose a statistical model for such data, derived from the structure of a Markov decision process. Adopting a Bayesian approach to inference, we show how latent variables of the model can be estimated, and how predictions about actions can be made, in a unified framework. A new Markov chain Monte Carlo (MCMC) sampler is devised for simulation from the posterior distribution. This step includes a parameter expansion step, which is shown to be essential for good convergence properties of the MCMC sampler. As an illustration, the method is applied to learning a human controller.

Bayesian Structured Prediction Using Gaussian Processes

We introduce a conceptually novel structured prediction model, GPstruct, which is kernelized, non-parametric and Bayesian, by design. We motivate the model with respect to existing approaches, among others, conditional random fields (CRFs), maximum margin Markov networks (M3N), and structured support vector machines (SVMstruct), which embody only a subset of its properties. We present an inference procedure based on Markov Chain Monte Carlo. The framework can be instantiated for a wide range of structured objects such as linear chains, trees, grids, and other general graphs. As a proof of concept, the model is benchmarked on several natural language processing tasks and a video gesture segmentation task involving a linear chain structure. We show prediction accuracies for GPstruct which are comparable to or exceeding those of CRFs and SVMstruct.

The search for the IFN-gamma receptor in fish: Functional and expression analysis of putative binding and signalling chains in rainbow trout Oncorhynchus mykiss

Interferons (IFNs), consisting of three major subfamilies, type I, type II (gamma) and type III (lambda) IFN, activate vertebrate antiviral defences once bound to their receptors. The three IFN subfamilies bind to different receptors, IFNAR1 and IFNAR2 for type I IFNs, IFN gamma R1 and IFN gamma R2 for type II IFN, and IL-28R1 and IL-10R2 for type III IFNs. In fish, although many types I and II IFN genes have been cloned, little is known about their receptors. In this report, two putative IFN-gamma receptor chains were identified and sequenced in rainbow trout (Oncorhynchus mykiss), and found to have many common characteristics with mammalian type II IFN receptor family members. The presented gene synteny analysis, phylogenetic tree analysis and ligand binding analysis all suggest that these molecules are the authentic IFN gamma Rs in fish. They are widely expressed in tissues, with IFN gamma R1 typically more highly expressed than IFN gamma R2. Using the trout RTG-2 cell line it was possible to show that the individual chains could be differentially modulated, with rIFN-gamma and rIL-1 beta down regulating IFN gamma R1 expression but up regulating IFN gamma R2 expression. Overexpression of the two receptor chains in RTG-2 cells revealed that the level of IFN gamma R2 transcript was crucial for responsiveness to rIFN-gamma, in terms of inducing gamma IP expression. Transfection experiments showed that the two putative receptors specifically bound to rIFN-gamma. These findings are discussed in the context of how the IFN gamma R may bind IFN-gamma in fish and the importance of the individual receptor chains to signal transduction. (c) 2009 Elsevier Ltd. All rights reserved.

Molecular cloning and characterization of common carp (Cyprinus carpio L.) TCR gamma and CD3 gamma/delta chains

Partial cDNA sequences of TCR gamma and CD3 gamma/delta were isolated from the thymus of common carp (Cyprinus carpio L.) by the method of suppression subtractive hybridization (SSH). Subsequently the full length cDNAs of carp TCR gamma and CD3 gamma/delta were obtained by means of 3' RACE and 5' RACE, respectively. The full length of carp TCR gamma chain is 1368 bp and encodes 326 amino acids including a signal peptide region of 19 amino acids and a transmembrane region of 23 amino acids at the C-terminal region from aa 291 to 313. The V region of carp TCR gamma contains 109 amino acids, the core motif FGXG in J segment was also found in carp TCR gamma. The C region of carp TCR gamma contains the characteristic CX6PX6WX45C motif. The CP region of carp TCR C gamma contains 37 amino acids. The full length of carp CD3 gamma/delta is 790 bp and encodes 175 amino acids including a signal peptide region of 17 amino acids and a transmembrane region of 23 amino acids from aa 93 to 115. Similar to other known CD3 gamma/delta s, four cysteine residues in the extracellular domain and an immunoreceptor tyrosine-based activation motif ITAM (YxxL/Ix6-8YxxL/I) in the intracellular domain are also included in carp CD3 gamma/delta. Differing from other known CD3 gamma/delta s, carp CD3 gamma/delta tacks the CXXCXE motif in the extracellular domain. RTPCR analysis demonstrated that the expression of TCR gamma gene was mainly in the thymus and gill of 6-month carp, but in 18-month carp, TCR gamma gene was detected in all the examined tissues. The expression of CD3 gamma/delta gene was detected in all examined tissues of 6 and 18-month carp; among them, the highest expression level was in the thymus of 6-month carp. In situ hybridization showed that CD3 gamma/delta-expressing cells were widely distributed in the head kidney, spleen and kidney of carp, whereas in the thymus, they were densely distributed in the lymphoid outer zone and scattered in the epithelioid inner zone. (c) 2007 Published by Etsevier Ltd.

Mass chains with passive interconnection: Complex iterative maps and scalability

This paper introduces the problem of passive control of a chain of N identical masses in which there is an identical passive connection between neighbouring masses and a similar connection to a movable point. The problem arises in the design of multi-storey buildings which are subjected to earthquake disturbances, but applies in other situations, for example vehicle platoons. The paper will study the scalar transfer functions from the disturbance to a given intermass displacement. It will be shown that these transfer functions can be conveniently represented in the form of complex iterative maps and that these maps provide a method to establish boundedness in N of the H ∞-norm of these transfer functions for certain choices of interconnection impedance. © 2013 IEEE.

DTADH and quantum critical phenomena caused by anisotropy and external magnetic field for spin-1/2 Heisenberg diamond chains

Using the transfer matrix renormalization group (TMRG) method, we study the connection between the first derivative of the thermal average of driving-term Hamiltonian (DTADH) and the trace of quantum critical behaviors at finite temperatures. Connecting with the exact diagonalization method, we give the phase diagrams and analyze the properties of each phase for both the ferromagnetic and anti-ferromagnetic frustrated J(3) anisotropy diamond chain models. The finite-temperature scaling behaviors near the critical regions are also investigated. Further, we show the critical behaviors driven by external magnetic field, analyze the formation of the 1/3 magnetic plateau and the influence of different interactions on those critical points for both the ferrimagnetic and anti-ferromagnetic distorted diamond chains.

Optical study of lateral carrier transfer in (In,Ga)As/GaAs quantum-dot chains

We have studied the lateral carrier transfer in a specially designed quantum dot chain structure by means of time-resolved photoluminescence (PL) and polarization PL. The PL decay time increases with temperature, following the T-1/2 law for the typical one-dimensional quantum system. The decay time depends strongly on the emission energy: it decreases as the photon energy increases. Moreover, a strong polarization anisotropy is observed. These results are attributed to the efficient lateral transfer of carriers along the chain direction. (c) 2008 American Institute of Physics.

Comparative study of optical properties between quantum dot chains band quantum dots

A comparative study of the steady-state and transient optical properties was made between InGaAs/GaAs quantum do chains (QDCs) and quantum dots (QDs). It was found that the photoluminescence (PL) decay time of QDCs exhibited a strong photon energy dependence, while it was less sensitive in QDs. The PL decay time increased much faster with the excitation power in the QDCs than that in QDs. When the excitation power was large enough, the PL decay time tended to be saturated. In addition, it was also found that the PL rise time was much shorter in QDCs than in QDs. All these experimental results show that there is a strong carrier coupling along the chain direction in the QD chain structure. The polarization PL measurements further confirm the carrier transfer process along the chain direction.

Optical properties of InGaAs/GaAs quantum chains

We have investigated the steady-state and transient optical properties of InGaAs/GaAs quantum chains and found that the photoluminescence (PL) decay time exhibits a strong photon energy dependence. It increases with the decrease of the emission energy. It is also found that the PL decay time increases with the excitation power. When the excitation power is large enough the PL decay time tends to be saturated. All these experimental results show that there is a strong carrier coupling along the chain direction in the quantum dot chain structure. The polarization PL measurements further confirm the carrier transfer process along the chain direction.

Optical and structural properties of self-assembled ZnO QD chains by L-MBE

ZnO complex 3D nano-structures have been self-organized on Al2O3 (0 0 0 1) substrate by laser molecular beam epitaxy (L-MBE). It is shown by AFM morphology that the structure is composed of ID quantum dot chains (QDCs) and larger nano-islands at the nodes of QDCs. The formation mechanism of the nano-structure is also investigated. XRD results indicate that the nano-structure is highly c-axis oriented, with the aligned in-plane oriented domains. Time-integrated photoluminescence (TIPL) of the sample shows obvious blue-shift and broadening of the near band-edge (NBE) emission at room temperature, which are related to the quantum confinement effects. Time-resolved PL (TRPL) result shows bi-exponential decay behavior of ZnO QDCs, with a fast decay time of 38.21 ps and a low decay time of 138.19ps, respectively, which is considered to be originated from the interdot coupling made by coherent emission and reabsorption of the photons in QDCs. (C) 2007 Elsevier B.V. All rights reserved.