Average control of Markov decision processes with Feller transition probabilities and general action spaces

This paper studies the average control problem of discrete-time Markov Decision Processes (MDPs for short) with general state space, Feller transition probabilities, and possibly non-compact control constraint sets A(x). Two hypotheses are considered: either the cost function c is strictly unbounded or the multifunctions A(r)(x) = {a is an element of A(x) : c(x, a) <= r} are upper-semicontinuous and compact-valued for each real r. For these two cases we provide new results for the existence of a solution to the average-cost optimality equality and inequality using the vanishing discount approach. We also study the convergence of the policy iteration approach under these conditions. It should be pointed out that we do not make any assumptions regarding the convergence and the continuity of the limit function generated by the sequence of relative difference of the alpha-discounted value functions and the Poisson equations as often encountered in the literature. (C) 2012 Elsevier Inc. All rights reserved.

Adaptive estimation of HMM transition probabilities

This paper presents new schemes for recursive estimation of the state transition probabilities for hidden Markov models (HMM's) via extended least squares (ELS) and recursive state prediction error (RSPE) methods. Local convergence analysis for the proposed RSPE algorithm is shown using the ordinary differential equation (ODE) approach developed for the more familiar recursive output prediction error (RPE) methods. The presented scheme converges and is relatively well conditioned compared with the ...

Adaptive estimation of hidden semi-Markov chains with parameterised transition probabilities and exponential decaying states

This paper develops maximum likelihood (ML) estimation schemes for finite-state semi-Markov chains in white Gaussian noise. We assume that the semi-Markov chain is characterised by transition probabilities of known parametric from with unknown parameters. We reformulate this hidden semi-Markov model (HSM) problem in the scalar case as a two-vector homogeneous hidden Markov model (HMM) problem in which the state consist of the signal augmented by the time to last transition. With this reformulation we apply the expectation Maximumisation (EM ) algorithm to obtain ML estimates of the transition probabilities parameters, Markov state levels and noise variance. To demonstrate our proposed schemes, motivated by neuro-biological applications, we use a damped sinusoidal parameterised function for the transition probabilities.

1. Ultrasonic studies of binary liquid structure in the critical region. Theory and experiment for the 2,6-lutidine/water system. 2. Hartree-Fock calculations of electric polarizabilities of some simple atoms and molecules, and their practicality. 3. Calculation of vibrational transition probabilities in collinear atom-diatom and diatom-diatom collisions with Lennard-Jones interaction

Part 1. Many interesting visual and mechanical phenomena occur in the critical region of fluids, both for the gas-liquid and liquid-liquid transitions. The precise thermodynamic and transport behavior here has some broad consequences for the molecular theory of liquids. Previous studies in this laboratory on a liquid-liquid critical mixture via ultrasonics supported a basically classical analysis of fluid behavior by M. Fixman (e. g., the free energy is assumed analytic in intensive variables in the thermodynamics)--at least when the fluid is not too close to critical. A breakdown in classical concepts is evidenced close to critical, in some well-defined ways. We have studied herein a liquid-liquid critical system of complementary nature (possessing a lower critical mixing or consolute temperature) to all previous mixtures, to look for new qualitative critical behavior. We did not find such new behavior in the ultrasonic absorption ascribable to the critical fluctuations, but we did find extra absorption due to chemical processes (yet these are related to the mixing behavior generating the lower consolute point). We rederived, corrected, and extended Fixman's analysis to interpret our experimental results in these more complex circumstances. The entire account of theory and experiment is prefaced by an extensive introduction recounting the general status of liquid state theory. The introduction provides a context for our present work, and also points out problems deserving attention. Interest in these problems was stimulated by this work but also by work in Part 3.

Part 2. Among variational theories of electronic structure, the Hartree-Fock theory has proved particularly valuable for a practical understanding of such properties as chemical binding, electric multipole moments, and X-ray scattering intensity. It also provides the most tractable method of calculating first-order properties under external or internal one-electron perturbations, either developed explicitly in orders of perturbation theory or in the fully self-consistent method. The accuracy and consistency of first-order properties are poorer than those of zero-order properties, but this is most often due to the use of explicit approximations in solving the perturbed equations, or to inadequacy of the variational basis in size or composition. We have calculated the electric polarizabilities of H₂, He, Li, Be, LiH, and N₂ by Hartree-Fock theory, using exact perturbation theory or the fully self-consistent method, as dictated by convenience. By careful studies on total basis set composition, we obtained good approximations to limiting Hartree-Fock values of polarizabilities with bases of reasonable size. The values for all species, and for each direction in the molecular cases, are within 8% of experiment, or of best theoretical values in the absence of the former. Our results support the use of unadorned Hartree-Pock theory for static polarizabilities needed in interpreting electron-molecule scattering data, collision-induced light scattering experiments, and other phenomena involving experimentally inaccessible polarizabilities.

Part 3. Numerical integration of the close-coupled scattering equations has been carried out to obtain vibrational transition probabilities for some models of the electronically adiabatic H₂-H₂ collision. All the models use a Lennard-Jones interaction potential between nearest atoms in the collision partners. We have analyzed the results for some insight into the vibrational excitation process in its dependence on the energy of collision, the nature of the vibrational binding potential, and other factors. We conclude also that replacement of earlier, simpler models of the interaction potential by the Lennard-Jones form adds very little realism for all the complication it introduces. A brief introduction precedes the presentation of our work and places it in the context of attempts to understand the collisional activation process in chemical reactions as well as some other chemical dynamics.

MEASUREMENT AND CALCULATION OF SPONTANEOUS EMISSION BRANCHING RATIOS, RELATIVE OSCILLATOR-STRENGTHS, TRANSITION-PROBABILITIES AND LEVEL LIFETIMES FOR ATOMIC URANIUM

We propose a laser induced sensitized fluorescence spectrometry for measuring the spontaneous emission branching ratios o?the transitions from the ten levels 5f36d7s7p-7M7, 5f36d7s7p-7L6, 5f37s27p-5K6, 5f26d27s2 - 5L7, 5f46d7s - 7L6, (17,070cm-1)-5L6, 5f26d27s2-5K6, 6d7s7p-7L5, 5f36d7s7p-7K5 and 5f26d27s2-5I5 to the ground state of atomic uranium (UI) for the first time. Their relative oscillator strengths have been measured by means of hollow cathode discharge (HCD) emission spectrometry. The radiative...

Feller transition functions, Resolvent Decomposition Theorems, and their application in unstable denumerable Markov processes

This paper surveys the recent progresses made in the field of unstable denumerable Markov processes. Emphases are laid upon methodology and applications. The important tools of Feller transition functions and Resolvent Decomposition Theorems are highlighted. Their applications particularly in unstable denumerable Markov processes with a single instantaneous state and Markov branching processes are illustrated.

Energy levels and transition probabilities for nitrogen-like Fe xx

Energies of the 700 lowest levels in Fe XX have been obtained using the multiconfiguration Dirac-Fock method. Configuration interaction method on the basis set of transformed radial orbitals with variable parameters taking into account relativistic corrections in the Breit-Pauli approximation was used to crosscheck our presented results. Transition probabilities, oscillator and line strengths are presented for electric dipole (E1), electric quadrupole (E2) and magnetic dipole (M1) transitions among these levels. The total radiative transition probabilities from each level are also provided. Results are compared with data compiled by NIST and with other theoretical work.

Relativistic allowed and forbidden transition probabilities for fluorine-like Fe XVIII

Energy levels and the corresponding transition probabilities for allowed and forbidden transitions among the levels of the ground configuration and first 23 excited configurations of fluorine-like Fe XVIII have been calculated using the multiconfigurational Dirac-Fock GRASP code. A total of 379 lowest bound levels of Fe XVIII is presented, and the energy levels are identified in spectroscopic notations. Transition probabilities, oscillator strengths and line strengths for electric dipole (E1), electric quadrupole (E2) and magnetic dipole (M1) transitions among these 379 levels are also presented. The calculated energy levels and transition probabilities are compared with experimental data.

Dirac-Fock energy levels and transition probabilities for oxygen-like Fe XIX

Multiconfigurational Dirac-Fock calculations are reported for 656 energy levels and the 214 840 electric dipole (E I), electric quadrupole (E2) and magnetic dipole (M1) transition probabilities in oxygen-like Fe xix. The spectroscopic notations as well as the total transition probabilities from each energy level are provided. Good agreement is found with data compiled by NIST.

Electron impact excitation of Mg VIII: Collision strengths, transition probabilities and theoretical EUV and soft X-ray line intensities for Mg VIII

Context: Mg VIII emission lines are observed in a range of astronomical objects such as the Sun, other cool stars and in the coronal line region of Seyfert galaxies. Under coronal conditions Mg VIII emits strongly in the extreme ultraviolet (EUV) and soft X-ray spectral regions which makes it an ideal ion for plasma diagnostics.

Aims. Two theoretical atomic models, consisting of 125 fine structure levels, are developed for the Mg VIII ion. The 125 levels arise from the 2s(2)2p, 2s(2)p2, 2p(3), 2s(2)3s, 2s(2)3p, 2s(2)3d, 2s2p3s, 2s2p3p, 2s2p3d, 2p(2)3s, 2p(2)3p and 2p(2)3d configurations. Electron impact excitation collision strengths and radiative transition probabilities are calculated for both Mg VIII models, compared with existing data, and the best model selected to generate a set of theoretical emission line intensities. The EUV lines, covering 312-790 angstrom, are compared with existing solar spectra (SERTS-89 and SUMER), while the soft X-ray transitions (69-97 angstrom) are examined for potential density diagnostic line ratios and also compared with the limited available solar and stellar observational data.

Methods. The R-matrix codes Breit-Pauli RMATRXI and RMATRXII are utilised, along with the PSTGF code, to calculate the collision strengths for two Mg VIII models. Collision strengths are averaged over a Maxwellian distribution to produce the corresponding effective collision strengths for use in astrophysical applications. Transition probabilities are also calculated using the CIV3 atomic structure code. The best data are then incorporated into the modelling code CLOUDY and line intensities generated for a range of electron temperatures and densities appropriate to solar and stellar coronal plasmas.

Results. The present effective collision strengths are compared with two previous calculations. Good levels of agreement are found with the most recent, but there are large differences with the other for forbidden transitions. The resulting line intensities compare favourably with the observed values from the SERTS-89 and SUMER spectra. Theoretical soft X-ray emission lines are presented and several density diagnostic line ratios examined, which are in reasonable agreement with the limited observational data available.

Oscillator strengths and transition probabilities for the W xlv ion

In this paper we present oscillator strengths and transition probabilities for W xlv transitions between levels arising from configurations 3d¹⁰4s²,4p²,4d², 3d¹⁰4k4l (k = s,p,d,f and l = p,d,f), 3d⁹4s²4l (l = p,d,f) and 3d⁹4s4p². The model used to calculate these contained all configurations which can be constructed from the available orbitals (up to n = 4), with either a 3d¹⁰ or 3d⁹ core. The calculations were performed with the configuration interaction CIV3 program with the inclusion of relativistic effects achieved through the use of the Breit-Pauli approximation. We compare our ab initio energy levels, oscillator strengths and transition rates with other experimental and theoretical values available in the literature. There is generally good agreement when only levels with 3d¹⁰ cores are considered. The literature is sparse for levels in which the 3d-subshell is opened: for the majority of the fine-structure lines considered, there is either no comparison data available or substantial differences are found. This paper also investigates how the inclusion of relativistic effects can result in a significant redistribution of the oscillator strength from the LS calculations.

Effect of pair interactions on transition probabilities between inactive and active states: achieving collective behaviour via pair interactions in social insects

To understand the evolution of well-organized social behaviour, we must first understand the mechanism by which collective behaviour establishes. In this study, the mechanisms of collective behaviour in a colony of social insects were studied in terms of the transition probability between active and inactive states, which is linked to mutual interactions. The active and inactive states of the social insects were statistically extracted from the velocity profiles. From the duration distributions of the two states, we found that 1) the durations of active and inactive states follow an exponential law, and 2) pair interactions increase the transition probability from inactive to active states. The regulation of the transition probability by paired interactions suggests that such interactions control the populations of active and inactive workers in the colony.