Average control of Markov decision processes with Feller transition probabilities and general action spaces

This paper studies the average control problem of discrete-time Markov Decision Processes (MDPs for short) with general state space, Feller transition probabilities, and possibly non-compact control constraint sets A(x). Two hypotheses are considered: either the cost function c is strictly unbounded or the multifunctions A(r)(x) = {a is an element of A(x) : c(x, a) <= r} are upper-semicontinuous and compact-valued for each real r. For these two cases we provide new results for the existence of a solution to the average-cost optimality equality and inequality using the vanishing discount approach. We also study the convergence of the policy iteration approach under these conditions. It should be pointed out that we do not make any assumptions regarding the convergence and the continuity of the limit function generated by the sequence of relative difference of the alpha-discounted value functions and the Poisson equations as often encountered in the literature. (C) 2012 Elsevier Inc. All rights reserved.

Inter-rater reliability of the evaluation of muscular chains associated with posture alterations in scoliosis

Background: In the Global postural re-education (GPR) evaluation, posture alterations are associated with anterior or posterior muscular chain impairments. Our goal was to assess the reliability of the GPR muscular chain evaluation. Methods: Design: Inter-rater reliability study. Fifty physical therapists (PTs) and two experts trained in GPR assessed the standing posture from photographs of five youths with idiopathic scoliosis using a posture analysis grid with 23 posture indices (PI). The PTs and experts indicated the muscular chain associated with posture alterations. The PTs were also divided into three groups according to their experience in GPR. Experts' results (after consensus) were used to verify agreement between PTs and experts for muscular chain and posture assessments. We used Kappa coefficients (K) and the percentage of agreement (%A) to assess inter-rater reliability and intra-class coefficients (ICC) for determining agreement between PTs and experts. Results: For the muscular chain evaluation, reliability was moderate to substantial for 12 PI for the PTs (% A: 56 to 82; K: 0.42 to 0.76) and perfect for 19 PI for the experts. For posture assessment, reliability was moderate to substantial for 12 PI for the PTs (% A > 60%; K: 0.42 to 0.75) and moderate to perfect for 18 PI for the experts (% A: 80 to 100; K: 0.55 to 1.00). The agreement between PTs and experts was good for most muscular chain evaluations (18 PI; ICC: 0.82 to 0.99) and PI (19 PI; ICC: 0.78 to 1.00). Conclusions: The GPR muscular chain evaluation has good reliability for most posture indices. GPR evaluation should help guide physical therapists in targeting affected muscles for treatment of abnormal posture patterns.

Finite-size corrections of the entanglement entropy of critical quantum chains

Using the density matrix renormalization group, we calculated the finite-size corrections of the entanglement alpha-Renyi entropy of a single interval for several critical quantum chains. We considered models with U(1) symmetry such as the spin-1/2 XXZ and spin-1 Fateev-Zamolodchikov models, as well as models with discrete symmetries such as the Ising, the Blume-Capel, and the three-state Potts models. These corrections contain physically relevant information. Their amplitudes, which depend on the value of a, are related to the dimensions of operators in the conformal field theory governing the long-distance correlations of the critical quantum chains. The obtained results together with earlier exact and numerical ones allow us to formulate some general conjectures about the operator responsible for the leading finite-size correction of the alpha-Renyi entropies. We conjecture that the exponent of the leading finite-size correction of the alpha-Renyi entropies is p(alpha) = 2X(epsilon)/alpha for alpha > 1 and p(1) = nu, where X-epsilon denotes the dimensions of the energy operator of the model and nu = 2 for all the models.

Entanglement of excited states in critical spin chains

Renyi and von Neumann entropies quantifying the amount of entanglement in ground states of critical spin chains are known to satisfy a universal law which is given by the conformal field theory (CFT) describing their scaling regime. This law can be generalized to excitations described by primary fields in CFT, as was done by Alcaraz et al in 2011 (see reference [1], of which this work is a completion). An alternative derivation is presented, together with numerical verifications of our results in different models belonging to the c = 1, 1/2 universality classes. Oscillations of the Renyi entropy in excited states are also discussed.

Multiple Island Chains in Primary Resonances.

We analyse the dynamics of a relativistic particle moving in a uniform magnetic field and perturbed by a stationary electrostatic wave. We show that a pulsed wave produces an infinite number of perturbing terms with the same winding number. The perturbation coupling alters the number of island chains as a function of the parameters of the wave. We also observe that the number of chains in is always even if the number of islands in each chain is odd.

Universal behavior of the Shannon mutual information of critical quantum chains

We consider the Shannon mutual information of subsystems of critical quantum chains in their ground states. Our results indicate a universal leading behavior for large subsystem sizes. Moreover, as happens with the entanglement entropy, its finite-size behavior yields the conformal anomaly c of the underlying conformal field theory governing the long-distance physics of the quantum chain. We study analytically a chain of coupled harmonic oscillators and numerically the Q-state Potts models (Q = 2, 3, and 4), the XXZ quantum chain, and the spin-1 Fateev-Zamolodchikov model. The Shannon mutual information is a quantity easily computed, and our results indicate that for relatively small lattice sizes, its finite-size behavior already detects the universality class of quantum critical behavior.

Reconocimiento de formas manuscritas con modelos ocultos de Markov

Programa de doctorado Tecnologías de las Telecomunicaciones

Modelización econométrica a través de cadenas de Markov de Monte Carlo (MCMC) en valoración contingente

Doctorado en Análisis Económico. Programa en Análisis Económico Aplicado

Parameterschätzung in zeitdiskreten ergodischen Markov-Prozessen am Beispiel des Cox-Ingersoll-Ross-Modells

In dieser Arbeit geht es um die Schätzung von Parametern in zeitdiskreten ergodischen Markov-Prozessen im allgemeinen und im CIR-Modell im besonderen. Beim CIR-Modell handelt es sich um eine stochastische Differentialgleichung, die von Cox, Ingersoll und Ross (1985) zur Beschreibung der Dynamik von Zinsraten vorgeschlagen wurde. Problemstellung ist die Schätzung der Parameter des Drift- und des Diffusionskoeffizienten aufgrund von äquidistanten diskreten Beobachtungen des CIR-Prozesses. Nach einer kurzen Einführung in das CIR-Modell verwenden wir die insbesondere von Bibby und Sørensen untersuchte Methode der Martingal-Schätzfunktionen und -Schätzgleichungen, um das Problem der Parameterschätzung in ergodischen Markov-Prozessen zunächst ganz allgemein zu untersuchen. Im Anschluss an Untersuchungen von Sørensen (1999) werden hinreichende Bedingungen (im Sinne von Regularitätsvoraussetzungen an die Schätzfunktion) für die Existenz, starke Konsistenz und asymptotische Normalität von Lösungen einer Martingal-Schätzgleichung angegeben. Angewandt auf den Spezialfall der Likelihood-Schätzung stellen diese Bedingungen zugleich lokal-asymptotische Normalität des Modells sicher. Ferner wird ein einfaches Kriterium für Godambe-Heyde-Optimalität von Schätzfunktionen angegeben und skizziert, wie dies in wichtigen Spezialfällen zur expliziten Konstruktion optimaler Schätzfunktionen verwendet werden kann. Die allgemeinen Resultate werden anschließend auf das diskretisierte CIR-Modell angewendet. Wir analysieren einige von Overbeck und Rydén (1997) vorgeschlagene Schätzer für den Drift- und den Diffusionskoeffizienten, welche als Lösungen quadratischer Martingal-Schätzfunktionen definiert sind, und berechnen das optimale Element in dieser Klasse. Abschließend verallgemeinern wir Ergebnisse von Overbeck und Rydén (1997), indem wir die Existenz einer stark konsistenten und asymptotisch normalen Lösung der Likelihood-Gleichung zeigen und lokal-asymptotische Normalität für das CIR-Modell ohne Einschränkungen an den Parameterraum beweisen.

Pseudodifferential analysis in Psi*-algebras on transmission spaces, infinite solving ideal chains and K-theory for conformally compact spaces

The present thesis is a contribution to the theory of algebras of pseudodifferential operators on singular settings. In particular, we focus on the $b$-calculus and the calculus on conformally compact spaces in the sense of Mazzeo and Melrose in connection with the notion of spectral invariant transmission operator algebras. We summarize results given by Gramsch et. al. on the construction of $Psi_0$-and $Psi*$-algebras and the corresponding scales of generalized Sobolev spaces using commutators of certain closed operators and derivations. In the case of a manifold with corners $Z$ we construct a $Psi*$-completion $A_b(Z,{}^bOmega^{1/2})$ of the algebra of zero order $b$-pseudodifferential operators $Psi_{b,cl}(Z, {}^bOmega^{1/2})$ in the corresponding $C*$-closure $B(Z,{}^bOmega^{12})hookrightarrow L(L^2(Z,{}^bOmega^{1/2}))$. The construction will also provide that localised to the (smooth) interior of Z the operators in the $A_b(Z, {}^bOmega^{1/2})$ can be represented as ordinary pseudodifferential operators. In connection with the notion of solvable $C*$-algebras - introduced by Dynin - we calculate the length of the $C*$-closure of $Psi_{b,cl}^0(F,{}^bOmega^{1/2},R^{E(F)})$ in $B(F,{}^bOmega^{1/2}),R^{E(F)})$ by localizing $B(Z, {}^bOmega^{1/2})$ along the boundary face $F$ using the (extended) indical familiy $I^B_{FZ}$. Moreover, we discuss how one can localise a certain solving ideal chain of $B(Z, {}^bOmega^{1/2})$ in neighbourhoods $U_p$ of arbitrary points $pin Z$. This localisation process will recover the singular structure of $U_p$; further, the induced length function $l_p$ is shown to be upper semi-continuous. We give construction methods for $Psi*$- and $C*$-algebras admitting only infinite long solving ideal chains. These algebras will first be realized as unconnected direct sums of (solvable) $C*$-algebras and then refined such that the resulting algebras have arcwise connected spaces of one dimensional representations. In addition, we recall the notion of transmission algebras on manifolds with corners $(Z_i)_{iin N}$ following an idea of Ali Mehmeti, Gramsch et. al. Thereby, we connect the underlying $C^infty$-function spaces using point evaluations in the smooth parts of the $Z_i$ and use generalized Laplacians to generate an appropriate scale of Sobolev spaces. Moreover, it is possible to associate generalized (solving) ideal chains to these algebras, such that to every $ninN$ there exists an ideal chain of length $n$ within the algebra. Finally, we discuss the $K$-theory for algebras of pseudodifferential operators on conformally compact manifolds $X$ and give an index theorem for these operators. In addition, we prove that the Dirac-operator associated to the metric of a conformally compact manifold $X$ is not a Fredholm operator.

Entanglement entropy in restricted integrable spin-chains

In questa tesi abbiamo presentato il calcolo dell’Entropia di Entanglement di un sistema quantistico unidimensionale integrabile la cui rappresentazione statistica é data dal modello RSOS, il cui punto critico é una realizzazione su reticolo di tutti i modelli conformi minimali. Sfruttando l’integrabilitá di questi modelli, abbiamo svolto il calcolo utilizzando la tecnica delle Corner Transfer Matrices (CTM). Il risultato ottenuto si discosta leggermente dalla previsione di J. Cardy e P. Calabrese ricavata utilizzando la teoria dei campi conformi descriventi il punto critico. Questa differenza é stata imputata alla non-unitarietá del modello studiato, in quanto la tecnica CTM studia il ground state, mentre la previsione di Cardy e Calabrese si focalizza sul vuoto conforme del modello: nel caso dei sistemi non-unitari questi due stati non coincidono, ma possono essere visti come eccitazioni l’uno dell’altro. Dato che l’Entanglement é un fenomeno genuinamente quantistico e il modello RSOS descrive un sistema statistico classico bidimensionale, abbiamo proposto una Hamiltoniana quantistica unidimensionale integrabile la cui rappresentazione statistica é data dal modello RSOS.