Statistical and harmonic analysis of gravity /

Project no. MO-011.

Gaussian process approximations of stochastic differential equations

Stochastic differential equations arise naturally in a range of contexts, from financial to environmental modeling. Current solution methods are limited in their representation of the posterior process in the presence of data. In this work, we present a novel Gaussian process approximation to the posterior measure over paths for a general class of stochastic differential equations in the presence of observations. The method is applied to two simple problems: the Ornstein-Uhlenbeck process, of which the exact solution is known and can be compared to, and the double-well system, for which standard approaches such as the ensemble Kalman smoother fail to provide a satisfactory result. Experiments show that our variational approximation is viable and that the results are very promising as the variational approximate solution outperforms standard Gaussian process regression for non-Gaussian Markov processes.

Research of Influence of Medication Preparations on the Process of Renewal of the Broken Equilibrium of Man Organism on the Doctor of Phytotherapy Workstation

In the given work by authors new approach to the exposure of degree of influencing of medications of vegetable origin in a time of renewal of broken equilibrium of man organism is offered. During realization of the given approach it is suggested to use the mathematical vehicle of.

Построение Логико-Вероятностных Моделей Временных Рядов с Использованием Цепей Маркова

The method of logic and probabilistic models constructing for multivariate heterogeneous time series is offered. There are some important properties of these models, e.g. universality. In this paper also discussed the logic and probabilistic models distinctive features in comparison with hidden Markov processes. The early proposed time series forecasting algorithm is tested on applied task.

O paradoxo da superdifusão de uma caminhada aleatória com memória exponencial

The random walk models with temporal correlation (i.e. memory) are of interest in the study of anomalous diffusion phenomena. The random walk and its generalizations are of prominent place in the characterization of various physical, chemical and biological phenomena. The temporal correlation is an essential feature in anomalous diffusion models. These temporal long-range correlation models can be called non-Markovian models, otherwise, the short-range time correlation counterparts are Markovian ones. Within this context, we reviewed the existing models with temporal correlation, i.e. entire memory, the elephant walk model, or partial memory, alzheimer walk model and walk model with a gaussian memory with profile. It is noticed that these models shows superdiffusion with a Hurst exponent H > 1/2. We study in this work a superdiffusive random walk model with exponentially decaying memory. This seems to be a self-contradictory statement, since it is well known that random walks with exponentially decaying temporal correlations can be approximated arbitrarily well by Markov processes and that central limit theorems prohibit superdiffusion for Markovian walks with finite variance of step sizes. The solution to the apparent paradox is that the model is genuinely non-Markovian, due to a time-dependent decay constant associated with the exponential behavior. In the end, we discuss ideas for future investigations.

Construcción y evaluación de un portafolio estructural, para obligatorias retiro programado, utilizando la metodología de frontera eficiente de Markowitz y la Teoría de Momentum

Este proyecto de investigación construye y evalúa la asignación de activos para el portafolio de Pensión Obligatoria de Retiro Programado, el cual atiende los retiros a los que un pensionado tiene derecho a través de su mesada pensional, utilizando el modelo de Frontera Eficiente de Markowitz, en combinación con la teoría de Momentum -- Para la ejecución del modelo se determinaron los activos de inversión admisibles en el Régimen de Inversiones -- Posteriormente, se construyen las matrices de rentabilidades, de restricciones, de varianzas y covarianzas, las cuales constituyen los insumos para ejecutar el modelo de optimización de portafolios de Markowitz -- A continuación, se realiza la selección de los portafolios obtenidos, teniendo en cuenta el nivel de volatilidad que el portafolio de Obligatorias Retiro Programado debe presentar; lo anterior, con el fin de cumplir con el objetivo de preservación del capital en la cuenta individual del pensionado, de manera que se pueda atender, de acuerdo a su esperanza de vida y la de sus beneficiarios, el pago de las mesadas pensionales que le correspondan -- El resultado obtenido corresponde a una asignación, en gran parte, en activos de Renta Fija expedidos por el Gobierno Nacional (TES), tanto en tasa fija como en tasa indexada a la UVR -- Adicionalmente, el modelo de optimización sugiere participaciones en activos de renta variable y, particularmente, no asigna recursos representativos en títulos de deuda privada indexados al IPC -- Esta investigación puede ser útil al momento de diseñar un portafolio base para Obligatorias Retiro Programado que, bajo una administración pasiva, permita cumplir el objetivo de otorgar a los pensionados una mesada para satisfacer las necesidades básicas

Risk-sensitive optimal control for Markov decision processes with monotone cost

The existence of an optimal feedback law is established for the risk-sensitive optimal control problem with denumerable state space. The main assumptions imposed are irreducibility and a near monotonicity condition on the one-step cost function. A solution can be found constructively using either value iteration or policy iteration under suitable conditions on initial feedback law.

Bounded parameter Markov Decision Processes with average reward criterion

Bounded parameter Markov Decision Processes (BMDPs) address the issue of dealing with uncertainty in the parameters of a Markov Decision Process (MDP). Unlike the case of an MDP, the notion of an optimal policy for a BMDP is not entirely straightforward. We consider two notions of optimality based on optimistic and pessimistic criteria. These have been analyzed for discounted BMDPs. Here we provide results for average reward BMDPs. We establish a fundamental relationship between the discounted and the average reward problems, prove the existence of Blackwell optimal policies and, for both notions of optimality, derive algorithms that converge to the optimal value function.

Wind-energy based path planning for electric unmanned aerial vehicles using Markov Decision Processes

Exploiting wind-energy is one possible way to ex- tend flight duration for Unmanned Arial Vehicles. Wind-energy can also be used to minimise energy consumption for a planned path. In this paper, we consider uncertain time-varying wind fields and plan a path through them. A Gaussian distribution is used to determine uncertainty in the Time-varying wind fields. We use Markov Decision Process to plan a path based upon the uncertainty of Gaussian distribution. Simulation results that compare the direct line of flight between start and target point and our planned path for energy consumption and time of travel are presented. The result is a robust path using the most visited cell while sampling the Gaussian distribution of the wind field in each cell.

Extending persistent monitoring by combining ocean models and Markov decision processes

Ocean processes are complex and have high variability in both time and space. Thus, ocean scientists must collect data over long time periods to obtain a synoptic view of ocean processes and resolve their spatiotemporal variability. One way to perform these persistent observations is to utilise an autonomous vehicle that can remain on deployment for long time periods. However, such vehicles are generally underactuated and slow moving. A challenge for persistent monitoring with these vehicles is dealing with currents while executing a prescribed path or mission. Here we present a path planning method for persistent monitoring that exploits ocean currents to increase navigational accuracy and reduce energy consumption.

Wind-energy based path planning for unmanned aerial vehicles using Markov decision processes

Exploiting wind-energy is one possible way to extend flight duration for Unmanned Arial Vehicles. Wind-energy can also be used to minimise energy consumption for a planned path. In this paper, we consider uncertain time-varying wind fields and plan a path through them. A Gaussian distribution is used to determine uncertainty in the Time-varying wind fields. We use Markov Decision Process to plan a path based upon the uncertainty of Gaussian distribution. Simulation results that compare the direct line of flight between start and target point and our planned path for energy consumption and time of travel are presented. The result is a robust path using the most visited cell while sampling the Gaussian distribution of the wind field in each cell.

Online hidden Markov model parameter estimation and minimax robust quickest change detection in uncertain stochastic processes

Stochastic (or random) processes are inherent to numerous fields of human endeavour including engineering, science, and business and finance. This thesis presents multiple novel methods for quickly detecting and estimating uncertainties in several important classes of stochastic processes. The significance of these novel methods is demonstrated by employing them to detect aircraft manoeuvres in video signals in the important application of autonomous mid-air collision avoidance.

Asymptotic behaviour of point processes with Markov-dependent intervals

This splitting techniques for MARKOV chains developed by NUMMELIN (1978a) and ATHREYA and NEY (1978b) are used to derive an imbedded renewal process in WOLD's point process with MARKOV-correlated intervals. This leads to a simple proof of renewal theorems for such processes. In particular, a key renewal theorem is proved, from which analogues to both BLACKWELL's and BREIMAN's forms of the renewal theorem can be deduced.

A reinforcement learning based algorithm for finite horizon Markov decision processes

We develop a simulation based algorithm for finite horizon Markov decision processes with finite state and finite action space. Illustrative numerical experiments with the proposed algorithm are shown for problems in flow control of communication networks and capacity switching in semiconductor fabrication.

An actor-critic algorithm with function approximation for discounted cost constrained Markov decision processes

We develop in this article the first actor-critic reinforcement learning algorithm with function approximation for a problem of control under multiple inequality constraints. We consider the infinite horizon discounted cost framework in which both the objective and the constraint functions are suitable expected policy-dependent discounted sums of certain sample path functions. We apply the Lagrange multiplier method to handle the inequality constraints. Our algorithm makes use of multi-timescale stochastic approximation and incorporates a temporal difference (TD) critic and an actor that makes a gradient search in the space of policy parameters using efficient simultaneous perturbation stochastic approximation (SPSA) gradient estimates. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal policy. (C) 2010 Elsevier B.V. All rights reserved.