33 resultados para Markov decision processes
em Consorci de Serveis Universitaris de Catalunya (CSUC), Spain
Resumo:
In this paper, we present a stochastic model for disability insurance contracts. The model is based on a discrete time non-homogeneous semi-Markov process (DTNHSMP) to which the backward recurrence time process is introduced. This permits a more exhaustive study of disability evolution and a more efficient approach to the duration problem. The use of semi-Markov reward processes facilitates the possibility of deriving equations of the prospective and retrospective mathematical reserves. The model is applied to a sample of contracts drawn at random from a mutual insurance company.
Resumo:
This paper investigates experimentally how organisational decision processes affect the moral motivations of actors inside a firm that must forego profits to reduce harming a third party. In a "vertical" treatment, one insider unilaterally sets the harm-reduction strategy; the other can only accept or quit. In a "horizontal" treatment, the insiders decide by consensus. Our 2-by-2 design also controls for communication effects. In our data, communication makes vertical firms more ethical; voice appears to mitigate "responsibility-alleviation" in that subordinates with voice feel responsible for what their firms do. Vertical firms are then more ethical than the horizontal firms for which our bargaining data reveal a dynamic form of responsibility-alleviation and our chat data indicate a strong "insider-outsider" effect.
Resumo:
We present a polyhedral framework for establishing general structural properties on optimal solutions of stochastic scheduling problems, where multiple job classes vie for service resources: the existence of an optimal priority policy in a given family, characterized by a greedoid (whose feasible class subsets may receive higher priority), where optimal priorities are determined by class-ranking indices, under restricted linear performance objectives (partial indexability). This framework extends that of Bertsimas and Niño-Mora (1996), which explained the optimality of priority-index policies under all linear objectives (general indexability). We show that, if performance measures satisfy partial conservation laws (with respect to the greedoid), which extend previous generalized conservation laws, then the problem admits a strong LP relaxation over a so-called extended greedoid polytope, which has strong structural and algorithmic properties. We present an adaptive-greedy algorithm (which extends Klimov's) taking as input the linear objective coefficients, which (1) determines whether the optimal LP solution is achievable by a policy in the given family; and (2) if so, computes a set of class-ranking indices that characterize optimal priority policies in the family. In the special case of project scheduling, we show that, under additional conditions, the optimal indices can be computed separately for each project (index decomposition). We further apply the framework to the important restless bandit model (two-action Markov decision chains), obtaining new index policies, that extend Whittle's (1988), and simple sufficient conditions for their validity. These results highlight the power of polyhedral methods (the so-called achievable region approach) in dynamic and stochastic optimization.
Resumo:
We present a polyhedral framework for establishing general structural properties on optimal solutions of stochastic scheduling problems, where multiple job classes vie for service resources: the existence of an optimal priority policy in a given family, characterized by a greedoid(whose feasible class subsets may receive higher priority), where optimal priorities are determined by class-ranking indices, under restricted linear performance objectives (partial indexability). This framework extends that of Bertsimas and Niño-Mora (1996), which explained the optimality of priority-index policies under all linear objectives (general indexability). We show that, if performance measures satisfy partial conservation laws (with respect to the greedoid), which extend previous generalized conservation laws, then theproblem admits a strong LP relaxation over a so-called extended greedoid polytope, which has strong structural and algorithmic properties. We present an adaptive-greedy algorithm (which extends Klimov's) taking as input the linear objective coefficients, which (1) determines whether the optimal LP solution is achievable by a policy in the given family; and (2) if so, computes a set of class-ranking indices that characterize optimal priority policies in the family. In the special case of project scheduling, we show that, under additional conditions, the optimal indices can be computed separately for each project (index decomposition). We further apply the framework to the important restless bandit model (two-action Markov decision chains), obtaining new index policies, that extend Whittle's (1988), and simple sufficient conditions for their validity. These results highlight the power of polyhedral methods (the so-called achievable region approach) in dynamic and stochastic optimization.
Resumo:
We show that if performance measures in a stochastic scheduling problem satisfy a set of so-called partial conservation laws (PCL), which extend previously studied generalized conservation laws (GCL), then the problem is solved optimally by a priority-index policy for an appropriate range of linear performance objectives, where the optimal indices are computed by a one-pass adaptive-greedy algorithm, based on Klimov's. We further apply this framework to investigate the indexability property of restless bandits introduced by Whittle, obtaining the following results: (1) we identify a class of restless bandits (PCL-indexable) which are indexable; membership in this class is tested through a single run of the adaptive-greedy algorithm, which also computes the Whittle indices when the test is positive; this provides a tractable sufficient condition for indexability; (2) we further indentify the class of GCL-indexable bandits, which includes classical bandits, having the property that they are indexable under any linear reward objective. The analysis is based on the so-called achievable region method, as the results follow fromnew linear programming formulations for the problems investigated.
Resumo:
We develop a mathematical programming approach for the classicalPSPACE - hard restless bandit problem in stochastic optimization.We introduce a hierarchy of n (where n is the number of bandits)increasingly stronger linear programming relaxations, the lastof which is exact and corresponds to the (exponential size)formulation of the problem as a Markov decision chain, while theother relaxations provide bounds and are efficiently computed. Wealso propose a priority-index heuristic scheduling policy fromthe solution to the first-order relaxation, where the indices aredefined in terms of optimal dual variables. In this way wepropose a policy and a suboptimality guarantee. We report resultsof computational experiments that suggest that the proposedheuristic policy is nearly optimal. Moreover, the second-orderrelaxation is found to provide strong bounds on the optimalvalue.
Resumo:
In this paper, we present a comprehensive study of different Independent Component Analysis (ICA) algorithms for the calculation of coherency and sharpness of electroencephalogram (EEG) signals, in order to investigate the possibility of early detection of Alzheimer’s disease (AD). We found that ICA algorithms can help in the artifact rejection and noise reduction, improving the discriminative property of features in high frequency bands (specially in high alpha and beta ranges). In addition to different ICA algorithms, the optimum number of selected components is investigated, in order to help decision processes for future works.
Resumo:
Stochastic processes defined by a general Langevin equation of motion where the noise is the non-Gaussian dichotomous Markov noise are studied. A non-FokkerPlanck master differential equation is deduced for the probability density of these processes. Two different models are exactly solved. In the second one, a nonequilibrium bimodal distribution induced by the noise is observed for a critical value of its correlation time. Critical slowing down does not appear in this point but in another one.
Resumo:
We address the question of how a third-party payer (e.g. an insurer) decides what providers to contract with. Three different mechanisms are studied and their properties compared. A first mechanism consists in the third-party payer setting up a bargaining procedure with both providers jointly and simultaneously. A second mechanism envisages the outcome of the same simultaneous bargaining but independently with every provider. Finally, the last mechanism is of different nature. It is the so-called "any willing provider" where the third-party payer announces a contract and every provider freely decides to sign it or not. The main finding is that the decision of the third-party payer depends on the surplus to be shared. When it is relatively high the third-party payer prefers the any willing provider system. When, on the contrary, the surplus is relatively low, the third-party payer will select one of the other two systems accor ing to how bargaining power is distributed.
Resumo:
Populations displaced as a result of mass violent conflict have become one of the most pressing humanitarian concerns of the last decades. They have also become one salient political issue as a perceived burden (in economic and security terms) and as an important piece in the shift towards a more interventionist paradigm in the international system, based on both humanitarian and security grounds. The saliency of these aspects has detracted attention from the analysis of the interactions between relocation processes and violent conflict. Violent conflict studies have also largely ignored those interactions as a result of the consideration of these processes as mere reaction movements determined by structural conditions. This article takes the view that individual’s agency is retained during such processes, and that it is consequential, calling for the need to introduce a micro perspective. Based on this, a model for the individual’s decision of return is presented. The model has the potential to account for the dynamics of return at both the individual and the aggregate level. And it further helps to grasp fundamental interconnections with violent conflict. Some relevant conclusions are derived for the case of Bosnia-Herzegovina and about the implications of the politicization of return.
Resumo:
The paper discusses maintenance challenges of organisations with a huge number of devices and proposes the use of probabilistic models to assist monitoring and maintenance planning. The proposal assumes connectivity of instruments to report relevant features for monitoring. Also, the existence of enough historical registers with diagnosed breakdowns is required to make probabilistic models reliable and useful for predictive maintenance strategies based on them. Regular Markov models based on estimated failure and repair rates are proposed to calculate the availability of the instruments and Dynamic Bayesian Networks are proposed to model cause-effect relationships to trigger predictive maintenance services based on the influence between observed features and previously documented diagnostics
Resumo:
An equation for mean first-passage times of non-Markovian processes driven by colored noise is derived through an appropriate backward integro-differential equation. The equation is solved in a Bourret-like approximation. In a weak-noise bistable situation, non-Markovian effects are taken into account by an effective diffusion coefficient. In this situation, our results compare satisfactorily with other approaches and experimental data.
Resumo:
We study steady-state correlation functions of nonlinear stochastic processes driven by external colored noise. We present a methodology that provides explicit expressions of correlation functions approximating simultaneously short- and long-time regimes. The non-Markov nature is reduced to an effective Markovian formulation, and the nonlinearities are treated systematically by means of double expansions in high and low frequencies. We also derive some exact expressions for the coefficients of these expansions for arbitrary noise by means of a generalization of projection-operator techniques.
Resumo:
First-passage time statistics for non-Markovian processes have heretofore only been developed for processes driven by dichotomous fluctuations that are themselves Markov. Herein we develop a new method applicable to Markov and non-Markovian dichotomous fluctuations and calculate analytic mean first-passage times for particular examples.
Resumo:
We present exact equations and expressions for the first-passage-time statistics of dynamical systems that are a combination of a diffusion process and a random external force modeled as dichotomous Markov noise. We prove that the mean first passage time for this system does not show any resonantlike behavior.