Efficient solutions to factored MDPs with imprecise transition probabilities

When modeling real-world decision-theoretic planning problems in the Markov Decision Process (MDP) framework, it is often impossible to obtain a completely accurate estimate of transition probabilities. For example, natural uncertainty arises in the transition specification due to elicitation of MOP transition models from an expert or estimation from data, or non-stationary transition distributions arising from insufficient state knowledge. In the interest of obtaining the most robust policy under transition uncertainty, the Markov Decision Process with Imprecise Transition Probabilities (MDP-IPs) has been introduced to model such scenarios. Unfortunately, while various solution algorithms exist for MDP-IPs, they often require external calls to optimization routines and thus can be extremely time-consuming in practice. To address this deficiency, we introduce the factored MDP-IP and propose efficient dynamic programming methods to exploit its structure. Noting that the key computational bottleneck in the solution of factored MDP-IPs is the need to repeatedly solve nonlinear constrained optimization problems, we show how to target approximation techniques to drastically reduce the computational overhead of the nonlinear solver while producing bounded, approximately optimal solutions. Our results show up to two orders of magnitude speedup in comparison to traditional ""flat"" dynamic programming approaches and up to an order of magnitude speedup over the extension of factored MDP approximate value iteration techniques to MDP-IPs while producing the lowest error of any approximation algorithm evaluated. (C) 2011 Elsevier B.V. All rights reserved.

Excited Electronic States, Transition Probabilities, and Radiative Lifetimes of CAs: A Theoretical Contribution and Challenge to Experimentalists

High-level CASSCF/MRCI calculations with a quintuple-zeta quality basis set are reported by characterizing for the first time a manifold of electronic states of the CAs radical yet to be investigated experimentally. Along with the potential energy curves and the associated spectroscopic constants, the dipole moment functions for selected electronic states as well as the transition dipole moment functions for the most relevant electronic transitions are also presented. Estimates of radiative transition probabilities and lifetimes complement this investigation, which also assesses the effect of spin-orbit interaction on the A (2)Pi state. Whenever pertinent, comparisons of similarities and differences with the isovalent CN and CP radicals are made.

Dynamic Fuzzy Logic Control of GeneticAlgorithm Probabilities

Genetic algorithms are commonly used to solve combinatorial optimizationproblems. The implementation evolves using genetic operators (crossover, mutation,selection, etc.). Anyway, genetic algorithms like some other methods have parameters(population size, probabilities of crossover and mutation) which need to be tune orchosen.In this paper, our project is based on an existing hybrid genetic algorithmworking on the multiprocessor scheduling problem. We propose a hybrid Fuzzy-Genetic Algorithm (FLGA) approach to solve the multiprocessor scheduling problem.The algorithm consists in adding a fuzzy logic controller to control and tunedynamically different parameters (probabilities of crossover and mutation), in anattempt to improve the algorithm performance. For this purpose, we will design afuzzy logic controller based on fuzzy rules to control the probabilities of crossoverand mutation. Compared with the Standard Genetic Algorithm (SGA), the resultsclearly demonstrate that the FLGA method performs significantly better.

On Qualitative Probabilities for Legal Reasoning about Evidence

A crucial aspect of evidential reasoning in crime investigation involves comparing the support that evidence provides for alternative hypotheses. Recent work in forensic statistics has shown how Bayesian Networks (BNs) can be employed for this purpose. However, the specification of BNs requires conditional probability tables describing the uncertain processes under evaluation. When these processes are poorly understood, it is necessary to rely on subjective probabilities provided by experts. Accurate probabilities of this type are normally hard to acquire from experts. Recent work in qualitative reasoning has developed methods to perform probabilistic reasoning using coarser representations. However, the latter types of approaches are too imprecise to compare the likelihood of alternative hypotheses. This paper examines this shortcoming of the qualitative approaches when applied to the aforementioned problem, and identifies and integrates techniques to refine them.

A Complexity Based Model for Quantifying Forensic Evidential Probabilities

An operational complexity model (OCM) is proposed to enable the complexity of both the cognitive and the computational components of a process to be determined. From the complexity of formation of a set of traces via a specified route a measure of the probability of that route can be determined. By determining the complexities of alternative routes leading to the formation of the same set of traces, the odds ratio indicating the relative plausibility of the alternative routes can be found. An illustrative application to a BitTorrent piracy case is presented, and the results obtained suggest that the OCM is capable of providing a realistic estimate of the odds ratio for two competing hypotheses. It is also demonstrated that the OCM can be straightforwardly refined to encompass a variety of circumstances.

Laws of large numbers for non-additive probabilities

We apply the concept of exchangeable random variables to the case of non-additive robability distributions exhibiting ncertainty aversion, and in the lass generated bya convex core convex non-additive probabilities, ith a convex core). We are able to rove two versions of the law of arge numbers (de Finetti's heorems). By making use of two efinitions. of independence we rove two versions of the strong law f large numbers. It turns out that e cannot assure the convergence of he sample averages to a constant. e then modal the case there is a true" probability distribution ehind the successive realizations of the uncertain random variable. In this case convergence occurs. This result is important because it renders true the intuition that it is possible "to learn" the "true" additive distribution behind an uncertain event if one repeatedly observes it (a sufficiently large number of times). We also provide a conjecture regarding the "Iearning" (or updating) process above, and prove a partia I result for the case of Dempster-Shafer updating rule and binomial trials.

Extinction, pollution, and the sustainable use of exhaustible and renewable resources

Single ownership of natural resources is conunon in many developing countries and socialist economies. The sole owner is usually the .state or society at large, and governments are responsible for either distributing exploitation rights or engaging in exploitation through their own corporations. • Under this circumstance, the notion of externality may not fully explain pollution problems existent in these nations. This paper studies the case where a single agent owns both exhaustible and renewable resources, and attempts to maximize its welfare. The resources are either perfect or imperfect substitutes. Initially, exhaustible resource extraction does not affect the renewable resource, and sustainable growth is attainable. A lactor of pollution flowing from the extraction of the nc.nrenewable resource into the growth of the renewable resource is introduced. The continuous exploitation of the exhaustible resource leads to the " optimal " extinction of the renewable resource, and sustainable growth is no longer reached. Regulation from a supra governmental agency such as an multinational institution may prove to be of utmost importance, if sustainability is to be achieved. The paper is divided into five sections. Section two provides a brief survey of the relevant literature. Section three presents the model without pollution. This factor is introduced in section four. The final section discusses some possible approaches for attaining sustainable growth, and contains the concluding remarks .

A Mismatch-Based Model for Memory Reconsolidation and Extinction in Attractor Networks

OSAN, R. , TORT, A. B. L. , AMARAL, O. B. . A mismatch-based model for memory reconsolidation and extinction in attractor networks. Plos One, v. 6, p. e23113, 2011.

Prior exposure to stress delays extinction but does not modify reinstatement of nicotine-induced conditioned place preference

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

The effect of elevated blood cortisol levels on the extinction of a conditioned stress response in rainbow trout

Following previously published observations that a conditioned response (CR) was lost more quickly by rainbow trout (Oncorhynchus mykiss) exhibiting a high responsiveness to stressors than by low responding individuals this study was designed to investigate the effects of exogenous cortisol on the retention of a CR in unselected rainbow trout. Fish held in isolation were conditioned over a 10-day period by pairing an innocuous signal (conditioned stimulus, CS: a water jet played on the surface of the tank water) with a mild stressor (unconditioned stimulus, US: 30 min of confinement). This resulted in a brief elevation of plasma cortisol levels (the CR) when the fish was exposed to the CS only. The effect of exogenous cortisol on the retention of the CR was evaluated by comparing the performance of fish that received cortisol-containing slow-release intraperitoneal implants, with fish receiving vehicle-only implants. Retention of the CR was assessed at intervals up to 35 days after conditioning ceased. The CR was considered to be evident when 30 min following presentation of the CS, mean plasma cortisol levels were significantly higher in conditioned than untrained fish. on day 1 both cortisol-implanted and vehicle-implanted conditioned fish exhibited a CR. However, from day 5 onwards the CR was observed only in the vehicle-implanted and conditioned group. This finding indicates that administration of cortisol accelerated the extinction of the CR in the cortisol-implanted fish, suggesting that elevated plasma cortisol levels can impair memory processes in rainbow trout. (c) 2006 Elsevier B.V. All rights reserved.

Dynamics of extinction in coupled populations of the flour beetle Tribolium castaneum

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Hyper abundant mesopredators and bird extinction in an Atlantic forest island

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Extinction of a conditioned response in rainbow trout selected for high or low responsiveness to stress

Two lines of rainbow trout (Oncorhynchus mykiss) that exhibit divergent endocrine responsiveness to stressors also display disparate behavioral traits. To investigate whether the high-responding (HR) and low-responding (LR) fish also differ in cognitive function, the rate of extinction of a conditioned response was compared between the two lines. Groups of HR and LR fish were exposed to a paired conditioned stimulus (CS- water off) and unconditioned stimulus (US; confinement stressor). After exposure to 18 CS-US pairings, at least 70% of individuals of both lines acquired a conditioned response (CR) manifested as an elevation of blood cortisol levels on presentation of the CS only. Post-conditioning, the fish were tested by presentation of the CS at weekly intervals, for 4 weeks, with no further reinforcement, and the extinction of the CR in the two lines was compared. The decline in mean plasma cortisol levels after exposure to the CS over successive tests suggested that the CR was retained for a shorter period among the HR (<14 days) than LR fish (<21 days). The frequency of individuals within each line whose plasma cortisol levels indicated a stress response when exposed to the CS was significantly greater among the LR than HR fish at 14 and 21 days with no HR fish falling into this category at 21 days. At 28 days post-conditioning, there were no HR fish and only three LR fish were categorized as stressed. These results suggest that there are differences in cognitive function between the two lines. Possible mechanisms underlying these differences are discussed. (C) 2004 Elsevier B.V. All rights reserved.