Sequential decision making in general state space models

Many problems in control and signal processing can be formulated as sequential decision problems for general state space models. However, except for some simple models one cannot obtain analytical solutions and has to resort to approximation. In this thesis, we have investigated problems where Sequential Monte Carlo (SMC) methods can be combined with a gradient based search to provide solutions to online optimisation problems. We summarise the main contributions of the thesis as follows. Chapter 4 focuses on solving the sensor scheduling problem when cast as a controlled Hidden Markov Model. We consider the case in which the state, observation and action spaces are continuous. This general case is important as it is the natural framework for many applications. In sensor scheduling, our aim is to minimise the variance of the estimation error of the hidden state with respect to the action sequence. We present a novel SMC method that uses a stochastic gradient algorithm to find optimal actions. This is in contrast to existing works in the literature that only solve approximations to the original problem. In Chapter 5 we presented how an SMC can be used to solve a risk sensitive control problem. We adopt the use of the Feynman-Kac representation of a controlled Markov chain flow and exploit the properties of the logarithmic Lyapunov exponent, which lead to a policy gradient solution for the parameterised problem. The resulting SMC algorithm follows a similar structure with the Recursive Maximum Likelihood(RML) algorithm for online parameter estimation. In Chapters 6, 7 and 8, dynamic Graphical models were combined with with state space models for the purpose of online decentralised inference. We have concentrated more on the distributed parameter estimation problem using two Maximum Likelihood techniques, namely Recursive Maximum Likelihood (RML) and Expectation Maximization (EM). The resulting algorithms can be interpreted as an extension of the Belief Propagation (BP) algorithm to compute likelihood gradients. In order to design an SMC algorithm, in Chapter 8 uses a nonparametric approximations for Belief Propagation. The algorithms were successfully applied to solve the sensor localisation problem for sensor networks of small and medium size.

The protective role of religious coping in adolescents' responses to poverty and sexual decision-making in rural Kenya.

In this study, we explored how adolescents in rural Kenya apply religious coping in sexual decision-making in the context of high rates of poverty and Human Immunodeficiency Virus (HIV). Semi-structured interviews were conducted with 34 adolescents. One-third (13) reported religious coping related to economic stress, HIV, or sexual decision-making; the majority (29) reported religious coping with these or other stressors. Adolescents reported praying for God to partner with them to engage in positive behaviors, praying for strength to resist unwanted behaviors, and passive strategies characterized by waiting for God to provide resources or protection from HIV. Adolescents in Sub-Saharan Africa may benefit from HIV prevention interventions that integrate and build upon their use of religious coping.

Decision making across social contexts: Competition increases preferences for risk in chimpanzees and bonobos

Context can have a powerful influence on decision-making strategies in humans. In particular, people sometimes shift their economic preferences depending on the broader social context, such as the presence of potential competitors or mating partners. Despite the important role of competition in primate conspecific interactions, as well as evidence that competitive social contexts impact primates' social cognitive skills, there has been little study of how social context influences the strategies that nonhumans show when making decisions about the value of resources. Here we investigate the impact of social context on preferences for risk (variability in payoffs) in our two closest phylogenetic relatives, chimpanzees, Pan troglodytes, and bonobos, Pan paniscus. In a first study, we examine the impact of competition on patterns of risky choice. In a second study, we examine whether a positive play context affects risky choices. We find that (1) apes are more likely to choose the risky option when making decisions in a competitive context; and (2) the play context did not influence their risk preferences. Overall these results suggest that some types of social contexts can shift patterns of decision making in nonhuman apes, much like in humans. Comparative studies of chimpanzees and bonobos can therefore help illuminate the evolutionary processes shaping human economic behaviour. © 2012 The Association for the Study of Animal Behaviour.

Decision making by senior social workers at point of first referral

In the United Kingdom there has been difficulty in implementing the family support provisions contained in the 1989 Children Act, largely because of continued emphasis on child protection activity by local authorities. There is an observable international tendency for child-care referrals to receive investigative response, resulting in families being traumatized and children's needs left unmet. There has been a lack of research into how child-care referrals are initially categorized by senior social workers. This paper reports on research undertaken in two Health and Social Services Trusts within Northern Ireland to ascertain if it might be possible to treat more initial referrals as 'child-care problem enquiries' as opposed to 'child protection investigations'. Results demonstrate that, while such potential may exist, a preoccupation with the management of risk could lead to the development of child-care problems receiving quasi-child protection responses. Consequently, changes in initial decision making may not have the full intended effects in terms of the organizational release of resources for family support or a lessening of the traumatic impact upon families.

Information gathering and decision making about resource value in animal contests

Contestants are predicted to adjust the cost of a fight in line with the perceived value of the resource and this provides a way of determining whether the resource has been assessed. An assessment of resource value is predicted to alter an animal's motivational state and we note different methods of measuring that state. We provide a categorical framework in which the degree of resource assessment may be evaluated and also note limitations of various approaches. We place studies in six categories: (1) cases of no assessment, (2) cases of internal state such as hunger influencing apparent value, (3) cases of the contestants differing in assessment ability, (4) cases of mutual and equal assessment of value, (5) cases where opponents differ in resource value and (6) cases of particularly complex assessment abilities that involve a comparison of the value of two resources. We examine the extent to which these studies support game theory predictions and suggest future areas of research. (C) 2008 The Association for the Study of Animal Behaviour. Published by Elsevier Ltd. All rights reserved.

Professional perspectives on decision making about the long-term care of older people

With the increasing pressure on social and health care resources, professionals have to be more explicit in their decision making regarding the long-term care of older people. This grounded theory study used 19 focus groups and nine semi-structured interviews (99 staff in total) to explore professional perspectives on this decision making. Focus group participants and interviewees comprised care managers, social workers, consultant geriatricians, general medical practitioners, community nurses, home care managers, occupational therapists and hospital discharge support staff. The emerging themes spanned context, clients, families and services. Decisions were often prompted by a crisis, hindering professionals seeking to make a measured assessment. Fear of burglary and assault, and the willingness and availability of family to help were major factors in decisions about living at home. Service availability in terms of public funding for community care, the availability of home care workers and workload pressures on primary care services influenced decision 'thresholds' regarding admission to institutional care. Assessment tools designed to assist decision making about the long-term care of older people need to take into account the critical aspects of individual fears and motivation, family support and the availability of publicly funded services as well as functional and medical needs.

The influence of evidence-based medicine training on decision-making in relation to over-the-counter medicines: a qualitative study

Objectives
To explore the role of evidence of effectiveness when making decisions about over-the-counter (OTC) medication and to ascertain whether evidence-based medicine training raised awareness in decision-making. Additionally, this work aimed to complement the findings of a previous study because all participants in this current study had received training in evidence-based medicine (unlike the previous participants).

Methods
Following ethical approval and an e-mailed invitation, face-to-face, semi-structured interviews were conducted with newly registered pharmacists (who had received training in evidence-based medicine as part of their MPharm degree) to discuss the role of evidence of effectiveness with OTC medicines. Interviews were recorded and transcribed verbatim. Following transcription, all data were entered into the NVivo software package (version 8). Data were coded and analysed using a constant comparison approach.

Key findings
Twenty-five pharmacists (7 males and 18 females; registered for less than 4 months) were recruited and all participated in the study. Their primary focus with OTC medicines was safety; sales of products (including those that lack evidence of effectiveness) were justified provided they did no harm. Meeting patient expectation was also an important consideration and often superseded evidence. Despite knowledge of the concept, and an awareness of ethical requirements, an evidence-based approach was not routinely implemented by these pharmacists. Pharmacists did not routinely utilize evidence-based resources when making decisions about OTC medicines and some felt uncomfortable discussing the evidence-base for OTC products with patients.

Conclusions
The evidence-based medicine training that these pharmacists received appeared to have limited influence on OTC decision-making. More work could be conducted to ensure that an evidence-based approach is routinely implemented in practice

Integrating textual analysis and evidential reasoning for decision making in Engineering design

Decision making is an important element throughout the life-cycle of large-scale projects. Decisions are critical as they have a direct impact upon the success/outcome of a project and are affected by many factors including the certainty and precision of information. In this paper we present an evidential reasoning framework which applies Dempster-Shafer Theory and its variant Dezert-Smarandache Theory to aid decision makers in making decisions where the knowledge available may be imprecise, conflicting and uncertain. This conceptual framework is novel as natural language based information extraction techniques are utilized in the extraction and estimation of beliefs from diverse textual information sources, rather than assuming these estimations as already given. Furthermore we describe an algorithm to define a set of maximal consistent subsets before fusion occurs in the reasoning framework. This is important as inconsistencies between subsets may produce results which are incorrect/adverse in the decision making process. The proposed framework can be applied to problems involving material selection and a Use Case based in the Engineering domain is presented to illustrate the approach. © 2013 Elsevier B.V. All rights reserved.

Individual Decision-Making and Inflation Persistence

The traditional task of a central bank is to preserve price stability and, in doing so, not to impair the real economy more than necessary. To meet this challenge, it is of great relevance whether inflation is only driven by inflation expectations and the current output gap or whether it is, in addition, influenced by past inflation. In the former case, as described by the New Keynesian Phillips curve, the central bank can immediately and simultaneously achieve price stability and equilibrium output, the so-called ‘divine coincidence’ (Blanchard and Galí 2007). In the latter case, the achievement of price stability is costly in terms of output and will be pursued over several periods. Similarly, it is important to distinguish this latter case, which describes ‘intrinsic’ inflation persistence, from that of ‘extrinsic’ inflation persistence, where the sluggishness of inflation is not a ‘structural’ feature of the economy but merely ‘inherited’ from the sluggishness of the other driving forces, inflation expectations and output. ‘Extrinsic’ inflation persistence is usually considered to be the less challenging case, as policy-makers are supposed to fight against the persistence in the driving forces, especially to reduce the stickiness of inflation expectations by a credible monetary policy, in order to reestablish the ‘divine coincidence’. The scope of this dissertation is to contribute to the vast literature and ongoing discussion on inflation persistence: Chapter 1 describes the policy consequences of inflation persistence and summarizes the empirical and theoretical literature. Chapter 2 compares two models of staggered price setting, one with a fixed two-period duration and the other with a stochastic duration of prices. I show that in an economy with a timeless optimizing central bank the model with the two-period alternating price-setting (for most parameter values) leads to more persistent inflation than the model with stochastic price duration. This result amends earlier work by Kiley (2002) who found that the model with stochastic price duration generates more persistent inflation in response to an exogenous monetary shock. Chapter 3 extends the two-period alternating price-setting model to the case of 3- and 4-period price durations. This results in a more complex Phillips curve with a negative impact of past inflation on current inflation. As simulations show, this multi-period Phillips curve generates a too low degree of autocorrelation and too early turnings points of inflation and is outperformed by a simple Hybrid Phillips curve. Chapter 4 starts from the critique of Driscoll and Holden (2003) on the relative real-wage model of Fuhrer and Moore (1995). While taking the critique seriously that Fuhrer and Moore’s model will collapse to a much simpler one without intrinsic inflation persistence if one takes their arguments literally, I extend the model by a term for inequality aversion. This model extension is not only in line with experimental evidence but results in a Hybrid Phillips curve with inflation persistence that is observably equivalent to that presented by Fuhrer and Moore (1995). In chapter 5, I present a model that especially allows to study the relationship between fairness attitudes and time preference (impatience). In the model, two individuals take decisions in two subsequent periods. In period 1, both individuals are endowed with resources and are able to donate a share of their resources to the other individual. In period 2, the two individuals might join in a common production after having bargained on the split of its output. The size of the production output depends on the relative share of resources at the end of period 1 as the human capital of the individuals, which is built by means of their resources, cannot fully be substituted one against each other. Therefore, it might be rational for a well-endowed individual in period 1 to act in a seemingly ‘fair’ manner and to donate own resources to its poorer counterpart. This decision also depends on the individuals’ impatience which is induced by the small but positive probability that production is not possible in period 2. As a general result, the individuals in the model economy are more likely to behave in a ‘fair’ manner, i.e., to donate resources to the other individual, the lower their own impatience and the higher the productivity of the other individual. As the (seemingly) ‘fair’ behavior is modelled as an endogenous outcome and as it is related to the aspect of time preference, the presented framework might help to further integrate behavioral economics and macroeconomics.

Optimal management of natural resources. Accounting for heterogeneity

Dynamic optimization methods have become increasingly important over the last years in economics. Within the dynamic optimization techniques employed, optimal control has emerged as the most powerful tool for the theoretical economic analysis. However, there is the need to advance further and take account that many dynamic economic processes are, in addition, dependent on some other parameter different than time. One can think of relaxing the assumption of a representative (homogeneous) agent in macro- and micro-economic applications allowing for heterogeneity among the agents. For instance, the optimal adaptation and diffusion of a new technology over time, may depend on the age of the person that adopted the new technology. Therefore, the economic models must take account of heterogeneity conditions within the dynamic framework. This thesis intends to accomplish two goals. The first goal is to analyze and revise existing environmental policies that focus on defining the optimal management of natural resources over time, by taking account of the heterogeneity of environmental conditions. Thus, the thesis makes a policy orientated contribution in the field of environmental policy by defining the necessary changes to transform an environmental policy based on the assumption of homogeneity into an environmental policy which takes account of heterogeneity. As a result the newly defined environmental policy will be more efficient and likely also politically more acceptable since it is tailored more specifically to the heterogeneous environmental conditions. Additionally to its policy orientated contribution, this thesis aims making a methodological contribution by applying a new optimization technique for solving problems where the control variables depend on two or more arguments --- the so-called two-stage solution approach ---, and by applying a numerical method --- the Escalator Boxcar Train Method --- for solving distributed optimal control problems, i.e., problems where the state variables, in addition to the control variables, depend on two or more arguments. Chapter 2 presents a theoretical framework to determine optimal resource allocation over time for the production of a good by heterogeneous producers, who generate a stock externalit and derives government policies to modify the behavior of competitive producers in order to achieve optimality. Chapter 3 illustrates the method in a more specific context, and integrates the aspects of quality and time, presenting a theoretical model that allows to determine the socially optimal outcome over time and space for the problem of waterlogging in irrigated agricultural production. Chapter 4 of this thesis concentrates on forestry resources and analyses the optimal selective-logging regime of a size-distributed forest.

International public goods through integrated natural resources management research in CGIAR partnerships

The CGIAR System conducts research to produce international public goods (IPG) that are of wide applicability creating a scientific base which speeds and broadens local adaptive development. Integrated natural resources management (INRM) research is sometimes seen to be very location specific and consequently does not lend itself readily to the production of IPGs. In this paper we analyse ways in which strategic approaches to INRM research can have broad international applicability and serve as useful foundations for the development of locally adapted technologies. The paper describes the evolution of the IPG concept within the CGIAR and elaborates on five major types of IPGs that have been generated from a varied set of recent INRM research efforts. CGIAR networks have both strengths and weaknesses in INRM research and application, with enormous differences in relative research and development capacities, responsibilities and data access of its partners, making programme process evolution critical to acceptance and participation. Many of the lessons learnt regarding challenges and corresponding IPG research approaches are relevant to designing and managing future multi-scale, multi-locational, coordinated INRM programmes involving broad-based partnerships to address complex environmental and livelihood problems for development.

Climate change and the British Uplands: evidence for decision-making

We summarise the work of an interdisciplinary network set up to explore the impacts of climate change in the British Uplands. In this CR Special, the contributors present the state of knowledge and this introduction synthesises this knowledge and derives implications for decision makers. The Uplands are valued semi-natural habitats, providing ecosystem services that have historically been taken for granted. For example, peat soils, which are mostly found in the Uplands, contain around 50% of the terrestrial carbon in the UK. Land management continues to be a driver of ecosystem service delivery. Degraded and managed peatlands are subject to erosion and carbon loss with negative impacts on biodiversity, carbon storage and water quality. Climate change is already being experienced in British Uplands and is likely to exacerbate these pressures. Climate envelope models suggest as much as 50% of British Uplands and peatlands will be exposed to climate stress by the end of the 21st century under low and high emissions scenarios. However, process-based models of the response of organic soils to this climate stress do not give a consistent indication of what this will mean for soil carbon: results range from a very slight increase in uptake, through a clear decline, to a net carbon loss. Preserving existing peat stocks is an important climate mitigation strategy, even if new peat stops forming. Preserving upland vegetation cover is a key win–win management strategy that will reduce erosion and loss of soil carbon, and protect a variety of services such as the continued delivery of a high quality water resource.

The limits of human development and the use of energy and natural resources

The development of nations is an unquestionable requirement. A lot of challenges concerning health, education and economy are present. A discussion on these development models has occupied the minds of decision makers in recent years. When energy supply and demand is considered, the situation becomes critical and the crucial question is: how to improve the quality of life of developing countries based on available models of development that are related to the life style of developed countries, for which the necessary use and waste of energy are present? How much energy is essential to humanity for not so as to endangering the survival conditions of future generations? the human development index (HDI) establishes the relationship among energy use, economic growth and social growth. Here it can be seen that 75% of the world population has a significant energy consumption potential. This is a strong reason to consider that the sustainable development concepts on energy policies are strategic to the future of the planet. This paper deals with the importance of seeking alternative development models for human development balance, natural resources conservation and environment through rational energy use concepts. (c) 2004 Elsevier Ltd. All rights reserved.