Bounded parameter Markov Decision Processes with average reward criterion

Bounded parameter Markov Decision Processes (BMDPs) address the issue of dealing with uncertainty in the parameters of a Markov Decision Process (MDP). Unlike the case of an MDP, the notion of an optimal policy for a BMDP is not entirely straightforward. We consider two notions of optimality based on optimistic and pessimistic criteria. These have been analyzed for discounted BMDPs. Here we provide results for average reward BMDPs. We establish a fundamental relationship between the discounted and the average reward problems, prove the existence of Blackwell optimal policies and, for both notions of optimality, derive algorithms that converge to the optimal value function.

The influence of climate variability on hydrological processes and surface and groundwater hydrochemistry : the tropical upper roper river catchment, Northern Territory, Australia

The Upper Roper River is one of the Australia’s unique tropical rivers which have been largely untouched by development. The Upper Roper River catchment comprises the sub-catchments of the Waterhouse River and Roper Creek, the two tributaries of the Roper River. There is a complex geological setting with different aquifer types. In this seasonal system, close interaction between surface water and groundwater contributes to both streamflow and sustaining ecosystems. The interaction is highly variable between seasons. A conceptual hydrogeological model was developed to investigate the different hydrological processes and geochemical parameters, and determine the baseline characteristics of water resources of this pristine catchment. In the catchment, long term average rainfall is around 850 mm and is summer dominant which significantly influences the total hydrological system. The difference between seasons is pronounced, with high rainfall up to 600 mm/month in the wet season, and negligible rainfall in the dry season. Canopy interception significantly reduces the amount of effective rainfall because of the native vegetation cover in the pristine catchment. Evaporation exceeds rainfall the majority of the year. Due to elevated evaporation and high temperature in the tropics, at least 600 mm of annual rainfall is required to generate potential recharge. Analysis of 120 years of rainfall data trend helped define “wet” and “dry periods”: decreasing trend corresponds to dry periods, and increasing trend to wet periods. The period from 1900 to 1970 was considered as Dry period 1, when there were years with no effective rainfall, and if there was, the intensity of rainfall was around 300 mm. The period 1970 – 1985 was identified as the Wet period 2, when positive effective rainfall occurred in almost every year, and the intensity reached up to 700 mm. The period 1985 – 1995 was the Dry period 2, with similar characteristics as Dry period 1. Finally, the last decade was the Wet period 2, with effective rainfall intensity up to 800 mm. This variability in rainfall over decades increased/decreased recharge and discharge, improving/reducing surface water and groundwater quantity and quality in different wet and dry periods. The stream discharge follows the rainfall pattern. In the wet season, the aquifer is replenished, groundwater levels and groundwater discharge are high, and surface runoff is the dominant component of streamflow. Waterhouse River contributes two thirds and Roper Creek one third to Roper River flow. As the dry season progresses, surface runoff depletes, and groundwater becomes the main component of stream flow. Flow in Waterhouse River is negligible, the Roper Creek dries up, but the Roper River maintains its flow throughout the year. This is due to the groundwater and spring discharge from the highly permeable Tindall Limestone and tufa aquifers. Rainfall seasonality and lithology of both the catchment and aquifers are shown to influence water chemistry. In the wet season, dilution of water bodies by rainwater is the main process. In the dry season, when groundwater provides baseflow to the streams, their chemical composition reflects lithology of the aquifers, in particular the karstic areas. Water chemistry distinguishes four types of aquifer materials described as alluvium, sandstone, limestone and tufa. Surface water in the headwaters of the Waterhouse River, the Roper Creek and their tributaries are freshwater, and reflect the alluvium and sandstone aquifers. At and downstream of the confluence of the Roper River, river water chemistry indicates the influence of rainfall dilution in the wet season, and the signature of the Tindall Limestone and tufa aquifers in the dry. Rainbow Spring on the Waterhouse River and Bitter Spring on the Little Roper River (known as Roper Creek at the headwaters) discharge from the Tindall Limestone. Botanic Walk Spring and Fig Tree Spring discharge into the Roper River from tufa. The source of water was defined based on water chemical composition of the springs, surface and groundwater. The mechanisms controlling surface water chemistry were examined to define the dominance of precipitation, evaporation or rock weathering on the water chemical composition. Simple water balance models for the catchment have been developed. The important aspects to be considered in water resource planning of this total system are the naturally high salinity in the region, especially the downstream sections, and how unpredictable climate variation may impact on the natural seasonal variability of water volumes and surface-subsurface interaction.

Refining animal models in fracture research: Seeking consensus in optimising both animal welfare and scientific validity for appropriate biomedical use

Background In an attempt to establish some consensus on the proper use and design of experimental animal models in musculoskeletal research, AOVET (the veterinary specialty group of the AO Foundation) in concert with the AO Research Institute (ARI), and the European Academy for the Study of Scientific and Technological Advance, convened a group of musculoskeletal researchers, veterinarians, legal experts, and ethicists to discuss, in a frank and open forum, the use of animals in musculoskeletal research. Methods The group narrowed the field to fracture research. The consensus opinion resulting from this workshop can be summarized as follows: Results & Conclusion Anaesthesia and pain management protocols for research animals should follow standard protocols applied in clinical work for the species involved. This will improve morbidity and mortality outcomes. A database should be established to facilitate selection of anaesthesia and pain management protocols for specific experimental surgical procedures and adopted as an International Standard (IS) according to animal species selected. A list of 10 golden rules and requirements for conduction of animal experiments in musculoskeletal research was drawn up comprising 1) Intelligent study designs to receive appropriate answers; 2) Minimal complication rates (5 to max. 10%); 3) Defined end-points for both welfare and scientific outputs analogous to quality assessment (QA) audit of protocols in GLP studies; 4) Sufficient details for materials and methods applied; 5) Potentially confounding variables (genetic background, seasonal, hormonal, size, histological, and biomechanical differences); 6) Post-operative management with emphasis on analgesia and follow-up examinations; 7) Study protocols to satisfy criteria established for a "justified animal study"; 8) Surgical expertise to conduct surgery on animals; 9) Pilot studies as a critical part of model validation and powering of the definitive study design; 10) Criteria for funding agencies to include requirements related to animal experiments as part of the overall scientific proposal review protocols. Such agencies are also encouraged to seriously consider and adopt the recommendations described here when awarding funds for specific projects. Specific new requirements and mandates related both to improving the welfare and scientific rigour of animal-based research models are urgently needed as part of international harmonization of standards.

Helping someone with problem drinking: Mental health first aid guidelines - a Delphi expert consensus study

Background Alcohol is a leading risk factor for avoidable disease burden. Research suggests that a drinker's social network can play an integral role in addressing hazardous (i.e., high-risk) or problem drinking. Often however, social networks do not have adequate mental health literacy (i.e., knowledge about mental health problems, like problem drinking, or how to treat them). This is a concern as the response that a drinker receives from their social network can have a substantial impact on their willingness to seek help. This paper describes the development of mental health first aid guidelines that inform community members on how to help someone who may have, or may be developing, a drinking problem (i.e., alcohol abuse or dependence). Methods A systematic review of the research and lay literature was conducted to develop a 285-item survey containing strategies on how to help someone who may have, or may be developing, a drinking problem. Two panels of experts (consumers/carers and clinicians) individually rated survey items, using a Delphi process. Surveys were completed online or via postal mail. Participants were 99 consumers, carers and clinicians with experience or expertise in problem drinking from Australia, Canada, Ireland, New Zealand, the United Kingdom, and the United States. Items that reached consensus on importance were retained and written into guidelines. Results The overall response rate across all three rounds was 68.7% (67.6% consumers/carers, 69.2% clinicians), with 184 first aid strategies rated as essential or important by ≥80% of panel members. The endorsed guidelines provide guidance on how to: recognize problem drinking; approach someone if there is concern about their drinking; support the person to change their drinking; respond if they are unwilling to change their drinking; facilitate professional help seeking and respond if professional help is refused; and manage an alcohol-related medical emergency. Conclusion The guidelines provide a consensus-based resource for community members seeking to help someone with a drinking problem. Improving community awareness and understanding of how to identify and support someone with a drinking problem may lead to earlier recognition of problem drinking and greater facilitation of professional help seeking.

Georg Calixtus, Isaac Casaubon, and the Consensus of Antiquity

This article explores the relationship between Georg Calixtus (1586-1656) and Isaac Casaubon (1559-1614). It does this in order to highlight an oversight in the existing literature concerning Calixtus, and to encourage scholars to revisit the work of Early Modern figures who have previously been considered only from modern disciplinary perspectives. By emphasizing the relationship between Calixtus and Casaubon, this article argues that Calixtus was potentially exposed to much broader circles of intellectual debate than has previously been considered, and that a reevaluation of his work in light of these debates is therefore in order.

Society, history and research processes

In this reflection on research processes a humanities researcher begins to ask questions about the cultural materialist dimensions of research activities. At the center of this exploration are questions relating to the ways in which personal histories and experiences inform particular research processes and the ways in which a researcher's habits of collecting and working with data are regulated by cultural and social practice. The reflection on personal research processes is located in terms of the ethics work of Michel Foucault that provides reminders about the role of modern bureaucracy in governing what appear to be personal processes.

Parallel implementation of stochastic simulation for large scale cellular processes

Experimental and theoretical studies have shown the importance of stochastic processes in genetic regulatory networks and cellular processes. Cellular networks and genetic circuits often involve small numbers of key proteins such as transcriptional factors and signaling proteins. In recent years stochastic models have been used successfully for studying noise in biological pathways, and stochastic modelling of biological systems has become a very important research field in computational biology. One of the challenge problems in this field is the reduction of the huge computing time in stochastic simulations. Based on the system of the mitogen-activated protein kinase cascade that is activated by epidermal growth factor, this work give a parallel implementation by using OpenMP and parallelism across the simulation. Special attention is paid to the independence of the generated random numbers in parallel computing, that is a key criterion for the success of stochastic simulations. Numerical results indicate that parallel computers can be used as an efficient tool for simulating the dynamics of large-scale genetic regulatory networks and cellular processes