Bounded parameter Markov Decision Processes with average reward criterion

Bounded parameter Markov Decision Processes (BMDPs) address the issue of dealing with uncertainty in the parameters of a Markov Decision Process (MDP). Unlike the case of an MDP, the notion of an optimal policy for a BMDP is not entirely straightforward. We consider two notions of optimality based on optimistic and pessimistic criteria. These have been analyzed for discounted BMDPs. Here we provide results for average reward BMDPs. We establish a fundamental relationship between the discounted and the average reward problems, prove the existence of Blackwell optimal policies and, for both notions of optimality, derive algorithms that converge to the optimal value function.

The influence of climate variability on hydrological processes and surface and groundwater hydrochemistry : the tropical upper roper river catchment, Northern Territory, Australia

The Upper Roper River is one of the Australia’s unique tropical rivers which have been largely untouched by development. The Upper Roper River catchment comprises the sub-catchments of the Waterhouse River and Roper Creek, the two tributaries of the Roper River. There is a complex geological setting with different aquifer types. In this seasonal system, close interaction between surface water and groundwater contributes to both streamflow and sustaining ecosystems. The interaction is highly variable between seasons. A conceptual hydrogeological model was developed to investigate the different hydrological processes and geochemical parameters, and determine the baseline characteristics of water resources of this pristine catchment. In the catchment, long term average rainfall is around 850 mm and is summer dominant which significantly influences the total hydrological system. The difference between seasons is pronounced, with high rainfall up to 600 mm/month in the wet season, and negligible rainfall in the dry season. Canopy interception significantly reduces the amount of effective rainfall because of the native vegetation cover in the pristine catchment. Evaporation exceeds rainfall the majority of the year. Due to elevated evaporation and high temperature in the tropics, at least 600 mm of annual rainfall is required to generate potential recharge. Analysis of 120 years of rainfall data trend helped define “wet” and “dry periods”: decreasing trend corresponds to dry periods, and increasing trend to wet periods. The period from 1900 to 1970 was considered as Dry period 1, when there were years with no effective rainfall, and if there was, the intensity of rainfall was around 300 mm. The period 1970 – 1985 was identified as the Wet period 2, when positive effective rainfall occurred in almost every year, and the intensity reached up to 700 mm. The period 1985 – 1995 was the Dry period 2, with similar characteristics as Dry period 1. Finally, the last decade was the Wet period 2, with effective rainfall intensity up to 800 mm. This variability in rainfall over decades increased/decreased recharge and discharge, improving/reducing surface water and groundwater quantity and quality in different wet and dry periods. The stream discharge follows the rainfall pattern. In the wet season, the aquifer is replenished, groundwater levels and groundwater discharge are high, and surface runoff is the dominant component of streamflow. Waterhouse River contributes two thirds and Roper Creek one third to Roper River flow. As the dry season progresses, surface runoff depletes, and groundwater becomes the main component of stream flow. Flow in Waterhouse River is negligible, the Roper Creek dries up, but the Roper River maintains its flow throughout the year. This is due to the groundwater and spring discharge from the highly permeable Tindall Limestone and tufa aquifers. Rainfall seasonality and lithology of both the catchment and aquifers are shown to influence water chemistry. In the wet season, dilution of water bodies by rainwater is the main process. In the dry season, when groundwater provides baseflow to the streams, their chemical composition reflects lithology of the aquifers, in particular the karstic areas. Water chemistry distinguishes four types of aquifer materials described as alluvium, sandstone, limestone and tufa. Surface water in the headwaters of the Waterhouse River, the Roper Creek and their tributaries are freshwater, and reflect the alluvium and sandstone aquifers. At and downstream of the confluence of the Roper River, river water chemistry indicates the influence of rainfall dilution in the wet season, and the signature of the Tindall Limestone and tufa aquifers in the dry. Rainbow Spring on the Waterhouse River and Bitter Spring on the Little Roper River (known as Roper Creek at the headwaters) discharge from the Tindall Limestone. Botanic Walk Spring and Fig Tree Spring discharge into the Roper River from tufa. The source of water was defined based on water chemical composition of the springs, surface and groundwater. The mechanisms controlling surface water chemistry were examined to define the dominance of precipitation, evaporation or rock weathering on the water chemical composition. Simple water balance models for the catchment have been developed. The important aspects to be considered in water resource planning of this total system are the naturally high salinity in the region, especially the downstream sections, and how unpredictable climate variation may impact on the natural seasonal variability of water volumes and surface-subsurface interaction.

Society, history and research processes

In this reflection on research processes a humanities researcher begins to ask questions about the cultural materialist dimensions of research activities. At the center of this exploration are questions relating to the ways in which personal histories and experiences inform particular research processes and the ways in which a researcher's habits of collecting and working with data are regulated by cultural and social practice. The reflection on personal research processes is located in terms of the ethics work of Michel Foucault that provides reminders about the role of modern bureaucracy in governing what appear to be personal processes.

Parallel implementation of stochastic simulation for large scale cellular processes

Experimental and theoretical studies have shown the importance of stochastic processes in genetic regulatory networks and cellular processes. Cellular networks and genetic circuits often involve small numbers of key proteins such as transcriptional factors and signaling proteins. In recent years stochastic models have been used successfully for studying noise in biological pathways, and stochastic modelling of biological systems has become a very important research field in computational biology. One of the challenge problems in this field is the reduction of the huge computing time in stochastic simulations. Based on the system of the mitogen-activated protein kinase cascade that is activated by epidermal growth factor, this work give a parallel implementation by using OpenMP and parallelism across the simulation. Special attention is paid to the independence of the generated random numbers in parallel computing, that is a key criterion for the success of stochastic simulations. Numerical results indicate that parallel computers can be used as an efficient tool for simulating the dynamics of large-scale genetic regulatory networks and cellular processes

Boundary weaving : the social structure and processes of organizational boundaries

Boundaries are an important field of study because they mediate almost every aspect of organizational life. They are becoming increasingly more important as organizations change more frequently and yet, despite the endemic use of the boundary metaphor in common organizational parlance, they are poorly understood. Organizational boundaries are under-theorized and researchers in related fields often simply assume their existence, without defining them. The literature on organizational boundaries is fragmented with no unifying theoretical basis. As a result, when it is recognized that an organizational boundary is "dysfunctional". there is little recourse to models on which to base remediating action. This research sets out to develop just such a theoretical model and is guided by the general question: "What is the nature of organizational boundaries?" It is argued that organizational boundaries can be conceptualised through elements of both social structure and of social process. Elements of structure include objects, coupling, properties and identity. Social processes include objectification, identification, interaction and emergence. All of these elements are integrated by a core category, or basic social process, called boundary weaving. An organizational boundary is a complex system of objects and emergent properties that are woven together by people as they interact together, objectifying the world around them, identifying with these objects and creating couplings of varying strength and polarity as well as their own fragmented identity. Organizational boundaries are characterised by the multiplicity of interconnections, a particular domain of objects, varying levels of embodiment and patterns of interaction. The theory developed in this research emerged from an exploratory, qualitative research design employing grounded theory methodology. The field data was collected from the training headquarters of the New Zealand Army using semi-structured interviews and follow up observations. The unit of analysis is an organizational boundary. Only one research context was used because of the richness and multiplicity of organizational boundaries that were present. The model arose, grounded in the data collected, through a process of theoretical memoing and constant comparative analysis. Academic literature was used as a source of data to aid theory development and the saturation of some central categories. The final theory is classified as middle range, being substantive rather than formal, and is generalizable across medium to large organizations in low-context societies. The main limitation of the research arose from the breadth of the research with multiple lines of inquiry spanning several academic disciplines, with some relevant areas such as the role of identity and complexity being addressed at a necessarily high level. The organizational boundary theory developed by this research replaces the typology approaches, typical of previous theory on organizational boundaries and reconceptualises the nature of groups in organizations as well as the role of "boundary spanners". It also has implications for any theory that relies on the concept of boundaries, such as general systems theory. The main contribution of this research is the development of a holistic model of organizational boundaries including an explanation of the multiplicity of boundaries . no organization has a single definable boundary. A significant aspect of this contribution is the integration of aspects of complexity theory and identity theory to explain the emergence of higher-order properties of organizational boundaries and of organizational identity. The core category of "boundary weaving". is a powerful new metaphor that significantly reconceptualises the way organizational boundaries may be understood in organizations. It invokes secondary metaphors such as the weaving of an organization's "boundary fabric". and provides managers with other metaphorical perspectives, such as the management of boundary friction, boundary tension, boundary permeability and boundary stability. Opportunities for future research reside in formalising and testing the theory as well as developing analytical tools that would enable managers in organizations to apply the theory in practice.

Knowledge-based urban development processes of an emerging knowledge city : Brisbane, Australia

In the global knowledge economy, to attract and retain knowledge-intensive industries and workers, cities produce various development strategies. Such strategising is an important development mechanism for cities to complete their transformation into knowledge cities. This paper discusses the critical connections between knowledge city foundations and integrated knowledge-based urban development strategies, and scrutinises Brisbane’s strategies in attracting and retaining investment and talent. The paper introduces a knowledge-based urban development assessment framework and uses this framework to provide a clearer understanding of Brisbane’s knowledge-based development processes and knowledge city transformation experience. The assessment framework particularly focuses on examining Brisbane’s four development processes, institutional, economic, socio-cultural and urban development, in detail. The findings reveal that although Brisbane is still in early stages of its transformation into a fully-fledged knowledge city, global orientation and achievements of Brisbane in strategising knowledge-based urban development are noteworthy.

Iron biogeochemistry and associated greenhouse gas evolution in a forested subtropical Australian coastal catchment : Poona Creek, Southeast Queensland

Iron (Fe) is the fourth most abundant element in the Earth’s crust. Excess Fe mobilization from terrestrial into aquatic systems is of concern for deterioration of water quality via biofouling and nuisance algal blooms in coastal and marine systems. Substantial Fe dissolution and transport involve alternate Fe(II) oxidation followed by Fe(III) reduction, with a diversity of Bacteria and Archaea acting as the key catalyst. Microbially-mediated Fe cycling is of global significance with regard to cycles of carbon (C), sulfur (S) and manganese (Mn). However, knowledge regarding microbial Fe cycling in circumneutral-pH habitats that prevail on Earth has been lacking until recently. In particular, little is known regarding microbial function in Fe cycling and associated Fe mobilization and greenhouse (CO2 and CH4, GHG) evolution in subtropical Australian coastal systems where microbial response to ambient variations such as seasonal flooding and land use changes is of concern. Using the plantation-forested Poona Creek catchment on the Fraser Coast of Southeast Queensland (SEQ), this research aimed to 1) study Fe cycling-associated bacterial populations in diverse terrestrial and aquatic habitats of a representative subtropical coastal circumneutral-pH (4–7) ecosystem; and 2) assess potential impacts of Pinus plantation forestry practices on microbially-mediated Fe mobilization, organic C mineralization and associated GHG evolution in coastal SEQ. A combination of wet-chemical extraction, undisturbed core microcosm, laboratory bacterial cultivation, microscopy and 16S rRNA-based molecular phylogenetic techniques were employed. The study area consisted primarily of loamy sands, with low organic C and dissolved nutrients. Total reactive Fe was abundant and evenly distributed within soil 0–30 cm profiles. Organic complexation primarily controlled Fe bioavailability and forms in well-drained plantation soils and water-logged, native riparian soils, whereas tidal flushing exerted a strong “seawater effect” in estuarine locations and formed a large proportion of inorganic Fe(III) complexes. There was a lack of Fe(II) sources across the catchment terrestrial system. Mature, first-rotation plantation clear-felling and second-rotation replanting significantly decreased organic matter and poorly crystalline Fe in well-drained soils, although variations in labile soil organic C fractions (dissolved organic C, DOC; and microbial biomass C, MBC) were minor. Both well-drained plantation soils and water-logged, native-vegetation soils were inhabited by a variety of cultivable, chemotrophic bacterial populations capable of C, Fe, S and Mn metabolism via lithotrophic or heterotrophic, (micro)aerobic or anaerobic pathways. Neutrophilic Fe(III)-reducing bacteria (FeRB) were most abundant, followed by aerobic, heterotrophic bacteria (heterotrophic plate count, HPC). Despite an abundance of FeRB, cultivable Fe(II)-oxidizing bacteria (FeOB) were absent in associated soils. A lack of links between cultivable Fe, S or Mn bacterial densities and relevant chemical measurements (except for HPC correlated with DOC) was likely due to complex biogeochemical interactions. Neither did variations in cultivable bacterial densities correlate with plantation forestry practices, despite total cultivable bacterial densities being significantly lower in estuarine soils when compared with well-drained plantation soils and water-logged, riparian native-vegetation soils. Given that bacterial Fe(III) reduction is the primary mechanism of Fe oxide dissolution in soils upon saturation, associated Fe mobilization involved several abiotic and biological processes. Abiotic oxidation of dissolved Fe(II) by Mn appeared to control Fe transport and inhibit Fe dissolution from mature, first-rotation plantation soils post-saturation. Such an effect was not observed in clear-felled and replanted soils associated with low SOM and potentially low Mn reactivity. Associated GHG evolution post-saturation mainly involved variable CO2 emissions, with low, but consistently increasing CH4 effluxes in mature, first-rotation plantation soil only. In comparison, water-logged soils in the riparian native-vegetation buffer zone functioned as an important GHG source, with high potentials for Fe mobilization and GHG, particularly CH4 emissions in riparian loam soils associated with high clay and crystalline Fe fractions. Active Fe–C cycling was unlikely to occur in lower-catchment estuarine soils associated with low cultivable bacterial densities and GHG effluxes. As a key component of bacterial Fe cycling, neutrophilic FeOB widely occurred in diverse aquatic, but not terrestrial, habitats of the catchment study area. Stalked and sheathed FeOB resembling Gallionella and Leptothrix were limited to microbial mat material deposited in surface fresh waters associated with a circumneutral-pH seep, and clay-rich soil within riparian buffer zones. Unicellular, Sideroxydans-related FeOB (96% sequence identity) were ubiquitous in surface and subsurface freshwater environments, with highest abundance in estuary-adjacent shallow coastal groundwater water associated with redox transition. The abundance of dissolved C and Fe in the groundwater-dependent system was associated with high numbers of cultivable anaerobic, heterotrophic FeRB, microaerophilic, putatively lithotrophic FeOB and aerobic, heterotrophic bacteria. This research represents the first study of microbial Fe cycling in diverse circumneutral-pH environments (terrestrial–aquatic, freshwater–estuarine, surface–subsurface) of a subtropical coastal ecosystem. It also represents the first study of its kind in the southern hemisphere. This work highlights the significance of bacterial Fe(III) reduction in terrestrial, and bacterial Fe(II) oxidation in aquatic catchment Fe cycling. Results indicate the risk of promotion of Fe mobilization due to plantation clear-felling and replanting, and GHG emissions associated with seasonal water-logging. Additional significant outcomes were also achieved. The first direct evidence for multiple biomineralization patterns of neutrophilic, microaerophilic, unicellular FeOB was presented. A putatively pure culture, which represents the first cultivable neutrophilic FeOB from the southern hemisphere, was obtained as representative FeOB ubiquitous in diverse catchment aquatic habitats.

An approximate Bayesian computation approach for estimating parameters of complex environmental processes in a cellular automata

Modelling an environmental process involves creating a model structure and parameterising the model with appropriate values to accurately represent the process. Determining accurate parameter values for environmental systems can be challenging. Existing methods for parameter estimation typically make assumptions regarding the form of the Likelihood, and will often ignore any uncertainty around estimated values. This can be problematic, however, particularly in complex problems where Likelihoods may be intractable. In this paper we demonstrate an Approximate Bayesian Computational method for the estimation of parameters of a stochastic CA. We use as an example a CA constructed to simulate a range expansion such as might occur after a biological invasion, making parameter estimates using only count data such as could be gathered from field observations. We demonstrate ABC is a highly useful method for parameter estimation, with accurate estimates of parameters that are important for the management of invasive species such as the intrinsic rate of increase and the point in a landscape where a species has invaded. We also show that the method is capable of estimating the probability of long distance dispersal, a characteristic of biological invasions that is very influential in determining spread rates but has until now proved difficult to estimate accurately.

Interactive processes link the multiple symptoms of fatigue in sport competition

Muscle physiologists often describe fatigue simply as a decline of muscle force and infer this causes an athlete to slow down. In contrast, exercise scientists describe fatigue during sport competition more holistically as an exercise-induced impairment of performance. The aim of this review is to reconcile the different views by evaluating the many performance symptoms/measures and mechanisms of fatigue. We describe how fatigue is assessed with muscle, exercise or competition performance measures. Muscle performance (single muscle test measures) declines due to peripheral fatigue (reduced muscle cell force) and/or central fatigue (reduced motor drive from the CNS). Peak muscle force seldom falls by >30% during sport but is often exacerbated during electrical stimulation and laboratory exercise tasks. Exercise performance (whole-body exercise test measures) reveals impaired physical/technical abilities and subjective fatigue sensations. Exercise intensity is initially sustained by recruitment of new motor units and help from synergistic muscles before it declines. Technique/motor skill execution deviates as exercise proceeds to maintain outcomes before they deteriorate, e.g. reduced accuracy or velocity. The sensation of fatigue incorporates an elevated rating of perceived exertion (RPE) during submaximal tasks, due to a combination of peripheral and higher CNS inputs. Competition performance (sport symptoms) is affected more by decision-making and psychological aspects, since there are opponents and a greater importance on the result. Laboratory based decision making is generally faster or unimpaired. Motivation, self-efficacy and anxiety can change during exercise to modify RPE and, hence, alter physical performance. Symptoms of fatigue during racing, team-game or racquet sports are largely anecdotal, but sometimes assessed with time-motion analysis. Fatigue during brief all-out racing is described biomechanically as a decline of peak velocity, along with altered kinematic components. Longer sport events involve pacing strategies, central and peripheral fatigue contributions and elevated RPE. During match play, the work rate can decline late in a match (or tournament) and/or transiently after intense exercise bursts. Repeated sprint ability, agility and leg strength become slightly impaired. Technique outcomes, such as velocity and accuracy for throwing, passing, hitting and kicking, can deteriorate. Physical and subjective changes are both less severe in real rather than simulated sport activities. Little objective evidence exists to support exercise-induced mental lapses during sport. A model depicting mind-body interactions during sport competition shows that the RPE centre-motor cortex-working muscle sequence drives overall performance levels and, hence, fatigue symptoms. The sporting outputs from this sequence can be modulated by interactions with muscle afferent and circulatory feedback, psychological and decision-making inputs. Importantly, compensatory processes exist at many levels to protect against performance decrements. Small changes of putative fatigue factors can also be protective. We show that individual fatigue factors including diminished carbohydrate availability, elevated serotonin, hypoxia, acidosis, hyperkalaemia, hyperthermia, dehydration and reactive oxygen species, each contribute to several fatigue symptoms. Thus, multiple symptoms of fatigue can occur simultaneously and the underlying mechanisms overlap and interact. Based on this understanding, we reinforce the proposal that fatigue is best described globally as an exercise-induced decline of performance as this is inclusive of all viewpoints.