Infinite-horizon policy-gradient estimation

Gradient-based approaches to direct policy search in reinforcement learning have received much recent attention as a means to solve problems of partial observability and to avoid some of the problems associated with policy degradation in value-function methods. In this paper we introduce GPOMDP, a simulation-based algorithm for generating a biased estimate of the gradient of the average reward in Partially Observable Markov Decision Processes (POMDPs) controlled by parameterized stochastic policies. A similar algorithm was proposed by Kimura, Yamamura, and Kobayashi (1995). The algorithm's chief advantages are that it requires storage of only twice the number of policy parameters, uses one free parameter β ∈ [0,1) (which has a natural interpretation in terms of bias-variance trade-off), and requires no knowledge of the underlying state. We prove convergence of GPOMDP, and show how the correct choice of the parameter β is related to the mixing time of the controlled POMDP. We briefly describe extensions of GPOMDP to controlled Markov chains, continuous state, observation and control spaces, multiple-agents, higher-order derivatives, and a version for training stochastic policies with internal states. In a companion paper (Baxter, Bartlett, & Weaver, 2001) we show how the gradient estimates generated by GPOMDP can be used in both a traditional stochastic gradient algorithm and a conjugate-gradient procedure to find local optima of the average reward. ©2001 AI Access Foundation and Morgan Kaufmann Publishers. All rights reserved.

Measuring bitrate and quality trade-off in a fast region-of-interest based video coding

Prevailing video adaptation solutions change the quality of the video uniformly throughout the whole frame in the bitrate adjustment process; while region-of-interest (ROI)-based solutions selectively retains the quality in the areas of the frame where the viewers are more likely to pay more attention to. ROI-based coding can improve perceptual quality and viewer satisfaction while trading off some bandwidth. However, there has been no comprehensive study to measure the bitrate vs. perceptual quality trade-off so far. The paper proposes an ROI detection scheme for videos, which is characterized with low computational complexity and robustness, and measures the bitrate vs. quality trade-off for ROI-based encoding using a state-of-the-art H.264/AVC encoder to justify the viability of this type of encoding method. The results from the subjective quality test reveal that ROI-based encoding achieves a significant perceptual quality improvement over the encoding with uniform quality at the cost of slightly more bits. Based on the bitrate measurements and subjective quality assessments, the bitrate and the perceptual quality estimation models for non-scalable ROI-based video coding (AVC) are developed, which are found to be similar to the models for scalable video coding (SVC).

Compact arrays for mobile platforms : trade-off between size and performance for SDMA and MIMO applications

Smart antenna receiver and transmitter systems consist of multi-port arrays with an individual receiver channel (including ADC) and an individual transmitter channel (including DAC)at every of the M antenna ports, respectively. By means of digital beamforming, an unlimited number of simultaneous complex-valued vector radiation patterns with M-1 degrees of freedom can be formed. Applications of smart antennas in communication systems include space-division multiple access. If both stations of a communication link are equipped with smart antennas (multiple-input-multiple-output, MIMO). multiple independent channels can be formed in a "multi-path-rich" environment. In this article, it will be shown that under certain circumstances, the correlation between signals from adjacent ports of a dense array (M + ΔM elements) can be kept as low as the correlation between signals from adjacent ports of a conventional array (M elements and half-wavelength pacing). This attractive feature is attained by means of a novel approach which employs a RF decoupling network at the array ports in order to form new ports which are decoupled and associated with mutually orthogonal (de-correlated) radiation patterns.

Life-history constraints in grassland plant species : a growth-defence trade-off is the norm

Plant growth can be limited by resource acquisition and defence against consumers, leading to contrasting trade-off possibilities. The competition-defence hypothesis posits a trade-off between competitive ability and defence against enemies (e.g. herbivores and pathogens). The growth-defence hypothesis suggests that strong competitors for nutrients are also defended against enemies, at a cost to growth rate. We tested these hypotheses using observations of 706 plant populations of over 500 species before and following identical fertilisation and fencing treatments at 39 grassland sites worldwide. Strong positive covariance in species responses to both treatments provided support for a growth-defence trade-off: populations that increased with the removal of nutrient limitation (poor competitors) also increased following removal of consumers. This result held globally across 4 years within plant life-history groups and within the majority of individual sites. Thus, a growth-defence trade-off appears to be the norm, and mechanisms maintaining grassland biodiversity may operate within this constraint.

Finite-SNR diversity-multiplexing trade-off of inter-vehicular cooperative diversity protocols

The finite-signal-to-noise ratio (SNR) diversity-multiplexing trade-off (DMT) of cooperative diversity protocols are investigated in vehicular networks based on cascaded Rayleigh fading. Lower bounds of DMT at finite SNR for orthogonal and non-orthogonal protocols are derived. The results showcase the first look into the achievable DMT trade-off of cooperative diversity in volatile vehicular environments. It is shown that the diversity gains are significantly suboptimal at realistic SNRs.

Reliability of partial ambiguity fixing with multiple GNSS constellations

Reliable ambiguity resolution (AR) is essential to Real-Time Kinematic (RTK) positioning and its applications, since incorrect ambiguity fixing can lead to largely biased positioning solutions. A partial ambiguity fixing technique is developed to improve the reliability of AR, involving partial ambiguity decorrelation (PAD) and partial ambiguity resolution (PAR). Decorrelation transformation could substantially amplify the biases in the phase measurements. The purpose of PAD is to find the optimum trade-off between decorrelation and worst-case bias amplification. The concept of PAR refers to the case where only a subset of the ambiguities can be fixed correctly to their integers in the integer least-squares (ILS) estimation system at high success rates. As a result, RTK solutions can be derived from these integer-fixed phase measurements. This is meaningful provided that the number of reliably resolved phase measurements is sufficiently large for least-square estimation of RTK solutions as well. Considering the GPS constellation alone, partially fixed measurements are often insufficient for positioning. The AR reliability is usually characterised by the AR success rate. In this contribution an AR validation decision matrix is firstly introduced to understand the impact of success rate. Moreover the AR risk probability is included into a more complete evaluation of the AR reliability. We use 16 ambiguity variance-covariance matrices with different levels of success rate to analyse the relation between success rate and AR risk probability. Next, the paper examines during the PAD process, how a bias in one measurement is propagated and amplified onto many others, leading to more than one wrong integer and to affect the success probability. Furthermore, the paper proposes a partial ambiguity fixing procedure with a predefined success rate criterion and ratio-test in the ambiguity validation process. In this paper, the Galileo constellation data is tested with simulated observations. Numerical results from our experiment clearly demonstrate that only when the computed success rate is very high, the AR validation can provide decisions about the correctness of AR which are close to real world, with both low AR risk and false alarm probabilities. The results also indicate that the PAR procedure can automatically chose adequate number of ambiguities to fix at given high-success rate from the multiple constellations instead of fixing all the ambiguities. This is a benefit that multiple GNSS constellations can offer.

Regional trade-offs between mine water and energy use : a water treatment case study

The mining industry faces concurrent pressures of reducing water use, energy consumption and greenhouse gas (GHG) emissions in coming years. However, the interactions between water and energy use, as well as GHG e missions have largely been neglected in modelling studies to date. In addition, investigations tend to focus on the unit operation scale, with little consideration of whole-of-site or regional scale effects. This paper presents an application of a hierarchical systems model (HSM) developed to represent water, energy and GHG emissions fluxes at scales ranging from the unit operation, to the site level, to the regional level. The model allows for the linkages between water use, energy use and GHG emissions to be examined in a fl exible and intuitive way, so that mine sites can predict energy and emissions impacts of water use reduction schemes and vice versa. This paper examines whether this approach can also be applied to the regional scale with multiple mine sites. The model is used to conduct a case study of several coal mines in the Bowen Basin, Australia, to compare the utility of centralised and decentralised mine water treatment schemes. The case study takes into account geographical factors (such as water pumping distances and elevations), economic factors (such as capital and operating cost curves for desalination treatment plants) and regional factors (such as regionally varying climates and associated variance in mine water volumes and quality). The case study results indicate that treatment of saline mine water incurs a trade-off between water and energy use in all cases. However, significant cost differences between centralised and decentralised schemes can be observed in a simple economic analysis. Further research will examine the possibility for deriving model up-scaling algorithms to reduce computational requirements.

The genetics of endurance : frequency of the ACTN3 R577X variant in Ironman World Championship athletes

Objectives To investigate the frequency of the ACTN3 R577X polymorphism in elite endurance triathletes, and whether ACTN3 R577X is significantly associated with performance time. Design Cross-sectional study. Methods Saliva samples, questionnaires, and performance times were collected for 196 elite endurance athletes who participated in the 2008 Kona Ironman championship triathlon. Athletes were of predominantly North American, European, and Australian origin. A one-way analysis of variance was conducted to compare performance times between genotype groups. Multiple linear regression analysis was performed to model the effect of questionnaire variables and genotype on performance time. Genotype and allele frequencies were compared to results from different populations using the chi-square test. Results Performance time did not significantly differ between genotype groups, and age, sex, and continent of origin were significant predictors of finishing time (age and sex: p < 5 × 10−6; continent: p = 0.003) though genotype was not. Genotype and allele frequencies obtained (RR 26.5%, RX 50.0%, XX 23.5%, R 51.5%, X 48.5%) were found to be not significantly different from Australian, Spanish, and Italian endurance athletes (p > 0.05), but were significantly different from Kenyan, Ethiopian, and Finnish endurance athletes (p < 0.01). Conclusions Genotype and allele frequencies agreed with those reported for endurance athletes of similar ethnic origin, supporting previous findings for an association between 577X allele and endurance. However, analysis of performance time suggests that ACTN3 does not alone influence endurance performance, or may have a complex effect on endurance performance due to a speed/endurance trade-off.

Sequential decision fusion of multibiometrics applied to text-dependent speaker verification for controlled errors

Reliability of the performance of biometric identity verification systems remains a significant challenge. Individual biometric samples of the same person (identity class) are not identical at each presentation and performance degradation arises from intra-class variability and inter-class similarity. These limitations lead to false accepts and false rejects that are dependent. It is therefore difficult to reduce the rate of one type of error without increasing the other. The focus of this dissertation is to investigate a method based on classifier fusion techniques to better control the trade-off between the verification errors using text-dependent speaker verification as the test platform. A sequential classifier fusion architecture that integrates multi-instance and multisample fusion schemes is proposed. This fusion method enables a controlled trade-off between false alarms and false rejects. For statistically independent classifier decisions, analytical expressions for each type of verification error are derived using base classifier performances. As this assumption may not be always valid, these expressions are modified to incorporate the correlation between statistically dependent decisions from clients and impostors. The architecture is empirically evaluated by applying the proposed architecture for text dependent speaker verification using the Hidden Markov Model based digit dependent speaker models in each stage with multiple attempts for each digit utterance. The trade-off between the verification errors is controlled using the parameters, number of decision stages (instances) and the number of attempts at each decision stage (samples), fine-tuned on evaluation/tune set. The statistical validation of the derived expressions for error estimates is evaluated on test data. The performance of the sequential method is further demonstrated to depend on the order of the combination of digits (instances) and the nature of repetitive attempts (samples). The false rejection and false acceptance rates for proposed fusion are estimated using the base classifier performances, the variance in correlation between classifier decisions and the sequence of classifiers with favourable dependence selected using the 'Sequential Error Ratio' criteria. The error rates are better estimated by incorporating user-dependent (such as speaker-dependent thresholds and speaker-specific digit combinations) and class-dependent (such as clientimpostor dependent favourable combinations and class-error based threshold estimation) information. The proposed architecture is desirable in most of the speaker verification applications such as remote authentication, telephone and internet shopping applications. The tuning of parameters - the number of instances and samples - serve both the security and user convenience requirements of speaker-specific verification. The architecture investigated here is applicable to verification using other biometric modalities such as handwriting, fingerprints and key strokes.

Parametric building development during early design stage

Objectives The objectives of this project were two-fold: • Assess the ease with which current architectural CAD systems supported the use ofparametric descriptions in defining building shape, engineering system performance and cost at the early stages of building design; • Assess the feasibility of implementing a software decision support system that allowed designers to trade-off the characteristics and configuration of various engineering systems to move towards a “global optimum” rather than considering each system in isolation and expecting humans to weigh up all of the costs and benefits. The first stage of the project consisted of using four different CAD systems to define building shells (envelopes) with different usages. These models were then exported into a shared database using the IFC information exchange specifications. The second stage involved the implementation of small computer programs that were able to estimate relevant system parameters based on performance requirements and the constraints imposed by the other systems. These are presented in a unified user interface that extracts the appropriate building shape parameters from the shared database Note that the term parametric in this context refers to the relationships among and between all elements of the building model - not just geometric associations - which will enable the desired coordination.

The new world of work and employment: fitness workers and what they want from the employment relationship

This paper reports on an empirically based study of the Queensland (Australia) health and fitness industry over 15 years (1993 -2008). This study traces the development of the new occupation of fitness instructor in a service industry which has evolved si nce the 1980s and is embedded in values of consumption and individualism. It is the new world of work. The data from the 1993 study was historically significant, capturing the conditions o f employment in an unregulated setting prior to the introduction of the first industrial a ward in that industry in 1994. Fitness workers bargained directly with employers over all a spects of the employment relationship without the constraints of industrial regulation or the presence of trade unions. The substantive outcomes of the employment relationship were a direct reflection of m anagerial prerogative and worker orientation and preference, and did not reflect the rewards and outcomes traditionally found in Australian workplaces. While the focus of the 1993 research was on exploring the employment relationship in a deregulated environment, an unusual phenomenon was identified: fitness workers happily trading-off what would be considere d standard working conditions for the opportunity to work (‘take the stage’). Since then, several streams of literature have evolved providing a new context for understanding this phenomenon in the fitness industry, including: the sociology of the body (Shilling 1993; Turner 1996); emotional (Hochschild 1984) and aesthetic labour (Warhurst et al 2000); the so cial relations of production and space (Lefebvre 1991; Moss 1995); body history (Helps 2007); the sociology of consumption (Saunders 1988; Baudrillard 1998; Ritzer 2004); and work identity (Du Gay 1996; Strangleman 2004). The 2008 survey instrument replicated the 1993 study but was additionally informed b y the new literature. Surveys were sent to 310 commercial fitness centres and 4,800 fitness workers across Queensland. Worker orientation appears unchanged, and industry working conditions still seem atypical despite regulation si nce 1994. We argue that for many fitness workers the goal is to gain access to the fitness centre economy. For this they are willing to trade-off standard conditions of employment, and exchange traditional employm ent rewards for m ore intrinsic psycho-social rewards gained the through e xp o sure of their physical capital (Bourdieu 1984) o r bo dily prowess to the adoration o f their gazing clients. Building on the tradition of emotional labour and aesthetic labour, this study introduces the concept of ocularcentric labour: a state in which labour’s quest for the psychosocial rewards gained from their own body image shapes the employment relationship. With ocularcentric labour the p sycho-social rewards have greater value for the worker than ‘hard’, core conditions of employment, and are a significant factor in bargaining and outcomes, often substituting fo r direct earnings. The wo rkforce profile (young, female, casual) and their expectations (psycho-social rewards of ado ration and celebrity) challenge traditional trade unions in terms of what they can deliver, given the fitness workers’ willingness to trade-off minimum conditions, hard-won by unions.

Info-gap theory and robust design of surveillance for invasive species : the case study of Barrow Island

Surveillance for invasive non-indigenous species (NIS) is an integral part of a quarantine system. Estimating the efficiency of a surveillance strategy relies on many uncertain parameters estimated by experts, such as the efficiency of its components in face of the specific NIS, the ability of the NIS to inhabit different environments, and so on. Due to the importance of detecting an invasive NIS within a critical period of time, it is crucial that these uncertainties be accounted for in the design of the surveillance system. We formulate a detection model that takes into account, in addition to structured sampling for incursive NIS, incidental detection by untrained workers. We use info-gap theory for satisficing (not minimizing) the probability of detection, while at the same time maximizing the robustness to uncertainty. We demonstrate the trade-off between robustness to uncertainty, and an increase in the required probability of detection. An empirical example based on the detection of Pheidole megacephala on Barrow Island demonstrates the use of info-gap analysis to select a surveillance strategy.

Asymmetrical DC link voltage configuration for a diode-clamped inverter

There is a trade off between a number of output voltage levels and the reliability and efficiency of a multilevel converter. A new configuration of diode-clamped multilevel inverters with a different combination of DC link capacitors voltage has been proposed in this paper. Two different symmetrical and asymmetrical unequal arrangements for a four-level diode-clamped inverter have been compared, in order to find an optimum arrangement with lower switching losses and optimised output voltage quality. The simulation and hardware results for a four-level inverter show that the asymmetrical configuration can obtain more output voltage levels with the same number of components compared with a conventional four-level inverter and this will lead to the reduction of the harmonic content of the output voltage. A new family of multi-output DC-DC converters with a simple control strategy has been utilised as a front-end converter to supply the DC link capacitor voltages for the optimised configuration.

Determination of optimal preventive maintenance strategy for serial production lines

A Split System Approach (SSA) based methodology is presented to assist in making optimal Preventive Maintenance decisions for serial production lines. The methodology treats a production line as a complex series system with multiple PM actions over multiple intervals. Both risk related cost and maintenance related cost are factored into the methodology as either deterministic or random variables. This SSA based methodology enables Asset Management (AM) decisions to be optimized considering a variety of factors including failure probability, failure cost, maintenance cost, PM performance, and the type of PM strategy. The application of this new methodology and an evaluation of the effects of these factors on PM decisions are demonstrated using an example. The results of this work show that the performance of a PM strategy can be measured by its Total Expected Cost Index (TECI). The optimal PM interval is dependent on TECI, PM performance and types of PM strategies. These factors are interrelated. Generally it was found that a trade-off between reliability and the number of PM actions needs to be made so that one can minimize Total Expected Cost (TEC) for asset maintenance.