Bounded parameter Markov Decision Processes with average reward criterion

Bounded parameter Markov Decision Processes (BMDPs) address the issue of dealing with uncertainty in the parameters of a Markov Decision Process (MDP). Unlike the case of an MDP, the notion of an optimal policy for a BMDP is not entirely straightforward. We consider two notions of optimality based on optimistic and pessimistic criteria. These have been analyzed for discounted BMDPs. Here we provide results for average reward BMDPs. We establish a fundamental relationship between the discounted and the average reward problems, prove the existence of Blackwell optimal policies and, for both notions of optimality, derive algorithms that converge to the optimal value function.

Beam steering with high front-to-back ratio and high directivity on small platforms using decoupled antenna pairs

Beam steering with high front-to-back ratio and high directivity on a small platform is proposed. Two closely spaced antenna pairs with eigenmode port decoupling are used as the basic radiating elements. Two orthogonal radiation patterns are obtained for each antenna pair. High front-to-back ratio and high directivity are achieved by combining the two orthogonal radiation patterns. With an infinite groundplane, a front-to-back ratio of 21 dB with a directivity of 9.8 dB can be achieved. Beam steering, at the expense of a slight decrease in directivity, is achieved by placing the two antenna pairs 0.5λ apart. The simulated half power beamwidth is 58°. A prototype was designed and the 2-D radiation patterns were measured. The prototype supports three directions of beam steering. The half power beamwidth was measured as 46°, 48°, and 50° for the three respective beam directions. The measured front-to-back ratio in azimuth plane is 8.5 dB, 8.0 dB and 7.6 dB, respectively.

Some novel techniques of parameter estimation for the dynamical models in biological systems

Inverse problems based on using experimental data to estimate unknown parameters of a system often arise in biological and chaotic systems. In this paper, we consider parameter estimation in systems biology involving linear and non-linear complex dynamical models, including the Michaelis–Menten enzyme kinetic system, a dynamical model of competence induction in Bacillus subtilis bacteria and a model of feedback bypass in B. subtilis bacteria. We propose some novel techniques for inverse problems. Firstly, we establish an approximation of a non-linear differential algebraic equation that corresponds to the given biological systems. Secondly, we use the Picard contraction mapping, collage methods and numerical integration techniques to convert the parameter estimation into a minimization problem of the parameters. We propose two optimization techniques: a grid approximation method and a modified hybrid Nelder–Mead simplex search and particle swarm optimization (MH-NMSS-PSO) for non-linear parameter estimation. The two techniques are used for parameter estimation in a model of competence induction in B. subtilis bacteria with noisy data. The MH-NMSS-PSO scheme is applied to a dynamical model of competence induction in B. subtilis bacteria based on experimental data and the model for feedback bypass. Numerical results demonstrate the effectiveness of our approach.

Exploring the parameter space of a rabbit ventricular action potential model to investigate the effect of variation on action potential and calcium transients

Computational models for cardiomyocyte action potentials (AP) often make use of a large parameter set. This parameter set can contain some elements that are fitted to experimental data independently of any other element, some elements that are derived concurrently with other elements to match experimental data, and some elements that are derived purely from phenomenological fitting to produce the desired AP output. Furthermore, models can make use of several different data sets, not always derived for the same conditions or even the same species. It is consequently uncertain whether the parameter set for a given model is physiologically accurate. Furthermore, it is only recently that the possibility of degeneracy in parameter values in producing a given simulation output has started to be addressed. In this study, we examine the effects of varying two parameters (the L-type calcium current (I(CaL)) and the delayed rectifier potassium current (I(Ks))) in a computational model of a rabbit ventricular cardiomyocyte AP on both the membrane potential (V(m)) and calcium (Ca(2+)) transient. It will subsequently be determined if there is degeneracy in this model to these parameter values, which will have important implications on the stability of these models to cell-to-cell parameter variation, and also whether the current methodology for generating parameter values is flawed. The accuracy of AP duration (APD) as an indicator of AP shape will also be assessed.

Parameter estimation for a phenomenological model of the cardiac action potential

The action potential (ap) of a cardiac cell is made up of a complex balance of ionic currents which flow across the cell membrane in response to electrical excitation of the cell. Biophysically detailed mathematical models of the ap have grown larger in terms of the variables and parameters required to model new findings in subcellular ionic mechanisms. The fitting of parameters to such models has seen a large degree of parameter and module re-use from earlier models. An alternative method for modelling electrically exciteable cardiac tissue is a phenomenological model, which reconstructs tissue level ap wave behaviour without subcellular details. A new parameter estimation technique to fit the morphology of the ap in a four variable phenomenological model is presented. An approximation of a nonlinear ordinary differential equation model is established that corresponds to the given phenomenological model of the cardiac ap. The parameter estimation problem is converted into a minimisation problem for the unknown parameters. A modified hybrid Nelder–Mead simplex search and particle swarm optimization is then used to solve the minimisation problem for the unknown parameters. The successful fitting of data generated from a well known biophysically detailed model is demonstrated. A successful fit to an experimental ap recording that contains both noise and experimental artefacts is also produced. The parameter estimation method’s ability to fit a complex morphology to a model with substantially more parameters than previously used is established.

Diversity in product familiarity across younger and older adults

This paper reports on the findings of a completed experiment examining levels of familiarity in younger and older adults. Research has shown that older adults use products less intuitively than younger adults, and that familiarity is an essential element of intuitive interaction. This finding influenced the decision to focus on familiarity and to investigate why older adults use products less intuitively than younger adults. By identifying and understanding the differences in familiarity, it is hypothesised that designers will be able to design more usable products for older adults. An empirical study was conducted, investigating the differences in familiarity between younger and older adults with contemporary products. Younger adults demonstrate significantly higher levels of familiarity compared to older adults, and the three groups of older adults demonstrated no significant differences between them. The implications of this finding is discussed.

Accurately simulating the production of radiotherapy portal images using non-zero beam angles

In this study, the delivery and portal imaging of one square-field and one conformal radiotherapy treatment was simulated using the Monte Carlo codes BEAMnrc and DOSXYZnrc. The treatment fields were delivered to a humanoid phantom from different angles by a 6 MV photon beam linear accelerator, with an amorphous-silicon electronic portal imaging device (a-Si EPID) used to provide images of the phantom generated by each field. The virtual phantom preparation code CTCombine was used to combine a computed-tomography-derived model of the irradiated phantom with a simple, rectilinear model of the a-Si EPID, at each beam angle used in the treatment. Comparison of the resulting experimental and simulated a-Si EPID images showed good agreement, within \[gamma](3%, 3 mm), indicating that this method may be useful in providing accurate Monte Carlo predictions of clinical a-Si EPID images, for use in the verification of complex radiotherapy treatments.

Grocery product pricing and Australian supermarket consumers : gender differences in perceived importance levels

Grocery shopping is an essential and routine activity. Although long regarded the responsibility of the female spouse, modern social and demographic shifts are causing men to become more engaged in this task. This is the first study to analyse gender differences with respect to the criterion of grocery product price within an Australian supermarket retail environment. A stratified sample of 140 male and 140 female grocery shoppers was surveyed. Results showed that men considered price attributes of products as being significantly lower in importance than did women. Additionally, men displayed lower levels of price nvolvement, reported referencing shelf price to a lesser extent, and gave lesser consideration to promotional tactics focusing on low price. Although men on average buy fewer items than do women, they spend more money for each item they purchase. This higher expenditure per item appears to be driven, at least in part, by a lack of price referencing. This research has implications for gender studies and consumer behaviour disciplines in relation to grocery shopping.

Feasibility of using health data sources to inform product safety surveillance in Queensland : a report for the Queensland Injury Prevention Council

This report provides an evaluation of the current available evidence-base for identification and surveillance of product-related injuries in children in Queensland. While the focal population was children in Queensland, the identification of information needs and data sources for product safety surveillance has applicability nationally for all age groups. The report firstly summarises the data needs of product safety regulators regarding product-related injury in children, describing the current sources of information informing product safety policy and practice, and documenting the priority product surveillance areas affecting children which have been a focus over recent years in Queensland. Health data sources in Queensland which have the potential to inform product safety surveillance initiatives were evaluated in terms of their ability to address the information needs of product safety regulators. Patterns in product-related injuries in children were analysed using routinely available health data to identify areas for future intervention, and the patterns in product-related injuries in children identified in health data were compared to those identified by product safety regulators. Recommendations were made for information system improvements and improved access to and utilisation of health data for more proactive approaches to product safety surveillance in the future.

Product labelling can confer sustained increases in expected and actual satiety

Expected satiety has been shown to play a key role in decisions around meal size. Recently it has become clear that these expectations can also influence the satiety that is experienced after a food has been consumed. As such, increasing the expected and actual satiety a food product confers without increasing its caloric content is of importance. In this study we sought to determine whether this could be achieved via product labelling. Female participants (N=75) were given a 223-kcal yoghurt smoothie for lunch. In separate conditions the smoothie was labelled as a diet brand, a highly-satiating brand, or an ‘own brand’ control. Expected satiety was assessed using rating scales and a computer-based ‘method of adjustment’, both prior to consuming the smoothie and 24 hours later. Hunger and fullness were assessed at baseline, immediately after consuming the smoothie, and for a further three hours. Despite the fact that all participants consumed the same food, the smoothie branded as highly-satiating was consistently expected to deliver more satiety than the other ‘brands’; this difference was sustained 24 hours after consumption. Furthermore, post-consumption and over three hours, participants consuming this smoothie reported significantly less hunger and significantly greater fullness. These findings demonstrate that the satiety that a product confers depends in part on information that is present around the time of consumption. We suspect that this process is mediated by changes to expected satiety. These effects may potentially be utilised in the development of successful weight-management products.