Confidence measurement in the light of signal detection theory

We compare three alternative methods for eliciting retrospective confidence in the context of a simple perceptual task: the Simple Confidence Rating (a direct report on a numerical scale), the Quadratic Scoring Rule (a post-wagering procedure), and the Matching Probability (MP; a generalization of the no-loss gambling method). We systematically compare the results obtained with these three rules to the theoretical confidence levels that can be inferred from performance in the perceptual task using Signal Detection Theory (SDT). We find that the MP provides better results in that respect. We conclude that MP is particularly well suited for studies of confidence that use SDT as a theoretical framework.

Introduction of a short writing task to measure and develop academic literacy in first year health undergraduates

This paper describes an initiative in the Faculty of Health at the Queensland University of Technology, Australia, where a short writing task was introduced to first year undergraduates in four courses including Public Health, Nursing, Social Work and Human Services, and Human Movement Studies. Over 1,000 students were involved in the trial. The task was assessed using an adaptation of the MASUS Procedure (Measuring the Academic Skills of University Students) (Webb & Bonanno, 1994). Feedback to the students including MASUS scores then enabled students to be directed to developmental workshops targeting their academic literacy needs. Students who achieved below the benchmark score were required to attend academic writing workshops in order to obtain the same summative 10% that was obtained by those who had achieved above the benchmark score. The trial was very informative, in terms of determining task appropriateness and timing, student feedback, student use of support, and student perceptions of the task and follow-up workshops. What we learned from the trial will be presented with a view to further refinement of this initiative.

Simple rules to contain an invasive species with a complex life cycle and high dispersal capacity

Designing practical rules for controlling invasive species is a challenging task for managers, particularly when species are long-lived, have complex life cycles and high dispersal capacities. Previous findings derived from plant matrix population analyses suggest that effective control of long-lived invaders may be achieved by focusing on killing adult plants. However, the cost-effectiveness of managing different life stages has not been evaluated. We illustrate the benefits of integrating matrix population models with decision theory to undertake this evaluation, using empirical data from the largest infestation of mesquite (Leguminosae: Prosopis spp) within Australia. We include in our model the mesquite life cycle, different dispersal rates and control actions that target individuals at different life stages with varying costs, depending on the intensity of control effort. We then use stochastic dynamic programming to derive cost-effective control strategies that minimize the cost of controlling the core infestation locally below a density threshold and the future cost of control arising from infestation of adjacent areas via seed dispersal. Through sensitivity analysis, we show that four robust management rules guide the allocation of resources between mesquite life stages for this infestation: (i) When there is no seed dispersal, no action is required until density of adults exceeds the control threshold and then only control of adults is needed; (ii) when there is seed dispersal, control strategy is dependent on knowledge of the density of adults and large juveniles (LJ) and broad categories of dispersal rates only; (iii) if density of adults is higher than density of LJ, controlling adults is most cost-effective; (iv) alternatively, if density of LJ is equal or higher than density of adults, management efforts should be spread between adults, large and to a lesser extent small juveniles, but never saplings. Synthesis and applications.In this study, we show that simple rules can be found for managing invasive plants with complex life cycles and high dispersal rates when population models are combined with decision theory. In the case of our mesquite population, focussing effort on controlling adults is not always the most cost-effective way to meet our management objective.

The NLP task at INEX 2004

The INEX workshop is concerned with evaluating the effectiveness of XML retrieval systems. In 2004 a natural language query task was added to the INEX Ad hoc track. Standard INEX Ad hoc topic titles are specified in NEXI -- a simplified and restricted subset of XPath, with a similar feel, and yet with a distinct IR flavour and interpretation. The syntax of NEXI is rigid and it imposes some limitations on the kind of information need that it can faithfully capture. At INEX 2004 the NLP question to be answered was simple -- is it practical to use a natural language query that is the equivalent of the formal NEXI title? The results of this experiment are reported and some information on the future direction of the NLP task is presented.

The moderating role of task characteristics in determining responses to a stressful work simulation

Two experimental studies were conducted to examine whether the stress-buffering effects of behavioral control on work task responses varied as a function of procedural information. Study 1 manipulated low and high levels of task demands, behavioral control, and procedural information for 128 introductory psychology students completing an in-basket activity. ANOVA procedures revealed a significant three-way interaction among these variables in the prediction of subjective task performance and task satisfaction. It was found that procedural information buffered the negative effects of task demands on ratings of performance and satisfaction only under conditions of low behavioral control. This pattern of results suggests that procedural information may have a compensatory effect when the work environment is characterized by a combination of high task demands and low behavioral control. Study 2 (N=256) utilized simple and complex versions of the in-basket activity to examine the extent to which the interactive relationship among task demands, behavioral control, and procedural information varied as a function of task complexity. There was further support for the stress-buffering role of procedural information on work task responses under conditions of low behavioral control. This effect was, however, only present when the in-basket activity was characterized by high task complexity, suggesting that the interactive relationship among these variables may depend on the type of tasks performed at work.

Task profiling : A task-Based approach to measuring the integration of ICT in the classroom

The measurement of ICT (information and communication technology) integration is emerging as an area of research interest with such systems as Education Queensland including it in their recently released list of research priorities. Studies to trial differing integration measurement instruments have taken place within Australia in the last few years, particularly Western Australia (Trinidad, Clarkson, & Newhouse, 2004; Trinidad, Newhouse & Clarkson, 2005), Tasmania (Fitzallen 2005) and Queensland (Finger, Proctor, & Watson, 2005). This paper will add to these investigations by describing an alternate and original methodological approach which was trialled in a small-scale pilot study conducted jointly by Queensland Catholic Education Commission (QCEC) and the Centre of Learning Innovation, Queensland University of Technology (QUT) in late 2005. The methodology described is based on tasks which, through a process of profiling, can be seen to be artefacts which embody the internal and external factors enabling and constraining ICT integration.