The need for ‘representative task design’ in evaluating efficacy of skills tests in sport : a comment on Russell, Benton and Kingsley (2010)

Russell, Benton and Kingsley (2010) recently suggested a new association football test comprising three different tasks for the evaluation of players' passing, dribbling and shooting skills. Their stated intention was to enhance ‘ecological validity’ of current association football skills tests allowing generalisation of results from the new protocols to performance constraints that were ‘representative’ of experiences during competitive game situations. However, in this comment we raise some concerns with their use of the term ‘ecological validity’ to allude to aspects of ‘representative task design’. We propose that in their paper the authors confused understanding of environmental properties, performance achievement and generalisability of the test and its outcomes. Here, we argue that the tests designed by Russell and colleagues did not include critical sources of environmental information, such as the active role of opponents, which players typically use to organise their actions during performance. Static tasks which are not representative of the competitive performance environment may lead to different emerging patterns of movement organisation and performance outcomes, failing to effectively evaluate skills performance in sport.

Video-based road commentary training improves hazard perception of young drivers in a dual task

This study used a video-based hazard perception dual task to compare the hazard perception skills of young drivers with middle aged, more experienced drivers and to determine if these skills can be improved with video-based road commentary training. The primary task required the participants to detect and verbally identify immediate hazard on video-based traffic scenarios while concurrently performing a secondary tracking task, simulating the steering of real driving. The results showed that the young drivers perceived fewer immediate hazards (mean = 75.2%, n = 24, 19 females) than the more experienced drivers (mean = 87.5%, n = 8, all females), and had longer hazard perception times, but performed better in the secondary tracking task. After the road commentary training, the mean percentage of hazards detected and identified by the young drivers improved to the level of the experienced drivers and was significantly higher than that of an age and driving experience matched control group. The results will be discussed in the context of psychological theories of hazard perception and in relation to road commentary as an evidence-based training intervention that seems to improve many aspects of unsafe driving behaviour in young drivers.

Integrating agency and resource dependence theory : firm profitability, industry regulation, and board task performance

Boards of directors are key governancemechanisms in organizations and fulfill twomain tasks:monitoringmanagers and firm performance, and providing advice and access to resources. In spite of a wealth of researchmuch remains unknown about how boards attend to the two tasks. This study investigates whether organizational (firm profitability) and environmental factors (industry regulation) affect board task performance. The data combine CEOs' responses to a questionnaire, and archival data from a sample of large Italian firms. Findings show that past firm performance is negatively associatedwith board monitoring and advice tasks; greater industry regulation enhances perceived board task performance; board monitoring and advice tasks tend to reinforce each other, despite their theoretical and practical distinction.

The NLP task at INEX 2004

The INEX workshop is concerned with evaluating the effectiveness of XML retrieval systems. In 2004 a natural language query task was added to the INEX Ad hoc track. Standard INEX Ad hoc topic titles are specified in NEXI -- a simplified and restricted subset of XPath, with a similar feel, and yet with a distinct IR flavour and interpretation. The syntax of NEXI is rigid and it imposes some limitations on the kind of information need that it can faithfully capture. At INEX 2004 the NLP question to be answered was simple -- is it practical to use a natural language query that is the equivalent of the formal NEXI title? The results of this experiment are reported and some information on the future direction of the NLP task is presented.

Overview of the NTCIR-10 cross-lingual link discovery task

This paper presents an overview of NTCIR-10 Cross-lingual Link Discovery (CrossLink-2) task. For the task, we continued using the evaluation framework developed for the NTCIR-9 CrossLink-1 task. Overall, recommended links were evaluated at two levels (file-to-file and anchor-to-file); and system performance was evaluated with metrics: LMAP, R-Prec and P@N.

NTCIR-10 crosslink-2 task : a link mining strategy

At NTCIR-10 we participated in the cross-lingual link discovery (CrossLink-2) task. In this paper we describe our systems for discovering cross-lingual links between the Chinese, Japanese, and Korean (CJK) Wikipedia and the English Wikipedia. The evaluation results show that our implementation of the cross-lingual linking method achieved promising results.

Implementation and removal of an affirmative-action quota : the impact on task assignment and workers’ skill acquisition

Both the United States and Canada have federal legislation that attempts to address employment inequities across specific target groups. The US has a long tradition of affirmative action, dating back to President Kennedy’s 1961 Executive Order; Canada enacted its Employment Equity Act in 1986. Employment Equity/Affirmative Action policy has attracted significant controversy, with high profile court cases and the repeal of state/provincial legislation. Coate and Loury (1993) examine the theoretical impact of introducing affirmative action. Unfortunately the theoretical impact of affirmative action is ambiguous. The current paper employs a laboratory experiment to shed empirical light on this theoretical ambiguity.

Retrieval of health advice on the web: AEHRC at ShARe/CLEF eHealth Evaluation Lab Task 3

This paper details the participation of the Australian e- Health Research Centre (AEHRC) in the ShARe/CLEF 2013 eHealth Evaluation Lab { Task 3. This task aims to evaluate the use of information retrieval (IR) systems to aid consumers (e.g. patients and their relatives) in seeking health advice on the Web. Our submissions to the ShARe/CLEF challenge are based on language models generated from the web corpus provided by the organisers. Our baseline system is a standard Dirichlet smoothed language model. We enhance the baseline by identifying and correcting spelling mistakes in queries, as well as expanding acronyms using AEHRC's Medtex medical text analysis platform. We then consider the readability and the authoritativeness of web pages to further enhance the quality of the document ranking. Measures of readability are integrated in the language models used for retrieval via prior probabilities. Prior probabilities are also used to encode authoritativeness information derived from a list of top-100 consumer health websites. Empirical results show that correcting spelling mistakes and expanding acronyms found in queries signi cantly improves the e ectiveness of the language model baseline. Readability priors seem to increase retrieval e ectiveness for graded relevance at early ranks (nDCG@5, but not precision), but no improvements are found at later ranks and when considering binary relevance. The authoritativeness prior does not appear to provide retrieval gains over the baseline: this is likely to be because of the small overlap between websites in the corpus and those in the top-100 consumer-health websites we acquired.

Identify disorders in health records using Conditional Random Fields and Metamap: AEHRC at ShARe/CLEF 2013 eHealth Evaluation Lab Task 1

The Australian e-Health Research Centre (AEHRC) recently participated in the ShARe/CLEF eHealth Evaluation Lab Task 1. The goal of this task is to individuate mentions of disorders in free-text electronic health records and map disorders to SNOMED CT concepts in the UMLS metathesaurus. This paper details our participation to this ShARe/CLEF task. Our approaches are based on using the clinical natural language processing tool Metamap and Conditional Random Fields (CRF) to individuate mentions of disorders and then to map those to SNOMED CT concepts. Empirical results obtained on the 2013 ShARe/CLEF task highlight that our instance of Metamap (after ltering irrelevant semantic types), although achieving a high level of precision, is only able to identify a small amount of disorders (about 21% to 28%) from free-text health records. On the other hand, the addition of the CRF models allows for a much higher recall (57% to 79%) of disorders from free-text, without sensible detriment in precision. When evaluating the accuracy of the mapping of disorders to SNOMED CT concepts in the UMLS, we observe that the mapping obtained by our ltered instance of Metamap delivers state-of-the-art e ectiveness if only spans individuated by our system are considered (`relaxed' accuracy).

Performance of a cognitive, but not visual, secondary task interacts with alcohol-induced balance impairment in novice and experienced motorcycle riders

The appropriateness of applying drink driving legislation to motorcycle riding has been questioned as there may be fundamental differences in the effects of alcohol on driving and motorcycling. It has been suggested that alcohol may redirect riders’ focus from higher-order cognitive skills such as cornering, judgement and hazard perception, to more physical skills such as maintaining balance. To test this hypothesis, the effects of low doses of alcohol on balance ability were investigated in a laboratory setting. The static balance of twenty experienced and twenty novice riders was measured while they performed either no secondary task, a visual (search) task, or a cognitive (arithmetic) task following the administration of alcohol (0%, 0.02%, and 0.05% BAC). Subjective ratings of intoxication and balance impairment increased in a dose-dependent manner in both novice and experienced motorcycle riders, while a BAC of 0.05%, but not 0.02%, was associated with impairments in static balance ability. This balance impairment was exacerbated when riders performed a cognitive, but not a visual, secondary task. Likewise, 0.05% BAC was associated with impairments in novice and experienced riders’ performance of a cognitive, but not a visual, secondary task, suggesting that interactive processes underlie balance and cognitive task performance. There were no observed differences between novice vs. experienced riders on static balance and secondary task performance, either alone or in combination. Implications for road safety and future ‘drink riding’ policy considerations are discussed.

Star performers : task and contextual performance are components, but are they enough?

Identifying outstanding performers or ‘stars’ is a critical component of managing talent. However, organizational effectiveness in this area is limited by the current lack of guidance about the behaviour and characteristics of stars. We address this gap by combining a conceptual analysis with an empirical study involving 174 managers. Conceptually we examine the alignment of managers’ perceptions of outstanding performance with the well established task and contextual performance model and find this framework accounts for a core element in managers’ judgments about outstanding performers. However, a second, more qualitative approach finds that other dimensions including being self-directed, and a willingness to lead are also important. Our findings are consistent with a long-term trend toward identifying work effectiveness with highly discretionary, psychological and behavioural elements, and we consider the implications of this for the study and management of high level, individual effectiveness.

One night CPAP withdrawal impairs performance at a driving simulator task faster than sleep restriction to 5 hours with treatment in OSA patients

Introduction Sleep restriction and missing 1 night’s continuous positive air pressure (CPAP) treatment are scenarios faced by obstructive sleep apnoea (OSA) patients, who must then assess their own fitness to drive. This study aims to assess the impact of this on driving performance. Method 11 CPAP treated participants (50–75 yrs), drove an interactive car simulator under monotonous motorway conditions for 2 hours on 3 afternoons, following;(i)normal night’s sleep (average 8.2 h) with CPAP (ii) sleep restriction (5 h), with CPAP (iii)normal length of sleep, without CPAP. Driving incidents were noted if the car came out of the designated driving lane. EEG was recorded continually and KSS reported every 200 seconds. Results Driving incidents: Incidents were more prevalent following CPAP withdrawal during hour 1, demonstrating a significant condition time interaction [F(6,60) = 3.40, p = 0.006]. KSS: At the start of driving participants felt sleepiest following CPAP withdrawal, by the end of the task KSS levels were similar following CPAP withdrawal and sleep restriction, demonstrating a significant condition, time interaction [F(3.94,39.41) = 3.39, p = 0.018]. EEG: There was a non significant trend for combined alpha and theta activity to be highest throughout the drive following CPAP withdrawal. Discussion CPAP withdrawal impairs driving simulator performance sooner than restricting sleep to 5 h with CPAP. Participants had insight into this increased sleepiness reflected by the higher KSS reported following CPAP withdrawal. In the practical terms of driving any one incident could be fatal. The earlier impairment reported here demonstrates the potential danger of missing CPAP treatment and highlights the benefit of CPAP treatment even when sleep time is short.

Task constraints

Newell (1985, 1986) identified the importance of interacting constraints on the emergent behaviours of learners or performers in sport as they assemble functional states of movement organisation in achieving task goals. Constraints, related to the person, task and environment, were defined as ‘boundaries or features that limit motion of the entity under consideration at any moment in time’ (Newell, 1986, p.347). Personal (or organismic) constraints include factors such as individual anthropometrics (height, weight, and limb lengths), fitness (e.g., strength, speed, aerobic capacity, and flexibility), mental skills (e.g. concentration, confidence, emotional control and motivation), perceptual and decisionmaking skills (e.g., recognising patterns of play, anticipation by reading the movements of opponents) and personality factors (e.g., risk taking or conservative behaviours). Newell (1986, p.350) distinguished between general environmental constraints, such as gravity, ambient temperature, natural light and altitude and task constraints, which are task specific and concerned with the goals of a specific activity. More recently, socio-cultural constraints (e.g., family support, cultural expectations and access to facilities) have also been considered as environmental constraints. Application of the constraints framework to the study of sport performance has led to task constraints being defined to include factors such as rules of games, equipment used, boundary playing areas and markings, nets and goals, the number of...

Revisiting sub-topic retrieval in the ImageCLEF 2009 photo retrieval task

Ranking documents according to the Probability Ranking Principle has been theoretically shown to guarantee optimal retrieval effectiveness in tasks such as ad hoc document retrieval. This ranking strategy assumes independence among document relevance assessments. This assumption, however, often does not hold, for example in the scenarios where redundancy in retrieved documents is of major concern, as it is the case in the sub–topic retrieval task. In this chapter, we propose a new ranking strategy for sub–topic retrieval that builds upon the interdependent document relevance and topic–oriented models. With respect to the topic– oriented model, we investigate both static and dynamic clustering techniques, aiming to group topically similar documents. Evidence from clusters is then combined with information about document dependencies to form a new document ranking. We compare and contrast the proposed method against state–of–the–art approaches, such as Maximal Marginal Relevance, Portfolio Theory for Information Retrieval, and standard cluster–based diversification strategies. The empirical investigation is performed on the ImageCLEF 2009 Photo Retrieval collection, where images are assessed with respect to sub–topics of a more general query topic. The experimental results show that our approaches outperform the state–of–the–art strategies with respect to a number of diversity measures.