A Systems-level approach to building sustainable assessment cultures : moderation, quality task design and dependability of judgement

This paper considers the conditions that are necessary at system and local levels for teacher assessment to be valid, reliable and rigorous. With sustainable assessment cultures as a goal, the paper examines how education systems can support local level efforts for quality learning and dependable teacher assessment. This is achieved through discussion of relevant research and consideration of a case study involving an evaluation of a cross-sectoral approach to promoting confidence in school-based assessment in Queensland, Australia. Building on the reported case study, essential characteristics for developing sustainable assessment cultures are presented, including: leadership in learning; alignment of curriculum, pedagogy and assessment; the design of quality assessment tasks and accompanying standards, and evidence-based judgement and moderation. Taken together, these elements constitute a new framework for building assessment capabilities and promoting quality assurance.

Relationship between the nature of the search task types and query reformulation behaviour

Success of query reformulation and relevant information retrieval depends on many factors, such as users’ prior knowledge, age, gender, and cognitive styles. One of the important factors that affect a user’s query reformulation behaviour is that of the nature of the search tasks. Limited studies have examined the impact of the search task types on query reformulation behaviour while performing Web searches. This paper examines how the nature of the search tasks affects users’ query reformulation behaviour during information searching. The paper reports empirical results from a user study in which 50 participants performed a set of three Web search tasks – exploratory, factorial and abstract. Users’ interactions with search engines were logged by using a monitoring program. 872 unique search queries were classified into five query types – New, Add, Remove, Replace and Repeat. Users submitted fewer queries for the factual task, which accounted for 26%. They completed a higher number of queries (40% of the total queries) while carrying out the exploratory task. A one-way MANOVA test indicated a significant effect of search task types on users’ query reformulation behaviour. In particular, the search task types influenced the manner in which users reformulated the New and Repeat queries.

Provisionality and the impossible task of painting

This thesis explores Raphael Rubinstein’s notion of provisionality as detailed in his influential article from 2009, Provisional Painting and his subsequent exhibition of the same name from 2011. Rubenstein’s writing is discussed in relation to modern art’s rhetoric around the many ‘deaths’ or ‘ends’ of painting as a contemporary art‐making medium, particularly in reference to Yve‐Alain Bois’ 1986 article, Painting: the task of mourning. While Rubenstein predominantly views the provisional via an abstract lens, it is through the work of Sigmar Polke and then Luc Tuymans, Peter Doig and Daniel Richter, that I build an argument to include the work of contemporary representational painters within his notion of provisionality. These new ideas of provisionality are then examined in the context of my recent paintings, which are viewed as contemporary examples of provisionality extended into the representational.

Use of a clay modeling task to reduce chocolate craving

Elaborated Intrusion theory (EI theory; Kavanagh, Andrade, & May, 2005) posits two main cognitive components in craving: associative processes that lead to intrusive thoughts about the craved substance or activity, and elaborative processes supporting mental imagery of the substance or activity. We used a novel visuospatial task to test the hypothesis that visual imagery plays a key role in craving. Experiment 1 showed that spending 10 min constructing shapes from modeling clay (plasticine) reduced participants' craving for chocolate compared with spending 10 min 'letting your mind wander'. Increasing the load on verbal working memory using a mental arithmetic task (counting backwards by threes) did not reduce craving further. Experiment 2 compared effects on craving of a simpler verbal task (counting by ones) and clay modeling. Clay modeling reduced overall craving strength and strength of craving imagery, and reduced the frequency of thoughts about chocolate. The results are consistent with EI theory, showing that craving is reduced by loading the visuospatial sketchpad of working memory but not by loading the phonological loop. Clay modeling might be a useful self-help tool to help manage craving for chocolate, snacks and other foods.

The need for ‘representative task design’ in evaluating efficacy of skills tests in sport : a comment on Russell, Benton and Kingsley (2010)

Russell, Benton and Kingsley (2010) recently suggested a new association football test comprising three different tasks for the evaluation of players' passing, dribbling and shooting skills. Their stated intention was to enhance ‘ecological validity’ of current association football skills tests allowing generalisation of results from the new protocols to performance constraints that were ‘representative’ of experiences during competitive game situations. However, in this comment we raise some concerns with their use of the term ‘ecological validity’ to allude to aspects of ‘representative task design’. We propose that in their paper the authors confused understanding of environmental properties, performance achievement and generalisability of the test and its outcomes. Here, we argue that the tests designed by Russell and colleagues did not include critical sources of environmental information, such as the active role of opponents, which players typically use to organise their actions during performance. Static tasks which are not representative of the competitive performance environment may lead to different emerging patterns of movement organisation and performance outcomes, failing to effectively evaluate skills performance in sport.

Video-based road commentary training improves hazard perception of young drivers in a dual task

This study used a video-based hazard perception dual task to compare the hazard perception skills of young drivers with middle aged, more experienced drivers and to determine if these skills can be improved with video-based road commentary training. The primary task required the participants to detect and verbally identify immediate hazard on video-based traffic scenarios while concurrently performing a secondary tracking task, simulating the steering of real driving. The results showed that the young drivers perceived fewer immediate hazards (mean = 75.2%, n = 24, 19 females) than the more experienced drivers (mean = 87.5%, n = 8, all females), and had longer hazard perception times, but performed better in the secondary tracking task. After the road commentary training, the mean percentage of hazards detected and identified by the young drivers improved to the level of the experienced drivers and was significantly higher than that of an age and driving experience matched control group. The results will be discussed in the context of psychological theories of hazard perception and in relation to road commentary as an evidence-based training intervention that seems to improve many aspects of unsafe driving behaviour in young drivers.

Integrating agency and resource dependence theory : firm profitability, industry regulation, and board task performance

Boards of directors are key governancemechanisms in organizations and fulfill twomain tasks:monitoringmanagers and firm performance, and providing advice and access to resources. In spite of a wealth of researchmuch remains unknown about how boards attend to the two tasks. This study investigates whether organizational (firm profitability) and environmental factors (industry regulation) affect board task performance. The data combine CEOs' responses to a questionnaire, and archival data from a sample of large Italian firms. Findings show that past firm performance is negatively associatedwith board monitoring and advice tasks; greater industry regulation enhances perceived board task performance; board monitoring and advice tasks tend to reinforce each other, despite their theoretical and practical distinction.

The NLP task at INEX 2004

The INEX workshop is concerned with evaluating the effectiveness of XML retrieval systems. In 2004 a natural language query task was added to the INEX Ad hoc track. Standard INEX Ad hoc topic titles are specified in NEXI -- a simplified and restricted subset of XPath, with a similar feel, and yet with a distinct IR flavour and interpretation. The syntax of NEXI is rigid and it imposes some limitations on the kind of information need that it can faithfully capture. At INEX 2004 the NLP question to be answered was simple -- is it practical to use a natural language query that is the equivalent of the formal NEXI title? The results of this experiment are reported and some information on the future direction of the NLP task is presented.

Overview of the NTCIR-10 cross-lingual link discovery task

This paper presents an overview of NTCIR-10 Cross-lingual Link Discovery (CrossLink-2) task. For the task, we continued using the evaluation framework developed for the NTCIR-9 CrossLink-1 task. Overall, recommended links were evaluated at two levels (file-to-file and anchor-to-file); and system performance was evaluated with metrics: LMAP, R-Prec and P@N.

NTCIR-10 crosslink-2 task : a link mining strategy

At NTCIR-10 we participated in the cross-lingual link discovery (CrossLink-2) task. In this paper we describe our systems for discovering cross-lingual links between the Chinese, Japanese, and Korean (CJK) Wikipedia and the English Wikipedia. The evaluation results show that our implementation of the cross-lingual linking method achieved promising results.

Implementation and removal of an affirmative-action quota : the impact on task assignment and workers’ skill acquisition

Both the United States and Canada have federal legislation that attempts to address employment inequities across specific target groups. The US has a long tradition of affirmative action, dating back to President Kennedy’s 1961 Executive Order; Canada enacted its Employment Equity Act in 1986. Employment Equity/Affirmative Action policy has attracted significant controversy, with high profile court cases and the repeal of state/provincial legislation. Coate and Loury (1993) examine the theoretical impact of introducing affirmative action. Unfortunately the theoretical impact of affirmative action is ambiguous. The current paper employs a laboratory experiment to shed empirical light on this theoretical ambiguity.

Retrieval of health advice on the web: AEHRC at ShARe/CLEF eHealth Evaluation Lab Task 3

This paper details the participation of the Australian e- Health Research Centre (AEHRC) in the ShARe/CLEF 2013 eHealth Evaluation Lab { Task 3. This task aims to evaluate the use of information retrieval (IR) systems to aid consumers (e.g. patients and their relatives) in seeking health advice on the Web. Our submissions to the ShARe/CLEF challenge are based on language models generated from the web corpus provided by the organisers. Our baseline system is a standard Dirichlet smoothed language model. We enhance the baseline by identifying and correcting spelling mistakes in queries, as well as expanding acronyms using AEHRC's Medtex medical text analysis platform. We then consider the readability and the authoritativeness of web pages to further enhance the quality of the document ranking. Measures of readability are integrated in the language models used for retrieval via prior probabilities. Prior probabilities are also used to encode authoritativeness information derived from a list of top-100 consumer health websites. Empirical results show that correcting spelling mistakes and expanding acronyms found in queries signi cantly improves the e ectiveness of the language model baseline. Readability priors seem to increase retrieval e ectiveness for graded relevance at early ranks (nDCG@5, but not precision), but no improvements are found at later ranks and when considering binary relevance. The authoritativeness prior does not appear to provide retrieval gains over the baseline: this is likely to be because of the small overlap between websites in the corpus and those in the top-100 consumer-health websites we acquired.

Identify disorders in health records using Conditional Random Fields and Metamap: AEHRC at ShARe/CLEF 2013 eHealth Evaluation Lab Task 1

The Australian e-Health Research Centre (AEHRC) recently participated in the ShARe/CLEF eHealth Evaluation Lab Task 1. The goal of this task is to individuate mentions of disorders in free-text electronic health records and map disorders to SNOMED CT concepts in the UMLS metathesaurus. This paper details our participation to this ShARe/CLEF task. Our approaches are based on using the clinical natural language processing tool Metamap and Conditional Random Fields (CRF) to individuate mentions of disorders and then to map those to SNOMED CT concepts. Empirical results obtained on the 2013 ShARe/CLEF task highlight that our instance of Metamap (after ltering irrelevant semantic types), although achieving a high level of precision, is only able to identify a small amount of disorders (about 21% to 28%) from free-text health records. On the other hand, the addition of the CRF models allows for a much higher recall (57% to 79%) of disorders from free-text, without sensible detriment in precision. When evaluating the accuracy of the mapping of disorders to SNOMED CT concepts in the UMLS, we observe that the mapping obtained by our ltered instance of Metamap delivers state-of-the-art e ectiveness if only spans individuated by our system are considered (`relaxed' accuracy).