Developing consistency in teacher judgment formation through online meeting centres

This paper investigates the complex interactions that occur as teachers meet online to justify and negotiate their assessment judgments of student work across relatively large and geographically dispersed populations. Drawing from sociocultural theories of learning and technology, the technology is positioned as playing a role in either supporting or hindering teachers reaching a common understanding of assessment standards. Meeting transcripts and interviews with the teachers have been qualitatively analysed in terms of the interactions that occurred and teachers’ perceptions of these interactions. While online meetings offer a partial solution to address the current demands of assessment in education, they also present new challenges as teachers meet, in an unfamiliar environment, to discuss student work.

Strategies to facilitate grading consistency in large classes

First year undergraduate university classes can be very large, and big student numbers often creates a challenge for instructors to ensure assignments are graded consistently across the cohort. This session describes and demonstrates the use of interactive audience response technology (ART) with assessors (rather than students) to moderate assignment grading. Results from preliminary research indicate this method of moderating the grading of assignments is effective, and achieves more consistent outcomes for students.

Moderation and consistency of teacher judgement : teachers’ views

Major curriculum and assessment reforms in Australia have generated research interest in issues related to standards, teacher judgement and moderation. This article is based on one related inquiry of a large-scale Australian Research Council Linkage project conducted in Queensland. This qualitative study analysed interview data to identify teachers’ views on standards and moderation as a means to achieving consistency of teacher judgement. A complementary aspect of the research involved a blind review that was conducted to determine the degree of teacher consistency without the experience of moderation. Empirical evidence was gained that most teachers, of the total interviewed, articulated a positive attitude towards the use of standards in moderation and perceived that this process produces consistency in teachers’ judgements. Context was identified as an important influential factor in teachers’ judgements and it was concluded that teachers’ assessment beliefs, attitudes and practices impact on their perceptions of the value of moderation practice and the extent to which consistency can be achieved.

On the consistency of multiclass classification methods

Binary classification methods can be generalized in many ways to handle multiple classes. It turns out that not all generalizations preserve the nice property of Bayes consistency. We provide a necessary and sufficient condition for consistency which applies to a large class of multiclass classification methods. The approach is illustrated by applying it to some multiclass methods proposed in the literature.

On the consistency of multiclass classification methods

Binary classification is a well studied special case of the classification problem. Statistical properties of binary classifiers, such as consistency, have been investigated in a variety of settings. Binary classification methods can be generalized in many ways to handle multiple classes. It turns out that one can lose consistency in generalizing a binary classification method to deal with multiple classes. We study a rich family of multiclass methods and provide a necessary and sufficient condition for their consistency. We illustrate our approach by applying it to some multiclass methods proposed in the literature.

Using technology to facilitate grading consistency in large classes

University classes in marketing are often large, and therefore require teams of teachers to cover all of the necessary activities. A major problem with teaching teams is the inconsistency that results from myriad individuals offering subjective opinions. This innovation uses the latest moderation techniques along with Audience Response Technology (ART) to enhance the learning experience by providing more consistent and reliable grading in large classes. Assessment items are moderated before they are graded in meetings that employ ART. Results show the process is effective when the teaching team is very large, or there is a diverse range of experienced and inexperienced teachers. This “behind the scenes” innovation is not immediately apparent to students, but results in more consistent grades, more useful feedback for students, and more confident graders.

Disaster triage : evidence, consistency and standard practice

Triage has military origins, with the initial purpose being to direct care to those most likely to be able to return to the battlefield. Conversely, modern disaster triage attempts to ensure both the prioritisation of care for those who need it most and a fair distribution of resources, so that health responders are able to ‘do the most for the most’...

Within-subject intra- and inter-method consistency of two experimental risk attitude elicitation methods

We compare the consistency of choices in two methods to used elicit risk preferences on an aggregate as well as on an individual level. We asked subjects to choose twice from a list of nine decision between two lotteries, as introduced by Holt and Laury (2002, 2005) alternating with nine decisions using the budget approach introduced by Andreoni and Harbaugh (2009). We find that while on an aggregate(subject pool) level the results are (roughly) consistent, on an individual(within-subject) level,behavior is far from consistent. Within each method as well as across methods we observe low correlations. This again questions the reliability of experimental risk elicitation measures and the ability to use results from such methods to control for the risk aversion of subjects when explaining e�ects in other experimental games.

Arts-health intersections : a model of practice for consistency in the arts-health sector

The field of Arts-Health practice and research has grown exponentially in the past 30 years. While researchers are using applied arts as the subject of investigation in research, the evaluation of practice and participant benefits has a limited general focus. In recent years, the field has witnessed a growing concentration on the evaluation of health outcomes, outputs and tangential benefits for participants engaging in Arts-Health practice. The wide range of methodological approaches applied arts practitioners implement make the field difficult to define. This article introduces the term Arts-Health intersections as a model of practice and framework to promote consistency in design, implementation and evaluative processes in applied arts programmes promoting health outcomes. The article challenges the current trend to solely evaluate health outcomes in the field, and promotes a concurrent and multidisciplinary methodological approach that can be adopted to promote evaluation, consistency and best practice in the field of Arts-Health intersections. The article provides a theoretical overview of Arts-Health intersections, and then takes this theoretical platform and details a best model of practice for developing Arts-Health intersections and presents this model as a guide.

Validation and consistency in the application of a trauma team activation system at a level 1 trauma centre

- The RAH was activated over 2500 trauma calls in 2009. This figure is over twice the number of calls put out by similar services. - Many trauma calls (in particular L2 trauma calls) from the existing system do not warrant activation of the trauma team - Sometimes trauma calls are activated for nontrauma reasons (eg rapid access to radiology, departmental pressures etc) - The excess of trauma calls has several deleterious effects particularly on time management for the trauma service staff: ward rounds/tertiary survey rounds, education, quality improvement, research

The consistency of crash involvement recall across time

Introduction Road safety researchers rely heavily on self-report data to explore the aetiology of crash risk. However, researchers consistently acknowledge a range of limitations associated with this methodological approach (e.g., self-report bias), which has been hypothesised to reduce the predictive efficacy of scales. Although well researched in other areas, one important factor often neglected in road safety studies is the fallibility of human memory. Given accurate recall is a key assumption in many studies, the validity and consistency of self-report data warrants investigation. The aim of the current study was to examine the consistency of self-report data of crash history and details of the most recent reported crash on two separate occasions. Materials & Method A repeated measures design was utilised to examine the self-reported crash involvement history of 214 general motorists over a two month period. Results A number of interesting discrepancies were noted in relation to number of lifetime crashes reported by the participants and the descriptions of their most recent crash across the two occasions. Of the 214 participants who reported having been involved in a crash, 35 (22.3%) reported a lower number of lifetime crashes as Time 2, than at Time 1. Of the 88 drivers who reported no change in number of lifetime crashes, 10 (11.4%) described a different most recent crash. Additionally, of the 34 reporting an increase in the number of lifetime crashes, 29 (85.3%) of these described the same crash on both occasions. Assessed as a whole, at least 47.1% of participants made a confirmed mistake at Time 1 or Time 2. Conclusions These results raise some doubt in regard to the accuracy of memory recall across time. Given that self-reported crash involvement is the predominant dependent variable used in the majority of road safety research, this issue warrants further investigation. Replication of the study with a larger sample size that includes multiple recall periods would enhance understanding into the significance of this issue for road safety methodology.

Within-subject intra- and inter-method consistency of two experimental risk attitude elicitation methods

We compare the consistency of choices in two methods used to elicit risk preferences on an aggregate as well as on an individual level. We ask subjects to choose twice from a list of nine decisions between two lotteries, as introduced by Holt and Laury (2002, 2005) alternating with nine decisions using the budget approach introduced by Andreoni and Harbaugh (2009). We find that, while on an aggregate (subject pool) level the results are consistent, on an individual (within-subject) level, behaviour is far from consistent. Within each method as well as across methods we observe low (simple and rank) correlations.