On the consistency of multiclass classification methods

Binary classification methods can be generalized in many ways to handle multiple classes. It turns out that not all generalizations preserve the nice property of Bayes consistency. We provide a necessary and sufficient condition for consistency which applies to a large class of multiclass classification methods. The approach is illustrated by applying it to some multiclass methods proposed in the literature.

On the consistency of multiclass classification methods

Binary classification is a well studied special case of the classification problem. Statistical properties of binary classifiers, such as consistency, have been investigated in a variety of settings. Binary classification methods can be generalized in many ways to handle multiple classes. It turns out that one can lose consistency in generalizing a binary classification method to deal with multiple classes. We study a rich family of multiclass methods and provide a necessary and sufficient condition for their consistency. We illustrate our approach by applying it to some multiclass methods proposed in the literature.

Using technology to facilitate grading consistency in large classes

University classes in marketing are often large, and therefore require teams of teachers to cover all of the necessary activities. A major problem with teaching teams is the inconsistency that results from myriad individuals offering subjective opinions. This innovation uses the latest moderation techniques along with Audience Response Technology (ART) to enhance the learning experience by providing more consistent and reliable grading in large classes. Assessment items are moderated before they are graded in meetings that employ ART. Results show the process is effective when the teaching team is very large, or there is a diverse range of experienced and inexperienced teachers. This “behind the scenes” innovation is not immediately apparent to students, but results in more consistent grades, more useful feedback for students, and more confident graders.

Disaster triage : evidence, consistency and standard practice

Triage has military origins, with the initial purpose being to direct care to those most likely to be able to return to the battlefield. Conversely, modern disaster triage attempts to ensure both the prioritisation of care for those who need it most and a fair distribution of resources, so that health responders are able to ‘do the most for the most’...

Within-subject intra- and inter-method consistency of two experimental risk attitude elicitation methods

We compare the consistency of choices in two methods to used elicit risk preferences on an aggregate as well as on an individual level. We asked subjects to choose twice from a list of nine decision between two lotteries, as introduced by Holt and Laury (2002, 2005) alternating with nine decisions using the budget approach introduced by Andreoni and Harbaugh (2009). We find that while on an aggregate(subject pool) level the results are (roughly) consistent, on an individual(within-subject) level,behavior is far from consistent. Within each method as well as across methods we observe low correlations. This again questions the reliability of experimental risk elicitation measures and the ability to use results from such methods to control for the risk aversion of subjects when explaining e�ects in other experimental games.

Arts-health intersections : a model of practice for consistency in the arts-health sector

The field of Arts-Health practice and research has grown exponentially in the past 30 years. While researchers are using applied arts as the subject of investigation in research, the evaluation of practice and participant benefits has a limited general focus. In recent years, the field has witnessed a growing concentration on the evaluation of health outcomes, outputs and tangential benefits for participants engaging in Arts-Health practice. The wide range of methodological approaches applied arts practitioners implement make the field difficult to define. This article introduces the term Arts-Health intersections as a model of practice and framework to promote consistency in design, implementation and evaluative processes in applied arts programmes promoting health outcomes. The article challenges the current trend to solely evaluate health outcomes in the field, and promotes a concurrent and multidisciplinary methodological approach that can be adopted to promote evaluation, consistency and best practice in the field of Arts-Health intersections. The article provides a theoretical overview of Arts-Health intersections, and then takes this theoretical platform and details a best model of practice for developing Arts-Health intersections and presents this model as a guide.

Validation and consistency in the application of a trauma team activation system at a level 1 trauma centre

- The RAH was activated over 2500 trauma calls in 2009. This figure is over twice the number of calls put out by similar services. - Many trauma calls (in particular L2 trauma calls) from the existing system do not warrant activation of the trauma team - Sometimes trauma calls are activated for nontrauma reasons (eg rapid access to radiology, departmental pressures etc) - The excess of trauma calls has several deleterious effects particularly on time management for the trauma service staff: ward rounds/tertiary survey rounds, education, quality improvement, research

The consistency of crash involvement recall across time

Introduction Road safety researchers rely heavily on self-report data to explore the aetiology of crash risk. However, researchers consistently acknowledge a range of limitations associated with this methodological approach (e.g., self-report bias), which has been hypothesised to reduce the predictive efficacy of scales. Although well researched in other areas, one important factor often neglected in road safety studies is the fallibility of human memory. Given accurate recall is a key assumption in many studies, the validity and consistency of self-report data warrants investigation. The aim of the current study was to examine the consistency of self-report data of crash history and details of the most recent reported crash on two separate occasions. Materials & Method A repeated measures design was utilised to examine the self-reported crash involvement history of 214 general motorists over a two month period. Results A number of interesting discrepancies were noted in relation to number of lifetime crashes reported by the participants and the descriptions of their most recent crash across the two occasions. Of the 214 participants who reported having been involved in a crash, 35 (22.3%) reported a lower number of lifetime crashes as Time 2, than at Time 1. Of the 88 drivers who reported no change in number of lifetime crashes, 10 (11.4%) described a different most recent crash. Additionally, of the 34 reporting an increase in the number of lifetime crashes, 29 (85.3%) of these described the same crash on both occasions. Assessed as a whole, at least 47.1% of participants made a confirmed mistake at Time 1 or Time 2. Conclusions These results raise some doubt in regard to the accuracy of memory recall across time. Given that self-reported crash involvement is the predominant dependent variable used in the majority of road safety research, this issue warrants further investigation. Replication of the study with a larger sample size that includes multiple recall periods would enhance understanding into the significance of this issue for road safety methodology.

Within-subject intra- and inter-method consistency of two experimental risk attitude elicitation methods

We compare the consistency of choices in two methods used to elicit risk preferences on an aggregate as well as on an individual level. We ask subjects to choose twice from a list of nine decisions between two lotteries, as introduced by Holt and Laury (2002, 2005) alternating with nine decisions using the budget approach introduced by Andreoni and Harbaugh (2009). We find that, while on an aggregate (subject pool) level the results are consistent, on an individual (within-subject) level, behaviour is far from consistent. Within each method as well as across methods we observe low (simple and rank) correlations.

‘Structuring’ judicial sentencing discretion : consistency, guidance or pandering to the punitive?

Until quite recently, most Australian jurisdictions gave statutory force to the principle of imprisonment as a sanction of last resort, reflecting its status as the most punitive sentencing option open to the court.1 That principle gave primary discretion as to whether incarceration was the most appropriate means of achieving the purpose of a sentence to the sentencing court, which received all of the information relevant to the offence, the offender and any victim(s). The disestablishment of this principle is symptomatic of an increasing erosion of judicial discretion with respect to sentencing, which appears to be resulting in some extremely punitive consequences.

Behavioral consistency, the homology assumption, and the problems of induction

The ultimate goal of profiling is to identify the major behavioral and personality characteristics to narrow the suspect pool. Inferences about offender characteristics can be accomplished deductively, based on the analysis of discrete offender behaviors established within a particular case. They can also be accomplished inductively, involving prediction based on abstract offender averages from group data (these methods and the logic on which they are based is detailed extensively in Chapters 2 and 4). As discussed, these two approaches are by no means equal.