Student self-assessment of performance : a useful learning tool for students and instructors?

Assurance of Learning is a focus for many Business Schools to meet AACSB requirements. Prior research has examined student self-assessment as a measure of learning in this context, but we propose instead that this is a useful ‘learning tool’ for students and instructors that can help improve student performance and curricula. We examined self-assessment in a third year undergraduate e-marketing course and found a positive, though weak relationship between student self-assessment and instructor ratings on intended learning goals and criteria for a written assignment. Students who self-assessed tended to perform better in the assessment item and course than those who did not self-assess. But less competent students were more inaccurate in their self-assessment, suggesting a need to enhance understanding of marking criteria and performance standards within this group. Implications for educators are discussed.

To what extent is the Mayer and Salovey (1997) model of emotional intelligence a useful predictor of leadership style and perceived leadership outcomes in Australian educational institutions?

Researchers have found that transformational leadership is related to positive outcomes in educational institutions. Hence, it is important to explore constructs that may predict leadership style in order to identify potential transformational leaders in assessment and selection procedures. Several studies in non-educational settings have found that emotional intelligence is a useful predictor of transformational leadership, but these studies have generally lacked methodological rigor and contextual relevance. This project, set in Australian educational institutions, employed a more rigorous methodology to answer the question: to what extent is the Mayer and Salovey (1997) model of emotional intelligence a useful predictor of leadership style and perceived leadership outcomes? The project was designed to move research in the field forward by using valid and reliable instruments, controlling for other predictors, obtaining an adequately sized sample of current leaders and collecting multiple ratings of their leadership behaviours. The study (N = 144 leaders and 432 raters) results indicated that emotional intelligence was not a useful predictor of leadership style and perceived leadership outcomes. In contrast, several of the other predictors in the study were found to predict leadership style.

Moderate sleep restriction in treated older male OSA participants: Greater impairment during monotonous driving compared with controls

Objectives To examine the effects on monotonous driving of normal sleep versus one night of sleep restriction in continuous positive airway pressure (CPAP) treated obstructive sleep apnoea (OSA) patients compared with age matched healthy controls. Methods Nineteen CPAP treated compliant male OSA patients (OSA-treated patients (OPs)), aged 50–75 years, and 20 healthy age-matched controls underwent both a normal night’s sleep and sleep restriction to 5 h (OPs remained on CPAP) in a counterbalanced design. All participants completed a 2 h afternoon monotonous drive in a realistic car simulator. Driving was monitored for sleepiness-related minor and major lane deviations, with ‘safe’ driving time being total time driven prior to first major lane deviation. EEGs were recorded continuously, and subjective sleepiness ratings were taken at regular intervals throughout the drive. Results After a normal night’s sleep, OPs and controls did not differ in terms of driving performance or in their ability to assess the levels of their own sleepiness, with both groups driving ‘safely’ for approximately 90 min. However, after sleep restriction, OPs had a significantly shorter (65 min) safe driving time and had to apply more compensatory effort to maintain their alertness compared with controls. They also underestimated the enhanced sleepiness. Nevertheless, apart from this caveat, there were generally close associations between subjective sleepiness, likelihood of a major lane deviation and EEG changes indicative of sleepiness. Conclusions With a normal night’s sleep, effectively treated older men with OSA drive as safely as healthy men of the same age. However, after restricted sleep, driving impairment is worse than that of controls. This suggests that, although successful CPAP treatment can alleviate potential detrimental effects of OSA on monotonous driving following normal sleep, these patients remain more vulnerable to sleep restriction.

Driver sleepiness—Comparisons between young and older men during a monotonous afternoon simulated drive

Young men figure prominently in sleep-related road crashes. Non-driving studies show them to be particularly vulnerable to sleep loss, compared with older men. We assessed the effect of a normal night's sleep vs. prior sleep restricted to 5 h, in a counterbalanced design, on prolonged (2 h) afternoon simulated driving in 20 younger (av. 23 y) and 19 older (av. 67 y) healthy men. Driving was monitored for sleepiness related lane deviations, EEGs were recorded continuously and subjective ratings of sleepiness taken every 200 s. Following normal sleep there were no differences between groups for any measure. After sleep restriction younger drivers showed significantly more sleepiness-related deviations and greater 4–11 Hz EEG power, indicative of sleepiness. There was a near significant increase in subjective sleepiness. Correlations between the EEG and subjective measures were highly significant for both groups, indicating good self-insight into increasing sleepiness. We confirm the greater vulnerability of younger drivers to sleep loss under prolonged afternoon driving.

Performance of a cognitive, but not visual, secondary task interacts with alcohol-induced balance impairment in novice and experienced motorcycle riders

The appropriateness of applying drink driving legislation to motorcycle riding has been questioned as there may be fundamental differences in the effects of alcohol on driving and motorcycling. It has been suggested that alcohol may redirect riders’ focus from higher-order cognitive skills such as cornering, judgement and hazard perception, to more physical skills such as maintaining balance. To test this hypothesis, the effects of low doses of alcohol on balance ability were investigated in a laboratory setting. The static balance of twenty experienced and twenty novice riders was measured while they performed either no secondary task, a visual (search) task, or a cognitive (arithmetic) task following the administration of alcohol (0%, 0.02%, and 0.05% BAC). Subjective ratings of intoxication and balance impairment increased in a dose-dependent manner in both novice and experienced motorcycle riders, while a BAC of 0.05%, but not 0.02%, was associated with impairments in static balance ability. This balance impairment was exacerbated when riders performed a cognitive, but not a visual, secondary task. Likewise, 0.05% BAC was associated with impairments in novice and experienced riders’ performance of a cognitive, but not a visual, secondary task, suggesting that interactive processes underlie balance and cognitive task performance. There were no observed differences between novice vs. experienced riders on static balance and secondary task performance, either alone or in combination. Implications for road safety and future ‘drink riding’ policy considerations are discussed.

Drinking and riding: is subjective workload related to performance?

Introduction: Within the context of road safety it is important that workload (the portion of a driver’s resources expended to perform a task) remains at a manageable level, preventing overloading and consequently performance decrements. Motorcyclists are over represented in crash statistics where the vehicle operator has a positive, low blood alcohol concentration (BAC) (e.g., 0.05%). The NASA task load index (NASA-TLX) comprises sub-scales that purportedly assess different aspects of subjective workload. It was hypothesized that, compared to a zero BAC condition, low BACs would be associated with increases in workload ratings, and decrements in riding performance. Method: Forty participants (20 novice, 20 experienced) completed simulated motorcycle rides in urban and rural scenarios under low dose BAC conditions (0.00%, 0.02%, 0.05% BAC), while completing a safety relevant peripheral detection task (PDT). Six sub-scales of the NASA-TLX were completed after each ride. Riding performance was assessed using standard deviation of lateral position (SDLP). Hazard perception was assessed by response time to the PDT. Results: Riding performance and hazard perception were affected by alcohol. There was a significant increase in SDLP in the urban scenario and of PDT reaction time in the rural scenario under 0.05% BAC compared to 0.00% BAC. Overall NASA-TLX score increased at 0.02% and 0.05% BAC in the urban environment only, with a trend for novices to rate workload higher than experienced riders. There was a significant main effect of sub-scale on workload ratings in both the urban and rural scenarios. Discussion: 0.05% BAC was associated with decrements in riding performance in the urban environment, decrements in hazard perception in the rural environment, and increases in overall ratings of subjective workload in the urban environment. The workload sub-scales of the NASA-TLX appear to be measuring distinct aspects of motorcycle riding-related workload. Issues of workload and alcohol impaired riding performance are discussed.

Do drinking and riding mix? Effects of legal doses of alcohol on balance ability in experienced motorcyclists

The appropriateness of applying drink driving legislation to motorcycle riding has been questioned as there may be fundamental differences in the effects of alcohol on these two activities. For example, while the distribution of blood alcohol content (BAC) levels among fatally injured male drivers compared to riders is similar, a greater proportion of motorcycle fatalities involve levels in the lower (0 to .10% BAC) range. Several psychomotor and higher-order cognitive skills underpinning riding performance appear to be significantly influenced by low levels of alcohol. For example, at low levels (.02 to .046% BAC), riders show significant increases in reaction time to hazardous stimuli, inattention to the riding task, performance errors such as leaving the roadway and a reduced ability to complete a timed course. It has been suggested that alcohol may redirect riders’ focus from higher-order cognitive skills to more physical skills such as maintaining balance. As part of a research program to investigate the potential benefits of introducing a zero, or reduced, BAC for all riders in Queensland regardless of their licence status, the effects of low doses of alcohol on balance ability were investigated in a laboratory setting. The static balance of ten experienced riders was measured while they performed either no secondary task, a visual search task, or a cognitive (arithmetic) task following the administration of alcohol (0; 0.02, and 0.05% BAC). Subjective ratings of intoxication and balance impairment increased in a dose-dependent manner; however, objective measures of static balance were negatively affected only at the .05% BAC dose. Performance on a concurrent secondary visual search task, but not a purely cognitive (arithmetic) task, improved postural stability across all BAC levels. Finally, the .05% BAC dose was associated with impaired performance on the cognitive (arithmetic) task, but not the visual search task, when participants were balancing, but neither task was impaired by alcohol when participants were standing on the floor. Implications for road safety and future ‘drink riding’ policy considerations are discussed.

Research capacity and culture in podiatry : early observations within Queensland Health

Background Research is a major driver of health care improvement and evidence-based practice is becoming the foundation of health care delivery. For health professions to develop within emerging models of health care delivery, it would seem imperative to develop and monitor the research capacity and evidence-based literacy of the health care workforce. This observational paper aims to report the research capacity levels of statewide populations of public-sector podiatrists at two different time points twelve-months apart. Methods The Research Capacity & Culture (RCC) survey was electronically distributed to all Queensland Health (Australia) employed podiatrists in January 2011 (n = 58) and January 2012 (n = 60). The RCC is a validated tool designed to measure indicators of research skill in health professionals. Participants rate skill levels against each individual, team and organisation statement on a 10-point scale (one = lowest, ten = highest). Chi-squared and Mann Whitney U tests were used to determine any differences between the results of the two survey samples. A minimum significance of p < 0.05 was used throughout. Results Thirty-seven (64%) podiatrists responded to the 2011 survey and 33 (55%) the 2012 survey. The 2011 survey respondents reported low skill levels (Median < 4) on most aspects of individual research aspects, except for their ability to locate and critically review research literature (Median > 6). Whereas, most reported their organisation’s skills to perform and support research at much higher levels (Median > 6). The 2012 survey respondents reported significantly higher skill ratings compared to the 2011 survey in individuals’ ability to secure research funding, submit ethics applications, and provide research advice, plus, in their organisation’s skills to support, fund, monitor, mentor and engage universities to partner their research (p < 0.05). Conclusions This study appears to report the research capacity levels of the largest populations of podiatrists published. The 2011 survey findings indicate podiatrists have similarly low research capacity skill levels to those reported in the allied health literature. The 2012 survey, compared to the 2011 survey, suggests podiatrists perceived higher skills and support to initiate research in 2012. This improvement coincided with the implementation of research capacity building strategies.

Rating defense mega-project success : the role of personal attributes and stakeholder relationships

In this paper, we develop and test a model of the relationships between mega-project leaders’ personal attributes and their ratings of project success in the context of the Australian defense industry. In our model, emotional intelligence, cognitive flexibility, and systemic thinking were hypothesized to be related to project success ratings, mediated by internal and external stakeholder relationships. We tested the model in an online survey study of 373 mega-project leaders. Results were that emotional intelligence and cognitive flexibility were found to be related to the quality of mega-project leader’s relationships with both internal and external stakeholders; and that these relationships in turn were found to be associated with the project leaders’ ratings of project success. We found however that systemic thinking had no relationship with either stakeholder relationships or ratings of project success. We conclude with a discussion of the implications of these findings and make recommendations for future research. In particular, additional research is needed to examine the contribution of a wider range of competencies on stakeholder relationships and project success in mega-projects, and that there is also a need for research in future to attempt to use more objective ratings of project success.

How to meet stakeholders' expectations, not manage them?

Evaluation practices in the higher education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology began a project, entitled REFRAME, to review its approach to evaluation, particularly the student survey system it had been using for the past five years. This presentation will outline the scholarly, evidence based methodology used to undertake institution-wide change, meet the needs of stakeholders suitable to the cultural needs of the institution. It is believed that this approach is broadly applicable to other institutions contemplating change with regard to evaluation of learning and teaching.

REFRAME: a new approach to evaluation in higher education

Evaluation practices in the higher education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology began a project, entitled REFRAME, to review its approach to evaluation, particularly the student survey system it had been using for the past five years. This presentation will outline the scholarly, evidence based methodology used to undertake institution-wide change, meet the needs of stakeholders suitable to the cultural needs of the institution. It is believed that this approach is broadly applicable to other institutions contemplating change with regard to evaluation of learning and teaching.

Characteristics of on-road driving performance of persons with central vision loss who use bioptic telescopes

Purpose. To compare the on-road driving performance of visually impaired drivers using bioptic telescopes with age-matched controls. Methods. Participants included 23 persons (mean age = 33 ± 12 years) with visual acuity of 20/63 to 20/200 who were legally licensed to drive through a state bioptic driving program, and 23 visually normal age-matched controls (mean age = 33 ± 12 years). On-road driving was assessed in an instrumented dual-brake vehicle along 14.6 miles of city, suburban, and controlled-access highways. Two backseat evaluators independently rated driving performance using a standardized scoring system. Vehicle control was assessed through vehicle instrumentation and video recordings used to evaluate head movements, lane-keeping, pedestrian detection, and frequency of bioptic telescope use. Results. Ninety-six percent (22/23) of bioptic drivers and 100% (23/23) of controls were rated as safe to drive by the evaluators. There were no group differences for pedestrian detection, or ratings for scanning, speed, gap judgments, braking, indicator use, or obeying signs/signals. Bioptic drivers received worse ratings than controls for lane position and steering steadiness and had lower rates of correct sign and traffic signal recognition. Bioptic drivers made significantly more right head movements, drove more often over the right-hand lane marking, and exhibited more sudden braking than controls. Conclusions. Drivers with central vision loss who are licensed to drive through a bioptic driving program can display proficient on-road driving skills. This raises questions regarding the validity of denying such drivers a license without the opportunity to train with a bioptic telescope and undergo on-road evaluation.

Rating defence major project success : the role of personal attributes and stakeholder relationships

In this paper we develop and test a model of the associations between major project managers' personal attributes and project success in the context of the Australian Defence industry. In our model, emotional intelligence, cognitive flexibility and systemic thinking were hypothesised to relate to project success, mediated by internal and external stakeholder relationships. The model was tested in an online survey with 373 major project managers. Emotional intelligence and cognitive flexibility were found to be related to the development, quality and effectiveness of major project managers' relationships with both internal and external stakeholders; and these in turn were associated with their ratings of project success. Systemic thinking, however, had no relationship with either stakeholder relationships or project success. Additional research is needed to examine the contribution of a wider range of personal attributes to stakeholder relationships and project success, and to assess whether this model is applicable in other industries and types of projects.

The inaccuracy of national character stereotypes

Consensual stereotypes of some groups are relatively accurate, whereas others are not. Previous work suggesting that national character stereotypes are inaccurate has been criticized on several grounds. In this article we (a) provide arguments for the validity of assessed national mean trait levels as criteria for evaluating stereotype accuracy and (b) report new data on national character in 26 cultures from descriptions (N= 3323) of the typical male or female adolescent, adult, or old person in each. The average ratings were internally consistent and converged with independent stereotypes of the typical culture member, but were weakly related to objective assessments of personality. We argue that this conclusion is consistent with the broader literature on the inaccuracy of national character stereotypes

Monitoring policy and actions on food environments : rationale and outline of the INFORMAS policy engagement and communication strategies

The International Network for Food and Obesity/non-communicable diseases Research, Monitoring and Action Support (INFORMAS) proposes to collect performance indicators on food policies, actions and environments related to obesity and non-communicable diseases. This paper reviews existing communications strategies used for performance indicators and proposes the approach to be taken for INFORMAS. Twenty-seven scoring and rating tools were identified in various fields of public health including alcohol, tobacco, physical activity, infant feeding and food environments. These were compared based on the types of indicators used and how they were quantified, scoring methods, presentation and the communication and reporting strategies used. There are several implications of these analyses for INFORMAS: the ratings/benchmarking approach is very commonly used, presumably because it is an effective way to communicate progress and stimulate action, although this has not been formally evaluated; the tools used must be trustworthy, pragmatic and policy-relevant; multiple channels of communication will be needed; communications need to be tailored and targeted to decision-makers; data and methods should be freely accessible. The proposed communications strategy for INFORMAS has been built around these lessons to ensure that INFORMAS's outputs have the greatest chance of being used to improve food environments.