Reframing evaluation of learning and teaching: An approach to change

Evaluation practices in the Higher Education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology (QUT) began a project which aims to reframe the organisation’s approach to the evaluation of learning and teaching. This paper describes the existing evaluation system; the emergence and early development of the project; and formulation of a conceptual framework identifying key dimensions of evaluation. It then compares the draft framework with other conceptualisations and models of evaluation identified in the literature, to determine its validity and suitability for supporting QUT’s plans for the future. Overall, the paper represents a structured evaluation of the REFRAME project at a particular point in its lifecycle. Given that the project follows an evidence based, practice-led process and applies an ongoing action research cycle, the findings are presented in the belief that QUT’s experience is broadly applicable to other institutions which may be contemplating change in relation to evaluation of learning and teaching.

Rating the Audience : the Business of Media

Knowing, measuring and understanding media audiences have become a multi-billion dollar business. But the convention that underpins that business, audience ratings, is in crisis. Rating the Audience is the first book to show why and how audience ratings research became a convention, an agreement, and the first to interrogate the ways that agreement is now under threat. Taking a historical approach, the book looks at the evolution of audience ratings and the survey industry. It goes on to analyse today's media environment, looking at the role of the internet and the increased difficulties it presents for measuring audiences. The book covers all the major players and controversies, such as Facebook's privacy rulings and Google's alliance with Nielsen. Offering the first real comparative study, it will be critical for media students and professionals.

Effects of acute multi-nutrient supplementation on rugby union match performance and recovery

Aim: To determine the effects of an acute multi-nutrient supplement on physiological, performance and recovery responses to intermittent-sprint running and muscular damage during rugby union matches. Methods: Using a randomised, double-blind, cross-over design, twelve male rugby union players ingested either 75 g of a comprehensive multi-nutrient supplement (SUPP), [Musashi] or 1 g of a taste and carbohydrate matched placebo (PL) for 5 days pre-competition. Competitive rugby union game running performance was then measured using 1 Hz GPS data (SPI10, SPI elite, GPSports), in addition to associated blood draws, vertical jump assessments and ratings of perceived muscular soreness (MS) pre, immediately post and 24 h post-competition. Baseline (BL) GPS data was collected during six competition rounds preceding data collection. Results: No significant differences were observed between supplement conditions for all game running, vertical jump, and ratings of perceived muscular soreness. However, effect size analysis indicated SUPP ingestion increased 1st half very high intensity running (VHIR) mean speed (d = 0.93) and 2nd half relative distance (m/min) (d = 0.97). Further, moderate increases in 2nd half VHIR distance (d = 0.73), VHIR m/min (d = 0.70) and VHIR mean speed (d = 0.56) in SUPP condition were also apparent. Moreover, SUPP demonstrated significant increases in 2nd half dist m/min, total game dist m/min and total game HIR m/min compared with BL data (P < 0.05). Further, large ES increases in VHIR time (d = 0.88) and moderate increases in 2nd half HIR m/min (d = 0.65) and 2nd half VHIR m/min (d = 0.74) were observed between SUPP and BL. Post-game aspartate aminotransferase (AST) (d = 1.16) and creatine kinase (CK) (d = 0.97) measures demonstrated increased ES values with SUPP, while AST and CK values correlated with 2nd half VHIR distance (r = −0.71 and r = −0.76 respectively). Elevated c-reactive protein (CRP) was observed post-game in both conditions, however was significantly blunted with SUPP (P = 0.05). Additionally, pre-game (d = 0.98) and post-game (d = 0.96) increases in cortisol (CORT) were apparent with SUPP. No differences were apparent between conditions for pH, lactate, glucose, HCO3, vertical jump assessments and MS (P > 0.05). Conclusion: These findings suggest SUPP may assist in the maintenance of VHIR speeds and distances covered during rugby union games, possibly via the buffering qualities of SUPP ingredients (i.e. caffeine, creatine, bicarbonate). While the mechanisms for these findings are unclear, the similar pH between conditions despite additional VHIR during SUPP may support this conclusion. Finally, correlations between increased work completed at very high intensities and muscular degradation in SUPP conditions, may mask any anti-catabolic properties of supplementation.

Minimum important difference between patients with rheumatoid arthritis : the patient's perspective

OBJECTIVE: To determine the point at which differences in clinical assessment scores on physical ability, pain and overall condition are sufficiently large to correspond to a subjective perception of a meaningful difference from the perspective of the patient. METHODS: Forty patients with a diagnosis of rheumatoid arthritis participated in an evening of clinical assessment and one-on-one conversations with each other regarding their arthritic condition. The assessments included tender and swollen joint counts, clinician and patient global assessments, participant assessment of pain and the Health Assessment Questionnaire (HAQ) on physical ability. After each conversation, participants rated themselves relative to their conversational partner on physical ability, pain and overall condition. These subjective comparative ratings were compared to the differences of the individual clinical assessments. RESULTS: In total there were 120 conversations. Generally participants judged themselves as less disabled than others. They rated themselves as "somewhat better" than their conversation partner when they had a (mean) 7% better score on the HAQ, 6% less pain, and 9% better global assessment. In contrast, they rated themselves as "somewhat worse" when they had a (mean) 16% worse score on the HAQ, 16% more pain, and 29% worse global assessment. CONCLUSIONS: Patients view clinically important differences in an asymmetric manner. These results can provide guidance in interpreting results and planning clinical trials.

Concurrent and longitudinal links between children’s and their friends’ antisocial and prosocial behavior in preschool

Concurrent and longitudinal links between children’s own and their nominated best friends’ antisocial and prosocial behavior were studied in a normative sample of 3–5-year-olds (N = 203). Moderating effects of age and gender were also explored. Subscales of the Strength and Difficulties Questionnaire (SDQ) were used to obtain teacher ratings of behavior for each target child and his/her nominated best friends. Nomination of best friends with higher levels of antisocial behavior and lower levels of prosocial behavior was concurrently linked to more antisocial behavior in boys. Nomination of highly prosocial best friends was concurrently linked to more prosocial behavior in both boys and girls. However, the study found no longitudinal effects of best friends’ behavior on target child’s behavior over a one-year period. A group of children who nominated no best friends at T1 were generally perceived as less prosocial, but not more antisocial, than other children. © 2011 Elsevier Inc. All rights reserved.

Intercultural competence in engineering education : who are we teaching?

BACKGROUND There is little doubt that our engineering graduates’ ability to identify cultural differences and their potential to impact on engineering projects, and to work effectively with these differences is of key importance in the modern engineering practice. Within engineering degree programs themselves there is also a significant need to recognise the impact of changing student and staff profiles on what happens in the classroom. The research described in this paper forms part of a larger project exploring issues of intercultural competence in engineering. PURPOSE This paper presents an observational and survey study of undergraduate and postgraduate engineering students from four institutions working in groups on tasks with a purely technical focus, or with a cultural and humanitarian element. The study sought to explore how students rate their own intercultural competence and team process and whether any differences exist depending on the nature of the task they are working on. We also investigated whether any differences were evident between groups of first year, second year and postgraduate students. DESIGN/METHOD The study used the miniCQS instrument (Ang & Van Dyne, 2008) and a Bales Interaction Process Analysis based scale (Bales, 1950; Carney, 1976) to collect students self ratings of group process, task management, and cultural experience and behaviour. The Bales IPA was also used for coding video observations of students working in groups. Survey data were used to form descriptive variables to compare outcomes across the different tasks and contexts. Observations analysed in Nvivo were used to provide commentary and additional detail on the quantitative data. RESULTS The results of the survey indicated consistent mean scores on each survey item for each group of students, despite vastly different tasks, student backgrounds and educational contexts. Some small, statistically significant mean differences existed, offering some basic insights into how task and student group composition could affect self ratings. Overall though, the results suggest minimal shift in how students view group function and their intercultural experience, irrespective of differing educational experience. CONCLUSIONS The survey results, contrasted with group observations, indicate that either students are not translating their experience (in the group tasks) into critical self assessment of their cultural competence and teamwork, or that they become more critical of team performance and cultural competence as their competence in these areas grows, so their ratings remain consistent. Both outcomes indicate that students need more intensive guidance to build their critical self and peer assessment skills in these areas irrespective of their year level of study.

From tiers to tables – enhancing student experience through collaborative learning spaces

BACKGROUND Collaborative and active learning have been clearly identified as ways students can engage in learning with each other and the academic staff. Traditional tier based lecture theatres and the didactic style they engender are not popular with students today as evidenced by the low attendance rates for lectures. Many universities are installing spaces designed with tables for group interaction with evolutions on spaces such as the TEAL (Technology Enabled Active Learning) (Massachusetts Institute of Technology, n.d.) and SCALE-UP (Student-Centred Activities for Large-Enrolment Undergraduate Programs) (North Carolina State University, n.d.) models. Technology advances in large screen computers and applications have also aided the move to these collaborative spaces. How well have universities structured learning using these spaces and how have students engaged with the content, technology, space and each other? This paper investigates the application of collaborative learning in such spaces for a cohort of 800+ first year engineers in the context of learning about and developing professional skills representative of engineering practice. PURPOSE To determine whether moving from tiers to tables enhances the student experience. Does utilising technology rich, activity based, collaborative learning spaces lead to positive experiences and active engagement of first year undergraduate engineering students? In developing learning methodology and approach in new learning spaces, what needs to change from a more traditional lecture and tutorial configuration? DESIGN/METHOD A post delivery review and analysis of outcomes was undertaken to determine how well students and tutors engaged with learning in new collaborative learning spaces. Data was gathered via focus group and survey of tutors, students survey and attendance observations. The authors considered the unit delivery approach along with observed and surveyed outcomes then conducted further review to produce the reported results. RESULTS Results indicate high participation in the collaborative sessions while the accompanying lectures were poorly attended. Students reported a high degree of satisfaction with the learning experience; however more investigation is required to determine the degree of improvement in retained learning outcomes. Survey feedback from tutors found that students engaged well in the activities during tutorials and there was an observed improvement in the quality of professional practice modelled by students during sessions. Student feedback confirmed the positive experiences in these collaborative learning spaces with 30% improvement in satisfaction ratings from previous years. CONCLUSIONS It is concluded that the right mix of space, technology and appropriate activities does engage students, improve participation and create a rich experience to facilitate potential for improved learning outcomes. The new Collaborative Teaching Spaces, together with integrated technology and tailored activities, has transformed the delivery of this unit and improved student satisfaction in tutorials significantly.

Risks of Carbon Fraud

Carbon credit markets are in the early stages of development and media headlines such as these illustrate emerging levels of concern and foreboding over the potential for fraudulent crime within these markets. Australian companies are continuing to venture into the largely unregulated voluntary carbon credit market to offset their emissions and / or give their customers the opportunity to be ‘carbon neutral’. Accordingly, the voluntary market has seen a proliferation of carbon brokers that offer tailored offset carbon products according to need and taste. With the instigation of the Australian compliance market and with pressure increasing for political responses to combat climate change, we would expect Australian companies to experience greater exposure to carbon products in both compliance and voluntary markets. This paper examines the risks of carbon fraud in these markets by reviewing cases of actual fraud and analysing and identifying contexts where risks of carbon fraud are most likely.

Sequential slotted amplify-and-forward with partial relay isolation

In this paper, we propose a novel relay ordering and scheduling strategy for the sequential slotted amplify-and-forward (SAF) protocol and evaluate its performance in terms of diversity-multiplexing trade-off (DMT). The relays between the source and destination are grouped into two relay clusters based on their respective locations. The proposed strategy achieves partial relay isolation and decreases the decoding complexity at the destination. We show that the DMT upper bound of sequential-SAF with the proposed strategy outperforms other amplify and forward protocols and is more practical compared to the relay isolation assumption made in the original paper [1]. Simulation result shows that the sequential-SAF protocol with the proposed strategy has better outage performance compared to the existing AF and non-cooperative protocols in high SNR regime.

Sequential slotted amplify-decode-and-forward

In this paper, we propose a novel slotted hybrid cooperative protocol named the sequential slotted amplify-decodeand-forward (SADF) protocol and evaluate its performance in terms of diversity-multiplexing trade-off (DMT). The relays between the source and destination are divided into two different groups and each relay either amplifies or decodes the received signal. We first compute the optimal DMT of the proposed protocol with the assumption of perfect decoding at the DF relays. We then derive the DMT closed-form expression of the proposed sequential-SADF and obtain the proximity gain bound for achieving the optimal DMT. With the proximity gain bound, we then found the distance ratio to achieve the optimal DMT performance. Simulation result shows that the proposed protocol with high proximity gain outperforms other cooperative communication protocols in high SNR regime.

Anomalous event detection using a semi-two dimensional hidden Markov model

The rapid increase in the deployment of CCTV systems has led to a greater demand for algorithms that are able to process incoming video feeds. These algorithms are designed to extract information of interest for human operators. During the past several years, there has been a large effort to detect abnormal activities through computer vision techniques. Typically, the problem is formulated as a novelty detection task where the system is trained on normal data and is required to detect events which do not fit the learned `normal' model. Many researchers have tried various sets of features to train different learning models to detect abnormal behaviour in video footage. In this work we propose using a Semi-2D Hidden Markov Model (HMM) to model the normal activities of people. The outliers of the model with insufficient likelihood are identified as abnormal activities. Our Semi-2D HMM is designed to model both the temporal and spatial causalities of the crowd behaviour by assuming the current state of the Hidden Markov Model depends not only on the previous state in the temporal direction, but also on the previous states of the adjacent spatial locations. Two different HMMs are trained to model both the vertical and horizontal spatial causal information. Location features, flow features and optical flow textures are used as the features for the model. The proposed approach is evaluated using the publicly available UCSD datasets and we demonstrate improved performance compared to other state of the art methods.

Spatio temporal feature evaluation for action recognition

Spatio-Temporal interest points are the most popular feature representation in the field of action recognition. A variety of methods have been proposed to detect and describe local patches in video with several techniques reporting state of the art performance for action recognition. However, the reported results are obtained under different experimental settings with different datasets, making it difficult to compare the various approaches. As a result of this, we seek to comprehensively evaluate state of the art spatio- temporal features under a common evaluation framework with popular benchmark datasets (KTH, Weizmann) and more challenging datasets such as Hollywood2. The purpose of this work is to provide guidance for researchers, when selecting features for different applications with different environmental conditions. In this work we evaluate four popular descriptors (HOG, HOF, HOG/HOF, HOG3D) using a popular bag of visual features representation, and Support Vector Machines (SVM)for classification. Moreover, we provide an in-depth analysis of local feature descriptors and optimize the codebook sizes for different datasets with different descriptors. In this paper, we demonstrate that motion based features offer better performance than those that rely solely on spatial information, while features that combine both types of data are more consistent across a variety of conditions, but typically require a larger codebook for optimal performance.

Intratherapist reliability in the rating of scapula posture in multiple planes of reference

Background: Evaluation of scapular posture is a fundamental component in the clinical evaluation of the upper quadrant. This study examined the intrarater reliability of scapular posture ratings. Methods: A test-retest reliability investigation was undertaken with one week between assessment sessions. At each session physical therapists conducted visual assessments of scapula posture (relative to the thorax) in five different scapula postural planes (plane of scapula, sagittal plane, transverse plane, horizontal plane, and vertical plane). These five plane ratings were performed for four different scapular posture perturbating conditions (rest, isometric shoulder; flexion, abduction, and external rotation). Results. A total of 100 complete scapular posture ratings (50 left, 50 right) were undertaken at each assessment. The observed agreement between the test and retest postural plane ratings ranged from 59% to 87%; 16 of the 20 plane-condition combinations exceeded 75% observed agreement. Kappa (and prevalence adjusted bias adjusted kappa) values were inconsistent across the postural planes and perturbating conditions. Conclusions: This investigation generally revealed fair to moderate intrarater reliability in the rating of scapular posture by visual inspection. However, enough disagreement between assessments was present to warrant caution when interpreting perceived changes in scapula position between longitudinal assessments using visual inspection alone.

Action research and unit improvement in higher education : a case study approach

Student satisfaction data has been collected on a national basis in Australia since 1972. In recent years this data has been used by federal government agencies to allocate funding, and by students in selecting their universities of choice. The purpose of this paper is to present the findings of an action research project designed to identify and implement unit improvement initiatives over a three year period for an underperforming unit. This research utilises student survey data and teacher reflections to identify areas of unit improvement, with a view to aligning learning experiences, teaching and assessment to learning outcomes and improved student satisfaction. This research concludes that whilst a voluntary student survey system may be imperfect, it nevertheless provides important data that can be utilised to the benefit of the unit, learning outcomes and student satisfaction ratings, as well as wider course related outcomes. Extrapolation of these findings is recommended to other underperforming units.

TS-fuzzy controlled DFIG based wind energy conversion system

This paper focuses on the implementation of the TS (Tagaki-Sugino) fuzzy controller for the active power and the DC capacitor voltage control of the Doubly Fed Induction Generator (DFIG) based wind generator. DFIG system is represented by a third-order model where electromagnetic transients of the stator are neglected. The effectiveness of the TS-fuzzy controller on the rotor speed oscillations and the DC capacitor voltage variations of the DFIG damping controller on converter ratings of the DFIG system is also investigated. The results of the time domain simulation studies are presented to elucidate the effectiveness of the TS-fuzzy controller compared with conventional PI controller in the DFIG system. The proposed TS-fuzzy controller can improve the fault ride through capability of DFIG compared to the conventional PI controller