The impact of increasing course enrolment on student evaluation of teaching in engineering education

Student evaluation of teaching (SET) is important, commonplace and may be used in staff performance management. The SET literature suggests that class size is a negative systematic influence on SET ratings. In this paper we investigate time-series SET data from a large first-year engineering class where a decline in SET ratings was observed over time as course enrolment increased. We observe a negative halo effect of increasing class size on mean SET ratings and conclude that increasing course enrolment leads to a significant reduction in all mean SET ratings, even when the course learning design remains essentially unchanged. We also find an additional differential effect of increasing course enrolment on mean SET ratings. We observe that the marginal reduction in mean SET ratings for each additional student in the course enrolment is greater for those aspects of the student learning experience that are likely to be most directly impacted by increasing class size. We provide implications for practice from these findings.

The systematic influences on student evaluation of teaching in engineering education

BACKGROUND Student evaluation of teaching (SET) has a long history, has grown in prevalence and importance over a period of decades, and is now common-place in many universities internationally. SET data are collected for a range of purposes, including: as diagnostic feedback to improve the quality of teaching and learning; as an input to staff performance management processes and personnel decisions such as promotion for staff; to provide information to prospective students in their selection of courses and programs; and as a source of data for research on teaching. Rovai et al. (2006) report that while SET research provides mixed results, there is evidence that, for course-related factors, smaller classes are rated more favourably than large classes, upper-year-level classes are rated more favourably than lower-year classes, and that there are rating differences between discipline areas. While additional course-related factors are also noted, other reviews of the literature on SET also identify these three factors as commonly reported systematic influences on SET ratings. The School of Engineering at Deakin University in Australia offers undergraduate and postgraduate engineering programs, and these programs are delivered in both on-campus and off-campus modes.PURPOSEThe paper presents a quantitative investigation of SET data for the School of Engineering at Deakin University to identify whether the commonly reported systematic influences on SET ratings of class size and year level are also observed here. The influence of online mode of offer is also explored.DESIGN/METHOD Deakin University’s Student Evaluation of Teaching and Units (SETU) questionnaire is administered to students enrolled in every unit of study every time that unit is offered, unless it is specially exempted. Following data collation, summary results are reported via a public website. The publicly available SETU data for all School of Engineering units of study were collected for a two year period. The collected data were subjected to analysis of variance (ANOVA) analysis to identify any significant systematic influences on mean student SETU ratings.RESULTS SETU data from 100 separate units of study over the two year period were collected, representing 3375 sets of SETU ratings, and covering unit enrolment sizes from 12 to 462 students. Although this was a modest sized investigation, significantly higher mean ratings for some SETU items were observed for units with small enrolments, for postgraduate level units compared to undergraduate level units, and for units offered in conventional mode compared to online mode of offer. The presence of the commonly observed systematic influences on SET ratings was confirmed.CONCLUSIONS While the use of SET data may have originally been primarily for formative purposes to improve teaching and learning, they are also increasingly used for summative judgements of teaching quality and teaching staff performance that have implications for personnel decision making. There may be an acceptance of the need for SET, however there remains no universal consensus as to what constitutes quality in university teaching and learning, and the increasing use of SET for high-stakes decision making puts pressure on institutions to ensure that their SET practices are sound, equitable and defensible.

An evaluation of the role of green marketing and a firm's internal practices for environmental sustainability

The management of a firm's green operations is increasingly important for marketing strategists. The purpose of this study is to investigate the cross-influences of green marketing strategy and the key internal green functional areas in a firm. We use the antecedents of marketing strategy and identify relationships between green marketing strategy and key supporting internal environmental operations of firms with respect to (1) green suppliers, (2) environmental resource management, (3) green research and development, and (4) environmental manufacturing processes and procedures. The statistical techniques of parallel analysis, factor analysis and multiple regressions are used to analyze data collected from 332 firms. The results identify that among the four functional areas of firms adopting green marketing strategy. Two are more likely to influence green marketing strategy – supplier selection and research and development. Implications are discussed. The findings contribute to the theory of green marketing strategy. Future research is recommended.

Evaluation of University of Queensland Library Spaces: three case studies at St Lucia Campus

Introduction: The paper reports on an evaluation study of spaces in the Social Sciences and Humanities (SSHL), Biological Sciences (BSL) and Walter Harrison Law (WHLL) Libraries of the University of Queensland (UQ). The study was part of an evaluation of the quality and patterns of use of spaces in UQ libraries, which aimed to propose recommendations for future improvements and decision-making. This paper presents a study of areas of weakness in existing SSHL spaces, and impacts of the refurbishment of spaces at BSL and WHLL on students’ experiences. The findings evidence a link between students’ learning experiences and the quality of library spaces.Methods: An online survey, “Students’ Experiences and Perceptions of Library Physical Spaces”, was designed to collect data from students. The survey questions addressed seven topics: (1) overall satisfaction with spaces in the UQ Libraries; (2) welcoming nature of the library entry; (3) lighting; (4) acoustics; (5) furniture; (6) wayfinding; and (7) preferences for different space types in the library.Results: 1505 students completed the surveys, with 1098 responses recorded to open-ended questions on why students visited the libraries and the physical characteristics of the spaces that influenced their experiences. Quantitative and qualitative analysis of the data elucidated key design challenges and considerations. In particular, the data suggested that the provision of individual quiet study spaces remains an important role of academic libraries, with Studying Alone identified as the most important reason for student visits.Conclusions: The findings highlight the importance of individual study spaces and propose a number of recommendations in relation to physical space design and management.Relevance: Academic libraries face the challenge to keep pace with change in relation to students’ demographics, pedagogy and technology. In doing so, it is important to have an understanding of the evolving patterns of students’ learning behaviours, space uses and preferences through ongoing evaluation of library spaces.

Patient deterioration education: Evaluation of face-to-face simulation and e-simulation approaches

Background: Simulation-based education is one strategy that may be used to teach nursing students to recognize and manage patient deterioration. Method: Final-year preregistration nursing students (n=97) completed three face-to-face laboratory-based team simulations with a simulated patient (actor) and 330 students individually completed a three-scenario Web-based simulation program: FIRST2ACTWeb™. Results: Both groups achieved moderate performance scores (means: face to face, 49%; e-simulation, 69%). Course evaluations were positive, skill gain showing a greater effect size in the face-to-face program than for e-simulation, and higher satisfaction and more positive appraisal. Conclusion: Face-to-face simulation and e-simulation are effective educational strategies with e-simulation offering greater feasibility. Either strategy is likely to add value to the learning experience.

Development and evaluation of a screening tool to identify people with diabetes at increased risk of medication problems relating to hypoglycaemia and medication non-adherence

Abstract Aims: To develop and evaluate a screening tool to identify people with diabetes at increased risk of medication problems relating to hypoglycaemia and medication non-adherence. Methods: A retrospective audit of attendances at a diabetes outpatient clinic at a public, teaching hospital over a 16-month period was conducted. Logistic regression was undertaken to examine risk factors associated with medication problems relating to hypoglycaemia and medication non-adherence and the most predictive set of factors comprise the Diabetes Medication Risk Screening Tool. Evaluating the tool involved assessing sensitivity and specificity, positive and negative predictive values, cut-off scores, inter-rater reliability, and content validity. Results: The Diabetes Medication Risk Screening Tool comprises seven predictive factors: age, living alone, English language, mental and behavioural problems, comorbidity index score, number of medications prescribed, and number of high-risk medications prescribed. The tool has 76.5% sensitivity, 59.5% specificity, and has a 65.1% positive predictive value, and a 71.8% negative predictive value. A score of 27 or more out of 62 was associated with high-risk of a medication problem. The inter-rater reliability of the tool was high (κ = 0.79, 95% CI 0.75 - 0.84) and the content validity index was 99.4%. Conclusion: The Diabetes Medication Risk Screening Tool has good psychometric properties and can proactively identify people with diabetes at greatest risk of medication problems relating to hypoglycaemia and medication non-adherence.

A parent-based intervention to promote healthy eating and active behaviours in pre-school children: evaluation of the MEND 2-4 randomized controlled trial.

BACKGROUND: There is a paucity of studies evaluating targeted obesity prevention interventions in pre-school children. OBJECTIVES: We conducted a randomized controlled trial to evaluate the efficacy of a parent-based obesity prevention intervention for pre-schoolers - MEND (Mind, Exercise, Nutrition … Do It!) 2-4 on child diet, eating habits, physical activity/sedentary behaviours, and body mass index (BMI). METHODS: Parent-child dyads attended 10 weekly 90-min workshops relating to nutrition, physical activity and behaviours, including guided active play and healthy snack time. Assessments were conducted at baseline, immediately post-intervention, and 6 and 12 months post-intervention; child intake of vegetables, fruit, beverages, processed snack foods, fussiness, satiety responsiveness, physical activity, sedentary behaviour and neophobia were assessed via parent proxy report. Parent and child height and weight were measured. RESULTS: Two hundred one parent-child dyads were randomized to intervention (n = 104) and control (n = 97). Baseline mean child age was 2.7 (standard deviation [SD] 0.6) years, and child BMI-for-age z-score (World Health Organization) was 0.66 (SD 0.88). We found significant positive group effects for vegetable (P = 0.01) and snack food (P = 0.03) intake, and satiety responsiveness (P = 0.047) immediately post-intervention. At 12 months follow-up, intervention children exhibited less neophobia (P = 0.03) than controls. CONCLUSION: Future research should focus on additional strategies to support parents to continue positive behaviour change. ACTRN12610000200088.

Evaluation of the Test-mate ChE (cholinesterase) field kit in acute organophosphorus poisoning

STUDY OBJECTIVE: Measurement of acetylcholinesterase (AChE) is recommended in the management of organophosphorus poisoning, which results in 200,000 deaths worldwide annually. The Test-mate ChE 400 is a portable field kit designed for detecting occupational organophosphorus exposure that measures RBC AChE and plasma cholinesterase (PChE) within 4 minutes. We evaluate Test-mate against a reference laboratory test in patients with acute organophosphorus self-poisoning. METHODS: This was a cross-sectional comparison study of 14 patients with acute organophosphorus poisoning between May 2007 and June 2008. RBC AChE and PChE were measured in 96 and 91 samples, respectively, with the Test-mate ChE field kit and compared with a reference laboratory, using the limits of agreement method (Bland and Altman), κ statistics, and Spearman's correlation coefficients. RESULTS: There was good agreement between the Test-mate ChE and the reference laboratory for RBC AChE. The mean difference (Test-mate-reference) was -0.62 U/g hemoglobin, 95% limits of agreement -10.84 to 9.59 U/g hemoglobin. Good agreement was also observed between the categories of mild, moderate, and severe RBC AChE inhibition (weighted κ 0.85; 95% confidence interval [CI] 0.83 to 0.87). Measurement of PChE also showed good agreement, with a mean difference (Test-mate-reference) of +0.06 U/mL blood, 95% limits of agreement -0.41 to 0.53 U/mL blood. Spearman's correlation coefficients were 0.87 (95% CI 0.81 to 0.91) for RBC AChE and 0.76 (95% CI 0.66 to 0.84) for PChE. Analysis for within-subject correlation of subjects did not change the limits of agreement. CONCLUSION: The Test-mate ChE field kit reliably provides rapid measurement of RBC AChE in acute organophosphorus poisoning.

A telephone support program to reduce costs and hospital admissions for patients at risk of readmissions: lessons from an evaluation of a complex health intervention

This study aimed to evaluate the effectiveness of a telephone health coaching and support service provided to members of an Australian private health insurance fund-Telephonic Complex Care Program (TCCP)-on hospital use and associated costs. A case-control pre-post study design was employed using propensity score matching. Private health insurance members (n=273) who participated in TCCP between April and December 2012 (cases) were matched (1:1) to members who had not previously been enrolled in the program or any other disease management programs offered by the insurer (n=232). Eligible members were community dwelling, aged ≥65 years, and had 2 or more hospital admissions in the 12 months prior to program enrollment. Preprogram variables that estimated the propensity score included: participant demographics, diagnoses, and hospital use in the 12 months prior to program enrollment. TCCP participants received one-to-one telephone support, personalized care plan, and referral to community-based services. Control participants continued to access usual health care services. Primary outcomes were number of hospital admission claims and total benefits paid for all health care utilizations in the 12 months following program enrollment. Secondary outcomes included change in total benefits paid, hospital benefits paid, ancillary benefits paid, and total hospital bed days over the 12 months post enrollment. Compared with matched controls, TCCP did not appear to reduce health care utilization or benefits paid in the 12 months following program enrollment. However, program characteristics and implementation may have impacted its effectiveness. In addition, challenges related to evaluating complex health interventions such as TCCP are discussed.

The National Perinatal Depression Initiative: An evaluation of access to general practitioners, psychologists and psychiatrists through the Medicare Benefits Schedule

OBJECTIVE: To evaluate the impact of the National Perinatal Depression Initiative on access to Medicare services for women at risk of perinatal mental illness. METHOD: Retrospective cohort study using difference-in-difference analytical methods to quantify the impact of the National Perinatal Depression Initiative policies on Medicare Benefits Schedule mental health usage by Australian women giving birth between 2006 and 2010. A random sample of women of reproductive age enrolled in Medicare who had not given birth where used as controls. The main outcome measures were the proportions of women giving birth each month who accessed a Medicare Benefits Schedule mental health items during the perinatal period (pregnancy through to the end of the first postnatal year) before and after the introduction of the National Perinatal Depression Initiative. RESULTS: The proportion of women giving birth who accessed at least one mental health item during the perinatal period increased from 88 to 141 per 1000 between 2007 and 2010. The difference-in-difference analysis showed that while there was an overall increase in Medicare Benefits Schedule mental health item access as a result of the National Perinatal Depression Initiative, this did not reach statistical significance. However, the National Perinatal Depression Initiative was found to significantly increase access in subpopulations of women, particularly those aged under 25 and over 34 years living in major cities. CONCLUSION: In the 2 years following its introduction, the National Perinatal Depression Initiative was found to have increased access to Medicare funded mental health services in particular groups of women. However, an overall increase across all groups did not reach statistical significance. Further studies are needed to assess the impact of the National Perinatal Depression Initiative on women during childbearing years, including access to tertiary care, the cost-effectiveness of the initiative, and mental health outcomes. It is recommended that new mental health policy initiatives incorporate a planned strategic approach to evaluation, which includes sufficient follow-up to assess the impact of public health strategies.

Economic evaluation of a psychological intervention for high distress cancer patients and carers: costs and quality-adjusted life years

OBJECTIVE: This study compared the cost-effectiveness of a psychologist-led, individualised cognitive behavioural intervention (PI) to a nurse-led, minimal contact self-management condition for highly distressed cancer patients and carers.

METHODS: This was an economic evaluation conducted alongside a randomised trial of highly distressed adult cancer patients and carers calling cancer helplines. Services used by participants were measured using a resource use questionnaire, and quality-adjusted life years were measured using the assessment of quality of life - eight-dimension - instrument collected through a computer-assisted telephone interview. The base case analysis stratified participants based on the baseline score on the Brief Symptom Inventory. Incremental cost-effectiveness ratio confidence intervals were calculated with a nonparametric bootstrap to reflect sampling uncertainty. The results were subjected to sensitivity analysis by varying unit costs for resource use and the method for handling missing data.

RESULTS: No significant differences were found in overall total costs or quality-adjusted life years (QALYs) between intervention groups. Bootstrapped data suggest the PI had a higher probability of lower cost and greater QALYs for both carers and patients with high distress at baseline. For patients with low levels of distress at baseline, the PI had a higher probability of greater QALYs but at additional cost. Sensitivity analysis showed the results were robust.

CONCLUSIONS: The PI may be cost-effective compared with the nurse-led, minimal contact self-management condition for highly distressed cancer patients and carers. More intensive psychological intervention for patients with greater levels of distress appears warranted.

An evaluation of the role of green marketing and a firm's internal practices for environmental sustainability

The management of a firm's green operations is increasingly important for marketing strategists. The purpose of this study is to investigate the cross-influences of green marketing strategy and the key internal green functional areas in a firm. We use the antecedents of marketing strategy and identify relationships between green marketing strategy and key supporting internal environmental operations of firms with respect to (1) green suppliers, (2) environmental resource management, (3) green research and development, and (4) environmental manufacturing processes and procedures. The statistical techniques of parallel analysis, factor analysis and multiple regressions are used to analyze data collected from 332 firms. The results identify that among the four functional areas of firms adopting green marketing strategy. Two are more likely to influence green marketing strategy – supplier selection and research and development. Implications are discussed. The findings contribute to the theory of green marketing strategy. Future research is recommended.

Development, implementation and evaluation of a clinical research engagement and leadership capacity building program in a large Australian health care service

BACKGROUND: Health professionals need to be integrated more effectively in clinical research to ensure that research addresses clinical needs and provides practical solutions at the coal face of care. In light of limited evidence on how best to achieve this, evaluation of strategies to introduce, adapt and sustain evidence-based practices across different populations and settings is required. This project aims to address this gap through the co-design, development, implementation, evaluation, refinement and ultimately scale-up of a clinical research engagement and leadership capacity building program in a clinical setting with little to no co-ordinated approach to clinical research engagement and education.

METHODS/DESIGN: The protocol is based on principles of research capacity building and on a six-step framework, which have previously led to successful implementation and long-term sustainability. A mixed methods study design will be used. Methods will include: (1) a review of the literature about strategies that engage health professionals in research through capacity building and/or education in research methods; (2) a review of existing local research education and support elements; (3) a needs assessment in the local clinical setting, including an online cross-sectional survey and semi-structured interviews; (4) co-design and development of an educational and support program; (5) implementation of the program in the clinical environment; and (6) pre- and post-implementation evaluation and ultimately program scale-up. The evaluation focuses on research activity and knowledge, attitudes and preferences about clinical research, evidence-based practice and leadership and post implementation, about their satisfaction with the program. The investigators will evaluate the feasibility and effect of the program according to capacity building measures and will revise where appropriate prior to scale-up.

DISCUSSION: It is anticipated that this clinical research engagement and leadership capacity building program will enable and enhance clinically relevant research to be led and conducted by health professionals in the health setting. This approach will also encourage identification of areas of clinical uncertainty and need that can be addressed through clinical research within the health setting.

Economic evaluation of a group-based exercise program for falls prevention among the older community-dwelling population

BACKGROUND: Falls among older people are of growing concern globally. Implementing cost-effective strategies for their prevention is of utmost importance given the ageing population and associated potential for increased costs of fall-related injury over the next decades. The purpose of this study was to undertake a cost-utility analysis and secondary cost-effectiveness analysis from a healthcare system perspective, of a group-based exercise program compared to routine care for falls prevention in an older community-dwelling population.

METHODS: A decision analysis using a decision tree model was based on the results of a previously published randomised controlled trial with a community-dwelling population aged over 70. Measures of falls, fall-related injuries and resource use were directly obtained from trial data and supplemented by literature-based utility measures. A sub-group analysis was performed of women only. Cost estimates are reported in 2010 British Pound Sterling (GBP).

RESULTS: The ICER of GBP£51,483 per QALY for the base case analysis was well above the accepted cost-effectiveness threshold of GBP£20,000 to £30,000 per QALY, but in a sensitivity analysis with minimised program implementation the incremental cost reached GBP£25,678 per QALY. The ICER value at 95% confidence in the base case analysis was GBP£99,664 per QALY and GBP£50,549 per QALY in the lower cost analysis. Males had a 44% lower injury rate if they fell, compared to females resulting in a more favourable ICER for the women only analysis. For women only the ICER was GBP£22,986 per QALY in the base case and was below the cost-effectiveness threshold for all other variations of program implementation. The ICER value at 95% confidence was GBP£48,212 in the women only base case analysis and GBP£23,645 in the lower cost analysis. The base case incremental cost per fall averted was GBP£652 (GBP£616 for women only). A threshold analysis indicates that this exercise program cannot realistically break even.

CONCLUSIONS: The results suggest that this exercise program is cost-effective for women only. There is no evidence to support its cost-effectiveness in a group of mixed gender unless the costs of program implementation are minimal. Conservative assumptions may have underestimated the true cost-effectiveness of the program.

Evaluation of a knowledge translation and exchange platform to advance non-communicable disease prevention

Coordinated systems are required to ensure evidence-informed practice and evaluation of community-based interventions (CBIs). Knowledge translation and exchange (KTE) strategies show promise, but these require evaluation. This paper describes implementation and evaluation of COOPS, a national KTE platform to support best practice in obesity prevention CBIs. A logic model guides KTE activities including knowledge brokering, networking, tailored communications, training, and needs assessments. A mixed-methods evaluation includes communications data, knowledge brokering database, annual survey of CBIs, pre- and post-event questionnaires, interviews, social network analysis, and case studies. This evaluation will contribute to understanding the process of implementing a KTE platform with CBIs and its reach, quality and effectiveness.