A conceptual model for reflecting on expected learning vs. demonstrated student performance

Educators are faced with many challenging questions in designing an effective curriculum. What prerequisite knowledge do students have before commencing a new subject? At what level of mastery? What is the spread of capabilities between bare-passing students vs. the top performing group? How does the intended learning specification compare to student performance at the end of a subject? In this paper we present a conceptual model that helps in answering some of these questions. It has the following main capabilities: capturing the learning specification in terms of syllabus topics and outcomes; capturing mastery levels to model progression; capturing the minimal vs. aspirational learning design; capturing confidence and reliability metrics for each of these mappings; and finally, comparing and reflecting on the learning specification against actual student performance. We present a web-based implementation of the model, and validate it by mapping the final exams from four programming subjects against the ACM/IEEE CS2013 topics and outcomes, using Bloom's Taxonomy as the mastery scale. We then import the itemised exam grades from 632 students across the four subjects and compare the demonstrated student performance against the expected learning for each of these. Key contributions of this work are the validated conceptual model for capturing and comparing expected learning vs. demonstrated performance, and a web-based implementation of this model, which is made freely available online as a community resource.

Emerging from student to practitioner : a cumulative development tool for students

PURPOSE: This pilot project’s aim was to trial a tool and process for developing students’ ability to engage in self-assessment using reflection on their clinical experiences, including feedback from workplace learning, in order to aid them in linking theory to practice and develop strategies to improve performance. BACKGROUND: In nursing education, students can experience a mismatch in performance compared to theoretical learning, this is referred to as the ‘theory practice gap’ (Scully 2011, Chan Chan & Liu 2011). One specific contributing factor seems to be students’ inability to engage in meaningful reflection and self-correcting behaviours. A self-assessment strategy was implemented within a third year clinical unit to ameliorate this mismatch with encouraging results, as students developed self-direction in addressing learning needs. In this pilot project the above strategy was adapted for implementation between different clinical units, to create a whole of course approach to integrating workplace learning. METHOD: The methodology underpinning this project is a scaffolded, supported reflective practice process. Improved self-assessment skills is achieved by students reflecting on and engaging with feedback, then mapping this to learning outcomes to identify where performance can be improved. Evaluation of this project includes: collation of student feedback identifying successful strategies along with barriers encountered in implementation; feedback from students and teachers via above processes and tools; and comparison of the number of learning contracts issued in clinical nursing units with similar cohorts. RESULTS: Results will be complete by May 2012 and include analysis of the data collected via the above evaluation methods. Other outcomes will include the refined process and tool, plus resources that should improve cost effectiveness without reducing student support. CONCLUSION: Implementing these tools and processes over the entire student’s learning package, will assist them to demonstrate progressive development through the course. Students will have learnt to understand feedback and integrate these skills for life-long learning.

Psychology students’ beliefs about integrating Complementary and Alternative Therapy (CAT) into their future psychology practice

We investigated whether belief-based differences exist between students who have strong and weak intentions to integrate complementary and alternative therapy (CAT) into future psychology practice by recommending CAT or specific CAT practitioners to clients. A cross-sectional methodology was used. Psychology undergraduate students (N = 106) participated in a paper-based questionnaire design to explore their underlying beliefs related to CAT integration. The study was undertaken at a major university in Queensland, Australia. The theory of planned behaviour belief-based framework guided the study. Multivariate analyses of variance examined the influence of behavioural, normative, and control beliefs on the strong and weak intention groups. A multiple regression analysis investigated the relative importance of these belief sets for predicting intentions. We found that clear differences emerged between strong and weak intenders on behavioural and normative beliefs, but not control beliefs. Strong intenders perceived the positive outcomes of integrating CAT, such as being able to offer clients a more holistic practice and having confidence in the practitioners/practices, as more likely to occur than weak intenders, and perceived the negative outcome of compromising my professional practice as less likely. Strong in-tenders were more likely than weak intenders to perceive that a range of important referents (e.g., clients) would think they should integrate CAT. Results of the regression analysis revealed the same pattern of results in that behavioural and normative beliefs, but not control beliefs, significantly predicted intentions. The findings from this study can be used to inform policy and educational initiatives that aim to encourage CAT use in psychology practice.

University Student Depression Inventory (USDI) : confirmatory factor analysis and review of psychometric properties

Background: The 30-item USDI is a self-report measure that assesses depressive symptoms among university students. It consists of three correlated three factors: Lethargy, Cognitive-Emotional and Academic motivation. The current research used confirmatory factor analysis to asses construct validity and determine whether the original factor structure would be replicated in a different sample. Psychometric properties were also examined. Method: Participants were 1148 students (mean age 22.84 years, SD = 6.85) across all faculties from a large Australian metropolitan university. Students completed a questionnaire comprising of the USDI, the Depression Anxiety Stress Scale (DASS) and Life Satisfaction Scale (LSS). Results: The three correlated factor model was shown to be an acceptable fit to the data, indicating sound construct validity. Internal consistency of the scale was also demonstrated to be sound, with high Cronbach Alpha values. Temporal stability of the scale was also shown to be strong through test-retest analysis. Finally, concurrent and discriminant validity was examined with correlations between the USDI and DASS subscales as well as the LSS, with sound results contributing to further support the construct validity of the scale. Cut-off points were also developed to aid total score interpretation. Limitations: Response rates are unclear. In addition, the representativeness of the sample could be improved potentially through targeted recruitment (i.e. reviewing the online sample statistics during data collection, examining the representativeness trends and addressing particular faculties within the university that were underrepresented). Conclusions: The USDI provides a valid and reliable method of assessing depressive symptoms found among university students.

The effects of linguistic factors on student performance on economics multiple choice questions

This paper proposes a framework to analyse performance on multiple choice questions with the focus on linguistic factors. Item Response Theory (IRT) is deployed to estimate ability and question difficulty levels. A logistic regression model is used to detect Differential Item Functioning questions. Probit models testify relationships between performance and linguistic factors controlling the effects of question construction and students’ background. Empirical results have important implications. The lexical density of stems affects performance. The use of non-Economics specialised vocabulary has differing impacts on the performance of students with different language backgrounds. The IRT-based ability and difficulty help explain performance variations.

A model for enhancing assessment and teaching practice at the coalface : Insights from a Fellow-In-Residence engagement program

Australian universities are currently engaging with new governmental policies and regulations that require them to demonstrate enhanced quality and accountability in teaching and research. The development of national academic standards for learning outcomes in higher education is one such instance of this drive for excellence. These discipline-specific standards articulate the minimum, or Threshold Learning Outcomes, to be addressed by higher education institutions so that graduating students can demonstrate their achievement to their institutions, accreditation agencies, and industry recruiters. This impacts not only on the design of Engineering courses (with particular emphasis on pedagogy and assessment), but also on the preparation of academics to engage with these standards and implement them in their day-to-day teaching practice on a micro level. This imperative for enhanced quality and accountability in teaching is also significant at a meso level, for according to the Australian Bureau of Statistics, about 25 per cent of teachers in Australian universities are aged 55 and above and more than 54 per cent are aged 45 and above (ABS, 2006). A number of institutions have undertaken recruitment drives to regenerate and enrich their academic workforce by appointing capacity-building research professors and increasing the numbers of early- and mid-career academics. This nationally driven agenda for quality and accountability in teaching permeates also the micro level of engineering education, since the demand for enhanced academic standards and learning outcomes requires both a strong advocacy for a shift to an authentic, collaborative, outcomes-focused education and the mechanisms to support academics in transforming their professional thinking and practice. Outcomes-focused education means giving greater attention to the ways in which the curriculum design, pedagogy, assessment approaches and teaching activities can most effectively make a positive, verifiable difference to students’ learning. Such education is authentic when it is couched firmly in the realities of learning environments, student and academic staff characteristics, and trustworthy educational research. That education will be richer and more efficient when staff works collaboratively, contributing their knowledge, experience and skills to achieve learning outcomes based on agreed objectives. We know that the school or departmental levels of universities are the most effective loci of changes in approaches to teaching and learning practices in higher education (Knight & Trowler, 2000). Heads of Schools are being increasingly entrusted with more responsibilities - in addition to setting strategic directions and managing the operational and sometimes financial aspects of their school, they are also expected to lead the development and delivery of the teaching, research and other academic activities. Guiding and mentoring individuals and groups of academics is one critical aspect of the Head of School’s role. Yet they do not always have the resources or support to help them mentor staff, especially the more junior academics. In summary, the international trend in undergraduate engineering course accreditation towards the demonstration of attainment of graduate attributes poses new challenges in addressing academic staff development needs and the assessment of learning. This paper will give some insights into the conceptual design, implementation and empirical effectiveness to date, of a Fellow-In-Residence Engagement (FIRE) program. The program is proposed as a model for achieving better engagement of academics with contemporary issues and effectively enhancing their teaching and assessment practices. It will also report on the program’s collaborative approach to working with Heads of Schools to better support academics, especially early-career ones, by utilizing formal and informal mentoring. Further, the paper will discuss possible factors that may assist the achievement of the intended outcomes of such a model, and will examine its contributions to engendering an outcomes-focussed thinking in engineering education.

Embedding environmental sustainability in the undergraduate chemistry curriculum : a case study

In spite of increasing attention devoted to the importance of embedding sustainability in university curricula, few Australian universities include specific green chemistry units, and there is no mention of green or sustainable chemistry concepts in the majority of units. In this paper, an argument is posited that all universities should embed sustainable chemistry within all Chemistry courses because it is the morally correct stance to minimise the harm of climate change. Attitudes of chemistry lecturers towards integrating sustainability into their teaching have been probed and it was found, using an established model, that personal environmental perspectives are critical to their attitude. Importantly, academic staff whose research has an environmental component were more likely to incorporate sustainability into their teaching while others struggled to find ways to do so even when they believed it to be important. This paper will recommend that resources are required to assist academic staff without a green chemistry research program to incorporate sustainability into their teaching and several suggestions are provided.

Skill development and social inclusion via peer assisted learning in first-year higher education : an evaluation of student experiences across two law units

Because professions seek graduates who can 'collaborate, share skills and knowledge, and communicate' (Kruck and Reif, 2001, p 37), it is important that university graduates are not equipped solely with the content knowledge of their discipline, but also with prospective employment skills. Furthermore, when students 'interact more in positive ways with their teachers and peers, they gain more in terms of essential skills and competencies, such as critical thinking, problem~solving [and] effective communication' (NSSE, 2000, p 2)./n this way, peer assisted fellowing has the potential to enhance students' professional development, and provide the social inclusion and engagement necessary for effective learning. This session describes two peer assisted learning models embedded within first year QUT Faculty of Law units. Through a partnership between teaching staff, student mentors and mentees, the models aim to facilitate student socialisation whilst supplementing understanding of substantive law with the development of academic and work·related skills. Mentor and mentee perceptions, and program implications, are considered.

Widening participation in university learning

This paper reports how one Australian university and the Queensland Department of Education and Training (DET) are working together to increase the number of school students from low socio-economic backgrounds enrolling in undergraduate university degrees. This innovative program involves university lecturers and school teachers working together in the delivery and assessment of four Bachelor of Education units (or subjects) to a cohort of Year eleven and twelve students at a secondary school. Focus group interviews collected data from 26 students, 7 parents, 4 school and 3 university staff to assess the effectiveness of the program. All stakeholders viewed the program as a highly valuable opportunity to experience university learning with 31 high school graduating students being made offers to enter full-time university in the 2010 and 2011. This positive result has particular significance in the current drive in Australia and elsewhere to increase the participation in higher education of young people from under-represented groups.

A snapshot of radiation therapy techniques and technology in Queensland : an aid to mapping undergraduate curriculum

Introduction: Undergraduate students studying the Bachelor of Radiation Therapy at Queensland University of Technology (QUT) attend clinical placements in a number of department sites across Queensland. To ensure that the curriculum prepares students for the most common treatments and current techniques in use in these departments, a curriculum matching exercise was performed. Methods: A cross-sectional census was performed on a pre-determined “Snapshot” date in 2012. This was undertaken by the clinical education staff in each department who used a standardized proforma to count the number of patients as well as prescription, equipment, and technique data for a list of tumour site categories. This information was combined into aggregate anonymized data. Results: All 12 Queensland radiation therapy clinical sites participated in the Snapshot data collection exercise to produce a comprehensive overview of clinical practice on the chosen day. A total of 59 different tumour sites were treated on the chosen day and as expected the most common treatment sites were prostate and breast, comprising 46% of patients treated. Data analysis also indicated that intensity-modulated radiotherapy (IMRT) use is relatively high with 19.6% of patients receiving IMRT treatment on the chosen day. Both IMRT and image-guided radiotherapy (IGRT) indications matched recommendations from the evidence. Conclusion: The Snapshot method proved to be a feasible and efficient method of gathering useful

Examining the Assessment Opportunities for Cultural Connectedness for Student Learning : A Sociocultural Analysis

The disparity that exists between the highest and lowest achievers together with deficit approaches to teaching, learning and assessment raise serious equity issues related to fairness, validity, culture and access, which were analysed in a recent Australian Research Council funded project. This chapter explores the potential that exists for teachers to work with Indigenous Teacher Assistants (ITAs) to secure cultural connectedness in teaching, learning and assessment of Indigenous students. The study was a design experiment, which was conducted in seven Catholic and Independent primary schools in northern Queensland and involved semi-structured focus group interviews with Year 4 and 6 Indigenous students, principals, teachers and Indigenous Teacher Assistants. Classroom observations and document analyses were also conducted. This corpus of data was analysed using a sociocultural theoretical lens. The use of a sociocultural analysis helped to identify cultural influences, Indigenous students’ funds of knowledge and values. The information from this analysis was made explicit to teachers to demonstrate how they could enhance their pedagogic and assessment practices by embracing and extending the cultural spaces for learning and teaching of Indigenous students. The way in which teachers construct their interactions for greater cultural connectedness and enhanced learning would appear to rely on relationship building with Indigenous staff, Indigenous students’ cultural knowledge, and improved understanding of assessment and related equity issues.

Problems and awareness of support services among University students

Although Australian universities have allocated significant resources toward the development of student support services, administrators have little systematic information about the problems undergraduate university students experience or students' knowledge about available support services. The author surveyed 441 students in an urban, nonresidential university to examine the prevalence of difficulties associated with learning, sexual harassment, discrimination, emotional distress, health problems, course and career concerns, financial difficulties, and difficulties with lecturers; he also assessed students' knowledge of support services in each of these areas. Course concerns were the most common problem, followed by emotional distress, worry about career choices, financial difficulties, and problems with lecturers. More than half of the students were unaware of the support services available to them to address a range of concerns from sexual harassment and discrimination to emotional distress. Approximately 20% of the students reported having used university counseling or career services. Implications for targeting specific areas for outreach programs are discussed.

Principals, Teachers, and Student's Perception of the Information and Communication Technology in Kuwait Secondary Schools (Rhetoric and reality)

ICT integration has been advocated to provide opportunities to improve students’ achievement and engagement through transforming the educational setting. A valuable tool that contributes in enhancing and developing students’ cognitive skills for lifelong learning, ICT integration has introduced a new educational philosophy, shifting the role of students into a more central position in the pedagogical processes. Kuwait, as with many other countries, has recently planned ICT integration to develop its citizen’s capacities. This study sought to capture the principals’, teachers’, and students’ perceptions of ICT integration in pedagogical activities, as well as how ICT is being used for learning and teaching activities in three ICT leading Kuwaiti secondary schools. Interviews with principals, teachers, and students were conducted, along with an open-ended questionnaire for the teachers, researcher observations, and document analysis. The findings revealed that ICT integration in Kuwait needed to be reinforced to accomplish the ICT integration objectives. A call for further support for teachers, and a reconsideration of the ICT integration strategies were also recommended.

The enactment of formative assessment in English language classrooms in two Chinese universities : teacher and student responses

The College English Curriculum Requirements (CECR), announced by the Chinese Ministry of Education in 2007, recommended the inclusion of formative assessment into the existing summative assessment framework of College English. This policy had the potential to fundamentally change the nature of assessment and its role in the teaching and learning of English in Chinese universities. In order to document and analyse these changes, case studies involving English language teachers and learners were undertaken in two Chinese Universities: one a Key university in the national capital; the other a non-Key university in a western province. The case study design incorporated classroom observations and interviews with English language teachers and their students. The type and focus of feedback and the engagement of students in assessment were analysed in the two contexts. Fundamental to the analysis was the concept of enactment, with the focus of this study on the ways that policy ideas and principles were enacted in the practices of the Chinese university classroom. Understandings of formative assessment as applied in contexts other than the predominantly Western, Anglophone contexts from where many of its principles derive, are offered.