Assessing performance and capability in the work place : focusing on essential skills and knowledge in student work placements

The determination of performance standards and assessment practices in regard to student work placements is an essential and important task. Inappropriate, inadequate, or excessively complex assessment tasks can influence levels of student engagement and the quality of learning outcomes. Critical to determining appropriate standards and assessment tasks is an understanding and knowledge of key elements of the learning environment and the extent to which opportunities are provided for students to engage in critical reflection and judgement of their own performance in the contexts of the work environment. This paper focuses on the development of essential skills and knowledge (capabilities) that provide evidence of learning in work placements by describing an approach taken in the science and technology disciplines. Assessment matrices are presented to illustrate a method of assessment for use within the context of the learning environment centred on work placements in science and technology. This study contributes to the debate on the meaning of professional capability, performance standards and assessment practices in work placement programs by providing evidence of an approach that can be adapted by other programs to achieve similar benefits. The approach may also be valuable to other learning contexts where capability and performance are being judged in situations that are outside a controlled teaching and learning environment i.e. in other life-wide learning contexts.

A methodological approach to modelling design led innovation across secondary education : an Australian case study

Incorporating design thinking as a generic capability at a school level is needed to ensure future generations are empowered for business innovation and active citizenship. This paper describes the methodology of an investigation into modelling design led innovation approaches from the business sector to secondary education, as part of a larger study. It builds on a previously discussed research agenda by outlining the scope, significance and limitations of currently available research in this area, examining an action research methodology utilising an Australian design immersion program case study, and discussing implications and future work. It employs a triangulated approach encompassing thematic analysis of qualitative data collection from student focus groups, semi-structured convergent interviews with teachers and facilitators, and student journals. Eventual outcomes will be reviewed and analysed within the framework of a proposed innovation matrix model for educational growth, synthesising principles responding to 21st century student outcomes. It is anticipated this research will inform a successful design led secondary education innovation model, facilitating new engagement frameworks between tertiary and secondary education sectors, as well as providing new insight into the suitability of action research in prototyping social innovation in Australia.

A conceptual model for reflecting on expected learning vs. demonstrated student performance

Educators are faced with many challenging questions in designing an effective curriculum. What prerequisite knowledge do students have before commencing a new subject? At what level of mastery? What is the spread of capabilities between bare-passing students vs. the top performing group? How does the intended learning specification compare to student performance at the end of a subject? In this paper we present a conceptual model that helps in answering some of these questions. It has the following main capabilities: capturing the learning specification in terms of syllabus topics and outcomes; capturing mastery levels to model progression; capturing the minimal vs. aspirational learning design; capturing confidence and reliability metrics for each of these mappings; and finally, comparing and reflecting on the learning specification against actual student performance. We present a web-based implementation of the model, and validate it by mapping the final exams from four programming subjects against the ACM/IEEE CS2013 topics and outcomes, using Bloom's Taxonomy as the mastery scale. We then import the itemised exam grades from 632 students across the four subjects and compare the demonstrated student performance against the expected learning for each of these. Key contributions of this work are the validated conceptual model for capturing and comparing expected learning vs. demonstrated performance, and a web-based implementation of this model, which is made freely available online as a community resource.

YuMi deadly maths program : efficacy and collaboration

Historically, perceptions about mathematics and how it is taught and learned in schools have been mixed and as a consequence have an influence on self efficacy. There are those of us who see mathematics as logical and an enjoyable subject to learn, whilst others see mathematics as irrelevant, difficult and contributing to their school failure. Research has shown that over-represented in the latter are Aboriginal and Torres Strait Islander, low SES and ESL students. These students are the focus of YuMi Deadly Centre (YDC) professional learning and research work at the Queensland University of Technology in Brisbane.

Emerging from student to practitioner : a cumulative development tool for students

PURPOSE: This pilot project’s aim was to trial a tool and process for developing students’ ability to engage in self-assessment using reflection on their clinical experiences, including feedback from workplace learning, in order to aid them in linking theory to practice and develop strategies to improve performance. BACKGROUND: In nursing education, students can experience a mismatch in performance compared to theoretical learning, this is referred to as the ‘theory practice gap’ (Scully 2011, Chan Chan & Liu 2011). One specific contributing factor seems to be students’ inability to engage in meaningful reflection and self-correcting behaviours. A self-assessment strategy was implemented within a third year clinical unit to ameliorate this mismatch with encouraging results, as students developed self-direction in addressing learning needs. In this pilot project the above strategy was adapted for implementation between different clinical units, to create a whole of course approach to integrating workplace learning. METHOD: The methodology underpinning this project is a scaffolded, supported reflective practice process. Improved self-assessment skills is achieved by students reflecting on and engaging with feedback, then mapping this to learning outcomes to identify where performance can be improved. Evaluation of this project includes: collation of student feedback identifying successful strategies along with barriers encountered in implementation; feedback from students and teachers via above processes and tools; and comparison of the number of learning contracts issued in clinical nursing units with similar cohorts. RESULTS: Results will be complete by May 2012 and include analysis of the data collected via the above evaluation methods. Other outcomes will include the refined process and tool, plus resources that should improve cost effectiveness without reducing student support. CONCLUSION: Implementing these tools and processes over the entire student’s learning package, will assist them to demonstrate progressive development through the course. Students will have learnt to understand feedback and integrate these skills for life-long learning.

Understanding QC and EQA : an authentic learning model for undergraduate students (Conference Abstract)

Introduction QC and EQA are integral to good pathology laboratory practice. Medical Laboratory Science students undertake a project exploring internal QC and EQA procedures used in chemical pathology laboratories. Each student represents an individual lab and the class group represents the peer group of labs performing the same assay using the same method. Methods Using a manual BCG assay for serum albumin, normal and abnormal controls are run with a patient sample over 7 weeks. The QC results are assessed each week using calculated z-scores and both 2S & 3S control rules to determine whether a run is ‘in control’. At the end of the 7 weeks a completed LJ chart is assessed using the Westgard Multirules. Students investigate causes of error and the implications for both lab practice and patient care if runs are not ‘in control’. Twice in the 7 weeks two EQA samples (with target values unknown) are assayed alongside the weekly QC and patient samples. Results from each student are collated and form the basis of an EQA program. ALP are provided and students complete a Youden Plot, which is used to analyse the performance of each ‘lab’ and the method to identify bias. Students explore the concept of possible clinical implications of a biased method and address the actions that should be taken if a lab is not in consensus with the peer group. Conclusion This project is a model of ‘real world’ practice in which student demonstrate an understanding of the importance of QC procedures in a pathology laboratory, apply and interpret statistics and QC rules and charts, apply critical thinking and analytical skills to quality performance data to make recommendations for further practice and improve their technical competence and confidence.

University Student Depression Inventory (USDI) : confirmatory factor analysis and review of psychometric properties

Background: The 30-item USDI is a self-report measure that assesses depressive symptoms among university students. It consists of three correlated three factors: Lethargy, Cognitive-Emotional and Academic motivation. The current research used confirmatory factor analysis to asses construct validity and determine whether the original factor structure would be replicated in a different sample. Psychometric properties were also examined. Method: Participants were 1148 students (mean age 22.84 years, SD = 6.85) across all faculties from a large Australian metropolitan university. Students completed a questionnaire comprising of the USDI, the Depression Anxiety Stress Scale (DASS) and Life Satisfaction Scale (LSS). Results: The three correlated factor model was shown to be an acceptable fit to the data, indicating sound construct validity. Internal consistency of the scale was also demonstrated to be sound, with high Cronbach Alpha values. Temporal stability of the scale was also shown to be strong through test-retest analysis. Finally, concurrent and discriminant validity was examined with correlations between the USDI and DASS subscales as well as the LSS, with sound results contributing to further support the construct validity of the scale. Cut-off points were also developed to aid total score interpretation. Limitations: Response rates are unclear. In addition, the representativeness of the sample could be improved potentially through targeted recruitment (i.e. reviewing the online sample statistics during data collection, examining the representativeness trends and addressing particular faculties within the university that were underrepresented). Conclusions: The USDI provides a valid and reliable method of assessing depressive symptoms found among university students.

The effects of linguistic factors on student performance on economics multiple choice questions

This paper proposes a framework to analyse performance on multiple choice questions with the focus on linguistic factors. Item Response Theory (IRT) is deployed to estimate ability and question difficulty levels. A logistic regression model is used to detect Differential Item Functioning questions. Probit models testify relationships between performance and linguistic factors controlling the effects of question construction and students’ background. Empirical results have important implications. The lexical density of stems affects performance. The use of non-Economics specialised vocabulary has differing impacts on the performance of students with different language backgrounds. The IRT-based ability and difficulty help explain performance variations.

Games for change and transformative learning : an ethnographic case study

This thesis aims to contribute to a better understanding of how serious games/games for change function as learning frameworks for transformative learning in an educational setting. This study illustrates how the meaning-making processes and learning with and through computer gameplay are highly contingent, and are significantly influenced by the uncertainties of the situational context. The study focuses on SCAPE, a simulation game that addresses urban planning and sustainability. SCAPE is based on the real-world scenario of Kelvin Grove Urban Village, an inner city redevelopment area in Brisbane, Queensland, Australia. The game is embedded within an educational program, and I thus account for the various gameplay experiences of different school classes participating in this program. The networks emerging from the interactions between students/players, educators, facilitators, the technology, the researcher, as well as the setting, result in unanticipated, controversial, and sometimes unintended gameplay experiences and outcomes. To unpack play, transformative learning and games, this study adopts an ecological approach that considers the magic circle of gameplay in its wider context. Using Actor-Network Theory as the ontological lens for inquiry, the methods for investigation include an extensive literature review, ethnographic participant observation of SCAPE, as well as student and teacher questionnaires, finishing with interviews with the designers and facilitators of SCAPE. Altogether, these methods address my research aim to better understand how the heterogeneous actors engage in the relationships in and around gameplay, and illustrate how their conflicting understandings enable, shape or constrain the (transformative) learning experience. To disentangle these complexities, my focus continuously shifts between the following modes of inquiry into the aims „h To describe and analyse the game as a designed artefact. „h To examine the gameplay experiences of players/students and account for how these experiences are constituted in the relationships of the network. „h To trace the meaning-making processes emerging from the various relations of players/students, facilitators, teachers, designers, technology, researcher, and setting, and consider how the boundaries of the respective ecology are configured and negotiated. „h To draw out the implications for the wider research area of game-based learning by using the simulation game SCAPE as an example for introducing gameplay to educational settings. Accounting in detail for five school classes, these accounts represent, each in its own right, distinct and sometimes controversial forms of engagement in gameplay. The practices and negotiations of all the assembled human and non-human actors highlight the contingent nature of gameplay and learning. In their sum, they offer distinct but by no means exhaustive examples of the various relationships that emerge from the different assemblages of human and non-human actors. This thesis, hence, illustrates that game-based learning in an educational setting is accompanied by considerable unpredictability and uncertainty. As ordinary life spills and leaks into gameplay experiences, group dynamics and the negotiations of technology, I argue that overly deterministic assertions of the game¡¦s intention, as well as a too narrowly defined understanding of the transformative learning outcome, can constrain our inquiries and hinder efforts to further elucidate and understand the evolving uncertainties around game-based learning. Instead, this thesis posits that playing and transformative learning are relational effects of the respective ecology, where all actors are networked in their (partial) enrolment in the process of translation. This study thus attempts to foreground the rich opportunities for exploring how game-based learning is assembled as a network of practices.

Leadership for creating a thinking school at Buranda State School

This article explores the role of principal leadership in creating a thinking school. It contributes to the school leadership literature by exploring the intersection of two important areas of study in education - school leadership and education for thinking - which is a particularly apt area of study, because effective school leadership is crucial if students are to learn to be critical and creative thinkers, yet this connection has not be widely investigated. We describe how one principal, Hinton, turned around an underperforming school by using critical and creative philosophical thinking as the focus for students, staff and parents. Then, drawing on the school leadership literature, the article describes seven attributes of school leadership beginning with four articulated by Leithwood and colleagues (2006) (building vision and setting direction; redesigning the organisation; understanding and developing people; managing the teaching and learning program), and adding three others (influence; self-development; and responding to context). This framework is then used in a case study format in a collaboration between practitioner and researchers to first explore evidence from empirical studies and personal reflection about Hinton's leadership of Buranda State School, and second to illuminate how these general features of school leadership apply to creating a thinking school. Based on the case study and using the general characteristics of school leadership, a framework for leading a thinking school is described. Because the framework is based on a turnaround school, this framework has wide applicability: to schools that are doing well as an indication of how to implement a contemporary approach to curriculum and pedagogy; and to schools that are underperforming and want a rigorous, high expectation and contemporary way to improve student learning.

Peer assisted learning, skills development and Generation Y : a case study of a first year undergraduate law unit

The majority of current first year university students belong to Generation Y. Consequently, research suggests that, in order to more effectively engage them, their particular learning preferences should be acknowledged in the organisation of their learning environments and in the support provided. These preferences are reflected in the Torts Student Peer Mentor Program, which, as part of the undergraduate law degree at the Queensland University of Technology, utilises active learning, structured sessions and teamwork to supplement student understanding of the substantive law of Torts with the development of life-long skills. This article outlines the Program, and its relevance to the learning styles and experiences of Generation Y first year law students transitioning to university, in order to investigate student perceptions of its effectiveness – both generally and, more specifically, in terms of the Program’s capacity to assist students to develop academic and work-related skills.

Widening participation in university learning

This paper reports how one Australian university and the Queensland Department of Education and Training (DET) are working together to increase the number of school students from low socio-economic backgrounds enrolling in undergraduate university degrees. This innovative program involves university lecturers and school teachers working together in the delivery and assessment of four Bachelor of Education units (or subjects) to a cohort of Year eleven and twelve students at a secondary school. Focus group interviews collected data from 26 students, 7 parents, 4 school and 3 university staff to assess the effectiveness of the program. All stakeholders viewed the program as a highly valuable opportunity to experience university learning with 31 high school graduating students being made offers to enter full-time university in the 2010 and 2011. This positive result has particular significance in the current drive in Australia and elsewhere to increase the participation in higher education of young people from under-represented groups.

Examining the Assessment Opportunities for Cultural Connectedness for Student Learning : A Sociocultural Analysis

The disparity that exists between the highest and lowest achievers together with deficit approaches to teaching, learning and assessment raise serious equity issues related to fairness, validity, culture and access, which were analysed in a recent Australian Research Council funded project. This chapter explores the potential that exists for teachers to work with Indigenous Teacher Assistants (ITAs) to secure cultural connectedness in teaching, learning and assessment of Indigenous students. The study was a design experiment, which was conducted in seven Catholic and Independent primary schools in northern Queensland and involved semi-structured focus group interviews with Year 4 and 6 Indigenous students, principals, teachers and Indigenous Teacher Assistants. Classroom observations and document analyses were also conducted. This corpus of data was analysed using a sociocultural theoretical lens. The use of a sociocultural analysis helped to identify cultural influences, Indigenous students’ funds of knowledge and values. The information from this analysis was made explicit to teachers to demonstrate how they could enhance their pedagogic and assessment practices by embracing and extending the cultural spaces for learning and teaching of Indigenous students. The way in which teachers construct their interactions for greater cultural connectedness and enhanced learning would appear to rely on relationship building with Indigenous staff, Indigenous students’ cultural knowledge, and improved understanding of assessment and related equity issues.

Principals, Teachers, and Student's Perception of the Information and Communication Technology in Kuwait Secondary Schools (Rhetoric and reality)

ICT integration has been advocated to provide opportunities to improve students’ achievement and engagement through transforming the educational setting. A valuable tool that contributes in enhancing and developing students’ cognitive skills for lifelong learning, ICT integration has introduced a new educational philosophy, shifting the role of students into a more central position in the pedagogical processes. Kuwait, as with many other countries, has recently planned ICT integration to develop its citizen’s capacities. This study sought to capture the principals’, teachers’, and students’ perceptions of ICT integration in pedagogical activities, as well as how ICT is being used for learning and teaching activities in three ICT leading Kuwaiti secondary schools. Interviews with principals, teachers, and students were conducted, along with an open-ended questionnaire for the teachers, researcher observations, and document analysis. The findings revealed that ICT integration in Kuwait needed to be reinforced to accomplish the ICT integration objectives. A call for further support for teachers, and a reconsideration of the ICT integration strategies were also recommended.