Assessing performance and capability in the work place : focusing on essential skills and knowledge in student work placements

The determination of performance standards and assessment practices in regard to student work placements is an essential and important task. Inappropriate, inadequate, or excessively complex assessment tasks can influence levels of student engagement and the quality of learning outcomes. Critical to determining appropriate standards and assessment tasks is an understanding and knowledge of key elements of the learning environment and the extent to which opportunities are provided for students to engage in critical reflection and judgement of their own performance in the contexts of the work environment. This paper focuses on the development of essential skills and knowledge (capabilities) that provide evidence of learning in work placements by describing an approach taken in the science and technology disciplines. Assessment matrices are presented to illustrate a method of assessment for use within the context of the learning environment centred on work placements in science and technology. This study contributes to the debate on the meaning of professional capability, performance standards and assessment practices in work placement programs by providing evidence of an approach that can be adapted by other programs to achieve similar benefits. The approach may also be valuable to other learning contexts where capability and performance are being judged in situations that are outside a controlled teaching and learning environment i.e. in other life-wide learning contexts.

Studios for the masses : can student colaboration replace the master-apprentice relationship in design instruction?

This action research project investigated the use of a collaborative learning approach for addressing issues associated with teaching urban design to large, diverse cohorts. As a case study, I observed two semesters of an urban design unit that I revised between 2011 and 2012 to incorporate collaborative learning activities. Data include instructional materials, participant observations, peer-reviews of collaborative learning activities, feedback from students and instructors and student projects. Themes that emerged through qualitative analysis include the challenge of removing inequalities inherent in the diverse cohort, the challenge of unifying project guidance and marking criteria, and the challenge of providing project guidance for a very large cohort. Most notably, the study revealed a need to clarify learning objectives relating to design principles in order to fully transition to and benefit from a collaborative learning model.

A conceptual model for reflecting on expected learning vs. demonstrated student performance

Educators are faced with many challenging questions in designing an effective curriculum. What prerequisite knowledge do students have before commencing a new subject? At what level of mastery? What is the spread of capabilities between bare-passing students vs. the top performing group? How does the intended learning specification compare to student performance at the end of a subject? In this paper we present a conceptual model that helps in answering some of these questions. It has the following main capabilities: capturing the learning specification in terms of syllabus topics and outcomes; capturing mastery levels to model progression; capturing the minimal vs. aspirational learning design; capturing confidence and reliability metrics for each of these mappings; and finally, comparing and reflecting on the learning specification against actual student performance. We present a web-based implementation of the model, and validate it by mapping the final exams from four programming subjects against the ACM/IEEE CS2013 topics and outcomes, using Bloom's Taxonomy as the mastery scale. We then import the itemised exam grades from 632 students across the four subjects and compare the demonstrated student performance against the expected learning for each of these. Key contributions of this work are the validated conceptual model for capturing and comparing expected learning vs. demonstrated performance, and a web-based implementation of this model, which is made freely available online as a community resource.

Emerging from student to practitioner : a cumulative development tool for students

PURPOSE: This pilot project’s aim was to trial a tool and process for developing students’ ability to engage in self-assessment using reflection on their clinical experiences, including feedback from workplace learning, in order to aid them in linking theory to practice and develop strategies to improve performance. BACKGROUND: In nursing education, students can experience a mismatch in performance compared to theoretical learning, this is referred to as the ‘theory practice gap’ (Scully 2011, Chan Chan & Liu 2011). One specific contributing factor seems to be students’ inability to engage in meaningful reflection and self-correcting behaviours. A self-assessment strategy was implemented within a third year clinical unit to ameliorate this mismatch with encouraging results, as students developed self-direction in addressing learning needs. In this pilot project the above strategy was adapted for implementation between different clinical units, to create a whole of course approach to integrating workplace learning. METHOD: The methodology underpinning this project is a scaffolded, supported reflective practice process. Improved self-assessment skills is achieved by students reflecting on and engaging with feedback, then mapping this to learning outcomes to identify where performance can be improved. Evaluation of this project includes: collation of student feedback identifying successful strategies along with barriers encountered in implementation; feedback from students and teachers via above processes and tools; and comparison of the number of learning contracts issued in clinical nursing units with similar cohorts. RESULTS: Results will be complete by May 2012 and include analysis of the data collected via the above evaluation methods. Other outcomes will include the refined process and tool, plus resources that should improve cost effectiveness without reducing student support. CONCLUSION: Implementing these tools and processes over the entire student’s learning package, will assist them to demonstrate progressive development through the course. Students will have learnt to understand feedback and integrate these skills for life-long learning.

University Student Depression Inventory (USDI) : confirmatory factor analysis and review of psychometric properties

Background: The 30-item USDI is a self-report measure that assesses depressive symptoms among university students. It consists of three correlated three factors: Lethargy, Cognitive-Emotional and Academic motivation. The current research used confirmatory factor analysis to asses construct validity and determine whether the original factor structure would be replicated in a different sample. Psychometric properties were also examined. Method: Participants were 1148 students (mean age 22.84 years, SD = 6.85) across all faculties from a large Australian metropolitan university. Students completed a questionnaire comprising of the USDI, the Depression Anxiety Stress Scale (DASS) and Life Satisfaction Scale (LSS). Results: The three correlated factor model was shown to be an acceptable fit to the data, indicating sound construct validity. Internal consistency of the scale was also demonstrated to be sound, with high Cronbach Alpha values. Temporal stability of the scale was also shown to be strong through test-retest analysis. Finally, concurrent and discriminant validity was examined with correlations between the USDI and DASS subscales as well as the LSS, with sound results contributing to further support the construct validity of the scale. Cut-off points were also developed to aid total score interpretation. Limitations: Response rates are unclear. In addition, the representativeness of the sample could be improved potentially through targeted recruitment (i.e. reviewing the online sample statistics during data collection, examining the representativeness trends and addressing particular faculties within the university that were underrepresented). Conclusions: The USDI provides a valid and reliable method of assessing depressive symptoms found among university students.

The effects of linguistic factors on student performance on economics multiple choice questions

This paper proposes a framework to analyse performance on multiple choice questions with the focus on linguistic factors. Item Response Theory (IRT) is deployed to estimate ability and question difficulty levels. A logistic regression model is used to detect Differential Item Functioning questions. Probit models testify relationships between performance and linguistic factors controlling the effects of question construction and students’ background. Empirical results have important implications. The lexical density of stems affects performance. The use of non-Economics specialised vocabulary has differing impacts on the performance of students with different language backgrounds. The IRT-based ability and difficulty help explain performance variations.

Skill development and social inclusion via peer assisted learning in first-year higher education : an evaluation of student experiences across two law units

Because professions seek graduates who can 'collaborate, share skills and knowledge, and communicate' (Kruck and Reif, 2001, p 37), it is important that university graduates are not equipped solely with the content knowledge of their discipline, but also with prospective employment skills. Furthermore, when students 'interact more in positive ways with their teachers and peers, they gain more in terms of essential skills and competencies, such as critical thinking, problem~solving [and] effective communication' (NSSE, 2000, p 2)./n this way, peer assisted fellowing has the potential to enhance students' professional development, and provide the social inclusion and engagement necessary for effective learning. This session describes two peer assisted learning models embedded within first year QUT Faculty of Law units. Through a partnership between teaching staff, student mentors and mentees, the models aim to facilitate student socialisation whilst supplementing understanding of substantive law with the development of academic and work·related skills. Mentor and mentee perceptions, and program implications, are considered.

Education for social responsibility: Ethics and imagination for engaging students and teachers in educational reform

In this chapter we make assumptions about the primary role of education for the life of its beneficiaries and for society. Undoubtedly, formal education plays an important role in enhancing the likelihood for participation in future social life, including enjoyment and employment, by the student as well as the development of the well being of society in general. Similarly, education is often seen as a main means for intergenerational transmission of knowledge and culture. However, as Dewey (1916) argues, in liberal societies, education has the capacity of enhancing democratic participation in society that goes beyond passive participation by its members. One can argue that the achievement of the ideals of democracy demands a free and strong education system. In other words, while education can function as an instrument to integrate students into the present society, it also has the potential to become an instrument for its transformation by means of which citizens can develop an understanding of how their society functions and a sense of agency towards its transformation. Arguably, this is what Freire (1985) meant when he talked about the role of education to “read and write” the world. A stream of progressive educators (e.g., Apple (2004), Freire, (1985), Giroux (2001) and McLaren (2002)) taught us that the reading of the world that is capable of leading into writing the world is a critical reading; i.e., a reading that poses “Why” questions and imagines “What else can be” (Carr & Kemmis, 1987).

Examining the Assessment Opportunities for Cultural Connectedness for Student Learning : A Sociocultural Analysis

The disparity that exists between the highest and lowest achievers together with deficit approaches to teaching, learning and assessment raise serious equity issues related to fairness, validity, culture and access, which were analysed in a recent Australian Research Council funded project. This chapter explores the potential that exists for teachers to work with Indigenous Teacher Assistants (ITAs) to secure cultural connectedness in teaching, learning and assessment of Indigenous students. The study was a design experiment, which was conducted in seven Catholic and Independent primary schools in northern Queensland and involved semi-structured focus group interviews with Year 4 and 6 Indigenous students, principals, teachers and Indigenous Teacher Assistants. Classroom observations and document analyses were also conducted. This corpus of data was analysed using a sociocultural theoretical lens. The use of a sociocultural analysis helped to identify cultural influences, Indigenous students’ funds of knowledge and values. The information from this analysis was made explicit to teachers to demonstrate how they could enhance their pedagogic and assessment practices by embracing and extending the cultural spaces for learning and teaching of Indigenous students. The way in which teachers construct their interactions for greater cultural connectedness and enhanced learning would appear to rely on relationship building with Indigenous staff, Indigenous students’ cultural knowledge, and improved understanding of assessment and related equity issues.

Multi-level knowledge transfer in software development outsourcing projects : the agency theory view

In recent years, software development outsourcing has become even more complex. Outsourcing partner have begun‘re- outsourcing’ components of their projects to other outsourcing companies to minimize cost and gain efficiencies, creating a multi-level hierarchy of outsourcing. This research in progress paper presents preliminary findings of a study designed to understand knowledge transfer effectiveness of multi-level software development outsourcing projects. We conceptualize the SD-outsourcing entities using the Agency Theory. This study conceptualizes, operationalises and validates the concept of Knowledge Transfer as a three-phase multidimensional formative index of 1) Domain knowledge, 2) Communication behaviors, and 3) Clarity of requirements. Data analysis identified substantial, significant differences between the Principal and the Agent on two of the three constructs. Using Agency Theory, supported by preliminary findings, the paper also provides prescriptive guidelines of reducing the friction between the Principal and the Agent in multi-level software outsourcing.

Principals, Teachers, and Student's Perception of the Information and Communication Technology in Kuwait Secondary Schools (Rhetoric and reality)

ICT integration has been advocated to provide opportunities to improve students’ achievement and engagement through transforming the educational setting. A valuable tool that contributes in enhancing and developing students’ cognitive skills for lifelong learning, ICT integration has introduced a new educational philosophy, shifting the role of students into a more central position in the pedagogical processes. Kuwait, as with many other countries, has recently planned ICT integration to develop its citizen’s capacities. This study sought to capture the principals’, teachers’, and students’ perceptions of ICT integration in pedagogical activities, as well as how ICT is being used for learning and teaching activities in three ICT leading Kuwaiti secondary schools. Interviews with principals, teachers, and students were conducted, along with an open-ended questionnaire for the teachers, researcher observations, and document analysis. The findings revealed that ICT integration in Kuwait needed to be reinforced to accomplish the ICT integration objectives. A call for further support for teachers, and a reconsideration of the ICT integration strategies were also recommended.

The enactment of formative assessment in English language classrooms in two Chinese universities : teacher and student responses

The College English Curriculum Requirements (CECR), announced by the Chinese Ministry of Education in 2007, recommended the inclusion of formative assessment into the existing summative assessment framework of College English. This policy had the potential to fundamentally change the nature of assessment and its role in the teaching and learning of English in Chinese universities. In order to document and analyse these changes, case studies involving English language teachers and learners were undertaken in two Chinese Universities: one a Key university in the national capital; the other a non-Key university in a western province. The case study design incorporated classroom observations and interviews with English language teachers and their students. The type and focus of feedback and the engagement of students in assessment were analysed in the two contexts. Fundamental to the analysis was the concept of enactment, with the focus of this study on the ways that policy ideas and principles were enacted in the practices of the Chinese university classroom. Understandings of formative assessment as applied in contexts other than the predominantly Western, Anglophone contexts from where many of its principles derive, are offered.

Ensuring student success : establishing a community of practice for culturally and linguistically diverse preservice teachers

This paper discusses the primacy of communities of practice within learning contexts at university and during practicum for culturally and linguistically diverse preservice teachers. The study illustrates that learning occurs when there are adequate opportunities for participation and practice. Data from interviews with 28 culturally and linguistically diverse preservice teachers illustrate that tensions created by social, cultural differences impact upon modes of identification and dimensions of practice (Lave & Wenger, 1991; Wenger, 1998). The study concludes by reiterating the importance of establishing proactive communities of practice to ensure success in learning and practice for this group of preservice teachers.

Good practice for safeguarding student learning engagement in higher education institutions : Final Report 2012

The project 'Good practice for safeguarding student learning engagement in higher education institutions' commenced in late 2010 as a Competitive Grant with funding provided by the Australian Learning and Teaching Council. The project is now overseen by the Office for Learning and Teaching within the Australian Department of Industry, Innovation, Science, Research and Tertiary Education. The project was completed in December 2012. The project was lead by QUT and comprised of the project team: Professor Karen Nelson, (project leader), Ms Tracy Creagh, (project manager) and Adjunct Professor John Clarke. Commencing in late 2010 the project invited a total of eight institutions across Australia and New Zealand (including QUT) who had either: existing programs and activities that monitored student learning engagement (MSLE); were in the early stages of implementing MSLE programs, or; who were piloting MSLE activities. As well, the project involved an advisory group and project evaluator comprising of academic and professional staff across two additional universities.