Assessing performance and capability in the work place : focusing on essential skills and knowledge in student work placements

The determination of performance standards and assessment practices in regard to student work placements is an essential and important task. Inappropriate, inadequate, or excessively complex assessment tasks can influence levels of student engagement and the quality of learning outcomes. Critical to determining appropriate standards and assessment tasks is an understanding and knowledge of key elements of the learning environment and the extent to which opportunities are provided for students to engage in critical reflection and judgement of their own performance in the contexts of the work environment. This paper focuses on the development of essential skills and knowledge (capabilities) that provide evidence of learning in work placements by describing an approach taken in the science and technology disciplines. Assessment matrices are presented to illustrate a method of assessment for use within the context of the learning environment centred on work placements in science and technology. This study contributes to the debate on the meaning of professional capability, performance standards and assessment practices in work placement programs by providing evidence of an approach that can be adapted by other programs to achieve similar benefits. The approach may also be valuable to other learning contexts where capability and performance are being judged in situations that are outside a controlled teaching and learning environment i.e. in other life-wide learning contexts.

Towards equity in the futures market : curriculum as a condition of access

This article provides a consideration of the problem of equity in education. In the first part of the discussion, the author draws on philosophical and sociological literatures to consider what equity means and its implications for education. Drawing on work by Burbules, Lord & Sherman, she looks to curriculum as a condition of access and the importance of learning support structures in bringing about equitable educational outcomes, conceived in terms of Amy Gutmanns’s democratic threshold. The paper offers a conceptual-theoretical model for thinking about the resourcing and curricular requirements for equity in contemporary liberal democratic societies, contrasting the social and economic policy mixes employed by governments situated at different points along a liberty/equality continuum.

Studios for the masses : can student colaboration replace the master-apprentice relationship in design instruction?

This action research project investigated the use of a collaborative learning approach for addressing issues associated with teaching urban design to large, diverse cohorts. As a case study, I observed two semesters of an urban design unit that I revised between 2011 and 2012 to incorporate collaborative learning activities. Data include instructional materials, participant observations, peer-reviews of collaborative learning activities, feedback from students and instructors and student projects. Themes that emerged through qualitative analysis include the challenge of removing inequalities inherent in the diverse cohort, the challenge of unifying project guidance and marking criteria, and the challenge of providing project guidance for a very large cohort. Most notably, the study revealed a need to clarify learning objectives relating to design principles in order to fully transition to and benefit from a collaborative learning model.

A conceptual model for reflecting on expected learning vs. demonstrated student performance

Educators are faced with many challenging questions in designing an effective curriculum. What prerequisite knowledge do students have before commencing a new subject? At what level of mastery? What is the spread of capabilities between bare-passing students vs. the top performing group? How does the intended learning specification compare to student performance at the end of a subject? In this paper we present a conceptual model that helps in answering some of these questions. It has the following main capabilities: capturing the learning specification in terms of syllabus topics and outcomes; capturing mastery levels to model progression; capturing the minimal vs. aspirational learning design; capturing confidence and reliability metrics for each of these mappings; and finally, comparing and reflecting on the learning specification against actual student performance. We present a web-based implementation of the model, and validate it by mapping the final exams from four programming subjects against the ACM/IEEE CS2013 topics and outcomes, using Bloom's Taxonomy as the mastery scale. We then import the itemised exam grades from 632 students across the four subjects and compare the demonstrated student performance against the expected learning for each of these. Key contributions of this work are the validated conceptual model for capturing and comparing expected learning vs. demonstrated performance, and a web-based implementation of this model, which is made freely available online as a community resource.

Emerging from student to practitioner : a cumulative development tool for students

PURPOSE: This pilot project’s aim was to trial a tool and process for developing students’ ability to engage in self-assessment using reflection on their clinical experiences, including feedback from workplace learning, in order to aid them in linking theory to practice and develop strategies to improve performance. BACKGROUND: In nursing education, students can experience a mismatch in performance compared to theoretical learning, this is referred to as the ‘theory practice gap’ (Scully 2011, Chan Chan & Liu 2011). One specific contributing factor seems to be students’ inability to engage in meaningful reflection and self-correcting behaviours. A self-assessment strategy was implemented within a third year clinical unit to ameliorate this mismatch with encouraging results, as students developed self-direction in addressing learning needs. In this pilot project the above strategy was adapted for implementation between different clinical units, to create a whole of course approach to integrating workplace learning. METHOD: The methodology underpinning this project is a scaffolded, supported reflective practice process. Improved self-assessment skills is achieved by students reflecting on and engaging with feedback, then mapping this to learning outcomes to identify where performance can be improved. Evaluation of this project includes: collation of student feedback identifying successful strategies along with barriers encountered in implementation; feedback from students and teachers via above processes and tools; and comparison of the number of learning contracts issued in clinical nursing units with similar cohorts. RESULTS: Results will be complete by May 2012 and include analysis of the data collected via the above evaluation methods. Other outcomes will include the refined process and tool, plus resources that should improve cost effectiveness without reducing student support. CONCLUSION: Implementing these tools and processes over the entire student’s learning package, will assist them to demonstrate progressive development through the course. Students will have learnt to understand feedback and integrate these skills for life-long learning.

University Student Depression Inventory (USDI) : confirmatory factor analysis and review of psychometric properties

Background: The 30-item USDI is a self-report measure that assesses depressive symptoms among university students. It consists of three correlated three factors: Lethargy, Cognitive-Emotional and Academic motivation. The current research used confirmatory factor analysis to asses construct validity and determine whether the original factor structure would be replicated in a different sample. Psychometric properties were also examined. Method: Participants were 1148 students (mean age 22.84 years, SD = 6.85) across all faculties from a large Australian metropolitan university. Students completed a questionnaire comprising of the USDI, the Depression Anxiety Stress Scale (DASS) and Life Satisfaction Scale (LSS). Results: The three correlated factor model was shown to be an acceptable fit to the data, indicating sound construct validity. Internal consistency of the scale was also demonstrated to be sound, with high Cronbach Alpha values. Temporal stability of the scale was also shown to be strong through test-retest analysis. Finally, concurrent and discriminant validity was examined with correlations between the USDI and DASS subscales as well as the LSS, with sound results contributing to further support the construct validity of the scale. Cut-off points were also developed to aid total score interpretation. Limitations: Response rates are unclear. In addition, the representativeness of the sample could be improved potentially through targeted recruitment (i.e. reviewing the online sample statistics during data collection, examining the representativeness trends and addressing particular faculties within the university that were underrepresented). Conclusions: The USDI provides a valid and reliable method of assessing depressive symptoms found among university students.

The effects of linguistic factors on student performance on economics multiple choice questions

This paper proposes a framework to analyse performance on multiple choice questions with the focus on linguistic factors. Item Response Theory (IRT) is deployed to estimate ability and question difficulty levels. A logistic regression model is used to detect Differential Item Functioning questions. Probit models testify relationships between performance and linguistic factors controlling the effects of question construction and students’ background. Empirical results have important implications. The lexical density of stems affects performance. The use of non-Economics specialised vocabulary has differing impacts on the performance of students with different language backgrounds. The IRT-based ability and difficulty help explain performance variations.

Modeling destination brand equity in an emerging long haul market

The destination branding literature emerged as recently as 1998, and there remains a dearth of empirical data that tests the effectiveness of brand campaigns over time. This paper reports the results of an investigation into consumer-based brand equity for Australia as a long haul destination in an emerging South American market. In spite of the high level of academic interest in the measurement of perceptions of destinations since the 1970s, few previous studies have examined perceptions held by South American consumers. Findings suggest that destination brand awareness, brand image, and brand value are positively related to brand loyalty for a long-haul destination. The results also indicate that Australia is a more compelling destination brand for previous visitors compared to non-visitors.

Identifying discourses of moderation in higher education

Moderation of student assessment is a critical component of teaching and learning in contemporary universities. Yet, despite this, it tends to be marked by idiosyncratic and sporadic processes informed by liminal understanding. This paper, in the light of forthcoming radical national requirements for the declaration of moderation processes in tertiary curricula in Australia, will present four discourses of moderation we identified in a recent study in a Faculty of Education in a large metropolitan university. The discourses are equity, justification, community building and accountability. Together, they will act as a starting point for academics to review their beliefs and attitudes towards the moderation of student assessment.

Measuring a destination’s brand equity between 2003 and 2012 using the consumer-based brand equity (CBBE) hierarchy

Given the increasing investments being made in brand development by destination marketing organisations (DMO) since the 1990s, including rebranding and repositioning, more research is needed to enhance understanding of how to effectively monitor destination brand performance over time. This paper reports the results of a study of brand performance of a competitive set of destinations, in their most important market, between 2003 and 2012. Brand performance was measured from the perspective of consumer perceptions, based on the concept of consumer-based brand equity (CBBE). A structured questionnaire was administered to different samples in 2003, 2007 and 2012. The results indicated minimal changes in perceptions of the five destinations over the 10 year period. Due to the commonality of challenges faced by DMOs worldwide, it is suggested the CBBE hierarchy provides destination marketers with a practical tool for evaluating brand performance over time; in terms of measures of effectiveness of past marketing communications, as well as indicators of future performance. In addition, and importantly, CBBE also provides transparent accountability measures for stakeholders. While the topic of destination image has been one of the most popular in the tourism literature, there has been a paucity of research published in relation to the temporal aspect of consumer perceptions. This is a rare investigation into the measurement of perceptions of destinations over a 10 year period.

Skill development and social inclusion via peer assisted learning in first-year higher education : an evaluation of student experiences across two law units

Because professions seek graduates who can 'collaborate, share skills and knowledge, and communicate' (Kruck and Reif, 2001, p 37), it is important that university graduates are not equipped solely with the content knowledge of their discipline, but also with prospective employment skills. Furthermore, when students 'interact more in positive ways with their teachers and peers, they gain more in terms of essential skills and competencies, such as critical thinking, problem~solving [and] effective communication' (NSSE, 2000, p 2)./n this way, peer assisted fellowing has the potential to enhance students' professional development, and provide the social inclusion and engagement necessary for effective learning. This session describes two peer assisted learning models embedded within first year QUT Faculty of Law units. Through a partnership between teaching staff, student mentors and mentees, the models aim to facilitate student socialisation whilst supplementing understanding of substantive law with the development of academic and work·related skills. Mentor and mentee perceptions, and program implications, are considered.

Standardising radiation therapy undergraduate competency levels

QUT Bachelor of Radiation Therapy students progress from first visiting a radiation therapy department to graduation and progression into the NPDP over a span of three years. Although there are clear guidelines as to expected competency level post-NPDP, there is still a variety of perceived levels prior to this. Staff and students feedback both suggest that different centres and within centres different staff have differing opinions of these levels. Indeed, many staff members object to the use of the word “competency” for a pre-NPODP undergraduate, preferring the term “achievement”. While it is acknowledged that students progress at different rates, it is vitally important for equity that staff expectations of students at different academic levels are identical. Provision of guidelines for different stages of progression are essential for equitable assessment and most assessments, including the NRTAT are complemented by statements to enable level to be determined. For the University-specific competency assessments some level of consensus between clinical staff is required, especially where students are placed at a large number of different placement sites. Aims The main aim of this initial study is to gauge staff opinions of levels of student progression in order to judge cross-centres consistency. A secondary objective is to evaluate the degree of correlation between staff seniority and perception of student levels. Informal feedback suggests that staff at or just post NPDP level have a different perception of student competency expectations than more senior staff. If these perceptions change with level it will make agreement of guidelines statements more challenging. Study Methods A standard evaluation questionnaire was provided to RT staff participating in ongoing updates to clinical assessment. As part of curriculum development staff were asked to provide anonymous and optional answers to further questions in order to audit current practice. This involved assigning level of student progression to different statements relating to tasks or competencies. After data collation, scores were assigned to level and totals used to rank statements according to perceived student level. Descriptive statistical analysis was used to identify which statements were easier to assign to student level and which were more ambiguous. Further sub-analysis was performed for each category of staff seniority to judge differences in perception. Strength of correlation between seniority and expectation was calculated to confirm or contradict the informal feedback. Results By collating different staff perceptions of competencies for different student levels commonly agreed statements can be used to define achievement level. This presentation outlines the results of the audit including statements that most staff perceived as relevant to a specific student group and statements that staff found to be harder to attribute. Strength of correlation between staff perception and seniority will be outlined where statistically significant.

The cost of being ethical : evidence from Islamic equity funds

This paper examines the relationship between financial performance and ethical screening intensity of a special class of ethical funds that is rooted in Islamic values – Islamic equity funds (IEFs). These faith-based ethical funds screen investments on compliance with Islamic values where conventional interest expense (riba), gambling (maysir), excessive uncertainty (gharar), and non-ethical (non-halal) products are prohibited. We test whether these extra screens affect the financial performance of IEFs relative to non-Islamic funds. Based on a large survivorship-free international sample of 387 Islamic funds, our results show that IEFs on average underperform conventional funds by 40 basis points per month, or 4.8% per year (supporting the underperformance hypothesis). While Islamic funds do not generally perform better during crisis periods, they outperformed conventional funds during the recent sub-prime crisis (supporting the outperformance hypothesis). Using holdings-based measures for ethical screening intensity, results show IEFs that apply more intensive screening perform worse, suggesting that there is a cost to being ethical.