Online self-and-peer-assessment for teamwork in architecture and business communications

There is considerable evidence of general student scepticism regarding the purpose of team assignments and high levels of concern for the fairness of assessment procedures when all members of a team receive the same grade. Some educators are similarly anxious about not only the validity of team grades, but also the need to assess ongoing team processes in addition to the final assignment product. This paper offers self-andpeer-assessment (SAPA) as a fair, valid and reliable method of producing information about ongoing team processes. The paper examines a pilot study investigating an online SAPA tool originally developed for a small class of architecture students. This tool is adapted for use for by students completing team assignment in two further architecture design units and for a very large class of 800 business communication students. The sample students studied on four campuses, as well as in off campus and offshore modes. The paper focuses on the initial stages of the study to demonstrate how researchers from very different backgrounds collaborated to adapt the online tool and implement a pilot study whilst maintaining both comparability of assessment and integrity of research design.

Online self and peer assessment in large, multi-campus, multi-cohort contexts

The increasing use of team assignments within higher education is well documented. The driving forces behind this include desires to facilitate reflective and collaborative learning, to develop generic teamwork skills for graduate employment and to reduce the grading workloads of faculty staff. Students however consistently report dissatisfaction when the assessment of team assignments produces a common grade for all team members. Self-andpeer-assessment (SAPA) is presented as a fair, valid and reliable method of producing information about ongoing team processes. This information can provide ongoing feedback to team members and rich formative data to instructors attempting to assess the team process and students’ teamwork skills. This data can also enable individualised summative assessment in dysfunctional teams or situations of uneven team member contributions. Whilst manual SAPA protocols can work effectively for smaller classes, computer-assisted SAPA offers a solution to the problems of large classes. This paper reports on the early stages of an online SAPA tool, originally developed for small classes of architecture students, adapted for use by very large business communication classes comprising up to 1000 students in a semester. This large unit is delivered on four Australian campuses as well as off-campus and in off-shore mode, by up to fourteen instructors at any one time. The paper documents how three researchers from very different backgrounds worked to create their own research team, implement a pilot study, and adapt the online tool, whilst adhering to comparability of assessment constraints and maintaining integrity of research design.

Improving laboratory learning through self and peer assessment of laboratory reports

The laboratory provides an opportunity for students to achieve many learning outcomes including to critically evaluate information, interpret and draw conclusions from scientific data, and communicate scientific results, information, or arguments. This paper describes a laboratory-writing task that involves self and peer evaluation. After discussion of the expectations of laboratory report writing during class, students self and peer evaluate reports. In a process similar to double-blind journal refereeing, students practise critically evaluating the quality of academic writing using a rubric. The summative assessment is based on how consistent their evaluations are with the evaluations of the same reports performed by their peers. The formative assessment is that students receive peer evaluations and feedback via a rubric on reports that they have written. The skill of critically evaluating their own reports is used to improve the laboratory reports in subsequent assessment tasks.

Prejudicial evaluation: bias in self-and-peer-assessments of teamwork contributions to design

This chapter considers a simple but important question: can students fairly assess each other’s individual contribution to team designs? The chapter focuses on a key problem when using online self-and-peerassessment to individualising design grades for team assignments, namely rater bias – the possibility of students being biased when assessing their own and their peers’ contributions. Three rater-bias issuesare considered in depth: (1) self-overmarking; (2) gender bias and gender differences; and (3) out-group bias in the peer assessment of international students in multicultural cohorts. Each issue is explored viathe analysis of eight years of quantitative data from the use of an online self-and-peer assessment tool. Evidence is found of self-overmarking and of out-group bias in nonhomogeneous cohorts. However, no evidence is found of gender bias. The chapter concludes with recommendations for design teachers around the assessment of individual contributions to teamwork using self-and-peer assessment.

Teacher and peer assessment of creativity in technology education

This study, part of a PhD thesis, investigates a number of aspects relating to technology education, creativity and the assessment of creativity and aesthetics. Following tuitional sessions, panels of upper primary school children were involved in assessing the creative and aesthetic characteristics of technological products made by other children. Teachers assessed a selection of student drawings for creativity and these results were compared against the corresponding products assessed by the panels of Year 5 children. Subjects were administered a test on their knowledge of technology and these scores were compared to product creativity scores. The results showed high correlations between the assessor ranks for both creativity and aesthetics although there was generally more consistency with creativity. While some subjects produced both creative drawings and products, others showed creativity in only one outcome. A further finding of the investigation identified the relationship between students' knowledge of technology and the creative product constructed by the older children in the study.

The architecture of peer assessment : do academically successful design students make good team mates in design assignments?

This paper considers the relationship between architecture and construction management students’ overall academic abilities (as measured by Weighted Average Marks [WAMs]), their peer ratings for contributions to team design assignments (as measured by an online Self-and-Peer-Assessment [SAPA] tool), and their specific abilities as building designers (as measured by grades in individual design assignments). The research was conducted to determine whether a student’s prior academic achievements might indicate how well they will work in teams. The research demonstrates a statistically significant relationship between WAMs and SAPA ratings indicating that academically successful students more often than not make good teammates. However, the study also highlights that when peers are assessing contributions to teamwork they are assessing skills and qualities in their teammates other than overall academic ability or the ability to design well. Whilst this study is largely located within the field of design, the findings are relevant to any group work where teachers aim to design assessment that unravels group and individual contribution.

Student peer and self assessment in the context of work integrated learning : making it work

Students are now experiencing opportunities in practice education where their learning is facilitated by professionals of varied backgrounds thus leading to issues of how to assess student performance while on placement. Educators have had to re-think the way in which students’ performance is assessed and to integrate the key graduate attributes of critical reflection, self and peer assessments and feedback. Using recommendations by Mason (1999) in relation to a collaborative group model for workplace learning and ‘The Self-Directed Learning Model’ by Gaiptman and Anthony (1989) the Occupational Wellness and Life Satisfaction (OWLS) program encourages students to reflect on their experiences in an environment of self and peer evaluation, focussing on the process of learning rather than purely on outcomes. Students are required to complete a self and peer assessment of their learning using a nationally recognised fieldwork evaluation instrument and develop a practice portfolio consisting of learning contract and supporting evidence for their self-assessment. Qualitative and quantitative data was collected via a questionnaire to alumni. The most frequently identified skills that were valued by respondents were autonomy and independence. Other benefits identified were facilitation of self directed learning, and ability to problem solve with colleagues and to share learning. In a higher education environment where lifelong learning and the ability to work collaboratively are valued graduate attributes, a focus on peer and self assessment within the context of work integrated learning contributes to graduates who are well placed to work in both traditional and newer and emerging areas of practice.

Sex does not matter: gender bias and gender differences in peer assessments of contributions to group work

This paper considers the possibility of gender bias in peer ratings for contributions to team assignments, as measured by an online self-and-peer assessment tool. The research was conducted to determine whether peer assessment led to reliable and fair marking outcomes. The methodology of Falchikov and Magin was followed in order to test their finding that gender has no discernable impact on peer ratings. Data from over 1500 participants at two universities enrolled in four different degree programmes were analysed. The research indicates an absence of gender bias in six case studies. The research also found that women received significantly higher ratings than men.

Using technology to improve peer review and collaborative conversations to benchmark academic standards

In 2010 the Australian government commissioned the Australian Learning and Teaching Council (ALTC) to undertake a national project to facilitate disciplinary development of threshold learning standards. The aim was to lay the foundation for all higher education providers to demonstrate to the new national higher education regulator, the Tertiary Education Quality and Standards Agency (TEQSA), that graduates achieved or exceeded minimum academic standards. Through a yearlong consultative process, representatives of employers, professional bodies, academics and students, developed learning standards applying to any Australian higher education provider. Willey and Gardner reported using a software tool, SPARKPLUS, in calibrating academic standards amongst teaching staff in large classes. In this paper, we investigate the effectiveness of this technology to promote calibrated understandings with the national accounting learning standards. We found that integrating the software with a purposely designed activity provided significant efficiencies in calibrating understandings about learning standards, developed expertise and a better understanding of what is required to meet these standards and how best to demonstrate them. The software and supporting calibration and assessment process can be adopted by other disciplines, including engineering, seeking to provide direct evidence about performance against learning standards. © 2012 IEEE.

Assessment practices in Scottish social work education: a practice audit of Scottish universities providing qualifying social work courses

Student assessment is a critical component of social work education. In the last ten years, the nature of assessment in social work education has changed considerably. Alongside innovations to more traditional forms of assessment such as essays and examinations, there has been increased emphasis on student participation in assessment in various forms of self- and peer-assessment. However, there is a dearth of published information on assessment strategies and methods employed in mainstream social work education programmes. The inception of the new qualifying courses across the UK has catalysed interest in the development of innovative assessment strategies and tools. In Scotland, the Scottish Institute for Excellence in Social Work Education (SIESWE) has commissioned research into innovative assessment strategies. This paper reports on the findings of a practice audit of current assessment practices in social work education in Scotland.

Graduate attributes as a focus for institution-wide curriculum renewal: innovations and challenges

Curtin University's Curriculum 2010 (C2010) initiative aimed at ensuring degree programs were excellent and sustainable. Before C2010, graduate attributes were not universally emphasised and indicators showed room for improvement in generic skills. C2010 focused on embedding graduate attributes through three strategies. The first was embedding graduate attributes in degree programs and mapping for constructive alignment of outcomes and assessments. The second strategy was related: programs were reviewed drawing on evidence including perspectives of graduates, employers and program teams on the relative importance of graduate attributes and the extent to which they were generally demonstrated or developed. The third strategy was a university-wide eportfolio system enabling students' self- and peer-assessment of graduate attributes. Since completion of C2010, proxy indicators show improvement. However, this paper highlights the challenges associated with graduate attributes and renewal. These include: academic staff engagement, the time needed for innovations to come to fruition and evidencing achievement of graduate attributes.

Making teamwork work

This paper explores team assignments as an increasingly common component of many university courses. It argues that a need exists for all instructors, particularly those involved in the teaching of business communication, to develop a multi-faceted teamwork perspective which encompasses every dimension of team assignments. Such a perspective should permeate assignment design, student training, assessment of team process as well as product, self and peer assessment and the individualisation of student scores. Each of these facets is discussed with reference to research literature with a number of practical suggestions provided as exemplars for how to make teamwork work.