Studying students' opinions : using surveys in writing program assessment

Administrators of writing programs are regularly faced with the problem of assessing the learning that students gain in their coursework. Many methods of assessment exist, but most have some problems associated with them related to the amount of time it takes to perform the study or the scope of the knowledge gained relative to number of participants or volume of information collected. This pilot study investigates the use of surveys of student opinion for their potential to assess composition instruction at Michigan Technological University. The primary goal of this pilot study is to test the effectiveness of using data collected in surveys to make recommendations for improvement of the composition program at Michigan Tech. The report concludes with recommendations for additional study and refinements to the instruments used.

Using core assessment criteria to improve essay writing

Harrington, Elander, Jo Lusher, Aiyegbayo, Pitt, , Hannah Robinson

Complex skills and academic writing:a review of evidence about the types of learning required to meet core assessment criteria

Assessment criteria are increasingly incorporated into teaching, making it important to clarify the pedagogic status of the qualities to which they refer. We reviewed theory and evidence about the extent to which four core criteria for student writing-critical thinking, use of language, structuring, and argument-refer to the outcomes of three types of learning: generic skills learning, a deep approach to learning, and complex learning. The analysis showed that all four of the core criteria describe to some extent properties of text resulting from using skills, but none qualify fully as descriptions of the outcomes of applying generic skills. Most also describe certain aspects of the outcomes of taking a deep approach to learning. Critical thinking and argument correspond most closely to the outcomes of complex learning. At lower levels of performance, use of language and structuring describe the outcomes of applying transferable skills. At higher levels of performance, they describe the outcomes of taking a deep approach to learning. We propose that the type of learning required to meet the core criteria is most usefully and accurately conceptualized as the learning of complex skills, and that this provides a conceptual framework for maximizing the benefits of using assessment criteria as part of teaching. © 2006 Taylor & Francis.

High Stakes and No Takers: The Impact of Florida Comprehensive Assessment Test (FCAT) Writing on Students’ and Teachers’ Perceptions of Writing

This ethnographic case study of ten students and their teachers concludes that the state writing test had a negative impact on students’ and teachers’ perceptions of writing in four categories: strengths and weaknesses in student writing, self-assessment of writing skills, factors impacting test scores, and motivation and attitudes toward writing.

Dancing between diversity and consistency : refining assessment in postgraduate degrees in dance

This publication is the culmination of a 2 year Australian Learning and Teaching Council's Project Priority Programs Research Grant which investigates key issues and challenges in developing flexible guidelines lines for best practice in Australian Doctoral and Masters by Research Examination, encompassing the two modes of investigation, written and multi-modal (practice-led/based) theses, their distinctiveness and their potential interplay. The aims of the project were to address issues of assessment legitimacy raised by the entry of practice-orientated dance studies into Australian higher degrees; examine literal embodiment and presence, as opposed to cultural studies about states of embodiment; foreground the validity of questions around subjectivity and corporeal intelligence/s and the reliability of artistic/aesthetic communications, and finally to celebrate ‘performance mastery’(Melrose 2003) as a rigorous and legitimate mode of higher research. The project began with questions which centred around: the functions of higher degree dance research; concepts of 'master-ness’ and ‘doctorateness’; the kinds of languages, structures and processes which may guide candidates, supervisors, examiners and research personnel; the purpose of evaluation/examination; addressing positive and negative attributes of examination. Finally the study examined ways in which academic/professional, writing/dancing, tradition/creation and diversity/consistency relationships might be fostered to embrace change. Over two years, the authors undertook a qualitative national study encompassing a triangulation of semi-structured face to face interviews and industry forums to gather views from the profession, together with an analysis of existing guidelines, and recent literature in the field. The most significant primary data emerged from 74 qualitative interviews with supervisors, examiners, research deans and administrators, and candidates in dance and more broadly across the creative arts. Qualitative data gathered from the two primary sources, was coded and analysed using the NVivo software program. Further perspectives were drawn from international consultant and dance researcher Susan Melrose, as well as publications in the field, and initial feedback from a draft document circulated at the World Dance Alliance Global Summit in July 2008 in Brisbane. Refinement of data occurred in a continual sifting process until the final publication was produced. This process resulted in a set of guidelines in the form of a complex dynamic system for both product and process oriented outcomes of multi-modal theses, along with short position papers on issues which arose from the research such as contested definitions, embodiment and ephemerality, ‘liveness’ in performance research higher degrees, dissolving theory/practice binaries, the relationship between academe and industry, documenting practices and a re-consideration of the viva voce.

The refractive folio : exegesis and assessment in the open studio

Folio submission is universally regarded as the most appropriate means for measuring a student’s performance in the studio. However, developing meaningful and defensible assessment criteria is persistent challenge for all tertiary art educators. In discipline-based studios, the parameters provided by medium and technique provide useful points of reference for assessing creative performance. But how can student performance be evaluated when there is no discipline-based framework to act as a point of reference? The ‘open’ studio approach to undergraduate teaching presents these and other pedagogical challenges. This paper discusses the innovative approaches to studio-based teaching and assessment at QUT. Vital to the QUT open studio model is the studio rationale – an exegetical document that establishes an individualised theoretical framework through which a student’s understandings can be, in part, evaluated. This paper argues that the exegetical folio effectively reconciles the frequently divergent imperatives of creative, professional and academic skills, while retaining the centrality of the studio as a site for the production of new material, processual and conceptual understandings.

Assessing reflection skills in law using criterion-referenced assessment

This paper focuses on the assessment of reflective practice, an issue that has not been fully explored within legal education literature. While the issue of how reflective practice should be taught is one that requires careful consideration, it is beyond the scope of this paper to consider both the teaching and the assessment of reflective practice. Part II of this paper conceptualises reflective practice, and Part III explores the benefits of reflective practice in legal education and the use of reflective writing to assess experiential learning in a legal context. Part IV considers the diverse issues that arise in assessing reflective practice and whether there is an objective method for assessing reflection. Part V of the paper examines the assessment of reflective practice in the context of an exemplar undergraduate law subject that uses a reflective report to assess students’ experiential learning during a court visit.14 Finally, Part VI offers a rubric to facilitate criterion-referenced assessment of reflective practice and thereby provides a framework for assessing reflection skills. It is suggested that the rubric is transferable not only to other law subjects but also to subjects in other disciplines.

Merging fact with fiction : a study of year 9 students' developing scientific literacy through the writing of hybridised scientific narratives on a socioscientific issue

International assessments of student science achievement, and growing evidence of students' waning interest in school science, have ensured that the development of scientific literacy continues to remain an important educational priority. Furthermore, researchers have called for teaching and learning strategies to engage students in the learning of science, particularly in the middle years of schooling. This study extends previous national and international research that has established a link between writing and learning science. Specifically, it investigates the learning experiences of eight intact Year 9 science classes as they engage in the writing of short stories that merge scientific and narrative genres (i.e., hybridised scientific narratives) about the socioscientific issue of biosecurity. This study employed a triangulation mixed methods research design, generating both quantitative and qualitative data, in order to investigate three research questions that examined the extent to which the students' participation in the study enhanced their scientific literacy; the extent to which the students demonstrated conceptual understanding of related scientific concepts through their written artefacts and in interviews about the artefacts; and the extent to which the students' participation in the project influenced their attitudes toward science and science learning. Three aspects of scientific literacy were investigated in this study: conceptual science understandings (a derived sense of scientific literacy), the students' transformation of scientific information in written stories about biosecurity (simple and expanded fundamental senses of scientific literacy), and attitudes toward science and science learning. The stories written by students in a selected case study class (N=26) were analysed quantitatively using a series of specifically-designed matrices that produce numerical scores that reflect students' developing fundamental and derived senses of scientific literacy. All students (N=152) also completed a Likert-style instrument (i.e., BioQuiz), pretest and posttest, that examined their interest in learning science, science self-efficacy, their perceived personal and general value of science, their familiarity with biosecurity issues, and their attitudes toward biosecurity. Socioscientific issues (SSI) education served as a theoretical framework for this study. It sought to investigate an alternative discourse with which students can engage in the context of SSI education, and the role of positive attitudes in engaging students in the negotiation of socioscientific issues. Results of the study have revealed that writing BioStories enhanced selected aspects of the participants' attitudes toward science and science learning, and their awareness and conceptual understanding of issues relating to biosecurity. Furthermore, the students' written artefacts alone did not provide an accurate representation of the level of their conceptual science understandings. An examination of these artefacts in combination with interviews about the students' written work provided a more comprehensive assessment of their developing scientific literacy. These findings support extensive calls for the utilisation of diversified writing-to-learn strategies in the science classroom, and therefore make a significant contribution to the writing-to-learn science literature, particularly in relation to the use of hybridised scientific genres. At the same time, this study presents the argument that the writing of hybridised scientific narratives such as BioStories can be used to complement the types of written discourse with which students engage in the negotiation of socioscientific issues, namely, argumentation, as the development of positive attitudes toward science and science learning can encourage students' participation in the discourse of science. The implications of this study for curricular design and implementation, and for further research, are also discussed.

Transferring knowledge through peer-reviewed assessment : the creation of a community of practice and the threats to its survival

While it is generally accepted in the learning and teaching literature that assessment is the single biggest influence on how students approach their learning, assessment methods within higher education are generally conservative and inflexible. Constrained by policy and accreditation requirements and the need for the explicit articulation of assessment standards for public accountability purposes, assessment tasks can fail to engage students or reflect the tasks students will face in the world of practice. Innovative assessment design can simultaneously deliver program objectives and active learning through a knowledge transfer process which increases student participation. This social constructivist view highlights that acquiring an understanding of assessment processes, criteria and standards needs active student participation. Within this context, a peer-assessed, weekly, assessment task was introduced in the first “serious” accounting subject offered as part of an undergraduate degree. The positive outcomes of this assessment innovation was that student failure rates declined 15%, tutorial participation increased fourfold, tutorial engagement increased six-fold and there was a 100% approval rating for the retention of the assessment task. In contributing to the core conference theme of “seismic” shifts within higher education, in stark contrast to the positive student response, staff-related issues of assessment conservatism and the necessity of meeting increasing research commitments, threatened the assessment task’s survival. These opposing forces to change have the potential to weaken the ability of higher education assessment arrangements to adequately serve either a new generation of students or the sector's community stakeholders.

Transferring knowledge through peer-reviewed assessment : the creation of a community of practice and the threats to its survival

While it is generally accepted in the learning and teaching literature that assessment is the single biggest influence on how students approach their learning, assessment methods within higher education are generally conservative and inflexible. Constrained by policy and accreditation requirements and the need for the explicit articulation of assessment standards for public accountability purposes, assessment tasks can fail to engage students or reflect the tasks students will face in the world of practice. Innovative assessment design can simultaneously deliver program objectives and active learning through a knowledge transfer process which increases student participation. This social constructivist view highlights that acquiring an understanding of assessment processes, criteria and standards needs active student participation. Within this context, a peer-assessed, weekly, assessment task was introduced in the first “serious” accounting subject offered as part of an undergraduate degree. The positive outcomes of this assessment innovation was that student failure rates declined 15%, tutorial participation increased fourfold, tutorial engagement increased six-fold and there was a 100% approval rating for the retention of the assessment task. In contributing to the core conference theme of “seismic” shifts within higher education, in stark contrast to the positive student response, staff-related issues of assessment conservatism and the necessity of meeting increasing research commitments, threatened the assessment task’s survival. These opposing forces to change have the potential to weaken the ability of higher education assessment arrangements to adequately serve either a new generation of students or the sector's community stakeholders.

‘Show Me Your Wiki and I’ll Show you Mine’: Using Online Interactive Media to Improve Academic Writing and Research in a Public Health Under-Graduate Cohort

The number of Internet users in Australia has been steadily increasing, with over 10.9 million people currently subscribed to an internet provider (ABS, 2011). Over the past year, the most avid users of the Internet were 15 – 24 year olds, with approximately 95% accessing the internet on a regular basis (ABS, Social Trends, 2011). While the internet has been described as fundamental to higher education students, social and leisure internet tools are also increasingly being used by these students to generate and maintain their social and professional networks and interactions (Duffy & Bruns 2006). Rapid technological advancements have enabled greater and faster access to information for learning and education (Hemmi et al, 2009; Glassman and Kang, 2011). As such, we sought to integrate interactive, online social media into the assessment profile of a Public Health undergraduate cohort at the Queensland University of Technology (QUT). The aim of this exercise was to engage students to both develop and showcase their research on a range of complex, contemporary health issues within the online forum of Wikispaces (http://www.wikispaces.com/) for review and critique by their peers. We applied Bandura’s Social Learning Theory (SLT) to analyse the interactive processes from which students developed deeper and more sustained learning, and via which their overall academic writing standards were raised. This paper outlines the assessment task, and the students’ feedback on their learning outcomes in relation to the Attentional, Retentional, Motor Reproduction, and Motivational Processes outlined by Bandura in SLT. We conceptualise the findings in a theoretical model, and discuss the implications for this approach within the broader tertiary environment.

Collaboration and competition on a Wiki : the praxis of online social learning to improve academic writing and research in under-graduate students

The number of internet users in Australia has been steadily increasing, with over 10.9 million people currently subscribed to an internet provider (ABS, 2011). Over the past year, the most avid users of the Internet were 15 – 24 year olds, with approximately 95% accessing the internet on a regular basis (ABS, Social Trends, 2011). While the internet, in particularly Web 2.0, has been described as fundamental to higher education students, social and leisure internet tools are also increasingly being used by these students to generate and maintain their social and professional networks and interactions (Duffy & Bruns, 2006). Rapid technological advancements have enabled greater and faster access to information for learning and education (Hemmi et al, 2009; Glassman & Kang, 2011). As such, we sought to integrate interactive, online social media into the assessment profile of a Public Health undergraduate cohort at the Queensland University of Technology (QUT). The aim of this exercise was to engage undergraduate students to both develop and showcase their research on a range of complex, contemporary health issues within the online forum of Wikispaces for review and critique by their peers. We applied Bandura’s Social Learning Theory (SLT) to analyse the interactive processes from which students developed deeper and more sustained learning, and via which their overall academic writing standards were enriched. This paper outlines the assessment task, and the students’ feedback on their learning outcomes in relation to the Attentional, Retentional, Motor Reproduction, and Motivational Processes outlined by Bandura in SLT. We conceptualise the findings in a theoretical model, and discuss the implications for this approach within the broader tertiary environment.