Using student feedback to construct an assessment rubric for a concept map in physiology

Constructing quality assessment rubrics can be challenging, especially when they are used for integrated, group-centered, applied learning. We describe a collaborative assessment task in which groups of second-year dentistry students developed a complex concept map. In groups of four, the students were given a written, simulated, medical history of a patient and required to construct a concept map illustrating relevant pathophysiological concepts and pharmacological interventions. This report describes a research project aimed at making educational goals of the task more explicit through investigating student and faculty member understandings of the criteria that might be used to assess the concept map. Information was gathered about the perceptions of students in relation to the learning goals associated with the task. These were compared with faculty member perceptions. The findings were used to develop an assessment rubric intended to be more accessible to learners. The new rubric used the language of both faculty members and students to more clearly represent expectations of each criterion and standard. This assessment rubric will be used in 2005 for the next phase of the project.

Modern methods for traditional tasks: Developing an electronic version of student fieldwork assessment at The University of Queensland.

Fieldwork placements are an integral part of many professional tertiary programmes. At The University of Queensland, Occupational Therapy students undertake block fieldwork affiliations off campus at a wide range of sites as part of their studies. Students’ fieldwork performance has traditionally been assessed using a hard copy format of the Student Placement Evaluation Form (SPEF), which is posted to the university on completion by the clinical supervisor. This project aimed to develop an electronic version of the UQ Occupational Therapy Student Placement Evaluation Form (SPEF), to allow the assessment to be completed and returned in an on line format. Practitioners had become very comfortable with using the existing print based form so in order to encourage and assist users to extend beyond their comfort zones, numerous steps were taken to ease the learning process including incorporating the existing page layout, consistent colour coding, considerable user instruction, testing and software enhancement cycles. Additionally, the e-version of the SPEF aimed to provide a range of benefits such as on screen assistance in the form of instructions, roll overs and feedback to supervisors, increased accuracy, faster completion, cost savings to the School, up to date design, improved security and confidential and anonymous storage of fieldwork results for potential future research.

The development and operation of the effective interpersonal relationship skills relevant to career development problems from staff assessment at an industrial research laboratory

This study concerns the application of a model of effective interpersonal relationships to problems arising from staff assessment at I.C.I. Ltd. Corporate Laboratory between 1972 and 1974. In collaboration with academic and industrial supervision, the study commenced with a survey of management and supervisor opinions about the effectiveness of current staff (work) relationships, with particular reference to the problem of recognising and developing creative potential. This survey emphasised a need to improve the relationships between staff in the staff assessment context. A survey of research into creativity emphasised the importance of the interpersonal environment for obtaining creative behaviour in an organisation context. A further survey of theories of how inter­personal behaviour related to personal creativity (therapeutic psychology) provided a model of effective interpersonal behaviour (Carkhuff, 1969) that could be applied to the organisation context of staff assessment. The objective of the project was redefined as a need to improve the conditions of interpersonal behaviour in relation to certain (career development) problems arising from staff assessment practices. In order to demonstrate the application of the model of effective interpersonal behaviour, the research student recorded interviews between himself and members of staff designed to develop and operate the dimensions of the model. Different samples of staff were used to develop the 'facilitative' and the 'action oriented' dimensions of bahaviour, and then for the operation of a helping programme (based on vocational guidance tests). These interactions have been analysed, according to the scales of measurement in the model ana the results are presented in case study form in this thesis. At each stage of the project, results and conclusions were presented to the sponsoring organisation (e.g. industrial supervisor) in order to assess their (subjective) opinion of relevance to the organ­ isation. Finally, recommendations on further actions towards general improvement of the work relationships in the laboratory were presented in a brief report to the sponsor.

Achieving continuous improvement through self-assessment

In 2007 the UK Office of Government Commerce was mandated to carry out Procurement Capability Reviews (PCR) across the 16 top spending UK Government Departments. Since then, this programme has evolved into a self assessment based approach which is markedly different from the original approach. Will the move from a centre-led strategic review of procurement capability to a department-led model based on self assessment continue to strengthen and improve procurement capability across Central Civil Government? OGC is currently working with UK Government Departments to carry out their PCRs using a self-assessment tool which incorporates qualitative and quantitative measures. Results are generated based on a capability maturity model. The results are assured independently. OGC expectations are that tangible and measurable capability improvements will be realised when departments embed the self-assessment model and implement the findings as part of a continuous improvement regime. This paper is a case study, using some relevant literature to reflect on past and possible future development of the PCR self assessment scheme.

Enhancing student learning experience through a novel electronic coursework assessment and feedback management system

Recent National Student Surveys revealed that many U.K. university students are dissatisfied with the timeliness and usefulness of the feedback received from their tutors. Ensuring timeliness in marking often results in a reduction in the quality of feedback. In Computer Science where learning relies on practising and learning from mistakes, feedback that pin-points errors and explains means of improvement is important to achieve a good student learning experience. Though suitable use of Information and Communication Technology should alleviate this problem, existing Virtual Learning Environments and e-Assessment applications such as Blackboard/WebCT, BOSS, MarkTool and GradeMark are inadequate to support a coursework assessment process that promotes timeliness and usefulness of feedback while maintaining consistency in marking involving multiple tutors. We have developed a novel Internet application, called eCAF, for facilitating an efficient and transparent coursework assessment and feedback process. The eCAF system supports detailed marking scheme editing and enables tutors to use such schemes to pin-point errors in students' work so as to provide helpful feedback efficiently. Tutors can also highlight areas in a submitted work and associate helpful feedback that clearly links to the identified mistakes and the respective marking criteria. In light of the results obtained from a recent trial of eCAF, we discuss how the key features of eCAF may facilitate an effective and efficient coursework assessment and feedback process.

Student input in the development of a marking scheme for improved engineering logbooks

Engineers logbooks are an important part of the CDIO process, as a prequel to the logooks they will be expected to keep when in industry. Previously however, students logbooks were insufficient and students did not appear to appreciate the importance of the logbooks or how they would be assessed. In an attempt to improve the students understanding and quality of logbooks, a group of ~100 1st year CDIO students were asked to collaboratively develop a marking matrix with the tutors. The anticipated outcome was that students would have more ownership in, and a deeper understanding of, the logbook and what is expected from the student during assessment. A revised marking matrix was developed in class and a short questionnaire was implemented on delivery of the adapted matrix to gauge the students response to the process. Marks from the logbooks were collected twice during teaching period one and two and compared to marks from previous years. This poster will deliver the methodology and outcomes for this venture.

An assessment of web-based textbook supplementary material to support student learning

There is an increasing trend by publishers to provide supplementary learning materials with text books in order to improve the learning experience and thus ultimately improve text book sales. This study will aim to establish the use of these materials and their relevance to students in terms of supporting student learning. The materials include multiple choice test banks, animated demonstrations, simulations, quizzes and electronic versions of the text. The study will focus on the extensive library of web-based learning materials available on the ‘WileyPlus’ web platform which accompanies the textbook ‘Operations Management’, 2nd edition authored by A. Greasley and published by John Wiley and Sons Ltd.

Can computer-based testing achieve quality and efficiency in assessment?

This paper reports findings of a two year study concerning the development and implementation of a general-purpose computer-based assessment (CBA) system at a UK University. Data gathering took place over a period of nineteen months, involving a number of formative and summative assessments. Approximately 1,000 students, drawn from undergraduate courses, were involved in the exercise. The techniques used in gathering data included questionnaires, observation, interviews and an analysis of student scores in both conventional examinations and computer-based assessments. Comparisons with conventional assessment methods suggest that the use of CBA techniques may improve the overall performance of students. However it is clear that the technique must not be seen as a "quick fix" for problems such as rising student numbers. If one accepts that current systems test only a relatively narrow range of skills, then the hasty implementation of CBA systems will result in a distorted and inaccurate view of student performance. In turn, this may serve to reduce the overall quality of courses and - ultimately - detract from the student learning experience. On the other hand, if one adopts a considered and methodical approach to computer-based assessment, positive benefits might include increased efficiency and quality, leading to improved student learning.

Specialist business skills of international students

This paper follows on from that presented at the last BEST conference in Edinburgh (Higson & Hamilton-Jones(2004)). At that stage, the authors outlined their initial research work with students studying on the yearlong International Foundation programmes. at three local FE Colleges allied to Aston University. The research (funded by the University's Teaching Quality Enhancement Funds (TQEF) involved questionnaires and interviews with staff and students (the latter all from overseas). it aimed to identify ways to improve the learning experience of students on the International Foundation programmes, to aid their smooth transition to full degree programmes in Business and Management and to improve the progression rates of such students while studying at Aston. The initial research findings were used to design a module for those students' progress to degree programmes in Aston Business School. This paper discusses how the module was designed, its content and the assessment methods used to help determine whether students are achieving the learning outcomes. The basic principle was to identify areas of study where the International Foundation Programme students needed help in order to improve their learning styles to assist them with the requirements of other modules that they would be studying during their time at Aston. Particular emphasis was put on the need to develop active learners who were not disadvantaged by their lack of awareness of UK culture and society and who were as comfortable performing written work under examination conditions or presenting orally as their UK counterparts. An additional aim was to prepare these students for the placement year which was a compulsory part of their degree. The module, therefore, comprises a range of inputs for a number of staff, a company visit, weekly reflective learning leading to Personal Development Plan (PDP) work, formal examinations, presentations, group work •and individual case studies. This paper also reports on the initial reaction of the students and tutors to the new learning experience with currently 30 participants undertaking the module. Provisional findings suggest that the International Foundation programme has prepared the students well for degree-level work and that as a group of international students they are much more analytical and, after studying the module interactive than their counterparts who have come directly onto Aston degrees. It has shown them still to be quite passive learners, comfortable with facts and lecture-style learning environments, but less comfortable when asked to use their own initiatives. Continuing progress needs to be made in terms of encouraging them to develop a reflective approach to learning with the students taking some time to feel comfortable with an analytical approach to learning. In addition, im account of the students' reactions to having to work through a formal (PDP) and the results of their first assessments will be provided. At Aston, this work is being used as a pilot to recognise good practice with regards to work with further groups of international students. it is hoped that this would have widespread application across the sector.

Student response to a pub quiz style first year psychology assessment

The deterioration in staff-student ratios in UK higher education has had a disproportionate impact on assessment and feedback, meaning that contemporary students may have fewer assessments and much less feedback than a generation ago (Gibbs, 2006). Early use of a quiz assessment may offer a blend of social benefits (social comparison, shared problem solving leading to engagement, belonging and continuation), academic benefits (early formative assessment, immediate feedback) and administrative benefits (on-the-spot verbal marking and feedback to 230 students simultaneously). This study sought student views on the acceptability and contribution to learning of the quiz. Social benefits were apparent but difficulties in creating questions to elicit deeper reasoning and problem solving are discussed and the quiz had limited pedagogic value in the eyes of participants. The use of assertion-reason questions are considered as a way of taking the table quiz to a higher level and extending its pedagogic value.