Enhancing student learning experience through a novel electronic coursework assessment and feedback management system

Recent National Student Surveys revealed that many U.K. university students are dissatisfied with the timeliness and usefulness of the feedback received from their tutors. Ensuring timeliness in marking often results in a reduction in the quality of feedback. In Computer Science where learning relies on practising and learning from mistakes, feedback that pin-points errors and explains means of improvement is important to achieve a good student learning experience. Though suitable use of Information and Communication Technology should alleviate this problem, existing Virtual Learning Environments and e-Assessment applications such as Blackboard/WebCT, BOSS, MarkTool and GradeMark are inadequate to support a coursework assessment process that promotes timeliness and usefulness of feedback while maintaining consistency in marking involving multiple tutors. We have developed a novel Internet application, called eCAF, for facilitating an efficient and transparent coursework assessment and feedback process. The eCAF system supports detailed marking scheme editing and enables tutors to use such schemes to pin-point errors in students' work so as to provide helpful feedback efficiently. Tutors can also highlight areas in a submitted work and associate helpful feedback that clearly links to the identified mistakes and the respective marking criteria. In light of the results obtained from a recent trial of eCAF, we discuss how the key features of eCAF may facilitate an effective and efficient coursework assessment and feedback process.

An assessment of web-based textbook supplementary material to support student learning

There is an increasing trend by publishers to provide supplementary learning materials with text books in order to improve the learning experience and thus ultimately improve text book sales. This study will aim to establish the use of these materials and their relevance to students in terms of supporting student learning. The materials include multiple choice test banks, animated demonstrations, simulations, quizzes and electronic versions of the text. The study will focus on the extensive library of web-based learning materials available on the ‘WileyPlus’ web platform which accompanies the textbook ‘Operations Management’, 2nd edition authored by A. Greasley and published by John Wiley and Sons Ltd.

Student response to a pub quiz style first year psychology assessment

The deterioration in staff-student ratios in UK higher education has had a disproportionate impact on assessment and feedback, meaning that contemporary students may have fewer assessments and much less feedback than a generation ago (Gibbs, 2006). Early use of a quiz assessment may offer a blend of social benefits (social comparison, shared problem solving leading to engagement, belonging and continuation), academic benefits (early formative assessment, immediate feedback) and administrative benefits (on-the-spot verbal marking and feedback to 230 students simultaneously). This study sought student views on the acceptability and contribution to learning of the quiz. Social benefits were apparent but difficulties in creating questions to elicit deeper reasoning and problem solving are discussed and the quiz had limited pedagogic value in the eyes of participants. The use of assertion-reason questions are considered as a way of taking the table quiz to a higher level and extending its pedagogic value.

Collaborative design of assessment criteria to improve undergraduate student engagement and performance

Student engagement is vital in enhancing the student experience and encouraging deeper learning. Involving students in the design of assessment criteria is one way in which to increase student engagement. In 2011, a marking matrix was used at Aston University (UK) for logbook assessment (Group One) in a project-based learning module. The next cohort of students in 2012 (Group Two) were asked to collaboratively redesign the matrix and were given a questionnaire about the exercise. Group Two initially scored a lower average logbook mark than Group One. However, Group Two showed the greatest improvement between assessments, and the quality of, and commitment to, logbooks was noticeably improved. Student input resulted in a more defined, tougher mark scheme. However, this provided an improved feedback system that gave more scope for self-improvement. The majority of students found the exercise incorporated their ideas, enhanced their understanding, and was useful in itself.

Encouraging student to reflect on their employment experience

The purpose behind this case study is to share with a wider audience of placement officers, tutors and those who are involved in the management of placement students or employment of graduates, the approach taken to encourage reflective learning in undergraduate placement students at Aston Business School. Reflective learning forms an important foundation of the placement year at Aston Business School, where a professional placement is a mandatory element of the four year degree, for all Home/EU students (optional for International students) who are taking a Single Honours degree (i.e. a fully business programme). The placement year is not compulsory for those students taking a Combined Honours degree (i.e. a degree where two unrelated subjects are studied), although approximately 50% of those students taking an Aston Business School subject opt to take a placement year. Students spend their year out undertaking a ‘proper’ job within a company or public sector organisation. They are normally paid a reasonable salary for their work (in 2004/5 the average advertised salary was £13,700 per annum). The placement year is assessed, carrying credits which amount to a contribution of 10% towards the students’ final degree. The assessment methods used require the students to submit an academic essay relating theory to practice, a factual report about the company which can be of use to future students, and a log book, the latter being the reflective piece of work. Encouragement to reflect on the placement year has always been an important feature of Aston Business School’s approach to learning. More recently, however, feedback from employers indicated that, although our students have excellent employability skills, “they do not think about them” (Aston Business School Advisory Panel, 2001). We, therefore, began some activities which would encourage students to go beyond the mere acquisition of skills and knowledge. This work became the basis of a programme of introductions to reflective learning, mentoring and awareness of different learning styles written up in Higson and Jones (2002). The idea was to get students used to the idea of reflection on their experiences well before they entered the placement year.

An assessment of the learning benefits of using a web-based learning environment when teaching accounting

The impact and use of information and communication technology on learning outcomes for accounting students is not well understood. This study investigates the impact of design features of Blackboard 1 used as aWeb-based Learning Environment (WBLE) in teaching undergraduate accounting students. Specifically, this investigation reports on a number of Blackboard design features (e.g. delivery of lecture notes, announcements, online assessment and model answers) used to deliver learning materials regarded as necessary to enhance learning outcomes. Responses from 369 on-campus students provided data to develop a regression model that seeks to explain enhanced participation and mental effort. The final regression shows that student satisfaction with the use of a WBLE is associated with five design features or variables. These include usefulness and availability of lecture notes, online assessment, model answers, and online chat.

Mapping teaching, learning and assessment in the MPharm in UK schools of pharmacy

Aim To undertake a national study of teaching, learning and assessment in UK schools of pharmacy. Design Triangulation of course documentation, 24 semi-structured interviews undertaken with 29 representatives from the schools and a survey of all final year students (n=1,847) in the 15 schools within the UK during 2003–04. Subjects and setting All established UK pharmacy schools and final year MPharm students. Outcome measures Data were combined and analysed under the topics of curriculum, teaching and learning, assessment, multi-professional teaching and learning, placement education and research projects. Results Professional accreditation was the main driver for curriculum design but links to preregistration training were poor. Curricula were consistent but offered little student choice. On average half the curriculum was science-based. Staff supported the science content but students less so. Courses were didactic but schools were experimenting with new methods of learning. Examinations were the principal form of assessment but the contribution of practice to the final degree ranged considerably (21–63%). Most students considered the assessment load to be about right but with too much emphasis upon knowledge. Assessment of professional competence was focused upon dispensing and pharmacy law. All schools undertook placement teaching in hospitals but there was little in community/primary care. There was little inter-professional education. Resources and logistics were the major limiters. Conclusions There is a need for an integrated review of the accreditation process for the MPharm and preregistration training and redefinition of professional competence at an undergraduate level.

The development and operation of the effective interpersonal relationship skills relevant to career development problems from staff assessment at an industrial research laboratory

This study concerns the application of a model of effective interpersonal relationships to problems arising from staff assessment at I.C.I. Ltd. Corporate Laboratory between 1972 and 1974. In collaboration with academic and industrial supervision, the study commenced with a survey of management and supervisor opinions about the effectiveness of current staff (work) relationships, with particular reference to the problem of recognising and developing creative potential. This survey emphasised a need to improve the relationships between staff in the staff assessment context. A survey of research into creativity emphasised the importance of the interpersonal environment for obtaining creative behaviour in an organisation context. A further survey of theories of how inter­personal behaviour related to personal creativity (therapeutic psychology) provided a model of effective interpersonal behaviour (Carkhuff, 1969) that could be applied to the organisation context of staff assessment. The objective of the project was redefined as a need to improve the conditions of interpersonal behaviour in relation to certain (career development) problems arising from staff assessment practices. In order to demonstrate the application of the model of effective interpersonal behaviour, the research student recorded interviews between himself and members of staff designed to develop and operate the dimensions of the model. Different samples of staff were used to develop the 'facilitative' and the 'action oriented' dimensions of bahaviour, and then for the operation of a helping programme (based on vocational guidance tests). These interactions have been analysed, according to the scales of measurement in the model ana the results are presented in case study form in this thesis. At each stage of the project, results and conclusions were presented to the sponsoring organisation (e.g. industrial supervisor) in order to assess their (subjective) opinion of relevance to the organ­ isation. Finally, recommendations on further actions towards general improvement of the work relationships in the laboratory were presented in a brief report to the sponsor.

Student input in the development of a marking scheme for improved engineering logbooks

Engineers logbooks are an important part of the CDIO process, as a prequel to the logooks they will be expected to keep when in industry. Previously however, students logbooks were insufficient and students did not appear to appreciate the importance of the logbooks or how they would be assessed. In an attempt to improve the students understanding and quality of logbooks, a group of ~100 1st year CDIO students were asked to collaboratively develop a marking matrix with the tutors. The anticipated outcome was that students would have more ownership in, and a deeper understanding of, the logbook and what is expected from the student during assessment. A revised marking matrix was developed in class and a short questionnaire was implemented on delivery of the adapted matrix to gauge the students response to the process. Marks from the logbooks were collected twice during teaching period one and two and compared to marks from previous years. This poster will deliver the methodology and outcomes for this venture.

Can computer-based testing achieve quality and efficiency in assessment?

This paper reports findings of a two year study concerning the development and implementation of a general-purpose computer-based assessment (CBA) system at a UK University. Data gathering took place over a period of nineteen months, involving a number of formative and summative assessments. Approximately 1,000 students, drawn from undergraduate courses, were involved in the exercise. The techniques used in gathering data included questionnaires, observation, interviews and an analysis of student scores in both conventional examinations and computer-based assessments. Comparisons with conventional assessment methods suggest that the use of CBA techniques may improve the overall performance of students. However it is clear that the technique must not be seen as a "quick fix" for problems such as rising student numbers. If one accepts that current systems test only a relatively narrow range of skills, then the hasty implementation of CBA systems will result in a distorted and inaccurate view of student performance. In turn, this may serve to reduce the overall quality of courses and - ultimately - detract from the student learning experience. On the other hand, if one adopts a considered and methodical approach to computer-based assessment, positive benefits might include increased efficiency and quality, leading to improved student learning.

Engaging employers as partners in work-based learning assessment:proposal for a quality enhancement framework

This paper seeks to advance research and practice related to the role of employers in all stages of the assessment process of work-based learning (WBL) within a tripartite relationship of higher education institution (HEI), student and employer. It proposes a research-informed quality enhancement framework to develop good practice in engaging employers as partners in assessment. The Enhancement Framework comprises three dimensions, each of which includes elements and questions generated by the experiences of WBL students, HEI staff and employers. The three dimensions of the Enhancement Framework are: 1. ‘premises of assessment’ encompassing issues of learning, inclusion, standards and value; 2. ‘practice’, encompassing stages of assessment made up of course design, assessment task, responsibilities, support, grading and feedback; 3. ‘communication of assessment’ with the emphasis on role clarity, language and pathways. With its prompt questions, the Enhancement Framework may be used as a capacity-building tool for promoting, sustaining, benchmarking and evaluating productive dialogue and critical reflection about assessment between WBL partners. The paper concludes by emphasising the need for professional development as well as policy and research development, so that assessment in WBL can more closely correspond to the potentially transformative nature of the learning experience.

Evaluating choices:the role of corpora in translation choices and target text assessment

Translation training in the university context needs to train students in the processes, in order to enhance and optimise the product as outcome of these processes. Evaluation of a target text as product has often been accused of being a subjective process, which does not easily lend itself to the type of feedback that could enable students to apply criteria more widely. For students, it often seems as though they make different inappropriate or incorrect choices every time they translate a new text, and the learning process appears unpredictable and haphazard. Within functionalist approaches to translation, with their focus on the target text in terms of functional adequacy to the intended purpose, as stipulated in the translation brief, there are guidelines for text production that can help to develop a more systematic approach not only to text production, but also to translation evaluation. In the context of a focus on user knowledge needs, target language conventions and acceptability, the use of corpora is an indispensable tool for the trainee translator. Evaluation can take place against the student's own reasoned selection process, based on hard evidence, against criteria which currently obtain in the TL and the TL culture. When trainee and evaluator work within the same guidelines, there is more scope for constructive learning and feedback.

Complex skills and academic writing:a review of evidence about the types of learning required to meet core assessment criteria

Assessment criteria are increasingly incorporated into teaching, making it important to clarify the pedagogic status of the qualities to which they refer. We reviewed theory and evidence about the extent to which four core criteria for student writing-critical thinking, use of language, structuring, and argument-refer to the outcomes of three types of learning: generic skills learning, a deep approach to learning, and complex learning. The analysis showed that all four of the core criteria describe to some extent properties of text resulting from using skills, but none qualify fully as descriptions of the outcomes of applying generic skills. Most also describe certain aspects of the outcomes of taking a deep approach to learning. Critical thinking and argument correspond most closely to the outcomes of complex learning. At lower levels of performance, use of language and structuring describe the outcomes of applying transferable skills. At higher levels of performance, they describe the outcomes of taking a deep approach to learning. We propose that the type of learning required to meet the core criteria is most usefully and accurately conceptualized as the learning of complex skills, and that this provides a conceptual framework for maximizing the benefits of using assessment criteria as part of teaching. © 2006 Taylor & Francis.