Improving E-Assessment Systems With A focus On Adaptive Feedback And Marking

Institutions involved in the provision of tertiary education across Europe are feeling the pinch. European universities, and other higher education (HE) institutions, must operate in a climate where the pressure of government spending cuts (Garben, 2012) is in stark juxtaposition to the EU’s strategy to drive forward and maintain a growth of student numbers in the sector (eurostat, 2015).

In order to remain competitive, universities and HE institutions are making ever-greater use of electronic assessment (E-Assessment) systems (Chatzigavriil et all, 2015; Ferrell, 2012). These systems are attractive primarily because they offer a cost-effect and scalable approach for assessment. In addition to scalability, they also offer reliability, consistency and impartiality; furthermore, from the perspective of a student they are most popular because they can offer instant feedback (Walet, 2012).

There are disadvantages, though.

First, feedback is often returned to a student immediately on competition of their assessment. While it is possible to disable the instant feedback option (this is often the case during an end of semester exam period when assessment scores must be can be ratified before release), however, this option tends to be a global ‘all on’ or ‘all off’ configuration option which is controlled centrally rather than configurable on a per-assessment basis.

If a formative in-term assessment is to be taken by multiple groups of
students, each at different times, this restriction means that answers to each question will be disclosed to the first group of students undertaking the assessment. As soon as the answers are released “into the wild” the academic integrity of the assessment is lost for subsequent student groups.

Second, the style of feedback provided to a student for each question is often limited to a simple ‘correct’ or ‘incorrect’ indicator. While this type of feedback has its place, it often does not provide a student with enough insight to improve their understanding of a topic that they did not answer correctly.

Most E-Assessment systems boast a wide range of question types including Multiple Choice, Multiple Response, Free Text Entry/Text Matching and Numerical questions. The design of these types of questions is often quite restrictive and formulaic, which has a knock-on effect on the quality of feedback that can be provided in each case.

Multiple Choice Questions (MCQs) are most prevalent as they are the most prescriptive and therefore most the straightforward to mark consistently. They are also the most amenable question types, which allow easy provision of meaningful, relevant feedback to each possible outcome chosen.
Text matching questions tend to be more problematic due to their free text entry nature. Common misspellings or case-sensitivity errors can often be accounted for by the software but they are by no means fool proof, as it is very difficult to predict in advance the range of possible variations on an answer that would be considered worthy of marks by a manual marker of a paper based equivalent of the same question.

Numerical questions are similarly restricted. An answer can be checked for accuracy or whether it is within a certain range of the correct answer, but unless it is a special purpose-built mathematical E-Assessment system the system is unlikely to have computational capability and so cannot, for example, account for “method marks” which are commonly awarded in paper-based marking.

From a pedagogical perspective, the importance of providing useful formative feedback to students at a point in their learning when they can benefit from the feedback and put it to use must not be understated (Grieve et all, 2015; Ferrell, 2012).

In this work, we propose a number of software-based solutions, which will overcome the limitations and inflexibilities of existing E-Assessment systems.

A tool for the evaluation of slaughter horse welfare during unloading

This study aimed to develop a scientific and practical tool to be used to assess horse welfare after commercial transport over long journeys. A set of physical, behavioural and environmental measures was selected, covering welfare aspects of both transport and unloading procedures. The protocol was field-tested on 51 intra-EU commercial transports arriving at different sites in Italy. Univariate analysis was implemented to look for associations between the input variables (environmental hazards potentially affecting the animal well-being during long transports) and the outcome variables (direct evaluation of the animal condition). No severe welfare impairments were recorded (ie dead on arrival, severe injuries, non-ambulatory animals), while milder ones were more frequent at unloading (eg slipping; 36.7%, reluctance to move; 9.6%). Correlations emerged between ramp slope and falling; type of ramp floor and slipping; fast gait and the presence of gaps between the ramp and the floor. The horses' behaviour was also related to the type of handling procedure used. The measures were repeatable and practical to apply and score during real-time unloading. This work provides a sound basis for a new and practical welfare assessment tool for horses travelling over long journeys. Careful and constant application of this protocol would provide stakeholders with the opportunity to track and monitor changes in the industry over time, as well as to identify high risk areas in transport routines.

Physical rehabilitation interventions for adult patients during critical illness: an overview of systematic reviews

Background Physical rehabilitation interventions aim to ameliorate the effects of critical illness-associated muscle dysfunction in survivors. We conducted an overview of systematic reviews (SR) evaluating the effect of these interventions across the continuum of recovery.

Methods Six electronic databases (Cochrane Library, CENTRAL, DARE, Medline, Embase, and Cinahl) were searched. Two review authors independently screened articles for eligibility and conducted data extraction and quality appraisal. Reporting quality was assessed and the Grading of Recommendations Assessment, Development and Evaluation approach applied to summarise overall quality of evidence.

Results Five eligible SR were included in this overview, of which three included meta-analyses. Reporting quality of the reviews was judged as medium to high. Two reviews reported moderate-to-high quality evidence of the beneficial effects of physical therapy commencing during intensive care unit (ICU) admission in improving critical illness polyneuropathy/myopathy, quality of life, mortality and healthcare utilisation. These interventions included early mobilisation, cycle ergometry and electrical muscle stimulation. Two reviews reported very low to low quality evidence of the beneficial effects of electrical muscle stimulation delivered in the ICU for improving muscle strength, muscle structure and critical illness polyneuropathy/myopathy. One review reported that due to a lack of good quality randomised controlled trials and inconsistency in measuring outcomes, there was insufficient evidence to support beneficial effects from physical rehabilitation delivered post-ICU discharge.

Conclusions Patients derive short-term benefits from physical rehabilitation delivered during ICU admission. Further robust trials of electrical muscle stimulation in the ICU and rehabilitation delivered following ICU discharge are needed to determine the long-term impact on patient care. This overview provides recommendations for design of future interventional trials and SR.