Improving E-Assessment Systems With A focus On Adaptive Feedback And Marking

Institutions involved in the provision of tertiary education across Europe are feeling the pinch. European universities, and other higher education (HE) institutions, must operate in a climate where the pressure of government spending cuts (Garben, 2012) is in stark juxtaposition to the EU’s strategy to drive forward and maintain a growth of student numbers in the sector (eurostat, 2015).

In order to remain competitive, universities and HE institutions are making ever-greater use of electronic assessment (E-Assessment) systems (Chatzigavriil et all, 2015; Ferrell, 2012). These systems are attractive primarily because they offer a cost-effect and scalable approach for assessment. In addition to scalability, they also offer reliability, consistency and impartiality; furthermore, from the perspective of a student they are most popular because they can offer instant feedback (Walet, 2012).

There are disadvantages, though.

First, feedback is often returned to a student immediately on competition of their assessment. While it is possible to disable the instant feedback option (this is often the case during an end of semester exam period when assessment scores must be can be ratified before release), however, this option tends to be a global ‘all on’ or ‘all off’ configuration option which is controlled centrally rather than configurable on a per-assessment basis.

If a formative in-term assessment is to be taken by multiple groups of
students, each at different times, this restriction means that answers to each question will be disclosed to the first group of students undertaking the assessment. As soon as the answers are released “into the wild” the academic integrity of the assessment is lost for subsequent student groups.

Second, the style of feedback provided to a student for each question is often limited to a simple ‘correct’ or ‘incorrect’ indicator. While this type of feedback has its place, it often does not provide a student with enough insight to improve their understanding of a topic that they did not answer correctly.

Most E-Assessment systems boast a wide range of question types including Multiple Choice, Multiple Response, Free Text Entry/Text Matching and Numerical questions. The design of these types of questions is often quite restrictive and formulaic, which has a knock-on effect on the quality of feedback that can be provided in each case.

Multiple Choice Questions (MCQs) are most prevalent as they are the most prescriptive and therefore most the straightforward to mark consistently. They are also the most amenable question types, which allow easy provision of meaningful, relevant feedback to each possible outcome chosen.
Text matching questions tend to be more problematic due to their free text entry nature. Common misspellings or case-sensitivity errors can often be accounted for by the software but they are by no means fool proof, as it is very difficult to predict in advance the range of possible variations on an answer that would be considered worthy of marks by a manual marker of a paper based equivalent of the same question.

Numerical questions are similarly restricted. An answer can be checked for accuracy or whether it is within a certain range of the correct answer, but unless it is a special purpose-built mathematical E-Assessment system the system is unlikely to have computational capability and so cannot, for example, account for “method marks” which are commonly awarded in paper-based marking.

From a pedagogical perspective, the importance of providing useful formative feedback to students at a point in their learning when they can benefit from the feedback and put it to use must not be understated (Grieve et all, 2015; Ferrell, 2012).

In this work, we propose a number of software-based solutions, which will overcome the limitations and inflexibilities of existing E-Assessment systems.

A standing ovation for Nigel: An informal study

This article analyses a series of emails thanking Nigel for his stewardship of JASSS and the characteristics of their authors. It identifies a correlation between two measures of author activity in social simulation research, but no pattern between these activity measures and the email timing. Instead, the sequence suggests a classic standing ovation effect.

Pharmacist-led feedback workshops increase appropriate prescribing of antimicrobials

Objectives To investigate whether and how structured feedback sessions can increase rates of appropriate antimicrobial prescribing by junior doctors.

Methods This was a mixed-methods study, with a conceptual orientation towards complexity and systems thinking. Fourteen junior doctors, in their first year of training, were randomized to intervention (feedback) and 21 to control (routine practice) groups in a single UK teaching hospital. Feedback on their antimicrobial prescribing was given, in writing and via group sessions. Pharmacists assessed the appropriateness of all new antimicrobial prescriptions 2 days per week for 6 months (46 days). The mean normalized prescribing rates of suboptimal to all prescribing were compared between groups using the t-test. Thematic analysis of qualitative interviews with 10 participants investigated whether and how the intervention had impact.

Results Data were collected on 204 prescriptions for 166 patients. For the intervention group, the mean normalized rate of suboptimal to all prescribing was 0.32 ± 0.36; for the control group, it was 0.68 ± 0.36. The normalized rates of suboptimal prescribing were significantly different between the groups (P = 0.0005). The qualitative data showed that individuals' prescribing behaviour was influenced by a complex series of dynamic interactions between individual and social variables, such as interplay between personal knowledge and the expectations of others.

Conclusions The feedback intervention increased appropriate prescribing by acting as a positive stimulus within a complex network of behavioural influences. Prescribing behaviour is adaptive and can be positively influenced by structured feedback. Changing doctors' perceptions of acceptable, typical and best practice could reduce suboptimal antimicrobial prescribing.

Transposing Musical Skill: Sonification of movement as concurrent augmented feedback enhances learning in a bimanual task

Concurrent feedback provided during acquisition can enhance performance of novel tasks. The ‘guidance hypothesis’ predicts that feedback provision leads to dependence and poor performance in its absence. However, appropriately-structured feedback information provided through sound (‘sonification’) may not be subject to this effect. We test this directly using a rhythmic bimanual shape-tracing task in which participants learned to move at a 4:3 timing ratio. Sonification of movement and demonstration was compared to two other learning conditions: (1) sonification of task demonstration alone and (2) completely silent practice (control). Sonification of movement emerged as the most effective form of practice, reaching significantly lower error scores than control. Sonification of solely the demonstration, which was expected to benefit participants by perceptually unifying task requirements, did not lead to better performance than control. Good performance was maintained by participants in the sonification condition in an immediate retention test without feedback, indicating that the use of this feedback can overcome the guidance effect. On a 24-hour retention test, performance had declined and was equal between groups. We argue that this and similar findings in the feedback literature are best explained by an ecological approach to motor skill learning which places available perceptual information at the highest level of importance.

The daily life of informal caregivers of elderly

Objectives: To identify the reasons why the informal caregivers to take care of the elderly; to identify enjoyed support and the problems/needs senses. Methods: A descriptive, exploratory and cross study. The sample was 366 informal caregivers of the elderly residents in the District of Évora (Alentejo). Applied a questionnaire, which identified the reasons that led to cohabitation, the difficulties experienced by caregivers, experienced changes in their health and support they receive. Results: Most caregivers are women, with a mean age of 54 years. The main reason of care was elderly disease. These caregivers have changed in relaxation and leisure activities, on the organization of day-to-day and on economic aspects. They receive support from health institutions, social security and firemen’s. Conclusions: Family is the support in the disease, despite the difficulties, particularly in relaxation and leisure activities, organization of day-to-day and economic matters. They requested support in healthcare, transportation and economic aid.

Uma abordagem curricular em matemática no 3º ciclo do ensino básico: um estudo de caso em geometria

Este trabalho debruça-se sobre as práticas de Ensino de Geometria no 3º ciclo do Ensino Básico tendo como principal foco de investigação as práticas de ensino da professora investigadora. O estudo empírico foi realizado em três turmas de 9º ano, da professora investigadora, de Abril a Junho de 2005, numa Escola da região de Trás-os-Montes (designada por Escola A). O trabalho realizado pela professora investigadora com alunos dessas turmas (no 7º em 2002/03, no 8º ano em 2003/04 e no 9º ano até Abril de 2005) constituiu o contexto precursor do estudo. O planeamento, desenvolvimento e reflexão sobre as práticas desenvolvidas foram realizados de forma colaborativa entre a professora investigadora e a critical friend, professora de Matemática a leccionar Matemática a turmas de 9º ano, numa outra Escola, da mesma região de Trás-os-Montes (designada por Escola B). A gestão curricular orientada por pressupostos legais em vigor e por resultados da investigação em Educação e Didáctica da Matemática foi realizada considerando o conhecimento informal e tácito de cada aluno; visou maximizar a aprendizagem matemática de todos os alunos em cada turma mas tendo a preocupação subjacente de proporcionar experiências de aprendizagem diversificadas de forma a favorecer todo o tipo de estilos de aprendizagem. O problema de investigação foi formulado a partir da seguinte questão: Como articular os esforços realizados e desenvolvidos pelo professor na sala de aula, de forma coerente e exequível, para promover aprendizagens significativas nos alunos, nomeadamente no domínio da Geometria? Face ao problema de investigação foram abordadas e estudadas as questões de investigação seguintes: 1. Quais são as características da experiência matemática proporcionada e qual é a sua relação com o que os alunos aprenderam? 2. Quais as características da avaliação implementada enquanto processo regulador das aprendizagens? 3. Qual a relação entre as tarefas de aprendizagem planificadas e as experiências de aprendizagem matemática proporcionadas? 4. Qual o papel das conferências com a critical friend (no desenvolvimento das experiências matemáticas proporcionadas) na gestão curricular? Este trabalho seguiu uma metodologia de investigação de natureza qualitativa baseado num estudo de caso, com uma vertente de investigação-acção, a partir de uma abordagem curricular em Geometria no 3º ciclo do Ensino Básico: a da professora investigadora. A observação participante e uma grande diversidade de documentos recolhidos (tarefas iniciais e reformuladas, cópias de relatórios, diários de bordo, transcrições das gravações áudio das conferências, etc.) constituíram as principais fontes de dados. Também foram considerados dados provenientes de instrumentos mais quantitativos, como de testes e de questionários. A análise de dados tomou como base a unidade de tempo de meio bloco de aulas (45 minutos) e as fases didácticas de realização de tarefas numa escala de investigação meso de forma a respeitar o trabalho na sala de aula e a sua complexidade. A triangulação de diversas fontes de dados permitiu apresentar: i) a experiência matemática proporcionada a partir das tarefas implementadas e os diferentes papéis assumidos pelo professor, pelos pequenos grupos de alunos ou pelo grupo turma; ii) as práticas avaliativas onde foram evidenciadas as diferentes formas de avaliação formativa e o feedback distribuído; iii) o envolvimento induzido e promovido nos alunos relativamente à sua aprendizagem matemática; iv) o papel crucial da critical friend na abordagem curricular em foco. A partir dos testes de competências e do questionário acerca do ensino, da avaliação e do modo de estudar dos alunos (QEAME) foi possível identificar os ganhos e o impacte desta abordagem curricular no desenvolvimento dos três tipos de constelações de competências e as percepções dos alunos relativamente à mediação realizada pela professora investigadora relativamente ao ensino, aprendizagem e métodos de estudo, respectivamente; a Listagem Dinâmica de Perguntas permitiu aumentar a consciência acerca dos itens trabalhados em sala de aula. A principal conclusão deste estudo é de que é possível implementar um currículo que desenvolva e favoreça a aquisição e desenvolvimento de competências na Geometria (desde as mais elementares até às de nível superior – nas diferentes constelações de reprodução, de conexão e de reflexão) onde o trabalho colaborativo entre os professores é fundamental na gestão e desenvolvimento curriculares. Como implicações para a investigação educacional em matemática surge a necessidade de se fomentar estudos investigacionais descritivos e holísticos em salas de aulas normais sobre as opções e razões das práticas de ensino, das actividades de aprendizagem e das diferentes formas de avaliação.

Educação informal intergeracional: o caso da transmissão dos usos e costumes da medicina tradicional

Dissertação de mest., Ciências da Educação, Faculdade de Ciências Humanas e Sociais, Universidade do Algarve, 2010

O que é para ti um tubarão? percepções das crianças sobre os tubarões em contexto (educação) informal de ciências

Dissertação de mest., Didáctica e Inovação no Ensino das Ciências (Biologia/Geologia), Faculdade de Ciências e Tecnologia, Univ. do Algarve, 2011

An embodied approach to language: the role of bodily feedback in discourse processing

Theories of embodied cognition argue that language processing arises not from amodal symbols that redescribe sensorimotor and affective experiences, but from partial simulations (reenactments) of modality-specific states. Recent findings on processing of words and sentences support such a stance emphasizing that the role of the body in the domain of language comprehension should not be overlooked or dismissed. The present research was conducted to extend prior work in two important ways. First, the role of simulation was tested with connected discourse rather than words or sentences presented in isolation. Second, both “online” and “offline” measures of discourse comprehension were taken. In Experiments 1 and 2 participants’ facial postures were manipulated to show that preparing the body for processing of emotion-congruent information improves discourse comprehension. In Experiment 3 the direction of body posture was manipulated to show that implicit properties of simulations, such as spatial dimension or location, are at least somewhat involved in processing of large language segments such as discourse. Finally, in Experiments 4 and 5 participants’ body movement and body posture were manipulated to show that even understanding of language describing metaphorical actions physically impossible to perform involves constructing a sensorimotor simulation of the described event. The major result was that compatibility between embodiment and language strongly modulated performance effectiveness in experiments on simulation of emotion and metaphorical action. The effect of simulation on comprehension of discourse implying spatial dimension was fragile. These findings support an embodied simulation account of cognition suggesting that sensorimotor and affective states are at least partially implicated in “online” and “offline” discourse comprehension.

Consistent Feedback for a Constructivist Pedagogy : a Study of Feedback on written Assessments in Cegep English Courses.

In order to determine how consistent feedback is within the English Cégep system, this study explores three key aspects of feedback provided to students: the amount of feedback provided, the nature of the feedback according to the defined criteria, and the relative importance of the three categories. Twenty-three teachers in four English departments participated in the study. Data was collected from a detailed questionnaire to give some context in terms of teacher training, experience, and assessment practices. Respondents were then asked to provide written feedback on a sample student essay. Their comments werw analysed in terms of the nature of the feedback, how consistent that feedback was between teachers, and how closely the feedback reflected the stated instructional objectives.