Improving E-Assessment Systems With A focus On Adaptive Feedback And Marking

Institutions involved in the provision of tertiary education across Europe are feeling the pinch. European universities, and other higher education (HE) institutions, must operate in a climate where the pressure of government spending cuts (Garben, 2012) is in stark juxtaposition to the EU’s strategy to drive forward and maintain a growth of student numbers in the sector (eurostat, 2015).

In order to remain competitive, universities and HE institutions are making ever-greater use of electronic assessment (E-Assessment) systems (Chatzigavriil et all, 2015; Ferrell, 2012). These systems are attractive primarily because they offer a cost-effect and scalable approach for assessment. In addition to scalability, they also offer reliability, consistency and impartiality; furthermore, from the perspective of a student they are most popular because they can offer instant feedback (Walet, 2012).

There are disadvantages, though.

First, feedback is often returned to a student immediately on competition of their assessment. While it is possible to disable the instant feedback option (this is often the case during an end of semester exam period when assessment scores must be can be ratified before release), however, this option tends to be a global ‘all on’ or ‘all off’ configuration option which is controlled centrally rather than configurable on a per-assessment basis.

If a formative in-term assessment is to be taken by multiple groups of
students, each at different times, this restriction means that answers to each question will be disclosed to the first group of students undertaking the assessment. As soon as the answers are released “into the wild” the academic integrity of the assessment is lost for subsequent student groups.

Second, the style of feedback provided to a student for each question is often limited to a simple ‘correct’ or ‘incorrect’ indicator. While this type of feedback has its place, it often does not provide a student with enough insight to improve their understanding of a topic that they did not answer correctly.

Most E-Assessment systems boast a wide range of question types including Multiple Choice, Multiple Response, Free Text Entry/Text Matching and Numerical questions. The design of these types of questions is often quite restrictive and formulaic, which has a knock-on effect on the quality of feedback that can be provided in each case.

Multiple Choice Questions (MCQs) are most prevalent as they are the most prescriptive and therefore most the straightforward to mark consistently. They are also the most amenable question types, which allow easy provision of meaningful, relevant feedback to each possible outcome chosen.
Text matching questions tend to be more problematic due to their free text entry nature. Common misspellings or case-sensitivity errors can often be accounted for by the software but they are by no means fool proof, as it is very difficult to predict in advance the range of possible variations on an answer that would be considered worthy of marks by a manual marker of a paper based equivalent of the same question.

Numerical questions are similarly restricted. An answer can be checked for accuracy or whether it is within a certain range of the correct answer, but unless it is a special purpose-built mathematical E-Assessment system the system is unlikely to have computational capability and so cannot, for example, account for “method marks” which are commonly awarded in paper-based marking.

From a pedagogical perspective, the importance of providing useful formative feedback to students at a point in their learning when they can benefit from the feedback and put it to use must not be understated (Grieve et all, 2015; Ferrell, 2012).

In this work, we propose a number of software-based solutions, which will overcome the limitations and inflexibilities of existing E-Assessment systems.

Educar (para) o pensar : desenvolvimento de competências reflexíveis em professores e alunos do 1º CEB : contributos da "Filosofia para Crianças"

O presente estudo inscreve-se na área científica da Formação de Professores, incidindo, particularmente, na compreensão do processo de desenvolvimento das competências reflexivas percebidas como factor de promoção do seu próprio desenvolvimento profissional e pessoal, do desenvolvimento da capacidade de pensar dos seus alunos, da revalorização dos processos curriculares de ensino-aprendizagem e de inovação dos contextos educacionais. Num contexto de complexidade, incerteza e mudança, importa repensar estratégias de formação de professores e de alunos para que possam constituir-se como fatores potenciadores do desenvolvimento da competência reflexiva. Estratégias que convocam, quer o professor, quer o aluno, para um tipo de questionamento de maior exigência reflexiva e consideradas potenciadoras do pensamento crítico, criativo e de cuidado com o outro, numa perspetiva educativa centrada no cuidar, que valoriza a dimensão humana, a atuação responsável, ética e solidária, em todos os planos da vida. Neste estudo propomo-nos retomar algumas das estratégias de formação já configuradas no movimento Filosofia para Crianças e que se constituíram como um programa de formação em contexto, no qual se procurou aprofundar e compreender as múltiplas dimensões e modos como interatuam os diferentes participantes da relação educativa em práticas curriculares reconfiguradas à luz dos pressupostos que sustentam este estudo. Do ponto de vista metodológico, a investigação inscreve-se num paradigma de natureza qualitativa e interpretativa, de matriz hermenêutica e ecológica, configurando uma abordagem de tipo complexo, e com características de estudo de caso, que considera indispensável a participação ativa do sujeito na construção do conhecimento próprio, bem como o carácter de imprevisibilidade e de recursividade das condições e subsistemas em que tal ocorre. No sentido de construir uma visão integrada do objeto em estudo, foram desenvolvidos procedimentos específicos (mixed-methods), nomeadamente análise documental, entrevista semiestruturada, observação participante e inquirição por questionário. O estudo, que decorreu na região centro do país, envolveu 5 professoras do 1.º Ciclo do Ensino Básico, 100 alunos do mesmo nível de ensino e os seus pais/encarregados de educação, inquiridos através de questionário e desenvolveu-se em duas fases. A primeira destinou-se à formação teórico-prática das professoras e, na segunda, foram desenvolvidas sessões práticas de Filosofia para Crianças com os alunos. Os portfolios reflexivos construídos pelos participantes e pela investigadora principal constituíram outra fonte da informação recolhida no estudo empírico. Os resultados do estudo situam-se a quatro níveis: no que respeita aos saberes básicos, ao perfil de competência dos professores, à sua formação e às estratégias e recursos considerados como potenciadores de um pensar de mais elevada qualidade. Quanto ao primeiro nível, o presente estudo releva o carácter estruturante e epistémico de aprender a pensar (bem), salientando que este se processa numa maior amplitude e profundidade dos conteúdos da própria reflexão, às quais subjaz uma visão ampla de cidadania planetária e socialmente comprometida, evidenciando uma ampliação do quadro referencial dos saberes básicos e considerados imprescindíveis para a educação dos cidadãos. A um segundo nível, salienta-se a exigência de um perfil de competência profissional que permita aos professores desenvolver nos seus alunos um pensar de qualidade e, simultaneamente, melhorar a sua própria competência reflexiva. Neste sentido, o estudo aponta para a continuidade das respostas que têm vindo a ser equacionadas por vários autores nacionais e internacionais que, ao abordarem a problemática da formação, do conhecimento profissional e do desenvolvimento identitário dos professores, têm acentuado a importância dos modelos crítico-reflexivos da formação e de uma supervisão ecológica, integradora, não standard e humanizada, no desenvolvimento das sociedades contemporâneas. Conforme os dados sugerem, admite-se que a formação integral dos cidadãos passa pela inclusão e interligação de diferentes áreas do conhecimento que, concertada e complementarmente, possam contribuir para o desenvolvimento da sensibilidade, do pensamento crítico e criativo, de uma cultura da responsabilidade e de uma atitude ética mais ativa e interventiva. Neste sentido, reafirma-se a importância de um trajeto formativo que promova a efetiva articulação entre teoria e a prática, o diálogo crítico-reflexivo entre saberes científicos e experiência, que focalize o profissional na sua praxis e saliente a sua conexão com o saber situado em contexto vivencial e didático- -pedagógico. Realça-se a pertinência de dinâmicas formativas que, a exemplo de “comunidades de investigação/aprendizagem”, na sua aceção de redes de formação que, na prossecução de projetos e propósitos comuns, incentivam a construção de itinerários próprios e de aprendizagens individuais, mobilizando processos investigativos pessoais e grupais. Evidencia-se a valorização de práticas promotoras da reflexão, do questionamento, da flexibilidade cognitiva como eixos estruturadores do pensamento e da ação profissional e como suporte do desenvolvimento profissional e pessoal, corroborando a importância dos processos transformadores decorrentes da experiência, da ação e da reflexão sobre ela. Finalmente, no que respeita às estratégias e recursos, os dados permitem corroborar a riqueza e o potencial do uso de portfolios reflexivos no desenvolvimento de competências linguísticas, comunicacionais, reflexivas e meta-reflexivas e o entendimento de que o processo de construção da identidade profissional ocorre e desenha-se numa dinâmica reflexiva- -prospetiva (re)confirmadora ou (re)configuradora de ideias, convicções, saberes e práticas, ou seja, identitária. De igual modo, a investigação releva a importância da construção de portfolios, por parte dos alunos, para o desenvolvimento da qualidade do seu pensamento, sublinhando-se o seu carácter inovador nesta área. Evidencia-se, ainda, a diversidade de estratégias que respeitem os interesses, necessidades e expectativas dos alunos e o seu contexto de vida, tal como o recurso a materiais diversificados que, atentos ao conteúdo da mensagem, possibilitem a autonomia do pensamento, o exercício efetivo da reflexão crítica e do questionamento, na sua articulação com as grandes questões que sempre despertaram a curiosidade humana e cuja atualidade permanece. Materiais e recursos que estabeleçam o diálogo entre razão e imaginação, entre saber e sensibilidade, que estimulem o envolvimento dos alunos na resolução de problemas e na procura conjunta de soluções e na construção de projetos individuais na malha dos projetos comuns. Reafirma-se, pois, a importância da humanização do saber, a educação pensada como vivência solidária de direitos e deveres. Uma perspetiva educacional humanista que assenta nas trajetórias de vida, na recuperação de experiências pessoais e singulares, que procura compreender a identidade como um processo em (re)elaboração permanente. O presente estudo integra-se na rede de cooperação científica Novos saberes básicos dos alunos no século XXI, novos desafios à formação de professores sendo que, e na linha das investigações produzidas neste âmbito, destaca que o alargamento das funções do professor do 1.º Ciclo do Ensino Básico, que colocando a tónica da ação pedagógica no como se aprende e para quê e na possibilidade de aprender e incorporar o imprevisível, incide no desenvolvimento de um conjunto de capacidades que vão para além das tradicionalmente associadas ao ensinar a ler, escrever e contar. Releva-se, pois, a pertinência da criação de ambientes educativos nos quais professores e alunos entreteçam, conjunta e coerentemente, conhecer, compreender, fazer, sentir, dizer, ver, ouvir e (con)viver em prol de uma reflexão que nos encaminhe no sentido de ser(mos) consciência.

Marking definiteness multiply: evidence from two varieties of Greek

Θέμα της μελέτης μας είναι οι πολυοριστικές δομές σε δύο νεοελληνικές ποικιλίες, την πρότυπη ελληνική (ΠΕ) και την καππαδοκική διάλεκτο (ΚΕ). Παρά την επιφανειακή ομοιότητα, οι δομές αυτές διαφέρουν ως προς τις συντακτικές και σημασιολογικές τους ιδιότητες. Για την ΠΕ υιοθετούμε την ανάλυση των Lekakou & Szendrői (2007, 2009, 2012, 2013), σύμφωνα με την οποία οι πολυοριστικές δομές είναι ένα είδος ονοματικής επεξήγησης, με την ιδιαιτερότητα ότι περιέχουν δομή ονοματικής απαλοιφής (noun ellipsis). Στην ΚΕ, η υποχρεωτική φύση του φαινομένου μας οδηγεί στην πρόταση ότι πρόκειται για ένα είδος μορφοσυντακτικής συμφωνίας. Συγκεκριμένα, τα άρθρα που συνοδεύουν το επίθετο είναι δείκτες ονοματικής συμφωνίας ως προς την οριστικότητα και προκύπτουν μετα-συντακτικά, στο θεωρητικό πλαίσιο της Κατανεμημένης Μορφολογίας. Υποστηρίζουμε ότι μια ενιαία ανάλυση της οριστικότητας στις δύο ποικιλίες είναι εφικτή, εφόσον δεχτούμε ότι σημασιολογική οριστικικότητα δεν εκφράζει κανένα από τα εκπεφρασμένα άρθρα, αλλά ένας φωνολογικά κενός τελεστής.

Rationality, External Norms and the Epistemic Value of Menus

Ever since Sen’s (1993; 1997) criticism on the notion of internal consistency or menu independence of choice, there exists a widespread perception that the standard revealed preference approach to the theory of rational choice has difficulties in coping with the existence of external norms, or the information a menu of choice might convey to a decision-maker, viz., the epistemic value of a menu. This paper provides a brief survey of possible responses to these criticisms of traditional rational choice theory. It is shown that a novel concept of norm-conditional rationalizability can neatly accommodate external norms within the standard framework of rationalizability theory. Furthermore, we illustrate that there are several ways of incorporating considerations regarding the epistemic value of opportunity sets into a generalized model of rational choice theory.

An Epistemic Argument in Support of Liberal Neutrality

My aim in the present paper is to develop a new kind of argument in support of the ideal of liberal neutrality. This argument combines some basic moral principles with a thesis about the relationship between the correct standards of justification for a belief/action and certain contextual factors. The idea is that the level of importance of what is at stake in a specific context of action determines how demanding the correct standards to justify an action based on a specific set of beliefs ought to be. In certain exceptional contexts –where the seriousness of harm in case of mistake and the level of an agent’s responsibility for the outcome of his action are specially high– a very small probability of making a mistake should be recognized as a good reason to avoid to act based on beliefs that we nonetheless affirm with a high degree of confidence and that actually justify our action in other contexts. The further steps of the argument consist in probing 1) that the fundamental state’s policies are such a case of exceptional context, 2) that perfectionist policies are the type of actions we should avoid, and 3) that policies that satisfy neutral standards of justification are not affected by the reasons which lead to reject perfectionist policies.

Epistemic deontologism and the voluntarist strategy against doxastic involuntarism

According to the deontological conception of epistemic justification, a belief is justified when it is our obligation or duty as rational creatures to believe it. However, this view faces an important objection according to which we cannot have such epistemic obligations since our beliefs are never under our voluntary control. One possible strategy against this argument is to show that we do have voluntary control over some of our beliefs, and that we therefore have epistemic obligations. This is what I call the voluntarist strategy. I examine it and argue that it is not promising. I show how the voluntarist attempts of Carl Ginet and Brian Weatherson fail, and conclude that it would be more fruitful for deontologists to look for a different strategy.

The Role of Epistemic Monitoring in Language Comprehension

In der psycholinguistischen Forschung ist die Annahme weitverbreitet, dass die Bewertung von Informationen hinsichtlich ihres Wahrheitsgehaltes oder ihrer Plausibilität (epistemische Validierung; Richter, Schroeder & Wöhrmann, 2009) ein strategischer, optionaler und dem Verstehen nachgeschalteter Prozess ist (z.B. Gilbert, 1991; Gilbert, Krull & Malone, 1990; Gilbert, Tafarodi & Malone, 1993; Herbert & Kübler, 2011). Eine zunehmende Anzahl an Studien stellt dieses Zwei-Stufen-Modell von Verstehen und Validieren jedoch direkt oder indirekt in Frage. Insbesondere Befunde zu Stroop-artigen Stimulus-Antwort-Kompatibilitätseffekten, die auftreten, wenn positive und negative Antworten orthogonal zum aufgaben-irrelevanten Wahrheitsgehalt von Sätzen abgegeben werden müssen (z.B. eine positive Antwort nach dem Lesen eines falschen Satzes oder eine negative Antwort nach dem Lesen eines wahren Satzes; epistemischer Stroop-Effekt, Richter et al., 2009), sprechen dafür, dass Leser/innen schon beim Verstehen eine nicht-strategische Überprüfung der Validität von Informationen vornehmen. Ausgehend von diesen Befunden war das Ziel dieser Dissertation eine weiterführende Überprüfung der Annahme, dass Verstehen einen nicht-strategischen, routinisierten, wissensbasierten Validierungsprozesses (epistemisches Monitoring; Richter et al., 2009) beinhaltet. Zu diesem Zweck wurden drei empirische Studien mit unterschiedlichen Schwerpunkten durchgeführt. Studie 1 diente der Untersuchung der Fragestellung, ob sich Belege für epistemisches Monitoring auch bei Informationen finden lassen, die nicht eindeutig wahr oder falsch, sondern lediglich mehr oder weniger plausibel sind. Mithilfe des epistemischen Stroop-Paradigmas von Richter et al. (2009) konnte ein Kompatibilitätseffekt von aufgaben-irrelevanter Plausibilität auf die Latenzen positiver und negativer Antworten in zwei unterschiedlichen experimentellen Aufgaben nachgewiesen werden, welcher dafür spricht, dass epistemisches Monitoring auch graduelle Unterschiede in der Übereinstimmung von Informationen mit dem Weltwissen berücksichtigt. Darüber hinaus belegen die Ergebnisse, dass der epistemische Stroop-Effekt tatsächlich auf Plausibilität und nicht etwa auf der unterschiedlichen Vorhersagbarkeit von plausiblen und unplausiblen Informationen beruht. Das Ziel von Studie 2 war die Prüfung der Hypothese, dass epistemisches Monitoring keinen evaluativen Mindset erfordert. Im Gegensatz zu den Befunden anderer Autoren (Wiswede, Koranyi, Müller, Langner, & Rothermund, 2013) zeigte sich in dieser Studie ein Kompatibilitätseffekt des aufgaben-irrelevanten Wahrheitsgehaltes auf die Antwortlatenzen in einer vollständig nicht-evaluativen Aufgabe. Die Ergebnisse legen nahe, dass epistemisches Monitoring nicht von einem evaluativen Mindset, möglicherweise aber von der Tiefe der Verarbeitung abhängig ist. Studie 3 beleuchtete das Verhältnis von Verstehen und Validieren anhand einer Untersuchung der Online-Effekte von Plausibilität und Vorhersagbarkeit auf Augenbewegungen beim Lesen kurzer Texte. Zusätzlich wurde die potentielle Modulierung dieser Effeke durch epistemische Marker, die die Sicherheit von Informationen anzeigen (z.B. sicherlich oder vielleicht), untersucht. Entsprechend der Annahme eines schnellen und nicht-strategischen epistemischen Monitoring-Prozesses zeigten sich interaktive Effekte von Plausibilität und dem Vorhandensein epistemischer Marker auf Indikatoren früher Verstehensprozesse. Dies spricht dafür, dass die kommunizierte Sicherheit von Informationen durch den Monitoring-Prozess berücksichtigt wird. Insgesamt sprechen die Befunde gegen eine Konzeptualisierung von Verstehen und Validieren als nicht-überlappenden Stufen der Informationsverarbeitung. Vielmehr scheint eine Bewertung des Wahrheitsgehalts oder der Plausibilität basierend auf dem Weltwissen – zumindest in gewissem Ausmaß – eine obligatorische und nicht-strategische Komponente des Sprachverstehens zu sein. Die Bedeutung der Befunde für aktuelle Modelle des Sprachverstehens und Empfehlungen für die weiterführende Forschung zum Vehältnis von Verstehen und Validieren werden aufgezeigt.

COMP1205 Technical Report Marking Scheme

This document outlines the grading boundaries for marking the technical report. It can be used for marking sample reports and peer review. The COMP1205 course team use it to show how they mark a sample report.

Production of tense marking in successive bilingual children: when do they converge with their monolingual peers?

Children with English as a second language (L2) with exposure of 18 months or less exhibit similar difficulties to children with Specific Language Impairment in tense marking, a marker of language impairment for English. This paper examines whether L2 children with longer exposure converge with their monolingual peers in the production of tense marking. 38 Turkish-English L2 children with a mean age of 7;8 and 33 monolingual age-matched controls completed the screening test of the Test of Early Grammatical Impairment (TEGI). The L2 children as a group were as accurate as the controls in the production of -ed, but performed significantly lower than the controls in the production of third person –s. Age and YoE affected the children’s performance. The highest age-expected performance on the TEGI was attested in eight and nine year-old children who had 4-6 YoE. L1 and L2 children performed better in regular compared to irregular verbs, but L2 children overregularized more than L1 children and were less sensitive to the phonological properties of verbs. The results show that tense marking and the screening test of the TEGI may be promising for differential diagnosis in eight and nine year-old L2 children with at least four YoE.

Electronic marking of statistics assessments for bioscience students

We describe the main features of a program written to perform electronic marking of quantitative or simple text questions. One of the main benefits is that it can check answers for being consistent with earlier errors, so can cope with a range of numerical questions. We summarise our experience of using it in a statistics course taught to 200 bioscience students.

Knowledge practices in design: The role of visual representations as 'epistemic objects'

We use a detailed study of the knowledge work around visual representations to draw attention to the multidimensional nature of `objects'. Objects are variously described in the literatures as relatively stable or in flux; as abstract or concrete; and as used within or across practices. We clarify these dimensions, drawing on and extending the literature on boundary objects, and connecting it with work on epistemic and technical objects. In particular, we highlight the epistemic role of objects, using our observations of knowledge work on an architectural design project to show how, in this setting, visual representations are characterized by a `lack' or incompleteness that precipitates unfolding. The conceptual design of a building involves a wide range of technical, social and aesthetic forms of knowledge that need to be developed and aligned. We explore how visual representations are used, and how these are meaningful to different stakeholders, eliciting their distinct contributions. As the project evolves and the drawings change, new issues and needs for knowledge work arise. These objects have an `unfolding ontology' and are constantly in flux, rather than fully formed. We discuss the implications for wider understandings of objects in organizations and for how knowledge work is achieved in practice.