External school evaluation : teachers’ perspectives : two case studies from Northern Portugal

The external evaluation of non-higher education schools in Portugal has been developed by the General Inspectorate of Education since 2006. A first cycle of evaluation was completed, covering all educational units in continental Portugal up to 2011. The model of evaluation has since been subject to alterations, and a second cycle of evaluation is now coming to an end. The current model of evaluation is based on documental analysis, analysis of students’ results, and panel interviews with a variety of representatives of the school community, and addresses three domains: results, provision of educational service and management. This paper is part of an ongoing research project, developed by 6 universities and supported by the Portuguese Foundation for Science and Technology (PTDC/CPE-CED/116674/2010) which intends to analyse the impacts and effects this process of external evaluation has had on Portuguese schools. This project includes a variety of perspectives and methodologies. In particular, we will focus on two case studies undertaken in two schools from the northern region of Portugal, and more specifically on the perspectives expressed by the teachers of those schools. These particular schools were chosen because they have been evaluated twice and represent different educational levels (basic and secondary), contexts and results. These case studies included the analysis of documental data, interviews to key informants and a questionnaire directed to teachers (n = 141) – the latter will be the main focus of this paper. Teachers are essential elements of the school community when considering the impacts of external evaluation, as any changes directed at teaching practices, student evaluation, among others are only possible through their direct action and implication. Therefore, their perceptions on the process and its impacts are crucial to the understanding of what does and does not change in schools as a consequence of external evaluation. Although teachers’ opinions are not homogenous and each school reveals a number of differences when it comes to teachers’ perceptions of School Evaluation, it was possible to stress some areas as the most and as the least consensual. Teachers in both schools agree External School Evaluation (ESE) is useful for the identification of the schools’ strengths and weaknesses, values students’ external evaluation results, imposes a model for schools internal evaluation (and in fact contributes to the very existence of internal evaluation practices), and contributes to schools improvement. However teachers in both schools do not believe ESE contributes to teachers’ autonomy produces changes in how curriculum is managed, or leads to innovative teaching practices. These results point to a greater emphasis on change at the levels of school management, self-evaluation and particularly internal evaluation, but little impact on the teaching practices. We believe the classroom is at the core of school practices and teaching processes are essential to any measure of school quality and to their impacts on student learning.

Self-regulated learning in higher education : strategies adopted by computer programming students

Trabalho apresentado em PAEE/ALE’2016, 8th International Symposium on Project Approaches in Engineering Education (PAEE) and 14th Active Learning in Engineering Education Workshop (ALE)

Effektivität der Qualifizierung von Teach First Fellows. Abschlussbericht zur summativen Evaluation der Sommerakademie von Teach First Deutschland

Teach First Fellows nehmen wie Lehrkräfte eine hohe Verantwortung für die weitere Entwicklung von Kindern und Jugendlichen wahr. Anders als Lehrkräfte verfügen sie jedoch nicht über eine grundständige Ausbildung für ihre spezifische Tätigkeit. Deshalb stellt sich die Frage, inwiefern es gelingen kann, durch eine sorgfältige Auswahl der Fellows, sowie durch eine intensive Kurzqualifizierung gute Voraussetzungen für den Erfolg ihrer Tätigkeit zu schaffen. Die vorliegende Studie evaluiert die Ergebnisse des Qualifizierungsprogramms, das Teach First Deutschland für seine Fellows entwickelt und implementiert hat. Dabei interessiert insbesondere, welches Wissen zur Unterstützung von Lernprozessen, welche Fähigkeiten zur Gestaltung von Unterricht und welche Erwartung von pädagogischer Wirksamkeit innerhalb eines halben Jahres bei den Fellows aufgebaut werden können. Um ein tieferes Verständnis der Wirkweise des Qualifizierungsprogramms zu ermöglichen, wurden neben den Ergebnissen auch dessen Prozesskomponenten einer Bewertung unterzogen. (DIPF/Orig.)

Assessing the Role of General Chemistry Learning in Higher Education

The inclusion of General Chemistry (GC) in the curricula of higher education courses in science and technology aims, on the one hand, to develop students' skills necessary for further studies and, on the other hand, to respond to the need of endowing future professionals of knowledge to analyze and solve multidisciplinary problems in a sustainable way. The participation of students in the evaluation of the role played by the GC in their training is crucial, and the analysis of the results can be an essential tool to increase success in the education of students and improving practices in various professions. Undeniably, this work will be focused on the development of an intelligent system to assess the role of GC. The computational framework is built on top of a Logic Programming approach to Knowledge Representation and Reasoning, complemented with a problem solving methodology moored on Artificial Neural Networks. The results so far obtained show that the proposed model stands for a good start, being its overall accuracy higher than 95%.

Communities of practice as professional and organizational development strategy in local public health organizations in Québec, Canada : an evaluation model

Communities of practice (CoPs) are among the professional development strategies most widely used in such fields as management and education. Though the approach has elicited keen interest, knowledge pertaining to its conceptual underpinnings is still limited, thus hindering proper assessment of CoPs' effects and the processes generating the latter. To address this shortcoming, this paper presents a conceptual model that was developed to evaluate an initiative based on a CoP strategy: Health Promotion Laboratories are a professional development intervention that was implemented in local public health organizations in Montreal (Quebec, Canada). The model is based on latest theories on work-group effectiveness and organizational learning and can be usefully adopted by evaluators who are increasingly called upon to illuminate decision-making about CoPs. Ultimately, validation of this conceptual model will help advance knowledge and practice pertaining to CoPs as well as professional and organizational development strategies in public health.

The Impact of Motor Learning on Motor Behavior and Cortical Dynamics in a Complex Stressful Social Environment

An economy of effort is a core characteristic of highly skilled motor performance often described as being effortless or automatic. Electroencephalographic (EEG) evaluation of cortical activity in elite performers has consistently revealed a reduction in extraneous associative cortical activity and an enhancement of task-relevant cortical processes. However, this has only been demonstrated under what are essentially practice-like conditions. Recently it has been shown that cerebral cortical activity becomes less efficient when performance occurs in a stressful, complex social environment. This dissertation examines the impact of motor skill training or practice on the EEG cortical dynamics that underlie performance in a stressful, complex social environment. Sixteen ROTC cadets participated in head-to-head pistol shooting competitions before and after completing nine sessions of skill training over three weeks. Spectral power increased in the theta frequency band and decreased in the low alpha frequency band after skill training. EEG Coherence increased in the left frontal region and decreased in the left temporal region after the practice intervention. These suggest a refinement of cerebral cortical dynamics with a reduction of task extraneous processing in the left frontal region and an enhancement of task related processing in the left temporal region consistent with the skill level reached by participants. Partitioning performance into ‘best’ and ‘worst’ based on shot score revealed that deliberate practice appears to optimize cerebral cortical activity of ‘best’ performances which are accompanied by a reduction in task-specific processes reflected by increased high-alpha power, while ‘worst’ performances are characterized by an inappropriate reduction in task-specific processing resulting in a loss of focus reflected by higher high-alpha power after training when compared to ‘best’ performances. Together, these studies demonstrate the power of experience afforded by practice, as a controllable factor, to promote resilience of cerebral cortical efficiency in complex environments.

Evaluation of Educational Environment for Medical Students of a Tertiary Pediatric Hospital in Tehran, Using DREEM Questionnaire

Background: Tertiary pediatric hospitals usually provide excellent clinical services, but such centers have a lot to do for educational perfection. Objectives: This study was performed to address under-graduate educational deficits and find feasible solutions. Patients and Methods: This cross-sectional study was done in a target population of 77 sixth year undergraduate medical students (response rate = 78%) who spent their 3-month pediatric rotation in the Children’s Medical Center, the Pediatrics Center of Excellence in Tehran, Iran. The Dundee ready educational environment measure (DREEM) instrument was used for assessing educational environment of this subspecialized pediatric hospital. Results: Among 60 students who answered the questionnaires, 24 were male (40%). Participants’ age ranged from 23 to 24 years. The mean total score was 95.8 (48%). Comparison of scores based on students’ knowledge showed no significant difference. Problematic areas were learning, academic self-perception, and social self-perception. Conclusions: Having an accurate schedule to train general practitioner, using new teaching methods, and providing a non-stressful atmosphere were suggested solutions.

A virtual agent to sustaining children's engagement in language learning

In this thesis we aimed to explore the potential of gamification - defined as “the use of game elements in non-game contexts” [30] - in increasing children's (aged 5 to 6) engagement with the task. This is mainly due to the fact that our world is living a technological era, and videogames are an example of this engagement by being able to maintain children’s (and adults) engagement for hours straight. For the purpose of limiting complexity, we only addressed the feedback element by introducing it with an anthropomorphic virtual agent (human-like aspect), because research shows that virtual agents (VA’s) can influence behavioural change [17], or even induce emotions on humans both through the use of feedback provided and their facial expressions, which can interpreted in the same way as of humans’ [2]. By pairing the VA with the gamification concept, we wanted to 1) create a VA that is likely to be well-received by children (appearance and behaviour), and 2) have the immediate feedback that games have, so we can give children an assessment of their actions in real-time, as opposed to waiting for feedback from someone (traditional teaching), and with this give students more chances to succeed [32, 43]. Our final system consisted on a virtual environment, where children formed words that corresponded to a given image. In order to measure the impact that the VA had on engagement, the system was developed in two versions: one version of the system was limited to provide a simple feedback environment, where the VA provided feedback, by responding with simple phrases (i.e. “correct” or “incorrect”); for the second version, the VA had a more complex approach where it tried to encourage children to complete the word – a motivational feedback - even when they weren’t succeeding. Lastly we conducted a field study with two groups of children, where one group tested the version with the simple feedback, and the other group tested the ‘motivational’ version of the system. We used a quantitative approach to analyze the collected data that measured the engagement, based on the number of tasks (words) completed and time spent with system. The results of the evaluation showed that the use of motivational feedback may carry a positive effect on engaging children.

A virtual agent to sustaining children's engagement in language learning

In this thesis we aimed to explore the potential of gamification - defined as “the use of game elements in non-game contexts” [30] - in increasing children's (aged 5 to 6) engagement with the task. This is mainly due to the fact that our world is living a technological era, and videogames are an example of this engagement by being able to maintain children’s (and adults) engagement for hours straight. For the purpose of limiting complexity, we only addressed the feedback element by introducing it with an anthropomorphic virtual agent (human-like aspect), because research shows that virtual agents (VA’s) can influence behavioural change [17], or even induce emotions on humans both through the use of feedback provided and their facial expressions, which can interpreted in the same way as of humans’ [2]. By pairing the VA with the gamification concept, we wanted to 1) create a VA that is likely to be well-received by children (appearance and behaviour), and 2) have the immediate feedback that games have, so we can give children an assessment of their actions in real-time, as opposed to waiting for feedback from someone (traditional teaching), and with this give students more chances to succeed [32, 43]. Our final system consisted on a virtual environment, where children formed words that corresponded to a given image. In order to measure the impact that the VA had on engagement, the system was developed in two versions: one version of the system was limited to provide a simple feedback environment, where the VA provided feedback, by responding with simple phrases (i.e. “correct” or “incorrect”); for the second version, the VA had a more complex approach where it tried to encourage children to complete the word – a motivational feedback - even when they weren’t succeeding. Lastly we conducted a field study with two groups of children, where one group tested the version with the simple feedback, and the other group tested the ‘motivational’ version of the system. We used a quantitative approach to analyze the collected data that measured the engagement, based on the number of tasks (words) completed and time spent with system. The results of the evaluation showed that the use of motivational feedback may carry a positive effect on engaging children.

Computer and laboratory simulation in the teaching of neonatal nursing: innovation and impact on learning

Objectives: to evaluate the cognitive learning of nursing students in neonatal clinical evaluation from a blended course with the use of computer and laboratory simulation; to compare the cognitive learning of students in a control and experimental group testing the laboratory simulation; and to assess the extracurricular blended course offered on the clinical assessment of preterm infants, according to the students. Method: a quasi-experimental study with 14 Portuguese students, containing pretest, midterm test and post-test. The technologies offered in the course were serious game e-Baby, instructional software of semiology and semiotechnique, and laboratory simulation. Data collection tools developed for this study were used for the course evaluation and characterization of the students. Nonparametric statistics were used: Mann-Whitney and Wilcoxon. Results: the use of validated digital technologies and laboratory simulation demonstrated a statistically significant difference (p = 0.001) in the learning of the participants. The course was evaluated as very satisfactory for them. The laboratory simulation alone did not represent a significant difference in the learning. Conclusions: the cognitive learning of participants increased significantly. The use of technology can be partly responsible for the course success, showing it to be an important teaching tool for innovation and motivation of learning in healthcare.

Deep learning models for the automatic analysis of volleyball plays

Collecting and analysing data is an important element in any field of human activity and research. Even in sports, collecting and analyzing statistical data is attracting a growing interest. Some exemplar use cases are: improvement of technical/tactical aspects for team coaches, definition of game strategies based on the opposite team play or evaluation of the performance of players. Other advantages are related to taking more precise and impartial judgment in referee decisions: a wrong decision can change the outcomes of important matches. Finally, it can be useful to provide better representations and graphic effects that make the game more engaging for the audience during the match. Nowadays it is possible to delegate this type of task to automatic software systems that can use cameras or even hardware sensors to collect images or data and process them. One of the most efficient methods to collect data is to process the video images of the sporting event through mixed techniques concerning machine learning applied to computer vision. As in other domains in which computer vision can be applied, the main tasks in sports are related to object detection, player tracking, and to the pose estimation of athletes. The goal of the present thesis is to apply different models of CNNs to analyze volleyball matches. Starting from video frames of a volleyball match, we reproduce a bird's eye view of the playing court where all the players are projected, reporting also for each player the type of action she/he is performing.

Shaping the future of learning using the student voice: we’re listening but are we hearing clearly?

Student voice data is a key factor as Manchester Metropolitan University strives to continually improve institutional technology enhanced learning (TEL) infrastructure. A bi-annual Institutional Student Survey enables students to communicate their experience of learning, teaching and assessment on programmes and specific units studied. Each cycle of the survey contains approximately 40–50,000 free text comments from students pertaining to what they appreciate and what they would like to see improved. A detailed thematic analysis of this data has identified 18 themes, arranged into six categories relating to the ‘Best’ aspects of courses, and 25 themes, arranged in seven categories in relation to aspects of courses considered to be ‘in need of improvement’. This student data was then used as a basis for semi-structured interviews with staff. Anecdotally, evidence suggested that student expectations and staff expectations around TEL and the virtual learning environment (VLE) differed. On-going evaluation of this work has highlighted a disconnect. In significant instances, academic colleagues seemingly misinterpret the student voice analysis and consequently struggle to respond effectively. In response to the analysis, the learning technologist's role has been to re-interpret the analysis and redevelop TEL staff development and training activities. The changes implemented have focused on: contextualising resources in VLE; making lectures more interactive; enriching the curriculum with audio–visual resources; and setting expectations around communications.

How should we measure online learning activity?

The proliferation of Web-based learning objects makes finding and evaluating resources a considerable hurdle for learners to overcome. While established learning analytics methods provide feedback that can aid learner evaluation of learning resources, the adequacy and reliability of these methods is questioned. Because engagement with online learning is different from other Web activity, it is important to establish pedagogically relevant measures that can aid the development of distinct, automated analysis systems. Content analysis is often used to examine online discussion in educational settings, but these instruments are rarely compared with each other which leads to uncertainty regarding their validity and reliability. In this study, participation in Massive Open Online Course (MOOC) comment forums was evaluated using four different analytical approaches: the Digital Artefacts for Learning Engagement (DiAL-e) framework, Bloom's Taxonomy, Structure of Observed Learning Outcomes (SOLO) and Community of Inquiry (CoI). Results from this study indicate that different approaches to measuring cognitive activity are closely correlated and are distinct from typical interaction measures. This suggests that computational approaches to pedagogical analysis may provide useful insights into learning processes.

Nurturing global collaboration and networked learning in higher education

We consider the principles of communities of practice (CoP) and networked learning in higher education, illustrated with a case study. iCollab has grown from an international community of practice connecting students and lecturers in seven modules across seven higher education institutions in six countries, to a global network supporting the exploration and evaluation of mobile web tools to engage in participatory curriculum development and supporting students in developing international collaboration and cooperation skills. This article explores the interplay of collaboration and cooperation, CoP and networked learning; describes how this interplay has operated in iCollab; and highlights opportunities and challenges of learning, teaching and interacting with students in networked publics in higher education.