Reliability assessment of a peer evaluation instrument in a team-based learning course

Objective: To evaluate the reliability of a peer evaluation instrument in a longitudinal team-based learning setting. Methods: Student pharmacists were instructed to evaluate the contributions of their peers. Evaluations were analyzed for the variance of the scores by identifying low, medium, and high scores. Agreement between performance ratings within each group of students was assessed via intra-class correlation coefficient (ICC). Results: We found little variation in the standard deviation (SD) based on the score means among the high, medium, and low scores within each group. The lack of variation in SD of results between groups suggests that the peer evaluation instrument produces precise results. The ICC showed strong concordance among raters. Conclusions: Findings suggest that our student peer evaluation instrument provides a reliable method for peer assessment in team-based learning settings.

Working for Families Phase 1 Evaluation 2004-2006 Executive Summary.

This report presents an evaluation of Phase 1 of the Working for Families Fund (WFF) covering 2004-06. WFF was established to invest in new initiatives to improve the employability of parents who have difficulties in participating in the labour market, specifically in employment, education or training. The Fund supported these parents through helping them find sustainable childcare solutions and through providing or accessing other relevant employability-related services. In rural areas, barriers created by poor transport, limited services and the lack of a critical mass of clients were also particularly important. WFF contributes to the Scottish Executive’s Closing the Opportunity Gap approach to tackling poverty and disadvantage, by improving rates of employment and economic activity, and to its commitment to eradicating child poverty within a generation.

Tutors as mediators of exchange student experiences: evaluation of the international student tutor system at the University of Turku

As academic student mobility is increasing, improving the functionality of international operations is recognised as a competitive advantage at tertiary education institutions. Although many scholars have researched the experiences of exchange students, the role of student tutors and their contribution to exchange students’ experiences is still an unknown factor. This research examines international tutoring at the University of Turku, and aims to understand better the way tutoring contributes to exchange experiences and to explore the functionality of the tutor system and discover areas for improvements. To achieve these goals, the research seeks to answer the fundamental research question: What is the role of tutors in mediating exchange experiences? The theoretical framework combines literature on mediating exchange experiences, the phenomenon of studying abroad, the process of adaptation, the importance of cross-cultural communication, and the role of student tutors as mediators. Based on the literature review, a theoretical model for studying the mediation of exchange experiences is introduced. The model’s applicability and validity is examined through a case study. Three methods were used in the empirical research: surveys, participant observations, and interviews. These methods provided extensive data from three major parties of the tutor system: tutors, exchange students, and the international office. The findings of the research reveal that tutoring – instrumental leading and social and cultural mediating – generates both negative and positive experiences depending on the individuals’ expectations, motivations, relationships, and the nature of the tutoring. Although functional, there are a few weaknesses in the tutor system. Tutors tend to act as effective instrumental leaders, but often fail to create a friendship and contribute to the exchange students’ experience through social and cultural mediation, which is significantly more important in the exchange students’ overall experience in terms of building networks, adapting, gaining emotional experiences, and achieving the stage of personal development and mental change. Based on the weaknesses, three improvements are suggested: (1) increasing comprehensive sharing of information, effective communication, and collective cooperation, (2) emphasising the importance of social and cultural mediation and increasing the frequency of interaction between tutors and exchange students, and (3) improving the recruitment and training, revising the process of reporting and rewarding, and finally, enhancing services and coordination.

External school evaluation : teachers’ perspectives : two case studies from Northern Portugal

The external evaluation of non-higher education schools in Portugal has been developed by the General Inspectorate of Education since 2006. A first cycle of evaluation was completed, covering all educational units in continental Portugal up to 2011. The model of evaluation has since been subject to alterations, and a second cycle of evaluation is now coming to an end. The current model of evaluation is based on documental analysis, analysis of students’ results, and panel interviews with a variety of representatives of the school community, and addresses three domains: results, provision of educational service and management. This paper is part of an ongoing research project, developed by 6 universities and supported by the Portuguese Foundation for Science and Technology (PTDC/CPE-CED/116674/2010) which intends to analyse the impacts and effects this process of external evaluation has had on Portuguese schools. This project includes a variety of perspectives and methodologies. In particular, we will focus on two case studies undertaken in two schools from the northern region of Portugal, and more specifically on the perspectives expressed by the teachers of those schools. These particular schools were chosen because they have been evaluated twice and represent different educational levels (basic and secondary), contexts and results. These case studies included the analysis of documental data, interviews to key informants and a questionnaire directed to teachers (n = 141) – the latter will be the main focus of this paper. Teachers are essential elements of the school community when considering the impacts of external evaluation, as any changes directed at teaching practices, student evaluation, among others are only possible through their direct action and implication. Therefore, their perceptions on the process and its impacts are crucial to the understanding of what does and does not change in schools as a consequence of external evaluation. Although teachers’ opinions are not homogenous and each school reveals a number of differences when it comes to teachers’ perceptions of School Evaluation, it was possible to stress some areas as the most and as the least consensual. Teachers in both schools agree External School Evaluation (ESE) is useful for the identification of the schools’ strengths and weaknesses, values students’ external evaluation results, imposes a model for schools internal evaluation (and in fact contributes to the very existence of internal evaluation practices), and contributes to schools improvement. However teachers in both schools do not believe ESE contributes to teachers’ autonomy produces changes in how curriculum is managed, or leads to innovative teaching practices. These results point to a greater emphasis on change at the levels of school management, self-evaluation and particularly internal evaluation, but little impact on the teaching practices. We believe the classroom is at the core of school practices and teaching processes are essential to any measure of school quality and to their impacts on student learning.

Governing the transition to higher levels of education and differences between achievement and school grades by gender

The author will explore the performance of boys and girls in external examinations in Slovenia at the beginning of upper secondary and tertiary education. These are critical points in students’ educational career at which he/she has to choose a school/university. Since both transitions are managed centrally by appropriate authorities, this is also a question of Educational Governance. Transitions between levels of education should, above all, assure fairness in selection procedures. At the point of transition to upper secondary schools we will explore differences between students’ achievements in various school subjects tested at the national assessment of knowledge (NA), and their school grades by gender. Since only school grades are used as admission criteria to upper secondary schools, this comparison of school grades with external and more objective measure of students’ achievement will show possible bias. In Slovenia admission to tertiary education usually consists of (externally assessed) Matura results and school grades in the last two years of upper secondary school. The author will compare the effects of both most commonly used measures of academic achievement on admission in view of gender differences. Study courses where selection procedure was actually applied will be of specific interest since they can show signs of (un)fairness. Results show signs of bias and build case for better Educational Governance. (DIPF/Orig.)

Self-regulated learning in higher education : strategies adopted by computer programming students

Trabalho apresentado em PAEE/ALE’2016, 8th International Symposium on Project Approaches in Engineering Education (PAEE) and 14th Active Learning in Engineering Education Workshop (ALE)

Effektivität der Qualifizierung von Teach First Fellows. Abschlussbericht zur summativen Evaluation der Sommerakademie von Teach First Deutschland

Teach First Fellows nehmen wie Lehrkräfte eine hohe Verantwortung für die weitere Entwicklung von Kindern und Jugendlichen wahr. Anders als Lehrkräfte verfügen sie jedoch nicht über eine grundständige Ausbildung für ihre spezifische Tätigkeit. Deshalb stellt sich die Frage, inwiefern es gelingen kann, durch eine sorgfältige Auswahl der Fellows, sowie durch eine intensive Kurzqualifizierung gute Voraussetzungen für den Erfolg ihrer Tätigkeit zu schaffen. Die vorliegende Studie evaluiert die Ergebnisse des Qualifizierungsprogramms, das Teach First Deutschland für seine Fellows entwickelt und implementiert hat. Dabei interessiert insbesondere, welches Wissen zur Unterstützung von Lernprozessen, welche Fähigkeiten zur Gestaltung von Unterricht und welche Erwartung von pädagogischer Wirksamkeit innerhalb eines halben Jahres bei den Fellows aufgebaut werden können. Um ein tieferes Verständnis der Wirkweise des Qualifizierungsprogramms zu ermöglichen, wurden neben den Ergebnissen auch dessen Prozesskomponenten einer Bewertung unterzogen. (DIPF/Orig.)

"Family Education and Support" program for families at psychosocial risk: the role of implementation process

This paper analyses the implementation characteristics of the Family Education and Support program, a theory-driven, needs-based, and evidence-based positive parenting program originally developed for the Andalusian family preservation services. The implementation process of 34 trials of the FAF program with 155 participants was analyzed. Cluster analyses were also performed to explore variability in implementation conditions from a comprehensive perspective. Results showed different implementation profiles that moderated the FAF effectiveness (namely lengthier interventions, higher program fidelity, and practitioners' positive perceptions and satisfaction with the program). The relevance of examining implementation process across several trials is discussed in order to distinguish core and non-core FAF components, as well as the need for combining faithful and adaptable implementations that guarantee the ecologic validity of evidence-based positive parenting programs.

Assessing the Role of General Chemistry Learning in Higher Education

The inclusion of General Chemistry (GC) in the curricula of higher education courses in science and technology aims, on the one hand, to develop students' skills necessary for further studies and, on the other hand, to respond to the need of endowing future professionals of knowledge to analyze and solve multidisciplinary problems in a sustainable way. The participation of students in the evaluation of the role played by the GC in their training is crucial, and the analysis of the results can be an essential tool to increase success in the education of students and improving practices in various professions. Undeniably, this work will be focused on the development of an intelligent system to assess the role of GC. The computational framework is built on top of a Logic Programming approach to Knowledge Representation and Reasoning, complemented with a problem solving methodology moored on Artificial Neural Networks. The results so far obtained show that the proposed model stands for a good start, being its overall accuracy higher than 95%.

Communities of practice as professional and organizational development strategy in local public health organizations in Québec, Canada : an evaluation model

Communities of practice (CoPs) are among the professional development strategies most widely used in such fields as management and education. Though the approach has elicited keen interest, knowledge pertaining to its conceptual underpinnings is still limited, thus hindering proper assessment of CoPs' effects and the processes generating the latter. To address this shortcoming, this paper presents a conceptual model that was developed to evaluate an initiative based on a CoP strategy: Health Promotion Laboratories are a professional development intervention that was implemented in local public health organizations in Montreal (Quebec, Canada). The model is based on latest theories on work-group effectiveness and organizational learning and can be usefully adopted by evaluators who are increasingly called upon to illuminate decision-making about CoPs. Ultimately, validation of this conceptual model will help advance knowledge and practice pertaining to CoPs as well as professional and organizational development strategies in public health.

Inequities in energy-balance related behaviours and family environmental determinants in European children: baseline results of the prospective EPHE evaluation study

BACKGROUND:Tackling inequalities in overweight, obesity and related determinants has become a top priority for the European research and policy agendas. Although it has been established that such inequalities accumulate from early childhood onward, they have not been studied extensively in children. The current article discusses the results of an explorative analysis for the identification of inequalities in behaviours and their determinants between groups with high and low socio-economic status. METHODS: This study is part of the Epode for the Promotion of Health Equity (EPHE) evaluation study, the overall aim of which is to assess the impact and sustainability of EPODE methodology to diminish inequalities in childhood obesity and overweight. Seven community-based programmes from different European countries (Belgium, Bulgaria, France, Greece, Portugal, Romania, The Netherlands) participate in the EPHE study. In each of the communities, children aged 6-8 years participated, resulting in a total sample of 1266 children and their families. A parental self-administrated questionnaire was disseminated in order to assess the socio-economic status of the household, selected energy balance-related behaviours (1. fruit and vegetable consumption; 2. soft drink/ fruit juices and water consumption; 3. screen time and 4. sleep duration) of the children and associated family environmental determinants. The Mann-Whitney U test and Pearson's chi-square test were used to test differences between the low and high education groups. The country-specific median was chosen as the cut-off point to determine the educational level, given the different average educational level in every country. RESULTS: Children with mothers of relatively high educational level consumed fruits and vegetables more frequently than their peers of low socio-economic status. The latter group of children had a higher intake of fruit juices and/or soft drinks and had higher screen time. Parental rules and home availability were consistently different between the two socio-economic groups in our study in all countries. However we did not find a common pattern for all behaviours and the variability across the countries was large. CONCLUSIONS: Our findings are indicative of socio-economic inequalities in our samples, although the variability across the countries was large. The effectiveness of interventions aimed at chancing parental rules and behaviour on health inequalities should be studied.

Evaluation of the quality of the teaching-learning process in undergraduate courses in Nursing

Objective: to identify aspects of improvement of the quality of the teaching-learning process through the analysis of tools that evaluated the acquisition of skills by undergraduate students of Nursing. Method: prospective longitudinal study conducted in a population of 60 second-year Nursing students based on registration data, from which quality indicators that evaluate the acquisition of skills were obtained, with descriptive and inferential analysis. Results: nine items were identified and nine learning activities included in the assessment tools that did not reach the established quality indicators (p<0.05). There are statistically significant differences depending on the hospital and clinical practices unit (p<0.05). Conclusion: the analysis of the evaluation tools used in the article "Nursing Care in Welfare Processes" of the analyzed university undergraduate course enabled the detection of the areas for improvement in the teaching-learning process. The challenge of education in nursing is to reach the best clinical research and educational results, in order to provide improvements to the quality of education and health care.

Goma Curriculum, A Character Education Paradigm: Composing a Text for Shaping Classroom Character Culture

The written text, and approaches to reading it, serves well as an analogy for the classroom space as a “text” that teachers are able to compose; and students are able to read, interpret meaning(s) of, and make responses to and about (Rosenblatt, 1988). Researchers point to ways in which the classroom can be conceptualized as a text to be evoked, experienced, and read (Freire & Macedo, 1987; Powell, 2009; Rosenblatt, 1988; Spears-Bunton & Powell, 2009). The present study analyzed secondary data including: 10 transcripts of teacher talks and six self-reports retrieved from the program evaluation archives of DOR Foundation. The data described six teachers’ classroom experiences subsequent to professional development centered on Goma character education curriculum that was used during a summer youth program located in South Georgia. Goma, an acronym that stands for Goal, Objective, Method, and Attitude, is a character education paradigm derived from The Inclusive Community Building Ellison Model, the theoretical framework used for this study. The Model identifies conflict resolution as one of its five foci (Hunt, Howard, & Rice, 1998). Hunt (2006) conceived Goma as part of a 7-Step unitary process, also named the 7-Step pathway, to demonstrate how conflict resolution is accomplished within a variety of contexts. Analysis of the data involved: (a) a priori coding of teacher talks transcripts using the components of the Goma 7-Step pathway as coding categories, (b) emergent coding of teacher talks transcripts for the types of experiences teachers evidenced, and (c) emergent coding of teachers’ self-reports for categories of teachers’ instructional activities. Results of the study showed positive influence of Goma curriculum on participating teachers and their instructional practices. Teachers were shown to have had cognitive, instructional, emotional, and social experiences that were most evident when they reported changes in their attitudes toward their students, themselves, and their instructional practices. The present study provided implications for classroom teachers wherein all aspects of teachers’ instructional practices can be guided by principles of positive character; and can be used to help compose the kinds of “texts” that may likely contribute to a classroom character culture.

Online education, accrediting standards, and student success: An examination of the relationship between the southern association of colleges and schools commission on colleges standards for online education and student success

Prior to 2000, there were less than 1.6 million students enrolled in at least one online course. By fall 2010, student enrollment in online distance education showed a phenomenal 283% increase to 6.1 million. Two years later, this number had grown to 7.1 million. In light of this significant growth and skepticism about quality, there have been calls for greater oversight of this format of educational delivery. Accrediting bodies tasked with this oversight have developed guidelines and standards for online education. ^ There is a lack of empirical studies that examine the relationship between accrediting standards and student success. The purpose of this study was to examine the relationship between the presence of Southern Association of Colleges and Schools Commission on College (SACSCOC) standards for online education in online courses, (a) student support services and (b) curriculum and instruction, and student success. An original 24-item survey with an overall reliability coefficient of .94 was administered to students (N=464) at Florida International University, enrolled in 24 university-wide undergraduate online courses during fall 2014, who rated the presence of these standards in their online courses. The general linear model was utilized to analyze the data. The results of the study indicated that the two standards, student support services and curriculum and instruction were both significantly and positively correlated with student success but with small R2 and strengths of association less than .35 and .20 respectively. Mixed results were produced from Chi-square tests for differences in student success between higher and lower rated online courses when controlling for various covariates such as discipline, gender, race/ethnicity, GPA, age, and number of online courses previously taken. A multiple linear regression analysis revealed that the curriculum and instruction standard was the only variable that accounted for a significant amount of unique variance in student success. Another regression test revealed that no significant interaction effect exists between the two SACSCOC standards and GPA in predicting student success. ^ The results of this study are useful for administrators, faculty, and researchers who are interested in accreditation standards for online education and how these standards relate to student success.^