A summative evaluation of a HIV/AIDS Early Childhood Care, Education and Development Teacher Training Workshop in Mongu, Zambia

Objective. To conduct a summative evaluation of an Early Childhood Care, Education and Development (ECCED) Teacher Training Workshop in Mongu, Zambia by assessing changes in knowledge, attitudes and intent to use the information. ^ Study design. A matched cohort survey design was used with additional qualitative data collected by structured observation of workshop sessions, daily facilitator and participant debriefs and participant interviews. ^ Results. Matching pre and post tests were completed by 27 individuals in addition to daily debriefs, structured workshop observation and participant interviews with 22% of the group. The participant population was predominantly female individuals aged 15-44 years old that had completed high school and additional post-secondary training, been teaching children aged 0 – 8 years for 2-5 years in the Western Province and received other HIV/AIDS and ECCED education. Pre-tests indicated a strong understanding of ECCED principles and misconceptions regarding HIV transmission, prevention and the disease's impact on early childhood development. The workshop was found to significantly increase the participants' knowledge of topics covered by the curriculum (paired t-test, N=27, p = 0.004, 95% CI 1.8, 8.6). Participants began with a more limited understanding of HIV/AIDS than ECCED, but the mean gain was much greater at 7.4 +/- 12.3 points. Significantly more participants believed at post-test that HIV/AIDS education should increase for future educators. The 77.8% of participants that increased their knowledge scores at post-test expressed significantly less fear of having a child with HIV/AIDS in the classroom (Independent Samples t-test, N= 27, p = 0.011). Overall participant fear decreased 15.5%. 92.6% and 88.9% of participants planned at post-test to respectively use and share the taught information in their daily professional lives and reported on innovative strategies to communicate with the community. ^ Conclusions. Teacher training workshops can significantly increase HIV/AIDS awareness and promote positive attitudes in educators working with children affected by HIV/AIDS. Using participant suggested teaching techniques such as poems and songs and translating the materials to the local language could assist future facilitators to both culturally and professionally relate to the workshop audience as well as increase participant capacity to share the information with the local community. ^

NAVET's Boxes - an Evaluation of the Post-Visit Loan Service at a Science Centre in Borås

Many pedagogues believe science centres to be a good complement to the more formal school teaching. For a visit to a science centre to be as educational as possible, there is a need for pre-visit information of some sort, a guided visit, and post-visit work. Many science centres offer loan services of different kinds. At Navet, a science centre in Borås, teachers can borrow boxes with experiments connected to the different themes they provide. The experiments are supposed to be a continuation of the visit and help settle the knowledge gained during the visit. This thesis is an evaluation of how the boxes function in the schools, and what the teachers think of them. The study was conducted through questionnaires and interviews with both teachers and the staff at Navet. The results of the study are very positive. Many teachers have been involved with Navet from the very beginning and they see a visit to Navet as an integrated part of their teaching. Some boxes work better than others and some might need clearer information, but overall the teachers see the boxes as timesavers, as a way to vary their teaching more easily, and as a help for teachers not specialized in mathematics and science.

Evaluation and Model of a Clinical Librarian Program

Clinical medical librarianship is entering its second decade, but little evaluative data has accrued in the literature. Variations from the original programs and novel new approaches have insured the survival of the program so far. The clinical librarian (CL) forms a vital link between the library and the health care professional, operating as an important information transfer agent. However, to further insure the survival of these vital programs, hard evaluative evidence is needed. The University of Texas Medical Branch (UTMB) at Galveston began a CL Program in 1978/79. An extensive three-year pre/post evaluation study was conducted using a specifically developed evaluation model, which, if adopted by others, will provide the needed comparative data. Both a pilot study, or formative evaluation, and a summative evaluation were conducted. The results of this evaluation confirmed many of the conclusions reported by other CL programs. Eight hypotheses were proposed at the beginning of this study. Data were collected and used to support acceptance or rejection of the null hypotheses, and conclusions were drawn according to the results. Implications relevant to the study conclusions and future trends in medical librarianship are also discussed in the closing chapter.

Assessing an experimental approach to industrial policy evaluation:Applying RCT+ to the case of Creative Credits

Experimental methods of policy evaluation are well-established in social policy and development eco-nomics but are rare in industrial and innovation policy. In this paper, we consider the arguments forapplying experimental methods to industrial policy measures, and propose an experimental policy eval-uation approach (which we call RCT+). This approach combines the randomised assignment of firmsto treatment and control groups with a longitudinal data collection strategy incorporating quantitativeand qualitative data (so-called mixed methods). The RCT+ approach is designed to provide a causativerather than purely summative evaluation, i.e. to assess both ‘whether’ and ‘how’ programme outcomesare achieved. In this paper, we assess the RCT+ approach through an evaluation of Creative Credits – aUK business-to-business innovation voucher initiative intended to promote new innovation partnershipsbetween SMEs and creative service providers. The results suggest the potential value of the RCT+ approachto industrial policy evaluation, and the benefits of mixed methods and longitudinal data collection.

Avaliação do desempenho docente : o papel da supervisão e as competências dos professores avaliadores

Dissertação apresentada à Escola Superior de Educação de Lisboa para obtenção do grau de Mestre em Ciências da Educação - Especialização em Supervisão em Educação

Métodos de Avaliação de Aprendizagens em História e Geografia do 3º Ciclo – Um Estudo Comparativo

A avaliação, no contexto de ensino-aprendizagem, ocupa um papel central e fulcral. Subdivide-se em diversas formas e modalidades que nos ajudam a compreender as formas de implementar momentos avaliativos e de conduzir práticas letivas. O presente relatório pretende efetuar uma contextualização teórica de alguns conceitos relacionados com a avaliação – nomeadamente a avaliação sumativa e a avaliação formativa – compreendendo a sua praticabilidade na atividade docente. A Prática de Ensino Supervisionada decorreu em duas escolas diferentes, e a minha prática letiva cingiu-se aos 8º e 9º anos do Ensino Básico. Desta forma, o trabalho apresentado efetua uma comparação entre duas práticas letivas e avaliativas, analisando a sua repercussão ao nível da aprendizagem dos alunos

Assessing evaluative measures for Internet-based courseware

Although there is a consensus in th~ literature on the many uses of the Internet in education, as well as the unique features of the Internet for presenting facts and information, there is no consensus on a standardized method for evaluating Internetbased courseware. Educators rarely have the opportunity to participate in the development of Internet-based courseware, yet they are encouraged to use the technology in their learning environments. This creates a need for summative evaluation methods for Internet-based health courseware. The purpose ofthis study was to assess evaluative measures for Internet-based courseware. Specifically, two entities were evaluated within the study: a) the outcome of the Internet-based courseware, and b) the Internet-based courseware itself. To this end, the Web site www.bodymatters.com was evaluated using two different approaches by two different cohorts. The first approach was a performance appraisal by a group of endusers. A positive, statistically significant change in the students performance was observed due to the intervention ofthe Web site. The second approach was a productoriented evaluation ofthe Web site with the use of a criterion-based checklist and an open-ended comments section. The findings indicate that a summative, criterion-based evaluation is best completed by a multidisciplinary team. The findi~gs also indicated that the two different cohorts reported different product-oriented appraisals of the Web site. The current research confirmed previous research that found that experts returning a poor evaluation of a Web site did not have a relationship to whether or not the end-users performance improved due to the intervention of the Web site.

Akzeptanz und Motivation beim selbstregulierten Lernen unter Einsatz von neuen Medien in der berufsbegleitenden Weiterbildung

Das Ziel der Dissertation war die Untersuchung des Computereinsatzes zur Lern- und Betreuungsunterstützung beim selbstgesteuerten Lernen in der Weiterbildung. In einem bisher konventionell durchgeführten Selbstlernkurs eines berufsbegleitenden Studiengangs, der an das Datenmanagement der Bürodatenverarbeitung heranführt, wurden die Kursunterlagen digitalisiert, die Betreuung auf eine online-basierte Lernbegleitung umgestellt und ein auf die neuen Lernmedien abgestimmtes Lernkonzept entwickelt. Dieses neue Lernkonzept wurde hinsichtlich der Motivation und der Akzeptanz von digitalen Lernmedien evaluiert. Die Evaluation bestand aus zwei Teilen: 1. eine formative, den Entwicklungsprozess begleitende Evaluation zur Optimierung der entwickelten Lernsoftware und des eingeführten Lernkonzeptes, 2. eine sowohl qualitative wie quantitative summative Evaluation der Entwicklungen. Ein zentraler Aspekt der Untersuchung war die freie Wahl der Lernmedien (multimediale Lernsoftware oder konventionelles Begleitbuch) und der Kommunikationsmedien (online-basierte Lernplattform oder die bisher genutzten Kommunikationskanäle: E-Mail, Telefon und Präsenztreffen). Diese Zweigleisigkeit erlaubte eine differenzierte Gegenüberstellung von konventionellen und innovativen Lernarrangements. Die Verbindung von qualitativen und quantitativen Vorgehensweisen, auf Grund derer die subjektiven Einstellungen der Probanden in das Zentrum der Betrachtung rückten, ließen einen Blickwinkel auf den Nutzen und die Wirkung der Neuen Medien in Lernprozessen zu, der es erlaubte einige in der Literatur als gängig angesehene Interpretationen in Frage zu stellen und neu zu diskutieren. So konnten durch eine Kategorisierung des Teilnehmerverhaltens nach online-typisch und nicht online-typisch die Ursache-Wirkungs-Beziehungen der in vielen Untersuchungen angeführten Störungen in Online-Seminaren verdeutlicht werden. In den untersuchten Kursen zeigte sich beispielsweise keine Abhängigkeit der Drop-out-Quote von den Lern- und Betreuungsformen und dass diese Quote mit dem neuen Lernkonzept nur geringfügig beeinflusst werden konnte. Die freie Wahl der Lernmedien führte zu einer gezielten Nutzung der multimedialen Lernsoftware, wodurch die Akzeptanz dieses Lernmedium stieg. Dagegen war die Akzeptanz der Lernenden gegenüber der Lernbegleitung mittels einer Online-Lernplattform von hoch bis sehr niedrig breit gestreut. Unabhängig davon reichte in allen Kursdurchgängen die Online-Betreuung nicht aus, so dass Präsenztreffen erbeten wurde. Hinsichtlich der Motivation war die Wirkung der digitalen Medien niedriger als erwartet. Insgesamt bieten die Ergebnisse Empfehlungen für die Planung und Durchführung von computerunterstützten, online-begleiteten Kursen.

TELMA: Technology enhanced learning environment for minimally invasive surgery

Background: Cognitive skills training for minimally invasive surgery has traditionally relied upon diverse tools, such as seminars or lectures. Web technologies for e-learning have been adopted to provide ubiquitous training and serve as structured repositories for the vast amount of laparoscopic video sources available. However, these technologies fail to offer such features as formative and summative evaluation, guided learning, or collaborative interaction between users. Methodology: The "TELMA" environment is presented as a new technology-enhanced learning platform that increases the user's experience using a four-pillared architecture: (1) an authoring tool for the creation of didactic contents; (2) a learning content and knowledge management system that incorporates a modular and scalable system to capture, catalogue, search, and retrieve multimedia content; (3) an evaluation module that provides learning feedback to users; and (4) a professional network for collaborative learning between users. Face validation of the environment and the authoring tool are presented. Results: Face validation of TELMA reveals the positive perception of surgeons regarding the implementation of TELMA and their willingness to use it as a cognitive skills training tool. Preliminary validation data also reflect the importance of providing an easy-to-use, functional authoring tool to create didactic content. Conclusion: The TELMA environment is currently installed and used at the Jesús Usón Minimally Invasive Surgery Centre and several other Spanish hospitals. Face validation results ascertain the acceptance and usefulness of this new minimally invasive surgery training environment.

CSR and the "underserving":a role for the state, civil society and business?

Purpose – The purpose of this paper is to demonstrate how the conceptual lens of corporate social responsibility (CSR), business and civil society can be used to explore “less popular causes” (in this case, a community-based public sector empirical study of initiatives with offenders) and, in particular, respond to the question used by Walzer “In which society can lives be best led?” Design/methodology/approach – This is a formative and summative evaluation study of a National Offender Management “community payback” offender scheme based in the UK using a mixed method, predominantly qualitative approach that integrates theory and practice. Findings – The paper finds that citizenship actions of front-line public sector employees, working in partnership with other agencies in the community, embody the essence of Walzer's notion of CSR and civil society by going beyond the call of duty to provide additional training and moral support for the community offenders. Originality/value – The paper contributes towards an understanding of how CSR and civil society debates can inform wider aspects of public policy and business through its application to areas of society that are perceived to be “challenging” and “undeserving”.

One or the other? Do we have to choose? : summative and-or formative evaluation. '¿Una u otra? ¿Tenemos que elegir? : evaluación formativa y-o sumativa'.

Se definen los conceptos de evaluación sumativa y evaluación formativa con el ánimo de ver si son tan diferentes y excluyentes. Se proponen caminos para combinar ambas y para incluir a los estudiantes en el proceso de evaluación.

Newham’s Every Child a Sports Person (NECaSP): A Summative Process Evaluation of a School-and-Community Based Intervention in East London, UK

Background: The NECaSP intervention aspires to increase sport and physical activity (PA) participation amongst young people in the UK. The aims of this paper are to report on a summative process evaluation of the NECaSP and make recommendations for future interventions. Methods: Seventeen schools provided data by students aged 11-13 (n=1,226), parents (n=192) and teachers (n= 14) via direct observation and questionnaires. Means, standard deviations and percentages were calculated for socio-demographic data. Qualitative data was analysed via directed content analysis and main themes identified. Results: Findings indicate further administrative, educational and financial support will help facilitate the success of the programme in improving PA outcomes for young people, and of other similar intervention programmes globally. Data highlighted the need to engage parents to increase likelihood of intervention success. Conclusions: One main strength of this study is the mixed-methods nature of the process evaluation. It is recommended that future school based interventions that bridge sports clubs and formal curriculum provision, should consider a more broad approach to the delivery of programmes throughout the academic year, school week and school day. Finally, changes in the school curriculum can be successful once all parties are involved (community, school, families).

Evaluation of the agricultural vocational training programmes conducted by the Krishi Vigyan Kendras (Farm Science Centres) in Indian Punjab

Krishin Vigyan Kendras-KVKs (Farm Science Centres) have been established by the Indian Council of Agricultural Research in 569 districts. The trust areas of KVKs are refinement and demonstration of technologies, and training of farmers and extension functionaries. Imparting vocational trainings in agriculture and allied fields for the rural youth is one of its mandates. The study was undertaken to do a formative and summative (outcome and impact) evaluation of the beekeeping and mushroom growing vocational training programmes in the Indian state of Punjab. One-group pre and post evaluation design was employed for conducting a formative and outcome evaluation. The knowledge tests were administered to 35 beekeeping and 25 mushroom cultivation trainees, before and after the training programmes organized in 2004. The trainees significantly gained in knowledge. A separate sample of 640 trainees, trained prior to 2004, was selected for finding the adoption status. Out of 640, a sample of 200 was selected by proportionate sampling technique out of three categories, namely: non-adopters, discontinued-adopters and continued-adopters for evaluating the long-term impact of these training programmes. Ex-post-facto one-shot case study design was applied for this impact analysis. The vocational training programmes have resulted in continued-adoption of beekeeping and mushroom cultivation enterprises by 20% and 51% trained farmers, respectively. Age and trainee occupation had significant influence on the adoption decision of beekeeping vocation, whereas education and family income significantly affected the adoption decision of mushroom cultivation. The continued adopters of beekeeping and mushroom growing had increased their family income by 49% and 24%, respectively. These training programmes are augmenting the dwindling farm income of the farmers in Indian Punjab.

The new final Clinical Skills examination in human medicine in Switzerland: Essential steps of exam development, implementation and evaluation, and central insights from the perspective of the national Working Group

Objective: Since 2011, the new national final examination in human medicine has been implemented in Switzerland, with a structured clinical-practical part in the OSCE format. From the perspective of the national Working Group, the current article describes the essential steps in the development, implementation and evaluation of the Federal Licensing Examination Clinical Skills (FLE CS) as well as the applied quality assurance measures. Finally, central insights gained from the last years are presented. Methods: Based on the principles of action research, the FLE CS is in a constant state of further development. On the foundation of systematically documented experiences from previous years, in the Working Group, unresolved questions are discussed and resulting solution approaches are substantiated (planning), implemented in the examination (implementation) and subsequently evaluated (reflection). The presented results are the product of this iterative procedure. Results: The FLE CS is created by experts from all faculties and subject areas in a multistage process. The examination is administered in German and French on a decentralised basis and consists of twelve interdisciplinary stations per candidate. As important quality assurance measures, the national Review Board (content validation) and the meetings of the standardised patient trainers (standardisation) have proven worthwhile. The statistical analyses show good measurement reliability and support the construct validity of the examination. Among the central insights of the past years, it has been established that the consistent implementation of the principles of action research contributes to the successful further development of the examination. Conclusion: The centrally coordinated, collaborative-iterative process, incorporating experts from all faculties, makes a fundamental contribution to the quality of the FLE CS. The processes and insights presented here can be useful for others planning a similar undertaking. Keywords: national final examination, licensing examination, summative assessment, OSCE, action research

Baseline evaluation of environmental physical activity and nutrition support of Texas Medical Center hospital worksites

Background: Obesity is a major health problem in the United States that has reached epidemic proportions. With most U.S adults spending the majority of their waking hours at work, the influence of the workplace environment on obesity is gaining in importance. Recent research implicates worksites as providing an 'obesogenic' environment as they encourage overeating and reduce the opportunity for physical activity. Objective: The aim of this study is to describe the nutrition and physical activity environment of Texas Medical Center (TMC) hospitals participating in the Shape Up Houston evaluation study to develop a scoring system to quantify the environmental data collected using the Environmental Assessment Tool (EAT) survey and to assess the inter-observer reliability of using the EAT survey. Methods: A survey instrument that was adapted from the Environmental Assessment Tool (EAT) developed by Dejoy DM et al in 2008 to measure the hospital environmental support for nutrition and physical activity was used for this study. The inter-observer reliability of using the EAT survey was measured and total percent agreement scores were computed. Most responses on the EAT survey are dichotomous (Yes and No) and these responses were coded with a '0' for a 'no' response and a '1' for a 'yes' response. A summative scoring system was developed to quantify these responses. Each hospital was given a score for each scale and subscale on the EAT survey in addition to a total score. All analyses were conducted using Stata 11 software. Results: High inter-observer reliability is observed using EAT. The percentage agreement scores ranged from 94.4%–100%. Only 2 of the 5 hospitals had a fitness facility onsite and scores for exercise programs and outdoor facilities available for hospital employees ranged from 0–62% and 0–37.5%, respectively. The healthy eating percentage for hospital cafeterias range from 42%–92% across the different hospitals while the healthy vending scores were 0%–40%. The total TMC 'healthy hospital' score was 49%. Conclusion: The EAT survey is a reliable instrument for measuring the physical activity and nutrition support environment of hospital worksites. The study results showed a large variability among the TMC hospitals in the existing physical activity and nutrition support environment. This study proposes cost effective policy changes that can increase environmental support to healthy eating and active living among TMC hospital employees.^