An evaluation framework for sustaining the impact of educational development

Notwithstanding significant efforts by international aid agencies, aid ineffectiveness became apparent in 1990s as the impact of continued development intervention did not endure the expected outcomes. Conventional monitoring and evaluation by those agencies is critiqued for focusing on measuring project outcomes and giving little attention to aspects of sustainability. As a result, devising a rigorous evaluation framework for educational development has been sought in light of recent paradigm shifts in international development. This paper reports on a case study of an Egyptian educational development project highlighting the importance of transforming the evaluation procedures to process evaluation so as to enhance project impact and longevity. This requires building evaluation capacity of the aid recipient country.

A Randomized Controlled Trial Evaluation of Time to Read, a Volunteer Tutoring Program for 8-9 Year Olds

Tutoring is commonly employed to prevent early reading failure, and evidence suggests that it can have a positive effect. This article presents findings from a large-scale (n = 734) randomized controlled trial evaluation of the effect of Time to Read—a volunteer tutoring program aimed at children aged 8 to 9 years—on reading comprehension, self-esteem, locus of control, enjoyment of learning, and future aspirations. The study found that the program had only a relatively small effect on children’s aspirations (effect size +0.17, 95% confidence interval [0.015, 0.328]) and no other outcomes. It is suggested that this lack of evidence found may be due to misspecification of the program logic model and outcomes identified and program-related factors, particularly the low dosage of the program.

Evaluation of the RB-RB/LB-LB mnemonic rule for recording optimally projected intraoral images of dental implants: an in vitro study

Objective: The aim of this study was to evaluate a simple mnemonic rule (the RB-RB/LB-LB rule) for recording intra-oral radiographs with optimal projection for the control of dental implants.Methods: 30 third-year dental students received a short lesson in the RB-RB/LB-LB mnemonic rule. The rule is as follows: if right blur then raise beam (RB-RB), i.e. if implant threads are blurred at the right side of the implant, the X-ray beam direction must be raised towards the ceiling to obtain sharp threads on both implant sides; if left blur then lower beam (LB-LB), i.e. if implant threads are blurred at the left side of the implant, the X-ray beam direction must be lowered towards the floor to obtain sharp threads on both implant sides. Intra-oral radiographs of four screw-type implants placed with different inclination in a Frasaco upper or lower jaw dental model (Frasaco GmbH, Tettnang, Germany) were recorded. The students were unaware of the inclination of the implants and were instructed to re-expose each implant, implementing the mnemonic rule, until an image of the implant with acceptable quality (subjectively judged by the instructor) was obtained. Subsequently, each radiograph was blindly assessed with respect to sharpness of the implant threads and assigned to one of four quality categories: (1) perfect, (2) not perfect, but clinically acceptable, (3) not acceptable and (4) hopeless.Results: For all implants, from one non-perfect exposure to the following, a higher score was obtained in 64% of the cases, 28% received the same score and 8% obtained a lower score. Only a small variation was observed among exposures of implants with different inclination. on average, two exposures per implant (range: one to eight exposures) were needed to obtain a clinically acceptable image.Conclusion: The RB-RB/LB-LB mnemonic rule for recording intra-oral radiographs of dental implants with a correct projection was easy to implement by inexperienced examiners. Dentomaxillofacial Radiology (2012) 41, 298-304. doi: 10.1259/dmfr/20861598

Service evaluation survey.

Each year the Lieutenant Governor is responsible for sending out and tallying the results of the "Service Evaluation Survey" designed to enable school boards to assess the Illinois State Board of Education (ISBE) and the Regional Offices of Education (ROEs). This year, in compliance with the School Code, and with the assistance of Evaluation Service Committee members, the substance of the survey has been updated and improved. The survey focuses on the primary service areas of ISBE and ROEs, including communication, certification, educational program support and finances. Additionally, this year the survey encouraged written comments, and the enclosed report details the points of view that were expressed.

Analysis of the education program approval process : a program evaluation /

"May 1988."

A study of the implementation of junior college *evaluation system in Taiwan

Since the establishment of the evaluation system in 1975, the junior colleges in the Republic of China (Taiwan), have gone through six formal evaluations. We know that evaluation in schooling, like quality control in businesses, should be a systematic, formal, and a continual process. It can doubtless serve as a strategy to refine the quality of education. The purpose of this research is to explore the current practice of junior college evaluation in Taiwan. This provides insight into the development of and quality of the current evaluation system. Moreover, this study also identified the source of problems with the current evaluation system and provided suggestion for improvements.^ In order to attain the above purposes, this research was undertaken in both theoretical and practical ways. First, theoretically, on the basis of a literature review, the theories of educational evaluation and, according to the course and principles of development, a view of the current practice in Taiwan. Secondly, in practice, by means of questionnaires, an analysis of the views of evaluation committeemen, junior college presidents, and administrators were obtained on evaluation models, methods, contents, organization, functions, criteria, grades reports, and others with suggestions for improvement. The summary of findings concludes that most evaluators and evaluatees think the purpose of evaluation can help the colleges explore their difficulties and problems. In addition, it was found that there is significant difference between the two groups regarding the evaluation methods, contents, organization, functions, criteria, grades reports and others, while analyzing these objective data forms the basis for an improved method of evaluation for Junior Colleges in Taiwan. ^

Assessment for learning in the accountability era : Queensland, Australia

Developments in school education in Australia over the past decade have witnessed the rise of national efforts to reform curriculum, assessment and reporting. Constitutionally the power to decide on curriculum matters still resides with the States. Higher stakes in assessment, brought about by national testing and international comparative analyses of student achievement data, have challenged State efforts to maintain the emphasis on assessment to promote learning while fulfilling accountability demands. In this article lessons from the Queensland experience indicate that it is important to build teachers' assessment capacity and their assessment literacy for the promotion of student learning. It is argued that teacher assessment can be a source of dependable results through moderation practice. The Queensland Studies Authority has recognised and supported the development of teacher assessment and moderation practice in the context of standards-driven, national reform. Recent research findings explain how the focus on learning can be maintained by avoiding an over-interpretation of test results in terms of innate ability and limitations and by encouraging teachers to adopt more tailored diagnosis of assessment data to address equity through focus on achievement for all. Such efforts are challenged as political pressures related to the Australian government’s implementation of national testing and national partnership funding arrangements tied to the performance of students at or below minimum standards become increasingly apparent.

External management of curriculum change

At a time of rapid curriculum reform, some schools are turning to external change agents for assistance, but what's the best way to initiate such an approach?

Evaluating leadership development using the Most Significant Change technique

Course evaluations are now a serious matter for universities trying to meet stakeholder needs and expectations, quality assurance, improvements and strategic decision making. Typically, students are invited to participate in surveys on how well the design and delivery aspects meet predetermined learning objectives, quality of teaching, and the types of improvements needed for future deliveries. We used the Most Significant Change technique to gather data on the impact of a leadership course on 18 Pacific Islanders who completed a Master of Education (Educational Leadership). Participants' views highlighted impacts that were of significance to the students and their workplaces. The findings demonstrate that the Most Significant Change technique offers a more comprehensive understanding of the impact of leadership development courses.

Feasibility of using technology to disseminate evidence to rural nurses and improve patient outcomes

Background: Rural African American women receive less frequent mammography screening and die of breast cancer at a higher rate than is seen in the general population. To overcome this disparity, it is necessary to assist rural providers in their efforts to influence women to obtain screening. Method: This study examined the feasibility of using distance education to disseminate knowledge about timely and appropriate mammography screening to rural nurses, using patient outcome data to evaluate the effectiveness of this intervention. Results: Overall, there was a decline in referrals and mammography screening, but the intervention group centers showed a smaller decline after the educational intervention than did the control group. Conclusion: The findings show the effect of dissemination of information and the feasibility of using patient outcome data for educational evaluation. Neighboring academic health centers and nursing schools should include in their mission the provision of educational programs for relatively isolated rural nurses.

A Prova Brasil e a contribuição para os processos de letramento e formação de alunos leitores

A avaliação da Educação Básica tem sido alvo de discussões em diversas esferas da sociedade e centra-se em dois aspectos principais: os quantitativos de controle e os organizacionais dos processos de ensino, ou seja, a avaliação pedagógica. O objetivo desta dissertação é refletir sobre a metodologia utilizada na Prova Brasil e as contribuições que esta avaliação pode oferecer a formação de alunos leitores. Há alguns trabalhos na área já que se trata de uma avaliação relativamente recente. Nossa pesquisa se diferencia das demais ao apresentar uma análise detalhada dos tópicos, descritores e itens que compõem a avaliação. O processo de pesquisa empreendido neste estudo perpassa quatro fases: na primeira etapa analisamos os aspectos particulares do sistema avaliativo nacional, na segunda fase refletimos sobre a construção da proposta avaliativa e sua apresentação, na terceira fase relacionamos os elementos presentes na proposta, analisando de que maneira estes elementos afetam positivamente ou negativamente a avaliação e na última fase apresentamos nossas considerações à luz das teorias linguísticas e pedagógicas. Utilizamos como corpus os itens de Língua Portuguesa referentes ao 5 ano de escolaridade liberados pelo INEP. Nossa pesquisa evidenciou que apesar de cada item ter como objetivo verificar a habilidade expressa por um único descritor, boa parte das questões apresenta problemas de seleção textual ou de elaboração fazendo com que determinadas habilidades não sejam efetivamente contempladas na avaliação, fato este que compromete diretamente a confiabilidade dos resultados apresentados. A contribuição social desta pesquisa reside na orientação para o trabalho com a multiplicidade de gêneros que circulam na sociedade, ressaltando a importância da formação plena do educando, não para que ele demonstre bom desempenho em avaliações, mas para que ele possa atuar em favor de sua própria vida e em benefício da sociedade em que está inserido, transformando-a

Qualidade na escola e qualidade da escola: as repercussões da cultura do exame em duas escolas públicas de Duque de Caxias

O trabalho discute a qualidade da escola aferida por instrumentos avaliativos externos e a qualidade na escola vivida no cotidiano, por meio de transações. Traz a contribuição de autores que discutem as correlações entre qualidade e avaliação educacional, bem como o advento do Índice de Desenvolvimento da Educação Básica (IDEB) como principal ferramenta de aferição da qualidade das escolas. Tem como questões: investigar de que forma a cultura do exame tem contribuído na produção de indicadores de qualidade na escola, quais os sentidos de qualidade atribuídos pela comunidade escolar ao conhecer os resultados das avaliações externas e de que forma as avaliações em larga escala interferem nas concepções de qualidade dos sujeitos no cotidiano escolar. A metodologia considera o cotidiano escolar em sua complexidade e, no viés do paradigma indiciário, percebe o sentido e o significado de práticas exercidas pela escola que, principalmente, estão sendo desprezadas na concepção que vem se criando do que é uma escola de qualidade. Conceituando a avaliação formativa como ferramenta para a promoção da qualidade escolar social, a pesquisa chama a atenção para o uso da avaliação em larga escala de forma vertical, pontual, inspirada nos parâmetros de neutralidade, centrada no produto, entendendo ser este o conhecimento a ser medido, o sujeito da educação. Na contramão, procura compreender os sentidos de qualidade entendidos pelos partícipes do projeto educativo da escola e a forma como esta qualidade é negociada, em sua operacionalização, vislumbrando práticas e proposições que apontam um movimento de avaliação interna nas escolas.

Avaliação em contextos de blended learning no ensino superior

The increasing complexity of the Information Society and the Bologna Declaration in the European context has led Higher Education (HE) institutions to revise their curricula courses, as far as the adoption of new strategies for teaching and learning as well as evaluation are concerned. It can also be emphasized that there has been a growing use of eLearning and of blended learning (bLearning) in HE, since these modes of training seem to be a very convenient option for lifelong learning. In this context, quality is taken as an essential goal in the development of the European Space for Education and Training, where HE institutions compete among themselves, and where evaluation is determinant as a promoter of this quality. Considering the problems summarized above, the research developed, based on four published scientific papers, intended to answer a set of research questions related to evaluation of bLearning contexts in HE. The study used diverse techniques and instruments (questionnaires, document analysis, and observation mediated technologies) spanning two methodological approaches: i) study of descriptive and exploratory nature and ii) case studies of bLearning modules. In the first approach an evaluation model for bLearning courses was developed, where we collected and analyzed, at a national level, the opinions of teachers with bLearning experience about the model dimensions. The case studies presented are post graduation curricular units, where bLearning teaching, learning and evaluation strategies were explored and evaluated, namely peer assessment. The main contributions of the first approach are: the process of questioning around the evaluation of bLearning courses, namely the quality assurance criteria for bLearning, as well as the model developed, providing a framework of theoretical, methodological and empirical elements that can be adapted in similar contexts. From the case studies emerged: the developed evaluation guidelines and the data collection instruments, in order to disseminate evaluation “best practices” that may be useful for other units in similar contexts. Regarding the recommendations about the evaluation of teaching of bLearning courses we emphasize: the use of versatile evaluation objects; the evaluation throughout the process and not just at the end; and the involvement of multiple evaluators, including students (whose feedback is essential to monitor the quality of teaching and learning). From the case studies we highlight: the need for discussion of evaluation frameworks to explore, and consequent increase in the transparency of the evaluation process; the increased interaction between groups; and the peer assessment as a strategy to promote active and autonomous learning. In addition to the contributions and recommendations for practice and research in the area of evaluation in bLearning contexts in HE, listed above, it also emerged from this study useful guidelines regarding educational evaluation in bLearning contexts, in order to improve the quality of teaching, learning and evaluation in such contexts.