One or the other? Do we have to choose? : summative and-or formative evaluation. '¿Una u otra? ¿Tenemos que elegir? : evaluación formativa y-o sumativa'.

Se definen los conceptos de evaluación sumativa y evaluación formativa con el ánimo de ver si son tan diferentes y excluyentes. Se proponen caminos para combinar ambas y para incluir a los estudiantes en el proceso de evaluación.

Design and formative evaluation of an information kiosk on cattle health for landless cattle owners

This paper describes and analyses the experience of designing, installing and evaluating a farmer-usable touch screen information kiosk on cattle health in a veterinary institution in Pondicherry. The contents of the kiosk were prepared based on identified demands for information on cattle health, arrived at through various stakeholders meetings. Information on these cattle diseases and conditions affecting the livelihoods of the poor was provided through graphics, text and audio back-up, keeping in mind the needs of landless and illiterate poor cattle owners. A methodology for kiosk evaluation based on the feedback obtained from kiosk facilitator, critical group reflection and individual users was formulated. The formative evaluation reveals the potential strength this ICT has in transferring information to the cattle owners in a service delivery centre. Such information is vital in preventing diseases and helps cattle owners to present and treat their animals at an early stage of disease condition. This in turn helps prevent direct and indirect losses to the cattle owners. The study reveals how an information kiosk installed at a government institution as a freely accessible source of information to all farmers irrespective of their class and caste can help in transfer of information among poor cattle owners, provided periodic updating, interactivity and communication variability are taken care of. Being in the veterinary centre, the kiosk helps stimulate dialogue, and facilitates demand of services based on the information provided by the kiosk screens.

An institutional study of the influence of ‘onlineness’ on student evaluation of teaching in a dual mode Australian university

Student evaluation of teaching (SET) is now commonplace in many universities internationally. The most common criticism of SET practices is that they are influenced by a number of non-teaching-related factors. More recently, there has been dramatic growth in online education internationally, but only limited research on the use of SET to evaluate online teaching. This paper presents a large-scale and detailed investigation, using the institutional SET data from an Australian university with a significant offering of wholly online units, and whose institutional SET instrument contains items relating to student perceptions of online technologies in teaching and learning. The relationship between educational technology and SET is not neutral. The mean ratings for the ‗online‘ aspects of SET are influenced by factors in the wider teaching and learning environment, and the overall perception of teaching quality is influenced by whether a unit is offered in wholly online mode or not.

Workplace learning coordinators program : formative evaluation : final report

The workplace learning coordinators program was established to increase the number of young people undertaking workplace learning placements, especially in industries that provide strong vocational outcomes. Additional attention has been paid to the potential benefits of the program for Koorie young people and other students who experience disadvantage. The program also seeks to improve the alignment between VCAL and VET provision and local industry needs, with a focus on indentifying new workplace learning opportunities with each service area’s employers. This formative evaluation sought to monitor the initial implementation of the WLC program during its first twelve months. Over this period the evaluators recorded the early teething challenges faced by WLCs during their first few months, and the eventual consolidation of their practices during the second half of the program’s first year. In reviewing all of the data collected for this evaluation, and the progressive responses of WLCs and stakeholders over the program’s first year, the evaluators consider that the model of service delivery is appropriate for what the program aims to achieve and recommend its continuance to 2013.

A critical evaluation and comparison of two formative measures of system quality using criterion variables

Evaluating the validity of formative variables has presented ongoing challenges for researchers. In this paper we use global criterion measures to compare and critically evaluate two alternative formative measures of System Quality. One model is based on the ISO-9126 software quality standard, and the other is based on a leading information systems research model. We find that despite both models having a strong provenance, many of the items appear to be non-significant in our study. We examine the implications of this by evaluating the quality of the criterion variables we used, and the performance of PLS when evaluating formative models with a large number of items. We find that our respondents had difficulty distinguishing between global criterion variables measuring different aspects of overall System Quality. Also, because formative indicators “compete with one another” in PLS, it may be difficult to develop a set of measures which are all significant for a complex formative construct with a broad scope and a large number of items. Overall, we suggest that there is cautious evidence that both sets of measures are valid and largely equivalent, although questions still remain about the measures, the use of criterion variables, and the use of PLS for this type of model evaluation.

Leading the evaluation of institutional online learning environments for quality enhancement in times of change

This paper reports on findings from a nationally funded project which aims to design and implement a quality management framework for online learning environments (OLEs). Evaluation is a key component of any quality management system and it is this aspect of the framework that is the focus of this paper. In developing the framework initial focus groups were conducted at the five participating institutions. These revealed that, although regarded as important, there did not appear to be a shared understanding of the nature and purpose of evaluation. A second series of focus groups revealed there were multiple perspectives arising from those with a vested interest in online learning. These perspectives will be outlined. Overall, how evaluation was undertaken was highly variable within and across the five institutions reflecting where they were at in relation to the development of their OLE.

Steps in the design, development and formative evaluation of obesity prevention-related behavior change trials

Obesity prevention interventions through dietary and physical activity change have generally not been effective. Limitations on possible program effectiveness are herein identified at every step in the mediating variable model, a generic conceptual framework for understanding how interventions may promote behavior change. To minimize these problems, and thereby enhance likely intervention effectiveness, four sequential types of formative studies are proposed: targeted behavior validation, targeted mediator validation, intervention procedure validation, and pilot feasibility intervention. Implementing these studies would establish the relationships at each step in the mediating variable model, thereby maximizing the likelihood that an intervention would work and its effects would be detected. Building consensus among researchers, funding agencies, and journal editors on distinct intervention development studies should avoid identified limitations and move the field forward.

Formative evaluation of baby makes 3

Baby Makes 3 was originally developed by Whitehorse Community Health Service and the City of Whitehorse as a program which seeks to prevent violence by promoting respect and equality between couples who have recently become parents for the first time. This is a significant but often stressful event in the life of families and it is not uncommon for couples to adopt ways of relating which reflect gendered norms and foster inequality.
By being offered as a program to all first-time parents by maternal and child health staff, the program in Whitehorse managed to engage with couples at this critical time, many of whom found the program enabled them to adopt greater equality in their relationships. As such, the initial Baby Makes 3 program has been found to be an effective and cost-efficient violence prevention strategy (Flynn, 2011).
The introduction of Baby Makes 3 to the Great South Coast Region is the first time this program has been implemented in a non-metropolitan setting. In addition to the original program aims, a number of additional aims and activities have been added, with the new project being known as Baby Makes 3 Plus.
Programs developed in urban contexts often face different challenges when implemented in rural and regional settings. Likewise, the factors for facilitating success may vary from those in an urban setting (Maidment and Bay, 2012). Hence, this formative evaluation aims to provide feedback on the initial implementation of the original Baby Makes 3 program in the Great South Coast Region with the expectation this will enable problems with implementation and/or opportunities for further enhancement of outcomes to be identified and appropriate action taken.

Formative Evaluation of the "Time Twisters": A Comic Book Based Obesity Prevention Curriculum for Elementary School Students

Introduction: Obesity is a public health problem, particularly in Hispanic children. Alternative media channels may offer the potential to motivate children to engage in health promoting behaviors. A comic book, “Time Twisters”, was developed to impact screen time use, physical activity, and dietary behavior for elementary school children and evaluated for acceptability and feasibility prior to implementation in a multi-component physical activity intervention. [See PDF for complete abstract]

Development and formative evaluation of an obesity prevention intervention for the pediatric primary care setting: Helping HAND (Healthy Activity and Nutrition Directions)

As the obesity epidemic continues to increase, the pediatric primary care office setting remains a relatively unexplored arena to offer obesity prevention interventions for children. The increased risk for adult obesity among 10 to 14 year-old children who are overweight, suggests obesity prevention programs should be introduced just before this age or early in this age period. Research is also accumulating on the importance of targeting parents along with children, since parents are in charge of the home environment for children. Therefore, the aim of this project was to develop an obesity prevention program called Helping HAND (Healthy Activity and Nutrition Directions) based on Social Cognitive Theory and authoritative parenting techniques for the pediatric primary care setting and conduct one-on-one interviews with parents as the initial formative evaluation of the intervention material for the obesity prevention intervention. A secondary aim of the project was to determine the feasibility of identifying appropriate subjects for the intervention, and conducting qualitative evaluations of the materials through recruitment through pediatric primary care settings. ^

A formative evaluation of the American Diabetes Association's Choosing Health and Sensible Exercise childhood obesity intervention

Background. The increasing prevalence of overweight among youth in the United States, and the parallel rise in related medical comorbidities has led to a growing need for efficient weight-management interventions. Purpose. The aim of this study was to evaluate the effects of the Choosing Health and Sensible Exercise (C.H.A.S.E.) childhood obesity prevention program on Body Mass Index (BMI), physical activity and dietary behaviors. Methods. This study utilized de-identified data collected during the fall 2006 session of the C.H.A.S.E. program. A total of 65 students at Woodview Elementary School and Deepwater Elementary School participated in this intervention. The C.H.A.S.E. program is a 10-week obesity prevention program that focuses on nutrition and physical activity education. Collection of height and weight data, and a health behavior survey was conducted during the first and last week of the intervention. Paired t-tests were used to determine statistically significant differences between pre- and post-intervention measurements. One-way analysis of variance was used to adjust for potential confounders, such as gender, age, BMI category ("normal weight", "at risk overweight", or "overweight"), and self-reported weight loss goals. Data were analyzed using STATA, v. 9.2. Results. A significant decrease in mean BMI (p< 0.05) was found after the 10-week intervention. While the results were statistically significant for the group as a whole, changes in BMI were not significant when stratified by age, sex, or ethnicity. The mean overall scores for the behavior survey did not change significantly pre- and post-intervention; however, significant differences were found in the dietary intention scale, indicating that students were more likely to intend to make healthier food choices (p<0.05). No statistically significant decreases in BMI were found when stratified for baseline BMI-for-age percentiles or baseline weight loss efforts (p>0.05). Conclusion. The results of this evaluation provide information that will be useful in planning and implementing an effective childhood obesity intervention in the future. Changes in the self-reported dietary intentions and BMI show that the C.H.A.S.E. program is capable of modifying food choice selection and decreasing BMI. Results from the behavior questionnaire indicate that students in the intervention program were making changes in a positive direction. Future implementation of the C.H.A.S.E. program, as well as other childhood obesity interventions, may want to consider incorporating additional strategies to increase knowledge and other behavioral constructs associated with decreased BMI. In addition, obesity prevention programs may want to increase parental involvement and increase the dose or intensity of the intervention. ^