Evaluating leadership development using the Most Significant Change technique

Course evaluations are now a serious matter for universities trying to meet stakeholder needs and expectations, quality assurance, improvements and strategic decision making. Typically, students are invited to participate in surveys on how well the design and delivery aspects meet predetermined learning objectives, quality of teaching, and the types of improvements needed for future deliveries. We used the Most Significant Change technique to gather data on the impact of a leadership course on 18 Pacific Islanders who completed a Master of Education (Educational Leadership). Participants' views highlighted impacts that were of significance to the students and their workplaces. The findings demonstrate that the Most Significant Change technique offers a more comprehensive understanding of the impact of leadership development courses.

Most Significant Change in conflict settings: staff development through monitoring and evaluation

Capoeira4Refugees is an NGO that uses the Afro-Brazilian art-form of Capoeira to promote psychosocial well-being in children affected by conflict and occupation. Capoeira4Refugees introduced the Most Significant Change (MSC) methodology to monitor and evaluate project implementation and impact across two locations in the Middle East. Analysis of interviews conducted with five field staff revealed that in line with, and building on previous research, MSC became an empowering tool that led to staff development. The potential for MSC to build staff reflexivity, independence and leadership has implications for other organisations working in conflict areas, particularly in situations of remote management.

Assessment of statistical change criteria used to define significant change in neuropsychological test performance following cardiac surgery

Objective: This paper compares four techniques used to assess change in neuropsychological test scores before and after coronary artery bypass graft surgery (CABG), and includes a rationale for the classification of a patient as overall impaired. Methods: A total of 55 patients were tested before and after surgery on the MicroCog neuropsychological test battery. A matched control group underwent the same testing regime to generate test–retest reliabilities and practice effects. Two techniques designed to assess statistical change were used: the Reliable Change Index (RCI), modified for practice, and the Standardised Regression-based (SRB) technique. These were compared against two fixed cutoff techniques (standard deviation and 20% change methods). Results: The incidence of decline across test scores varied markedly depending on which technique was used to describe change. The SRB method identified more patients as declined on most measures. In comparison, the two fixed cutoff techniques displayed relatively reduced sensitivity in the detection of change. Conclusions: Overall change in an individual can be described provided the investigators choose a rational cutoff based on likely spread of scores due to chance. A cutoff value of ≥20% of test scores used provided acceptable probability based on the number of tests commonly encountered. Investigators must also choose a test battery that minimises shared variance among test scores.

Report of the workshop on process monitoring and significant change. Bangkok, Thailand 26-28 June 2002

The STREAM Initiative is a process rather than a project, and its focus is on learning and building on learning, not the achievement of pre-determined objectives. An overarching goal of STREAM is to facilitate changes that support poor people who manage aquatic resources. A key objective of STREAM is policy change, which in itself is complex and difficult to monitor. Two further layers of complexity relate to the regional scope of the Initiative and the collaborative involvement of stakeholders, all of which need to be accountable for their work. The objectives of this workshop are consistent with the aims of the STREAM Initiative and can be summerized as follows: 1- Familiarizing everyone in the regional STREAM Initiative with work being done in process monitoring and significant change. 2- Discussion and development of a practical information system that enables (i) the monitoring of development processes and significant changes occurring within the STREAM Initiative, and (ii) learning to inform STREAM implementation and other stakeholders. (PDF has 59 pages.)

Method of detecting a significant change of scene

A significant change of scene in a gradually changing scene is detected with the aid of a least one camera means for capturing digital images of the scene. A current image of the scene is formed together with a present weighted reference image which is formed from a plurality of previous images of the scene. Cell data is established based on the current image and the present weighted reference image. The cell data is statistically analysed so as to be able to identify at least one difference corresponding to a significant change of scene. When identified, an indication of such significant change of scene is provided.

The most significant deficiencies in macro and micro nutrients in adolescents living with HIV/AIDS in antiretroviral therapy

Objective: To evaluate macro and micronutrients intake of adolescents living with HIV/AIDS in use of antiretroviral therapy and compare it to the Dietary Reference Intakes. Methodology: Cross-sectional study conducted with adolescents of both genders with HIV/AIDS, assessing the dietary composition of macro and micronutrients, using the 24h dietary recall. Results: 39 adolescents, average age of 15 years, 51.3% males. The participants intake of total calories, total fiber (g/d), liposoluble vitamins (A, D, E, K), vitamin B5 (mg/d), vitamin B9 (mg/d), vitamin C (mg/d), calcium (mg/d), phosphorus (mg/d), potassium (mg/d), and magnesium (mg/d) was lower than recommended. The percentages of intake lower than recommended were 79.5% for calories, 82.1% for total fibers, 89.7% for vitamin A, 100% for vitamin D, 87.2% for vitamin E, 100% for vitamin K, 71.8% for vitamin B5, 82.1% for vitamin B9, 76.9% for vitamin C, 92.3% for calcium, 61.5% for phosphorus, 97.4% for potassium, and 76.9% for magnesium. The participants ingested more carbohydrates (g), proteins (g), vitamins B2 (mg/d), B3 (mg/d), B8 (mg/d) and sodium (g/d) than recommended, the percentages above the recommendations being 92.3% for carbohydrates, 64.1% for proteins and vitamin B2, 56.4% for vitamin B3, 82.1% for vitamin B8, and 59% for sodium. The remaining nutrients were within the amounts recommended by the DRIs. Conclusion: Food intake was inadequate as compared to the recommendations of the International Nutrition Guidelines.

Teaching teachers for the future : how, what, why, and what next?

This paper contextualises the Teaching Teachers of the Future (TTF) Project and acts as a preamble for the TTF stream of papers at ACEC2012. It discusses the aims and objectives of the project, its genesis in a changing educational and political landscape, the use of TPACK as a theoretical scaffold, and briefly report on the operations of the various components and partners. Further, it will discuss the research opportunities afforded by the project including a national survey of all pre-service teachers in Australia gauging their TPACK confidence and the use of the Most Significant Change (MSC) methodology. Finally the paper will discuss the outcomes of the project and its future.

Teaching teachers for the future : how, what, why, and what next?

This paper contextualises the Teaching Teachers of the Future (TTF) Project and acts as a preamble for the TTF stream of papers at ACEC2012. It discusses the aims and objectives of the project, its genesis in a changing educational and political landscape, the use of TPACK as a theoretical scaffold, and briefly report on the operations of the various components and partners. Further, it will discuss the research opportunities afforded by the project including a national survey of all pre-service teachers in Australia gauging their TPACK confidence and the use of the Most Significant Change (MSC) methodology. Finally the paper will discuss the outcomes of the project and its future.