Citizen science and natural resource governance: program design for vernal pool policy innovation

Effective natural resource policy depends on knowing what is needed to sustain a resource and building the capacity to identify, develop, and implement flexible policies. This retrospective case study applies resilience concepts to a 16-year citizen science program and vernal pool regulatory development process in Maine, USA. We describe how citizen science improved adaptive capacities for innovative and effective policies to regulate vernal pools. We identified two core program elements that allowed people to act within narrow windows of opportunity for policy transformation, including (1) the simultaneous generation of useful, credible scientific knowledge and construction of networks among diverse institutions, and (2) the formation of diverse leadership that promoted individual and collective abilities to identify problems and propose policy solutions. If citizen science program leaders want to promote social-ecological systems resilience and natural resource policies as outcomes, we recommend they create a system for internal project evaluation, publish scientific studies using citizen science data, pursue resources for program sustainability, and plan for leadership diversity and informal networks to foster adaptive governance.

Evaluation of financial incentives as a quality improvement strategy in the public hospital context : clinician attitudes, design variables, and economic costs

In 2008, a three-year pilot ‘pay for performance’ (P4P) program, known as ‘Clinical Practice Improvement Payment’ (CPIP) was introduced into Queensland Health (QHealth). QHealth is a large public health sector provider of acute, community, and public health services in Queensland, Australia. The organisation has recently embarked on a significant reform agenda including a review of existing funding arrangements (Duckett et al., 2008). Partly in response to this reform agenda, a casemix funding model has been implemented to reconnect health care funding with outcomes. CPIP was conceptualised as a performance-based scheme that rewarded quality with financial incentives. This is the first time such a scheme has been implemented into the public health sector in Australia with a focus on rewarding quality, and it is unique in that it has a large state-wide focus and includes 15 Districts. CPIP initially targeted five acute and community clinical areas including Mental Health, Discharge Medication, Emergency Department, Chronic Obstructive Pulmonary Disease, and Stroke. The CPIP scheme was designed around key concepts including the identification of clinical indicators that met the set criteria of: high disease burden, a well defined single diagnostic group or intervention, significant variations in clinical outcomes and/or practices, a good evidence, and clinician control and support (Ward, Daniels, Walker & Duckett, 2007). This evaluative research targeted Phase One of implementation of the CPIP scheme from January 2008 to March 2009. A formative evaluation utilising a mixed methodology and complementarity analysis was undertaken. The research involved three research questions and aimed to determine the knowledge, understanding, and attitudes of clinicians; identify improvements to the design, administration, and monitoring of CPIP; and determine the financial and economic costs of the scheme. Three key studies were undertaken to ascertain responses to the key research questions. Firstly, a survey of clinicians was undertaken to examine levels of knowledge and understanding and their attitudes to the scheme. Secondly, the study sought to apply Statistical Process Control (SPC) to the process indicators to assess if this enhanced the scheme and a third study examined a simple economic cost analysis. The CPIP Survey of clinicians elicited 192 clinician respondents. Over 70% of these respondents were supportive of the continuation of the CPIP scheme. This finding was also supported by the results of a quantitative altitude survey that identified positive attitudes in 6 of the 7 domains-including impact, awareness and understanding and clinical relevance, all being scored positive across the combined respondent group. SPC as a trending tool may play an important role in the early identification of indicator weakness for the CPIP scheme. This evaluative research study supports a previously identified need in the literature for a phased introduction of Pay for Performance (P4P) type programs. It further highlights the value of undertaking a formal risk assessment of clinician, management, and systemic levels of literacy and competency with measurement and monitoring of quality prior to a phased implementation. This phasing can then be guided by a P4P Design Variable Matrix which provides a selection of program design options such as indicator target and payment mechanisms. It became evident that a clear process is required to standardise how clinical indicators evolve over time and direct movement towards more rigorous ‘pay for performance’ targets and the development of an optimal funding model. Use of this matrix will enable the scheme to mature and build the literacy and competency of clinicians and the organisation as implementation progresses. Furthermore, the research identified that CPIP created a spotlight on clinical indicators and incentive payments of over five million from a potential ten million was secured across the five clinical areas in the first 15 months of the scheme. This indicates that quality was rewarded in the new QHealth funding model, and despite issues being identified with the payment mechanism, funding was distributed. The economic model used identified a relative low cost of reporting (under $8,000) as opposed to funds secured of over $300,000 for mental health as an example. Movement to a full cost effectiveness study of CPIP is supported. Overall the introduction of the CPIP scheme into QHealth has been a positive and effective strategy for engaging clinicians in quality and has been the catalyst for the identification and monitoring of valuable clinical process indicators. This research has highlighted that clinicians are supportive of the scheme in general; however, there are some significant risks that include the functioning of the CPIP payment mechanism. Given clinician support for the use of a pay–for-performance methodology in QHealth, the CPIP scheme has the potential to be a powerful addition to a multi-faceted suite of quality improvement initiatives within QHealth.

Development of an innovative technology-based youth passenger safety program : an evidence-based approach

Young drivers are overrepresented in motor vehicle crash rates, and their risk increases when carrying similar aged passengers. Graduated Driver Licensing strategies have demonstrated effectiveness in reducing fatalities among young drivers, however complementary approaches may further reduce crash rates. Previous studies conducted by the researchers have shown that there is considerable potential for a passenger focus in youth road safety interventions, particularly involving the encouragement of young passengers to intervene in their peers’ risky driving (Buckley, Chapman, Sheehan & Davidson, 2012). Additionally, this research has shown that technology-based applications may be a promising means of delivering passenger safety messages, particularly as young people are increasingly accessing web-based and mobile technologies. This research describes the participatory design process undertaken to develop a web-based road safety program, and involves feasibility testing of storyboards for a youth passenger safety application. Storyboards and framework web-based materials were initially developed for a passenger safety program, using the results of previous studies involving online and school-based surveys with young people. Focus groups were then conducted with 8 school staff and 30 senior school students at one public high school in the Australian Capital Territory. Young people were asked about the situations in which passengers may feel unsafe and potential strategies for intervening in their peers’ risky driving. Students were also shown the storyboards and framework web-based material and were asked to comment on design and content issues. Teachers were also shown the material and asked about their perceptions of program design and feasibility. The focus group data will be used as part of the participatory design process, in further developing the passenger safety program. This research describes an evidence-based approach to the development of a web-based application for youth passenger safety. The findings of this research and resulting technology will have important implications for the road safety education of senior high school students.

Development of a school connectedness component of an adolescent injury prevention program : lessons for implementation

The school environment plays an important role in shaping adolescent outcomes, and research increasingly demonstrates the need to target the school social context in health promotion programs. This paper describes the research process undertaken to design a school connectedness component of an injury prevention program for early adolescents, Skills for Preventing Injury in Youth (SPIY). The connectedness component takes the form of a professional development workshop for teachers on increasing students’ connectedness to school, and this paper describes the research process used to construct program material. It also describes the methods used to encourage teachers’ implementation of connectedness strategies following program delivery. A multi-stage process of data collection included, (i) surveys with 540 Grade 9 students to examine links between school connectedness and risk-related injury, (ii) a systematic literature review of previously-evaluated school connectedness programs to determine key strategies that encourage implementation fidelity and program effectiveness, and (iii) interviews with 14 high school teachers to understand current use of connectedness strategies and ideas for program design. Findings from each stage are discussed in terms of how results informed the program design. The survey data provided information from which to frame program content, and the results of the systematic review demonstrated effective program strategies. The teacher interview data also provided program content incorporating target participants’ views and aligning with their priorities, which is important to ensure effective implementation of program strategies. A comprehensive design process provides an understanding of methods for, and may encourage, teachers’ future implementation of program strategies.

Game On: Know Alcohol - Co-creating a tailored alcohol social marketing program

Research demonstrates that education programs are more effective when their target audiences and other key stakeholder groups are involved in the design. The majority of programs continue to be researcher and expert driven. This study extends upon previous research by employing a co-creation research design to offer a consumer driven alternative to education program design. Two co-creation groups involving twenty 14-16 year old Year 10 students who had previously participated in the Game On:Know Alcohol (GOKA) program, which aims to moderate alcohol drinking attitudes and behaviour, were conducted. Analysis revealed that a co-created GOKA program will differ substantially from the researcher and expert driven program that is currently being field tested. Students prefer interactive activities and activities that engage and challenge. Co-creation offers the potential to contest researcher and expert views and may offer to assist in the generation of new insights for the development of education programs.

Supporting Reuse and Evolution in Software Design

Program design is an area of programming that can benefit significantly from machine-mediated assistance. A proposed tool, called the Design Apprentice (DA), can assist a programmer in the detailed design of programs. The DA supports software reuse through a library of commonly-used algorithmic fragments, or cliches, that codifies standard programming. The cliche library enables the programmer to describe the design of a program concisely. The DA can detect some kinds of inconsistencies and incompleteness in program descriptions. It automates detailed design by automatically selecting appropriate algorithms and data structures. It supports the evolution of program designs by keeping explicit dependencies between the design decisions made. These capabilities of the DA are underlaid bya model of programming, called programming by successive elaboration, which mimics the way programmers interact. Programming by successive elaboration is characterized by the use of breadth-first exposition of layered program descriptions and the successive modifications of descriptions. A scenario is presented to illustrate the concept of the DA. Technques for automating the detailed design process are described. A framework is given in which designs are incrementally augmented and modified by a succession of design steps. A library of cliches and a suite of design steps needed to support the scenario are presented.

Dependency Directed Reasoning for Complex Program Understanding

Artificial Intelligence research involves the creation of extremely complex programs which must possess the capability to introspect, learn, and improve their expertise. Any truly intelligent program must be able to create procedures and to modify them as it gathers information from its experience. [Sussman, 1975] produced such a system for a 'mini-world'; but truly intelligent programs must be considerably more complex. A crucial stepping stone in AI research is the development of a system which can understand complex programs well enough to modify them. There is also a complexity barrier in the world of commercial software which is making the cost of software production and maintenance prohibitive. Here too a system which is capable of understanding complex programs is a necessary step. The Programmer's Apprentice Project [Rich and Shrobe, 76] is attempting to develop an interactive programming tool which will help expert programmers deal with the complexity involved in engineering a large software system. This report describes REASON, the deductive component of the programmer's apprentice. REASON is intended to help expert programmers in the process of evolutionary program design. REASON utilizes the engineering techniques of modelling, decomposition, and analysis by inspection to determine how modules interact to achieve the desired overall behavior of a program. REASON coordinates its various sources of knowledge by using a dependency-directed structure which records the justification for each deduction it makes. Once a program has been analyzed these justifications can be summarized into a teleological structure called a plan which helps the system understand the impact of a proposed program modification.

Did children’s perceptions of an after-school social learning program predict change in their behavior?

This research explored the influence of children’s perceptions of a pro-social behavior after-school program on actual change in the children’s behavioral outcomes over the program’s duration. Children’s perceptions of three program processes were collected as well as self-reported pro-social and anti-social behavior before and after the program. Statistical models showed that: Positive perceptions of the program facilitators’ dispositions significantly predicted reductions in anti-social behavior; and positive perceptions with the program activities significantly predicted gains in pro-social behavior. The children’s perceptions of their peers’ behavior in the sessions were not found to a significant predictor of behavioral change. The two significant perceptual indicators predicted a small percentage of the change in the behavioral outcomes. However, as after-school social learning programs have a research history of problematic implementation children’s perceptions should be considered in future program design, evaluation and monitoring.

How does contract design affect the uptake of microcredit among the ultra-poor? : experimental evidence from the river islands of Northern Bangladesh

Despite the professed claims of microcredit alleviating poverty, little is known about what kind of credit contract is suitable for extremely poor households, also called the ultra-poor. To fill this knowledge gap, we initiated a field experiment in the river islands of northern Bangladesh, where a substantial portion of dwellers could be categorized as ultra-poor due to cyclic floods. We randomly offered four types of loans to such dwellers: regular small cash loans with one-year maturity, large cash loans with three-year maturity both with and without a one-year grace period, and in-kind livestock loans with three-year maturity and a one-year grace period. We compared uptake rates as well as the determinants of uptake and found that the uptake rate is the lowest for the regular contract, followed by the in-kind contract. Contrary to prior belief, we also found that the microcredit demand by the ultra-poor is not necessarily small, and in particular the ultra-poor are significantly more likely to join a microcredit program than the moderately poor if a grace period with longer maturity is attached to a large amount of credit, irrespective of whether the credit is provided in cash or in kind. This paper provides evidence that a typical microcredit contract with one-year maturity and without a grace period is not attractive to the ultra-poor. Microfinance institutions may need to design better credit contracts to address the poor's needs.

A longitudinal study of Project Trio: A dropout prevention program in a large urban environment

The subject of dropout prevention/reduction is deservedly receiving attention as a problem that, if not resolved, could threaten our national future.^ This study investigates a small segment of the overall dropout problem, which has apparently unique features of program design and population selection. The evidence presented here should add to the knowledge bank of this complicated problem.^ Project Trio was one of a number of dropout prevention programs and activities which were conducted in Dade County school years 1984-85 and 1985-86, and it is here investigated longitudinally through the end of the 1987-88 school year. It involved 17 junior and senior high schools, and 27 programs, 10 the first year and 17 the second, with over 1,000 total students, who had been selected by the schools from a list of the "at risk" students provided by the district, and were divided approximately evenly into the classical research design of an experimental group and the control group, which following standard procedure was to take the regular school curriculum. No school had more than 25 students in either group.^ Each school modified the basic design of the project to accommodate the individual school characteristics and the perceived needs of their students; however all schools projects were to include some form of academic enhancement, counseling and career awareness study.^ The conclusion of this study was that the control group had a significantly lower dropout rate than the experimental group. Though impossible to make a certain determination of the reasons for this unexpected result, it appears from evidence presented that one cause may have been inadequate administration at the local level.^ This study was also a longitudinal investigation of the "at risk" population as a whole for the three and four year period, to determine if academic factors were present in records may be used to identify dropout proneness.^ A significant correlation was found between dropping out and various measures including scores on the Quality of School Life Instrument, attendance, grade point averages, mathematics grades, and overage in grade, important identifiers in selection for dropout prevention programs. ^

A case study on a design-informed developmental evaluation

Developmental evaluation (DE) is an evaluation approach that aims to support the development of an innovation (Patton, 1994, 2011). This aim is achieved through supporting clients’ information needs through evaluative inquiry as they work to develop and refine the innovation. While core concepts and principles are beginning to be articulated and refined, challenges remain as to how to focus a developmental evaluation beyond those knowledge frameworks most immediate to clients to support innovation development. Anchoring a DE in knowledge frameworks other than those of the clients might direct attention to issues not yet obvious to clients, but which might further the goal of supporting innovation development if attended to. Drawing concepts and practices from the field of design may be one avenue with which to inform developmental evaluation in achieving its aim. Through a case study methodology, this research seeks to understand the nuances of operationalizing the guiding principles of DE as well as to investigate the utility, feasibility, and consequences of integrating design concepts and practices into developmental evaluation (design-informed developmental evaluation, “DI-DE”). It does so by documenting the efforts of a design-informed developmental evaluator and a task force of educators and researchers in a Faculty of Education as they work to develop a graduate-level education program. A systematic review into those purposeful efforts made to introduce DI-DE thinking into task force deliberations, and an analysis into the responses and consequences of those efforts shed light on what it had meant to practice DI-DE. As a whole, this research on evaluation is intended to further contemporary thinking about the closely coupled relationship between program development and evaluation in complex and dynamic environments.

Eight month post program completion: change in risk factors for chronic disease amongst participants in a four-month pedometer workplace health program

Objective: To evaluate whether participation in a 4-month, pedometer-based, physical activity, workplace health program is associated with long-term sustained improvements in risk factors for type 2 diabetesand cardiovascular disease, 8 months after the completion of the program.Design and Methods: A sample size of 720 was required. 762 Australian adults employed in primarily sedentary occupations and voluntarily enrolled in a workplace program were recruited. Demographic, behavioral, anthropometric and biomedical measurements were completed at baseline, 4 and 12 months.Results: About 76% of participants returned at 12 months. Sustained improvements at 12 months were observed for self-reported vegetable intake, self-reported sitting time and independently measured bloodpressure. Modest improvements from baseline in self-reported physical activity and independently measured waist circumference at 12 months indicated that the significant improvements observed immediately after the health program could not be sustained. Approximately half of those not meetingguidelines for physical activity, waist circumference and blood pressure at baseline, were meeting guidelines at 12 months.Conclusions: Participation in this 4-month, pedometer-based, physical activity, workplace health program was associated with sustained improvements in chronic disease risk factors at 12 months. These results indicate that such programs can have a long-term benefit and thus a potential role to play in population prevention of chronic disease.

A multi-family group intervention for adolescent depression: the BEST MOOD program

Depression is the most common mental disorder for young people, and it is associated with educational underachievement, self-harm, and suicidality. Current psychological therapies for adolescent depression are usually focused only on individual-level change and often neglect family or contextual influences. The efficacy of interventions may be enhanced with a broader therapeutic focus on family factors such as communication, conflict, support, and cohesion. This article describes a structured multi-family group approach to the treatment of adolescent depression: Behaviour Exchange Systems Therapy for adolescent depression (BEST MOOD). BEST MOOD is a manualized intervention that is designed to address both individual and family factors in the treatment of adolescent depression. BEST MOOD adopts a family systems approach that also incorporates psychoeducation and elements of attachment theories. The program consists of eight multifamily group therapy sessions delivered over 2 hours per week, where parents attend the first four sessions and young people and siblings join from week 5. The program design is specifically aimed to engage youth who are initially resistant to treatment and to optimize youth and family mental health outcomes. This article presents an overview of the theoretical model, session content, and evaluations to date, and provides a case study to illustrate the approach.