A case study on a design-informed developmental evaluation

Developmental evaluation (DE) is an evaluation approach that aims to support the development of an innovation (Patton, 1994, 2011). This aim is achieved through supporting clients’ information needs through evaluative inquiry as they work to develop and refine the innovation. While core concepts and principles are beginning to be articulated and refined, challenges remain as to how to focus a developmental evaluation beyond those knowledge frameworks most immediate to clients to support innovation development. Anchoring a DE in knowledge frameworks other than those of the clients might direct attention to issues not yet obvious to clients, but which might further the goal of supporting innovation development if attended to. Drawing concepts and practices from the field of design may be one avenue with which to inform developmental evaluation in achieving its aim. Through a case study methodology, this research seeks to understand the nuances of operationalizing the guiding principles of DE as well as to investigate the utility, feasibility, and consequences of integrating design concepts and practices into developmental evaluation (design-informed developmental evaluation, “DI-DE”). It does so by documenting the efforts of a design-informed developmental evaluator and a task force of educators and researchers in a Faculty of Education as they work to develop a graduate-level education program. A systematic review into those purposeful efforts made to introduce DI-DE thinking into task force deliberations, and an analysis into the responses and consequences of those efforts shed light on what it had meant to practice DI-DE. As a whole, this research on evaluation is intended to further contemporary thinking about the closely coupled relationship between program development and evaluation in complex and dynamic environments.

An integrative case study of positive youth development in a recreational community sport program

Participation in organized activities is associated with many markers of positive youth development (PYD) such as improved self-esteem, social competence, and academic achievement (Mahoney et al., 2009). Sport is an extra-curricular activity that is particularly popular among youth, as nearly three quarters of Canadian children and adolescents are engaged in organized sport or physical activity (CFLRI, 2012). Much of the research in this area has examined sport programs which are explicitly structured to promote life skills or PYD outcomes; however, the sustainability of such programs has recently been called into question (Turnnidge, Hancock, & Côté, 2014). The purpose of this program of research was to conduct an in-depth case study of a successful, sustainable community youth basketball league. Study 1 was a qualitative descriptive exploration of coaches’ perceptions of the basketball league. This study provided a general overview of the structure and perceived benefits of the league, from the perspective of current coaches. Coaches highlighted the unique emphasis that the league places on fun and positive immediate sport experiences. Study 2 expanded on selected themes from Study 1, namely, the salience of the league’s culture and tendency for former players to return to the league as volunteers. This study used an ethnographic approach wherein the first author volunteered as an assistant coach on a basketball team over the course of a six-month season. This approach enabled him to gain considerable first hand insight into the organizational culture of the basketball league. The third and final study adopted a quantitative approach using both systematic observation and questionnaires to investigate the relationship between PYD outcomes and observed athlete behaviour during basketball games. A cluster analysis revealed the presence of two distinct groups of athletes characterized by relatively high and low perceptions of PYD outcomes, which were also associated with varying behavioural characteristics during competition. The results of these three studies provide a detailed blueprint of a successful youth sport program that has been sustained over 60 years. While not without limitations, many characteristics of the league should prove useful in structuring youth sport programs in other contexts.

From Unemployment to Sustainable Livelihoods in Ottawa's Technology Sector (Working Paper 44)

Since submission of the draft report to the OECD‐LEED Program on 30 June 2007, a unique seven‐years retrospective study of the unemployed high technology workers was released by Statistics Canada.1 Drawing upon Statistics Canada’s confidential Longitudinal Worker File – itself constructed from four administrative data sources that linked Records of Employment and tax filer information by the Social Insurance Number and firm‐level data by a company identifier – this study was able to identify and trace the re‐employment of those permanently laid off in the high‐tech industry by location. The findings are stunning.