Assessing fun in serious games: Formative evaluation of games from two diabetes prevention interventions

Purpose. To determine the usability of two video games to prevent type 2 diabetes and obesity among youth through analysis of data collected during alpha-testing. ^ Subjects. Ten children aged 9 to 12 were selected for three 2-hour alpha testing sessions.^ Methods. "Escape from Diab" and "Nanoswarm" were designed to change dietary and physical inactivity behaviors, based on a theoretical framework of mediating variables obtained from social cognitive theory, self-determination theory, elaboration likelihood model, and behavioral inoculation theory. Thirteen mini-games developed by the software company were divided into 3 groups based on completion date. Children tested 4-5 mini-games in each of three sessions. Observed game play was followed by a scripted interview. Results from observation forms and interview transcripts were tabulated and coded to determine usability. Suggestions for game modifications were delivered to the software design firm, and a follow-up table reports rationale for inclusion or exclusion of such modifications.^ Results. Participants were 50% frequent video game players and 20% non game-players. Most (60%) were female. The mean grade (indicating likeability as a subset of usability) across all games given by children was significantly greater than a neutral grade of 80% (89%, p < 0.01), indicating a positive likeability score. The games on average also received positive ratings for fun, helpfulness of instructions and length compared to neutral values (midpoint on likert scales) (all p < 0.01). Observation notes indicated that participants paid attention to the instructions, did not appear to have much difficulty with the games, and were "not frustrated", "not bored", "very engaged", "not fidgety" and "very calm" (all p < 0.01). The primary issues noted in observations and interviews were unclear instructions and unclear purpose of some games. Player suggestions primarily involved ways to make on screen cues more visible or noticeable, instructions more clear, and games more elaborate or difficult.^ Conclusions. The present study highlights the importance of alpha testing video game components for usability prior to completion to enhance usability and likeability. Results indicate that creating clear instructions, making peripheral screen cues more eye-catching or noticeable, and vigorously stating the purpose of the game to improve understandability are important elements. However, future interventions will each present unique materials and user-interfaces and should therefore also be thoroughly alpha-tested. ^

Youth empowerment evaluation to improve afterschool programming

Community-based participatory research necessitates that community members act as partners in decision making and mutual learning and discovery. In the same light, for programs/issues involving youth, youth should be partners in knowledge sharing and evaluation (Checkoway & Richards-Schuster, 2004). This study is a youth-focused empowerment evaluation for the Successful Youth program. Successful Youth is a multi-component youth development after-school program for Latino middle school youth, created with the goal of reducing teen pregnancy. An empowerment evaluation is collaborative and participatory (Balcazar and Harper 2003). The three steps of an empowerment evaluation are: (1) defining mission, (2) taking stock, and (3) planning for the future (Fetterman 2001).^ In a program where youth are developing leadership skills, making choices, and learning how to self reflect and evaluate, the empowerment evaluation could not be more aligned with promoting and enhancing these skills. In addition, an empowerment evaluation is designed to "foster improvement and self-determination" and "build capacity" (Fetterman 2001). Four empowerment groups were conducted with approximately 6-9 Latino 7th grade students per group. All participants were enrolled in the Successful Youth program. Results indicate points where students' perceptions of the program were aligned with the program's mission and where gaps were identified. Students offered recommendations for program improvements. Additionally, students enjoyed expressing their feelings about the program and appreciated that their opinions were valued. Youth recommendations will be brought to program staff; and, where possible, gaps will be addressed. Empowerment evaluations with youth will continue during the duration of the program so that youth involvement and input remains integral in the evaluation and to ascertain whether the program's goals are being met. ^