Evaluation management : a sourcebook of readings /

Mode of access: Internet.

Evaluation forum /

"A review journal for employment and training professionals focusing on policy analysis and evaluation research in education, occupational training, work and welfare, and economic development."

Respecting and fulfilling the right of post-primary pupils to consent to participate in trials and evaluative research: A discussion paper

This paper provides an introduction to issues surrounding the participation rights of young people in research and the implications of their growing involvement in research as well as providing a discourse on the ethical implications related to consent. The unique contribution of this paper is that it considers children’s rights in respect to the increasing opportunities for young people to take part in evaluation research. The aim of this paper, therefore, is to acknowledge the growing involvement for young people in research and the implications of ensuring that their rights of participation are respected. Secondly we will consider the children’s rights legislation and our obligations as researchers to implement this. Finally we will explore consent as an issue in its own right as well as the practicalities of accessing participants. This paper will postulate that any research about young people should involve and prioritise at all stages of the research process; including participation in decision-making. We conclude by identifying five key principles, which we believe can help to facilitate the fulfilment of post-primary pupils’ ability to consent to participate in trials and evaluative research.

Juliane Lamprecht: Rekonstruktiv-responsive Evaluation in der Praxis, Neue Perspektiven dokumentarischer Evaluationsforschung. Wiesbaden: VS Verlag für Sozialwissenschaften 2012 [Rezension]

Rezension von: Juliane Lamprecht: Rekonstruktiv-responsive Evaluation in der Praxis, Neue Perspektiven dokumentarischer Evaluationsforschung, Wiesbaden: VS Verlag für Sozialwissenschaften 2012 (305 S.; ISBN 978-3-5311-7985-8)

Relations among Topic Knowledge, Individual Interest, and Relational Reasoning, and Critical Thinking in Maternity Nursing

Critical thinking in learners is a goal of educators and professional organizations in nursing as well as other professions. However, few studies in nursing have examined the role of the important individual difference factors topic knowledge, individual interest, and general relational reasoning strategies in predicting critical thinking. In addition, most previous studies have used domain-general, standardized measures, with inconsistent results. Moreover, few studies have investigated critical thinking across multiple levels of experience. The major purpose of this study was to examine the degree to which topic knowledge, individual interest, and relational reasoning predict critical thinking in maternity nurses. For this study, 182 maternity nurses were recruited from national nursing listservs explicitly chosen to capture multiple levels of experience from prelicensure to very experienced nurses. The three independent measures included a domain-specific Topic Knowledge Assessment (TKA), consisting of 24 short-answer questions, a Professed and Engaged Interest Measure (PEIM), with 20 questions indicating level of interest and engagement in maternity nursing topics and activities, and the Test of Relational Reasoning (TORR), a graphical selected response measure with 32 items organized in scales corresponding to four forms of relational reasoning: analogy, anomaly, antithesis, and antinomy. The dependent measure was the Critical Thinking Task in Maternity Nursing (CT2MN), composed of a clinical case study providing cues with follow-up questions relating to nursing care. These questions align with the cognitive processes identified in a commonly-used definition of critical thinking in nursing. Reliable coding schemes for the measures were developed for this study. Key findings included a significant correlation between topic knowledge and individual interest. Further, the three individual difference factors explained a significant proportion of the variance in critical thinking with a large effect size. While topic knowledge was the strongest predictor of critical thinking performance, individual interest had a moderate significant effect, and relational reasoning had a small but significant effect. The findings suggest that these individual difference factors should be included in future studies of critical thinking in nursing. Implications for nursing education, research, and practice are discussed.

The experience of China-educated nurses working in Australia : a symbolic interactionist perspective

Transnational nurse migration is a growing phenomenon. However, relatively little is known about the experiences of immigrant nurses and particularly about non-English speaking background nurses who work in more economically developed countries. Informed by a symbolic interactionist framework, this research explored the experience of China-educated nurses working in the Australian health care system. Using a modified constructivist grounded theory method, the main source of data were 46 face to face in-depth interviews with 28 China-educated nurses in two major cities in Australia. The key findings of this research are fourfold. First, the core category developed in this study is reconciling different realities, which inserts a theoretical understanding beyond the concepts of acculturation, assimilation, and integration. Second, in contrast to the dominant discourse which reduces the experience of immigrant nurses to language and culture, this research concludes that it was not just about language and nor was it simply about culture. Third, rather than focus on the negative aspects of difference as in the immigration literature and in the practice of nursing, this research points to the importance of recognising the social value of difference. Finally, the prevailing view that the experience of immigrant nurses is largely negative belies its complexities. This research concludes that it is naïve to define the experience as either good or bad. Rather, ambivalence was the essential feature of the experience and a more appropriate theoretical concept. This research produced a theoretical understanding of the experience of China-educated nurses working in Australia. The findings may not only inform Chinese nurses who wish to immigrate but also contribute to the implementation of more effective support services for immigrant nurses in Australian health care organisations.

Getting to know a stranger-rural nurses' experiences of mentoring : a grounded theory

Background: Mentoring is often proposed as a solution to the problem of successfully recruiting and retaining nursing staff. The aim of this constructivist grounded theory study was to explore Australian rural nurses' experiences of mentoring. Design: The research design used was reflexive in nature resulting in a substantive, constructivist grounded theory study. Participants: A national advertising campaign and snowball sampling were used to recruit nine participants from across Australia. Participants were rural nurses who had experience in mentoring others. Methods: Standard grounded theory methods of theoretical sampling, concurrent data collection and analysis using open, axial and theoretical coding and a story line technique to develop the core category and category saturation were used. To cultivate the reflexivity required of a constructivist study, we also incorporated reflective memoing, situational analysis mapping techniques and frame analysis. Data was generated through eleven interviews, email dialogue and shared situational mapping. Results: Cultivating and growing new or novice rural nurses using supportive relationships such as mentoring was found to be an existing, integral part of experienced rural nurses' practice, motivated by living and working in the same communities. Getting to know a stranger is the first part of the process of cultivating and growing another. New or novice rural nurses gain the attention of experienced rural nurses through showing potential or experiencing a critical incidence. Conclusions: The problem of retaining nurses is a global issue. Experienced nurses engaged in clinical practice have the potential to cultivate and grow new or novice nurses-many already do so. Recognising this role and providing opportunities for development will help grow a positive, supportive work environment that nurtures the experienced nurses of tomorrow.

Grounded theory : a methodological spiral from positivism to postmodernism

Aim. Our aim in this paper is to explain a methodological/methods package devised to incorporate situational and social world mapping with frame analysis, based on a grounded theory study of Australian rural nurses' experiences of mentoring. Background. Situational analysis, as conceived by Adele Clarke, shifts the research methodology of grounded theory from being located within a postpositivist paradigm to a postmodern paradigm. Clarke uses three types of maps during this process: situational, social world and positional, in combination with discourse analysis. Method. During our grounded theory study, the process of concurrent interview data generation and analysis incorporated situational and social world mapping techniques. An outcome of this was our increased awareness of how outside actors influenced participants in their constructions of mentoring. In our attempts to use Clarke's methodological package, however, it became apparent that our constructivist beliefs about human agency could not be reconciled with the postmodern project of discourse analysis. We then turned to the literature on symbolic interactionism and adopted frame analysis as a method to examine the literature on rural nursing and mentoring as secondary form of data. Findings. While we found situational and social world mapping very useful, we were less successful in using positional maps. In retrospect, we would argue that collective action framing provides an alternative to analysing such positions in the literature. This is particularly so for researchers who locate themselves within a constructivist paradigm, and who are therefore unwilling to reject the notion of human agency and the ability of individuals to shape their world in some way. Conclusion. Our example of using this package of situational and social worlds mapping with frame analysis is intended to assist other researchers to locate participants more transparently in the social worlds that they negotiate in their everyday practice. © 2007 Blackwell Publishing Ltd.

Understanding the role of knowledge in the practice of expert nephrology nurses in Australia

This paper, which is abstracted from a larger study into the acquisition and exercise of nephrology nursing expertise, aims to explore the role of knowledge in expert practice. Using grounded theory methodology, the study involved 17 registered nurses who were practicing in a metropolitan renal unit in New South Wales, Australia. Concurrent data collection and analysis was undertaken, incorporating participants' observations and interviews. Having extensive nephrology nursing knowledge was a striking characteristic of a nursing expert. Expert nurses clearly relied on and utilized extensive nephrology nursing knowledge to practice. Of importance for nursing, the results of this study indicate that domain-specific knowledge is a crucial feature of expert practice.

Evaluating parent-child group music therapy programmes: challenges and successes for Sing & Grow

Sing & Grow is an early intervention music therapy programme for families with children from birth to 3 years of age, who are socially, economically, or physically disadvantaged. It aims to improve parenting skills and confidence, promote positive parent–child interactions, stimulate child development, and provide social networking opportunities. Music and song activities are used in a therapeutic context to enhance parenting skills, improve parent–child interactions, provide essential developmental stimulation for children, promote social support for parenting, and strengthen links between parents and community services.

Validating the is-impact model in the Malaysian public sector

This research is one of several ongoing studies conducted within the IT Professional Services (ITPS) research programme at Queensland University of Technology (QUT). In 2003, ITPS introduced the IS-Impact model, a measurement model for measuring information systems success from the viewpoint of multiple stakeholders. The model, along with its instrument, is robust, simple, yet generalisable, and yields results that are comparable across time, stakeholders, different systems and system contexts. The IS-Impact model is defined as “a measure at a point in time, of the stream of net benefits from the Information System (IS), to date and anticipated, as perceived by all key-user-groups”. The model represents four dimensions, which are ‘Individual Impact’, ‘Organizational Impact’, ‘Information Quality’ and ‘System Quality’. The two Impact dimensions measure the up-to-date impact of the evaluated system, while the remaining two Quality dimensions act as proxies for probable future impacts (Gable, Sedera & Chan, 2008). To fulfil the goal of ITPS, “to develop the most widely employed model” this research re-validates and extends the IS-Impact model in a new context. This method/context-extension research aims to test the generalisability of the model by addressing known limitations of the model. One of the limitations of the model relates to the extent of external validity of the model. In order to gain wide acceptance, a model should be consistent and work well in different contexts. The IS-Impact model, however, was only validated in the Australian context, and packaged software was chosen as the IS understudy. Thus, this study is concerned with whether the model can be applied in another different context. Aiming for a robust and standardised measurement model that can be used across different contexts, this research re-validates and extends the IS-Impact model and its instrument to public sector organisations in Malaysia. The overarching research question (managerial question) of this research is “How can public sector organisations in Malaysia measure the impact of information systems systematically and effectively?” With two main objectives, the managerial question is broken down into two specific research questions. The first research question addresses the applicability (relevance) of the dimensions and measures of the IS-Impact model in the Malaysian context. Moreover, this research question addresses the completeness of the model in the new context. Initially, this research assumes that the dimensions and measures of the IS-Impact model are sufficient for the new context. However, some IS researchers suggest that the selection of measures needs to be done purposely for different contextual settings (DeLone & McLean, 1992, Rai, Lang & Welker, 2002). Thus, the first research question is as follows, “Is the IS-Impact model complete for measuring the impact of IS in Malaysian public sector organisations?” [RQ1]. The IS-Impact model is a multidimensional model that consists of four dimensions or constructs. Each dimension is represented by formative measures or indicators. Formative measures are known as composite variables because these measures make up or form the construct, or, in this case, the dimension in the IS-Impact model. These formative measures define different aspects of the dimension, thus, a measurement model of this kind needs to be tested not just on the structural relationship between the constructs but also the validity of each measure. In a previous study, the IS-Impact model was validated using formative validation techniques, as proposed in the literature (i.e., Diamantopoulos and Winklhofer, 2001, Diamantopoulos and Siguaw, 2006, Petter, Straub and Rai, 2007). However, there is potential for improving the validation testing of the model by adding more criterion or dependent variables. This includes identifying a consequence of the IS-Impact construct for the purpose of validation. Moreover, a different approach is employed in this research, whereby the validity of the model is tested using the Partial Least Squares (PLS) method, a component-based structural equation modelling (SEM) technique. Thus, the second research question addresses the construct validation of the IS-Impact model; “Is the IS-Impact model valid as a multidimensional formative construct?” [RQ2]. This study employs two rounds of surveys, each having a different and specific aim. The first is qualitative and exploratory, aiming to investigate the applicability and sufficiency of the IS-Impact dimensions and measures in the new context. This survey was conducted in a state government in Malaysia. A total of 77 valid responses were received, yielding 278 impact statements. The results from the qualitative analysis demonstrate the applicability of most of the IS-Impact measures. The analysis also shows a significant new measure having emerged from the context. This new measure was added as one of the System Quality measures. The second survey is a quantitative survey that aims to operationalise the measures identified from the qualitative analysis and rigorously validate the model. This survey was conducted in four state governments (including the state government that was involved in the first survey). A total of 254 valid responses were used in the data analysis. Data was analysed using structural equation modelling techniques, following the guidelines for formative construct validation, to test the validity and reliability of the constructs in the model. This study is the first research that extends the complete IS-Impact model in a new context that is different in terms of nationality, language and the type of information system (IS). The main contribution of this research is to present a comprehensive, up-to-date IS-Impact model, which has been validated in the new context. The study has accomplished its purpose of testing the generalisability of the IS-Impact model and continuing the IS evaluation research by extending it in the Malaysian context. A further contribution is a validated Malaysian language IS-Impact measurement instrument. It is hoped that the validated Malaysian IS-Impact instrument will encourage related IS research in Malaysia, and that the demonstrated model validity and generalisability will encourage a cumulative tradition of research previously not possible. The study entailed several methodological improvements on prior work, including: (1) new criterion measures for the overall IS-Impact construct employed in ‘identification through measurement relations’; (2) a stronger, multi-item ‘Satisfaction’ construct, employed in ‘identification through structural relations’; (3) an alternative version of the main survey instrument in which items are randomized (rather than blocked) for comparison with the main survey data, in attention to possible common method variance (no significant differences between these two survey instruments were observed); (4) demonstrates a validation process of formative indexes of a multidimensional, second-order construct (existing examples mostly involved unidimensional constructs); (5) testing the presence of suppressor effects that influence the significance of some measures and dimensions in the model; and (6) demonstrates the effect of an imbalanced number of measures within a construct to the contribution power of each dimension in a multidimensional model.

Development and review of vignettes representing older people with cognitive impairment

This paper describes a process undertaken to develop and review five clinical vignettes to be used in geriatric nursing educational research. The purpose of this process was to provide valid depictions of delirium and its subtypes and distinguish delirium from dementia. Five vignettes depicting hospital bedside interactions between nursing staff, family, and an older patient who displayed signs of one of the following conditions: delirium (hyper and hypo-active types respectively), dementia, or delirium (both types) superimposed on dementia were constructed. Vignette accuracy and reliability was established using a multistage process that culminated in formal review by a group of ten international nursing and medical delirium experts. The final five vignettes accurately depicted the given scenario as agreed by the experts and were at an appropriate level of simplicity and clarity. Given the increased interest in vignettes for both nursing research and educational purposes, the described method of vignette development and review has the ability to assist other vignette developers in creating reliable representations of their desired clinical scenarios.

Peer-led diabetes self-management programme for community-dwelling older people in China : study protocol for a quasi-experimental design

Aim. A protocol for a new peer-led self-management programme for communitydwelling older people with diabetes in Shanghai, China. Background. The increasing prevalence of type 2 diabetes poses major public health challenges. Appropriate education programmes could help people with diabetes to achieve self-management and better health outcomes. Providing education programmes to the fast growing number of people with diabetes present a real challenge to Chinese healthcare system, which is strained for personnel and funding shortages. Empirical literature and expert opinions suggest that peer education programmes are promising. Design. Quasi-experimental. Methods. This study is a non-equivalent control group design (protocol approved in January, 2008). A total of 190 people, with 95 participants in each group, will be recruited from two different, but similar, communities. The programme, based on Social Cognitive Theory, will consist of basic diabetes instruction and social support and self-efficacy enhancing group activities. Basic diabetes instruction sessions will be delivered by health professionals, whereas social support and self-efficacy enhancing group activities will be led by peer leaders. Outcome variables include: self-efficacy, social support, self-management behaviours, depressive status, quality of life and healthcare utilization, which will be measured at baseline, 4 and 12 weeks. Discussion. This theory-based programme tailored to Chinese patients has potential for improving diabetes self-management and subsequent health outcomes. In addition, the delivery mode, through involvement of peer leaders and existing community networks,is especially promising considering healthcare resource shortage in China.