Development and interrater reliability testing of a telephone interview training programme for Australian nurse interviewers

Summary Background The final phase of a three phase study analysing the implementation and impact of the nurse practitioner role in Australia (the Australian Nurse Practitioner Project or AUSPRAC) was undertaken in 2009, requiring nurse telephone interviewers to gather information about health outcomes directly from patients and their treating nurse practitioners. A team of several registered nurses was recruited and trained as telephone interviewers. The aim of this paper is to report on development and evaluation of the training process for telephone interviewers. Methods The training process involved planning the content and methods to be used in the training session; delivering the session; testing skills and understanding of interviewers post-training; collecting and analysing data to determine the degree to which the training process was successful in meeting objectives and post-training follow-up. All aspects of the training process were informed by established educational principles. Results Interrater reliability between interviewers was high for well-validated sections of the survey instrument resulting in 100% agreement between interviewers. Other sections with unvalidated questions showed lower agreement (between 75% and 90%). Overall the agreement between interviewers was 92%. Each interviewer was also measured against a specifically developed master script or gold standard and for this each interviewer achieved a percentage of correct answers of 94.7% or better. This equated to a Kappa value of 0.92 or better. Conclusion The telephone interviewer training process was very effective and achieved high interrater reliability. We argue that the high reliability was due to the use of well validated instruments and the carefully planned programme based on established educational principles. There is limited published literature on how to successfully operationalise educational principles and tailor them for specific research studies; this report addresses this knowledge gap.

Recruiting and engaging new mothers in nutrition research studies : lessons from the Australian NOURISH randomised controlled trial

Background: Despite important implications for the budgets, statistical power and generalisability of research findings, detailed reports of recruitment and retention in randomised controlled trials (RCTs) are rare. The NOURISH RCT evaluated a community-based intervention for first-time mothers that promoted protective infant feeding practices as a primary prevention strategy for childhood obesity. The aim of this paper is to provide a detailed description and evaluation of the recruitment and retention strategies used. Methods: A two stage recruitment process designed to provide a consecutive sampling framework was used. First time mothers delivering healthy term infants were initially approached in postnatal wards of the major maternity services in two Australian cities for consent to later contact (Stage 1). When infants were about four months old mothers were re-contacted by mail for enrolment (Stage 2), baseline measurements (Time 1) and subsequent random allocation to the intervention or control condition. Outcomes were assessed at infant ages 14 months (Time 2) and 24 months (Time 3). Results: At Stage 1, 86% of eligible mothers were approached and of these women, 76% consented to later contact. At Stage 2, 3% had become ineligible and 76% could be recontacted. Of the latter, 44% consented to full enrolment and were allocated. This represented 21% of mothers screened as eligible at Stage 1. Retention at Time 3 was 78%. Mothers who did not consent or discontinued the study were younger and less likely to have a university education. Conclusions: The consent and retention rates of our sample of first time mothers are comparable with or better than other similar studies. The recruitment strategy used allowed for detailed information from non-consenters to be collected; thus selection bias could be estimated. Recommendations for future studies include being able to contact participants via mobile phone (particular text messaging), offering home visits to reduce participant burden and considering the use of financial incentives to support participant retention.

Will the application of spatial multi criteria evaluation technique enhance the quality of decision-making to resolve boundary conflicts in the Philippines?

Systematic studies that evaluate the quality of decision-making processes are relatively rare. Using the literature on decision quality, this research develops a framework to assess the quality of decision-making processes for resolving boundary conflicts in the Philippines. The evaluation framework breaks down the decision-making process into three components (the decision procedure, the decision method, and the decision unit) and is applied to two ex-post (one resolved and one unresolved) and one ex-ante cases. The evaluation results from the resolved and the unresolved cases show that the choice of decision method plays a minor role in resolving boundary conflicts whereas the choice of decision procedure is more influential. In the end, a decision unit can choose a simple method to resolve the conflict. The ex-ante case presents a follow-up intended to resolve the unresolved case for a changing decision-making process in which the associated decision unit plans to apply the spatial multi criteria evaluation (SMCE) tool as a decision method. The evaluation results from the ex-ante case confirm that the SMCE has the potential to enhance the decision quality because: a) it provides high quality as a decision method in this changing process, and b) the weaknesses associated with the decision unit and the decision procedure of the unresolved case were found to be eliminated in this process.

An integrated educational model for continuing nurse education

Aim This paper reports on the development and evaluation of an integrated clinical learning model to inform ongoing education for surgical nurses. The research aim was to evaluate the effectiveness of implementing a Respiratory Skills Update (ReSKU) education program, in the context of organisational utility, on improving surgical nurses' practice in the area of respiratory assessment. Background Continuous development and integration of technological innovations and research in the healthcare environment mandate the need for continuing education for nurses. Despite an increased worldwide emphasis on this, there is scant empirical evidence of program effectiveness. Methods A quasi experimental pre test, post test non–equivalent control group design evaluated the impact of the ReSKU program on surgical nurses' clinical practice. The 2008 study was conducted in a 400 bed regional referral public hospital and was consistent with contemporary educational approaches using multi-modal, interactive teaching strategies. Findings The study demonstrated statistically significant differences between groups regarding reported use of respiratory skills, three months after ReSKU program attendance. Between group data analysis indicated that the intervention group's reported beliefs and attitudes pertaining to subscale descriptors showed statistically significant differences in three of the six subscales. Conclusion The construct of critical thinking in the clinical context, combined with clinical reasoning and purposeful reflection, was a powerful educational strategy to enhance competency and capability in clinicians.

A critical evaluation and comparison of two formative measures of system quality using criterion variables

Evaluating the validity of formative variables has presented ongoing challenges for researchers. In this paper we use global criterion measures to compare and critically evaluate two alternative formative measures of System Quality. One model is based on the ISO-9126 software quality standard, and the other is based on a leading information systems research model. We find that despite both models having a strong provenance, many of the items appear to be non-significant in our study. We examine the implications of this by evaluating the quality of the criterion variables we used, and the performance of PLS when evaluating formative models with a large number of items. We find that our respondents had difficulty distinguishing between global criterion variables measuring different aspects of overall System Quality. Also, because formative indicators “compete with one another” in PLS, it may be difficult to develop a set of measures which are all significant for a complex formative construct with a broad scope and a large number of items. Overall, we suggest that there is cautious evidence that both sets of measures are valid and largely equivalent, although questions still remain about the measures, the use of criterion variables, and the use of PLS for this type of model evaluation.

Evaluation of Trippple : a field analysis

This paper illustrates a field research performed with a team of experts involved in the evaluation of Trippple, a system aimed at supporting the different phases of a tourist trip, in order to provide feedback and insights, both on the functionalities already implemented (that at the time of evaluation were available only as early and very unstable prototypes), and on the functionalities still to be implemented. We show how the involvement of professionals helped to focus on challenging aspects, instead of less important, cosmetic, issues and resulted profitable in terms of early feedback, issues spotted, and improvements suggested

Evaluating the QUT Homestay Program : lessons learnt in providing quality services to international students

The QUT Homestay Program is an essential part of the university’s commitment to meet the accommodation needs of international students. Despite the importance of this style of accommodation, there is very little research addressing issues related to homestay arrangements. The program at Queensland University of Technology (QUT) was evaluated in 2002 to develop a continuous improvement framework to ensure provision of quality homestay services to international students. This paper presents an overview of the evaluation and key lessons learnt in providing quality homestay services to international students. It will cover social and cross-cultural issues faced by providers and international students in the homestay environment, the homestay support needs, program information, policies, procedures and code of practice governing the program.

Evaluation of Eph receptor and ephrin expression within the human cornea and limbus

Eph receptor tyrosine kinases and their ligands, the ephrins, regulate the development and maintenance of multiple organs but little is known about their potential role within the cornea. The purpose of this study was to perform a thorough investigation of Eph/ephrin expression within the human cornea including the limbal stem cell niche. Initially, immunohistochemistry was performed on human donor eyes to determine the spatial distribution of Eph receptors and ephrins in the cornea and limbus. Patterns of Eph/ephrin gene expression in (1) immortalised human corneal endothelial (B4G12) or corneal epithelial (HCE-T) cell lines, and (2) primary cultures of epithelial or stromal cells established from the corneal limbus of cadaveric eye tissue were then assessed by reverse transcription (RT) PCR. Limbal epithelial or stromal cells from primary cultures were also assessed for evidence of Eph/ephrin-reactivity by immunofluorescence. Immunoreactivity for ephrinA1 and EphB4 was detected in the corneal endothelium of donor eyes. EphB4 was also consistently detected in the limbal and corneal epithelium and in cells located in the stroma of the peripheral cornea. Expression of multiple Eph/ephrin genes was detected in immortalised corneal epithelial and endothelial cell lines. Evidence of Eph/ephrin gene expression was also demonstrated in primary cultures of human limbal stromal (EphB4, B6; ephrinA5) and epithelial cells (EphA1, A2; ephrinA5, B2) using both RT-PCR and immunofluorescence. The expression of Eph receptors and ephrins within the human cornea and limbus is much wider than previously appreciated and suggests multiple potential roles for these molecules in the maintenance of normal corneal architecture.

Tracking new directions in the evaluation of postgraduate supervision

This article focuses on the evaluation of thesis supervision and highlights the vast range of problem areas presently documented as significant areas of concern for both graduate students and their supervisors. Additionally, the authors report on a study completed in 1995, which surveyed all Australian universities about current evaluative practices in postgraduate supervision. The conclusion of this study was that the conduct of such evaluations appears to be minimal and is primarily designed to obtain an indicator of the general “health” of a university's postgraduate supervision rather than to foster improved supervisory practices. As part of the same study, the authors conducted University faculty-based student and supervisor focus groups. Key issues emerging from these focus groups were: (1) the importance of relational aspects of supervision as the student communicates over the long term with one or more supervisors; (2) the importance of systematic feedback, monitoring, and evaluation to the supervisory process; and (3) the lack of strategies to facilitate this evaluative feedback process. On the basis of these findings, the authors designed evaluative strategies to facilitate regular ongoing feedback between students and supervisors.

Treating skin tears in nursing home residents : a pilot study comparing four types of dressings

A pilot study was conducted to compare four types of dressings used to treat skin tears in nursing home residents. Wounds treated with a non-occlusive dressing healed more quickly than those dressed with occlusive dressings. The results suggest that ease of use and product wastage are important considerations when treating skin tears. The pilot study also highlights the need for further research into skin tear management and the need for ongoing education for nurses regarding skin integrity risk assessment and product information.

Phenomenography : opening a new territory for library and information science research

Phenomenography is a research approach devised to allow the investigation of varying ways in which people experience aspects of their world. Whilst growing attention is being paid to interpretative research in LIS, it is not always clear how the outcomes of such research can be used in practice. This article explores the potential contribution of phenomenography in advancing the application of phenomenological and hermeneutic frameworks to LIS theory, research and practice. In phenomenography we find a research toll which in revealing variation, uncovers everyday understandings of phenomena and provides outcomes which are readily applicable to professional practice. THe outcomes may be used in human computer interface design, enhancement, implementation and training, in the design and evaluation of services, and in education and training for both end users and information professionals. A proposed research territory for phenomenography in LIS includes investigating qualitative variation in the experienced meaning of: 1) information and its role in society 2) LIS concepts and principles 3) LIS processes and; 4) LIS elements.

Evaluation of the "reconceptualising early mathematics learning" project

The Pattern and Structure Mathematics Awareness Project (PASMAP) has investigated the development of patterning and early algebraic reasoning among 4 to 8 year olds over a series of related studies. We assert that an awareness of mathematical pattern and structure enables mathematical thinking and simple forms of generalisation from an early age. The project aims to promote a strong foundation for mathematical development by focusing on critical, underlying features of mathematics learning. This paper provides an overview of key aspects of the assessment and intervention, and analyses of the impact of PASMAP on students’ representation, abstraction and generalisation of mathematical ideas. A purposive sample of four large primary schools, two in Sydney and two in Brisbane, representing 316 students from diverse socio-economic and cultural contexts, participated in the evaluation throughout the 2009 school year and a follow-up assessment in 2010. Two different mathematics programs were implemented: in each school, two Kindergarten teachers implemented the PASMAP and another two implemented their regular program. The study shows that both groups of students made substantial gains on the ‘I Can Do Maths’ assessment and a Pattern and Structure Assessment (PASA) interview, but highly significant differences were found on the latter with PASMAP students outperforming the regular group on PASA scores. Qualitative analysis of students’ responses for structural development showed increased levels for the PASMAP students; those categorised as low ability developed improved structural responses over a relatively short period of time.

Calling for help : a responsive evaluation of Child Health Line

The project examined the responsiveness of the telenursing service provided by the Child Health Line (hereinafter referred to as CHL). It aimed to provide an account of population usage of the service, the call request types and the response of the service to the calls. In so doing, the project extends the current body of knowledge pertaining to the provision of parenting support through telenursing. Approximately 900 calls to the CHL were audio-recorded over the December 2005-2006 Christmas-New Year period. A protocol was developed to code characteristics of the call, the interactional features between the caller and nurse call-taker, and the extent to which there was (a) agreement on problem definition and the plan of action and (b) interactional alignment between nurse and caller. A quantitative analysis examined the frequencies of the main topics covered in calls to the CHL and any statistical associations between types of calls, length of calls and nurse-caller alignment. In addition, a detailed qualitative analysis was conducted on a subset of calls dealing with the nurse management of calls seeking medical advice and information. Key findings include: • Overall, 74% of the calls discussed parenting and child development issues, 48% discussed health/medical issues, and 16% were information-seeking calls. • More specifically: o 21% discussed health/medical and parenting and child development issues. o 3% discussed parenting and information-seeking issues. o 5% discussed health/medical, parenting/development and information issues. o 18% exclusively focussed on health and medical issues and therefore were outside the remit of the intended scope of the CHL. These calls caused interactional dilemmas for the nurse call-takers as they simultaneously dealt with parental expectations for help and the CHL guidelines indicating that offering medical advice was outside the remit of the service. • Most frequent reasons for calling were to discuss sleep, feeding, normative infant physical functions and parenting advice. • The average length of calls to the CHL was 7 minutes. • Longer calls were more likely to involve nurse call-takers giving advice on more than one topic, the caller displaying strong emotions, the caller not specifically providing the reason for the call, and the caller discussing parenting and developmental issues. • Shorter calls were characterised by the nurse suggesting that the child receive immediate medical attention, the nurse emphasising the importance or urgency of the plan of action, the caller referring to or requesting confirmation of a diagnosis, and caller and nurse call-taker discussion of health and medical issues. • The majority of calls, 92%, achieved parent-nurse alignment by the conclusion of the call. However, 8% did not. • The 8% of calls that were not aligned require further quantitative and qualitative investigation of the interactional features. The findings are pertinent in the current context where Child Health Line now resides within 13HEALTH. These findings indicate: 1. A high demand for parenting advice. 2. Nurse call-takers have a high level of competency in dealing with calls about parenting and normal child development, which is the remit of the CHL. 3. Nurse call-takers and callers achieve a high degree of alignment when both parties agree on a course of action. 4. There is scope for developing professional practice in calls that present difficulties in terms of call content, interactional behaviour and call closure. Recommendations of the project: 1. There are numerous opportunities for further research on interactional aspects of calls to the CHL, such as further investigations of the interactional features and the association of the features to alignment and nonalignment. The rich and detailed insights into the patterns of nurse-parent interactions were afforded by the audio-recording and analysis of calls to the CHL. 2. The regular recording of calls would serve as a way of increasing understanding of the type and nature of calls received, and provide a valuable training resource. Recording and analysing calls to CHL provides insight into the operation of the service, including evidence about the effectiveness of triaging calls. 3. Training in both recognising and dealing with problem calls may be beneficial. For example, calls where the caller showed strong emotion, appeared stressed, frustrated or troubled were less likely to be rated as aligned calls. In calls where the callers described being ‘at their wits end’, or responded to each proposed suggestion with ‘I’ve tried that’, the callers were fairly resistant to advice-giving. 4. Training could focus on strategies for managing calls relating to parenting support and advice, and parental well-being. The project found that these calls were more likely to be rated as being nonaligned. 5. With the implementation of 13HEALTH, future research could compare nurse-parent interaction following the implementation of triaging. Of the calls, 21% had both medical and parenting topics discussed and 5.3% discussed medical, parenting and information topics. Added to this, in 12% of calls, there was ambiguity between the caller and nurse call-taker as to whether the problem was medical or behavioural.

Revealing performance heterogeneity through knowledge management maturity evaluation : a capability-based approach

The dynamic capabilities view (DCV) focuses on renewal of firms’ strategic knowledge resources so as to sustain competitive advantage within turbulent markets. Within the context of the DCV, the focus of knowledge management (KM) is to develop the KMC through deploying knowledge governance mechanisms that are conducive to facilitating knowledge processes so as to produce superior business performance over time. The essence of KM performance evaluation is to assess how well the KMC is configured with knowledge governance mechanisms and processes that enable a firm to achieve superior performance through matching its knowledge base with market needs. However, little research has been undertaken to evaluate KM performance from the DCV perspective. This study employed a survey study design and adopted hypothesis-testing approaches to develop a capability-based KM evaluation framework (CKMEF) that upholds the basic assertions of the DCV. Under the governance of the framework, a KM index (KMI) and a KM maturity model (KMMM) were derived not only to indicate the extent to which a firm’s KM implementations fulfill its strategic objectives, and to identify the evolutionary phase of its KMC, but also to bench-mark the KMC in the research population. The research design ensured that the evaluation framework and instruments have statistical significance and good generalizabilty to be applied in the research population, namely construction firms operating in the dynamic Hong Kong construction market. The study demonstrated the feasibility of quantitatively evaluating the development of the KMC and revealing the performance heterogeneity associated with the development.

A generic framework and runtime environment for development and evaluation of behavioral biometrics solutions

Increasing use of computerized systems in our daily lives creates new adversarial opportunities for which complex mechanisms are exploited to mend the rapid development of new attacks. Behavioral Biometrics appear as one of the promising response to these attacks. But it is a relatively new research area, specific frameworks for evaluation and development of behavioral biometrics solutions could not be found yet. In this paper we present a conception of a generic framework and runtime environment which will enable researchers to develop, evaluate and compare their behavioral biometrics solutions with repeatable experiments under the same conditions with the same data.