A critical evaluation and comparison of two formative measures of system quality using criterion variables

Evaluating the validity of formative variables has presented ongoing challenges for researchers. In this paper we use global criterion measures to compare and critically evaluate two alternative formative measures of System Quality. One model is based on the ISO-9126 software quality standard, and the other is based on a leading information systems research model. We find that despite both models having a strong provenance, many of the items appear to be non-significant in our study. We examine the implications of this by evaluating the quality of the criterion variables we used, and the performance of PLS when evaluating formative models with a large number of items. We find that our respondents had difficulty distinguishing between global criterion variables measuring different aspects of overall System Quality. Also, because formative indicators “compete with one another” in PLS, it may be difficult to develop a set of measures which are all significant for a complex formative construct with a broad scope and a large number of items. Overall, we suggest that there is cautious evidence that both sets of measures are valid and largely equivalent, although questions still remain about the measures, the use of criterion variables, and the use of PLS for this type of model evaluation.

Recruitment into diabetes prevention programs : what is the impact of errors in self-reported measures of obesity?

Background
Error in self-reported measures of obesity has been frequently described, but the effect of self-reported error on recruitment into diabetes prevention programs is not well established. The aim of this study was to examine the effect of using self-reported obesity data from the Finnish diabetes risk score (FINDRISC) on recruitment into the Greater Green Triangle Diabetes Prevention Project (GGT DPP).

Methods
The GGT DPP was a structured group-based lifestyle modification program delivered in primary health care settings in South-Eastern Australia. Between 2004–05, 850 FINDRISC forms were collected during recruitment for the GGT DPP. Eligible individuals, at moderate to high risk of developing diabetes, were invited to undertake baseline tests, including anthropometric measurements performed by specially trained nurses. In addition to errors in calculating total risk scores, accuracy of self-reported data (height, weight, waist circumference (WC) and Body Mass Index (BMI)) from FINDRISCs was compared with baseline data, with impact on participation eligibility presented.

Results
Overall, calculation errors impacted on eligibility in 18 cases (2.1%). Of n = 279 GGT DPP participants with measured data, errors (total score calculation, BMI or WC) in self-report were found in n = 90 (32.3%). These errors were equally likely to result in under- or over-reported risk. Under-reporting was more common in those reporting lower risk scores (Spearman-rho = −0.226, p-value < 0.001). However, underestimation resulted in only 6% of individuals at high risk of diabetes being incorrectly categorised as moderate or low risk of diabetes.

Conclusions
Overall FINDRISC was found to be an effective tool to screen and recruit participants at moderate to high risk of diabetes, accurately categorising levels of overweight and obesity using self-report data. The results could be generalisable to other diabetes prevention programs using screening tools which include self-reported levels of obesity.

Human performance measures for interactive haptic-audio-visual interfaces

Virtual reality and simulation are becoming increasingly important in modern society and it is essential to improve our understanding of system usability and efficacy from the users’ perspective. This paper introduces a novel evaluation method designed to assess human user capability when undertaking technical and procedural training using virtual training systems. The evaluation method falls under the user-centred design and evaluation paradigm and draws on theories of cognitive, skillbased and affective learning outcomes. The method focuses on user interaction with haptic-audio-visual interfaces and the complexities related to variability in users’ performance, and the adoption and acceptance of the technologies. A large scale user study focusing on object assembly training tasks involving selecting, rotating, releasing, inserting and manipulating 3D objects was performed. The study demonstrated the advantages of the method in obtaining valuable multimodal information for accurate and comprehensive evaluation of virtual training system efficacy. The study investigated how well users learn, perform, adapt to and perceive the virtual training. The results of the study revealed valuable aspects of the design and evaluation of virtual training systems contributing to an improved understanding of more usable virtual training systems.

Competency measurements : testing convergent validity for two measures

Aim.  This paper is a report of a study to investigate whether the Australian National Competency Standards for Registered Nurses demonstrate correlations with the Finnish Nurse Competency Scale. Background.  Competency assessment has become popular as a key regulatory requirement and performance indicator. The term competency, however, does not have a globally accepted definition and this has the potential to create controversy, ambiguity and confusion. Variations in meaning and definitions adopted in workplaces and educational settings will affect the interpretation of research findings and have implications for the nursing profession. Method.  A non-experimental cross-sectional survey design was used with a convenience sample of 116 new graduate nurses in 2005. The second version of the Australian National Competency Standards and the Nurse Competency Scale was used to elicit responses to self-assessed competency in the transitional year (first year as a Registered Nurse). Findings.  Correlational analysis of self-assessed levels of competence revealed a relationship between the Australian National Competency Standards (ANCI) and the Nurse Competency Scale (NCS). The correlational relation between ANCI domains and NCS factors suggests that these scales are indeed used to measure related dimensions. A statistically significant relationship (r = 0·75) was found between the two competency measures. Conclusion.  Although the finding of convergent validity is insufficient to establish construct validity for competency as used in both measures in this study, it is an important step towards this goal. Future studies on relationships between competencies must take into account the validity and reliability of the tools.

Associations between measures of adiposity over 10 years and patella cartilage in population-based asymptomatic women

Objective: Osteoarthritis (OA) most commonly affects the patellofemoral compartment of the knee, and is a major cause of pain and disability. Structural changes that evolve prior to the onset of symptoms can be visualised using magnetic resonance imaging (MRI). There is little known information about the role of adiposity on the early structural changes in the patella cartilage in younger, asymptomatic adult females.

Methods: One hundred and sixty asymptomatic women (20–49 years) participating in the Geelong Osteoporosis Study underwent knee MRI (2006–8). Weight and body mass index (BMI) were measured 10 years prior (1994–7, baseline) and at the time of MRI (current), with change over the period calculated (current–baseline). Relationships between measures of adiposity and patella cartilage volume and defects were examined.

Results: After adjustment for age and patella bone volume, there was a reduction of 13 ml (95% confidence interval (95% CI), −25.7, −0.55) in patella cartilage volume for every 1 unit increase in current BMI, and a reduction of 27 ml (95% CI −52.6, −1.5) per BMI unit increase over 10 years (P=0.04 for both). No significant association was observed between baseline BMI and patella cartilage volume (P=0.16). Increased baseline and current weight and BMI were associated with increased prevalence of patella cartilage defects (all P<0.001).

Conclusions: Adiposity and weight gain during midlife are associated with detrimental structural change at the patella in young to middle-aged healthy non-osteoarthritic women. Maintaining a healthy weight and avoiding weight gain in younger asymptomatic women may be important in the prevention of patellofemoral OA.