The relative and absolute reliability of two balance performance measures in chronic stroke patients

Purpose. To examine the relative reliability and absolute reliability of the Berg Balance Scale (BBS) and the Postural Assessment Scale for Stroke Patients (PASS) in chronic stroke patients. Method. A total of 52 mild to moderate stroke patients, who had a stroke more than 6 months previously, participated in the study. Both balance measures were administered twice, seven days apart, to the patients. A relative reliability index (intra-class correlation coefficient, ICC2,1) was used to examine the level of agreement between test and retest. Absolute reliability indices, including the Bland and Altman method, the standard error of measurement (SEM), and the smallest real differences (SRD), were used to define the extent to which a balance score varies on test-retest measurements. Results. Test-retest agreements were high (ICC2,1: BBS = 0.98; PASS = 0.97), indicating excellent agreement from a relative perspective. The SEM of the BBS and PASS, representing the smallest change threshold that indicates a real improvement for a group of individuals, were 2.4 and 1.1, respectively. The SRD of the BBS and PASS were 6.7 and 3.2, respectively, exhibiting the smallest change threshold that indicates a real improvement for a single individual. Conclusions. The test-retest agreements of the BBS and PASS were high in mild to moderate chronic stroke patients. The thresholds of both measures to detect real change are acceptable in research and clinical settings.

The use of scoring rubrics to determine clinical performance in the operating suite

Summary : This research evolved out of the need to examine the validity and inter-rater reliability of a set of performance-based scoring rubrics designed to measure competencies within the operating suite.

Method : Both holistic and analytical rubrics were developed aligned to the ACORN Standard [Australian College of Operating Room Nurses Standard NR4, 2004. ACORN Competency Standards for Perioperative Nurses: Standard NR4: The Instrument Nurse in the Perioperative Environment. Australian College of Operating Room Nurses Ltd, Adelaide] and underpinned by the Dreyfus model (1981). Three video clips that captured varying performance of nurses performing as instrument nurses in the operating suite were recorded and used as prompts by expert raters, who judged the performance using the rubrics.

Results : The study found that the holistic rubrics led to more consistent judgments than the analytical rubrics, yet the latter provided more diagnostic information for intervention purposes. Despite less consistency, the Analytical Observation Form had sufficient construct validity to satisfy the requirements of criterion referencing as determined by the Item Separation Index (Rasch, 1960), including high internal consistency and greater inter-rater reliability when average ratings were used.

Conclusion : The study was an empirical investigation of the use of concomitant Analytical and Holistic Rubrics to determine various levels of performance in the operating suite including inter-rater reliability. The methodology chosen was theoretically sound and sufficiently flexible to be used to develop other competencies within the operating suite.

Parsimonious and efficient assessment of health-related quality of life in osteoarthritis research: validation of the Assessment of Quality of Life (AQoL) instrument

Background: The Assessment of Quality of Life (AQoL) utility instrument was psychometrically developed for the general population. This study aimed to explore its potential as an osteoarthritis (OA) outcome measure.

Methods: WOMAC, Lequesne index, SF-36, Visual analogue scales and the AQoL were administered to 222 people with OA. The ability of each questionnaire to detect differences between groups was based on (i) self-rated health (SRH) and, (ii) differences between people on an orthopedic waiting list (WL) vs people with OA in the community (C). Comparisons included effect size, relative efficiency and receiver operator characteristic curves.

Results: All instruments detected differences between groups; however no one instrument exhibited superior efficiency. The AQoL demonstrated strong psychometric properties.

Conclusion: The AQoL has equivalent performance to comparator questionnaires commonly used in OA research and would be a useful adjunct to well-established disease specific scales. The AQoL has important advantages; brevity (12 items), facilitates comparisons between disease groups, and delivers a utility score that can be used in health economic evaluations.

The second curriculum opera : Arias, assessment and methodological traces

Drawing on contemporary forms of qualitative research such as performance ethnography (McCall, 2000), autoethnography (Ellis & Bochner, 2000; Ellis & Flaherty, 1992) and using narrative and writing as forms of inquiry (Richardson, 1990; 1992; 1995a; 1995b; 1997; 1999; 2000), this research project constructs a 'learning through' (Gardner, 1983; 1993; 1995; 1999; 2003a; 2003b) approach to curriculum within pre-service teacher education. During 2002 we initiated the first curriculum opera (Dixon & White, 2003; Dixon, White, & Smerdon, 2003) in our Faculty of Education with thirty-seven students. In 2003 we developed this learning and teaching approach with twice as many students. We also reconsidered assessment of students involved in the opera for overall theoretical consistency. As students increasingly took control, they 'imagined curriculum' (Doll & Gough, 2002) and transformed their exploration of identity in the 'process of becoming' teachers (Britzman, 2003). In this paper, we outline the project and the learning involved. We also indicate future directions for learning and teaching in preservice teacher education as well as the potential uses and misuses of teacher assessment through portfolio.

Mathematical power of special-needs pupils : an ICT-based dynamic assessment format to reveal weak pupils' learning potential

This paper reports a study aimed at revealing special-educational-needs pupils' learning potential by means of an ICT-based assessment including a dynamic visual tool that might help pupils when solving mathematics problems. The study focused on subtraction problems up to 100, which require 'borrowing'. These problems, in which the value of the ones-digit of the subtrahend is larger than the ones-digit of the minuend, are known as a serious difficulty for weak pupils in mathematics. Seven of such problems from a standardised test were placed in the ICT environment. Data were collected from two test conditions: the standardised written test format and the ICT version of the test items including the tool that provided pupils with a set of virtual manipulatives. The 37 pupils involved in the study were 8–12 years old and from two special-education schools in the Netherlands. Comparison of the performance scores in the two formats showed that an ICT-based assessment format, including a dynamic visual tool, can reveal weak pupils' learning potential and strategy use. The study also pointed out that 'partial-tool use', ie, not carrying out the complete subtraction operation with the tool, can provide sufficient support to find the correct answer.

How do we compare? - Applying UK pay for performance indicators to an Australian general practice

BACKGROUND United Kingdom general practitioners receive payment based on their performance in multiple clinical indicators. We set out to apply the same indicators in an Australian general practice to benchmark our performance and to see how much work was required to obtain the data.

METHODS Clinical indicators for the 2008–2009 UK Quality and Outcomes Framework (QOF) cycle were examined and achievement levels measured in a large rural Australian general practice, mainly by computer searching of the clinical database.

RESULTS Outcome measures were obtainable for 79 out of 80 indicators. Manual perusal of computer records was required for 16 indicators. Data collection takes approximately 130 hours. The Australian general practice achieved 66% of available pay for performance points compared to the UK average of 97%.

DISCUSSION United Kingdom QOF clinical data is obtainable relatively easily in a well computerised Australian rural general practice. The exercise identified significant areas in which clinical performance could be improved.

Variables influencing the mathematics performance of first-year tertiary students : a case study

The need to understand which factors most strongly affect performance in first-year mathematics programs at Khon Kaen University (KKU), in North Eastern Thailand, provided the main focus of the study which is described. First-year mathematics students in the 1990-1991 academic year, from four KKU faculty groups (Medicine and Nursing, Agriculture, Science and Education, and Engineering) were involved in this study. Research literatures addressing variables which were likely to influence performance in early tertiary mathematical study, and variables associated with difficulties in learning mathematics at the transition from upper secondary school to tertiary studies, were reviewed. The first major aim of the study was to identify the variables which were good predictors of first-year mathematics performance at KKU. Results from stepwise multiple regression analyses indicated that the following predictor variables were statistically significant and entered the regression equations for most Faculty groups: School Mathematics Achievement, Self-Esteem, Study Habits in Mathematics, and Faculty of Study. Other predictor variables that sometimes entered regression equations (depending on the Faculty group) were Socio-Economic-Status, Mathematics Language Competence, Mathematics Confidence, Attitude Towards Mathematics, and Gender. Depending on Faculty group, the statistically significant variables accounted for between 11% and 74% of scores on fist-year KKU mathematics examinations. The predictor variables contributed much more to the variance of scores on first-semester mathematics examinations than to the variance of scores on second-semester mathematics examinations. It was also found that scores on the Direct Entry Examination Mathematics test (administered by KKU) and the School Mathematics Achievement test (developed and administered by the author) had stronger correlations with first-year KKU mathematics performance than did scores on the National Entry Examination Mathematics tests (administered by the Thai Ministry of University Affairs). Scores on the three pre-university mathematics achievement test instruments were better predictors of first-semester mathematics performance than of second-semester mathematics performance. It was found that the mean Mathematics Confidence of male students was statistically significantly higher than that of female students, but there were no statistically significant gender differences in Mathematics Misplaced Confidence. Only about 30% of the main sample ( 30% of the male and 30% of the female sample groups) had appropriate confidence in mathematics, that is, they thought their answers were correct when they were, in fact, correct, and they thought they were wrong when they were, in fact, incorrect. So far as Faculty performance differences were concerned, Engineering students had the highest Mathematics Confidence scores, followed by the Medicine and Nursing group of students and the Science and Education group students. Agriculture students had the lowest mean Mathematics Confidence score. No statistically significant differences occurred in Mathematics Misplaced Confidence between different Faculty groups. The second main aim of the study was to investigate why many first-year students experienced difficulties in coping with their mathematics units. A small group of senior secondary mathematics teachers, university mathematics lecturers, and first-year mathematics students were interviewed during the first semester of the 1990-1991 academic year. Interviews were conducted by the author according to a questionnaire format, and were aimed at identifying factors causing difficulty in the transition from senior secondary to university mathematical study. The analysis of the quantitative data together with the interview data indicated that the major sources of difficulty were associated with: (a) students' mathematical abilities; (b) curriculum content; (c) course organisation; (d) students' study habits; (e) instructional styles; and (f) assessment procedures. The results of the investigation are discussed in the light of the relevant literature and related research. The study concludes with recommendations which are addressed to mathematics teachers and education administrators in senior secondary schools in Thailand, to the Thai Ministry of Education, and to the KKU Department of Mathematics.

Developing an ‘as performing’ building assessment

The building profession is increasingly becoming more demanding with respect to building environmental performance. Intentions are to provide best practices into our buildings. In part, this is a response due to the Australian government and other independent organisations that have developed policy on rating tools and performance ranking measures, all with the intention of accomplishing environmentally sustainable buildings.

With rating systems endorsing innovative environmental design solutions, it could be asked: Are our buildings really operating as rated? Do we know whether our designs are in compliance with what was calculated or simulated? Is there a feedback loop informing the design process on successes or failures in our designs or mechanical services?

While ratings continue to focus on ‘by design’ or ‘as built’ rewards, few tools acknowledge perhaps the more crucial bottom line: ‘as performing’. With the exception of an AGBR (Australian Green Building Rating) scheme on actual annual energy consumption, there appears to be no ‘as performing’ assessment. Furthermore, practically every building is a prototype (a one-off) and requires commissioning, programming and scheduling of its services. It would certainly appear that as stakeholders (the procurers, owners, facilities managers and users) of the newly built environment, that what we really want to know is actual on-site confirmation of performance. It is the objective of the Mobile Architecture and Built Environment Laboratory (MABEL), to provide such a service.

Trade liberalisation and economic growth in Fiji : an empirical assessment using the ARDL approach

This study investigates the effect of trade liberalization on economic performance in Fiji using a Cobb-Douglas production function, which is expanded to take into account political instability and trade liberalization. The long run results conform to theoretical expectations, except for the contribution of labour force, which is negatively related to real Gross Domestic Product. We attribute this to the rapid and consistent emigration of skilled labour following the 1987 coups. While human capital was found to be the most influential variable, exports and investment were found to be weakly related to Gross Domestic Product. The key finding is that the dummy variable for signing the IMF agreement in 1984 had a statistically significant positive effect on real Gross Domestic Product in the long run, but the short run effects of signing the agreement as well as the short run and long run effects of implementing the agreement in 1986 were statistically insignificant.

Significance of environment in the assessment of sustainable development : the case for South West Victoria

The assessment of sustainable development is often based on the three pillars of sustainability model or triple bottom line using a set of indicators that evaluate the social, economic and environmental systems. It is thought that by measuring the performance of each system information can be gained about the sustainability of the whole system. However, this represents a disconnect between sustainability theory and the practice of sustainability evaluation as there is no attempt to evaluate if this assumption is true. During the development of a sustainability assessment framework for south west Victoria, Australia, it has become evident that this approach to sustainability assessment does not provide an accurate evaluation of system sustainability. Throughout this project, from stakeholder prioritisation of indicators to final multiple criteria analysis of sustainability, the environmental indicators were found to be the most important for the region’s sustainability. As a consequence, the assessment produced shows that in south west Victoria, sustainability is largely determined by the condition of the environment. This finding highlights the current disconnection between theory and the reality of sustainability. Thus, we discuss a framework for sustainability assessment that attempts to re-connect theory to practice.

Monitoring the performance of a microkernel based operating system and supported applications

The message passing microkernel based operating system is a class of operating system in which the policies are implemented using servers whose co-operation is supported by using a small hardware dependent layer called a microkernel. This thesis addresses the issue of reducing the performance deficiency by documenting the synthesis, development, implementation and assessment of a methodology and mechanisms for monitoring the performance of a message passing microkernel based operating system and supported processes. The methodology has been extensively evaluated to ensure that it can be used successfully to monitor performance.

An examination of the consistency of investigative interviewers' performance

Three original studies were conducted to examine the consistency of investigative-interviewer performance across similar and distinct interview tasks. Overall, the degree of consistency depended on numerous factors, including the nature of the event and interview paradigm, the precision of the assessment measure and whether group versus individual stability was measured. The portfolio highlights and discusses some of the complexities that can arise when conducting a parenting capacity assessment with a patient who has, or is suspected of having, an intellectual disability. Four case studies are presented.

Work readiness in graduate recruitment and selection : a review of current assessment methods

Graduate recruitment and selection differs from other contexts in that graduate applicants generally lack job-related experience. Recent research has highlighted that employers are placing increasing value on graduates being work ready. Work readiness is believed to be indicative of graduate potential in terms of long term job performance and career advancement. A review of the literature has found that current graduate recruitment and selection practices lack the rigour and construct validity to effectively assess work readiness. In addition, the variety of interchangeable terms and definitions articulated by employers and academics on what constitutes work readiness suggests the need to further refine this construct. This paper argues that work readiness is an important selection criterion, and should be examined systematically in the graduate assessment process, as a construct in itself. The ineffectiveness of current assessment methods in being able to measure work readiness supports the need to develop a specific measure of work readiness that will allow more effective decision practices and potentially predict long term job capacity and performance.

The assessment of тт-тт selective stationary phases for two-dimensional HPLC analysis of foods : application to the analysis of coffee

Differences between alkyl, dipole–dipole, hydrogen bonding, and π-π selective surfaces represented by non-resonance and resonance π-stationary phases have been assessed for the separation of ‘Ristretto’ café espresso by employing 2DHPLC techniques with C18 phase selectivity detection. Geometric approach to factor analysis (GAFA) was used to measure the detected peaks (N), spreading angle (β), correlation, practical peak capacity (n_p) and percentage usage of the separations space, as an assessment of selectivity differences between regional quadrants of the two-dimensional separation plane. Although all tested systems were correlated to some degree to the C18 dimension, regional measurement of separation divergence revealed that performance of specific systems was better for certain sample components. The results illustrate that because of the complexity of the ‘real’ sample obtaining a truly orthogonal two-dimensional system for complex samples of natural origin may be practically impossible.

Examination of the stability and consistency of investigative interviewer performance across similar mock interview contexts

Purpose. Three studies examined the degree to which investigative interviewers’ adherence to best-practice guidelines is consistent across similar mock interviews.
Method. In each study, two interviews were administered within a period of several hours. Further, group and individual stability of interviewer performance was analysed, and performance was measured by calculating the proportion of open-ended and leading questions as well as the presence of predetermined problem behaviours. The studies varied depending on the type of interview paradigm employed. Interviewer performance in Study 1was measured in a group context where participants rotated between the role of interviewer, child respondent, and observer. In Study 2, an adult played the role of a child recalling abuse but this occurred in isolation (participants did not observe others or play the child). Study 3 was similar to Study 2 except that in each interview an unfamiliar child aged 5–7 years recalled an innocuous event.
Results. Interviewer performance was relatively stable across tasks, although the strength of the relationship between measures varied across analyses. Improvement in open-ended question usage occurred in Study 1 but not Studies 2 and 3. Irrespective of the assessment context, the dichotomous rating scale yielded greater consistency than when questions were tallied. Further, group stability overestimated individual stability. The practical implications of these findings for trainers and researchers are discussed.