Designing Assessment for Quality Learning

- Considers broad-scale assessment approaches and how they impact on educational opportunity and outcomes. - Brings together internationally recognised scholars providing new insights into assessment for learning improvement and accountability. - Presents different theoretical and methodological perspectives influential in the field of assessment, learning and social change. - Contributes to the theorising of assessment in contexts characterised by heightened accountability requirements and constant change. This book brings together internationally recognised scholars with an interest in how to use the power of assessment to improve student learning and to engage with accountability priorities at both national and global levels. It includes distinguished writers who have worked together for some two decades to shift the assessment paradigm from a dominant focus on assessment as measurement towards assessment as central to efforts to improve learning. These writers have worked with the teaching profession and, in so doing, have researched and generated key insights into different ways of understanding assessment and its relationship to learning. The volume contributes to the theorising of assessment in contexts characterised by heightened accountability requirements and constant change. The book’s structure and content reflect already significant and growing international interest in assessment as contextualised practice, as well as theories of learning and teaching that underpin and drive particular assessment approaches. Learning theories and practices, assessment literacies, teachers’ responsibilities in assessment, the role of leadership, and assessment futures are the organisers within the book’s structure and content. The contributors to this book have in common the view that quality assessment, and quality learning and teaching are integrally related. Another shared view is that the alignment of assessment with curriculum, teaching and learning is linchpin to efforts to improve both learning opportunities and outcomes for all. Essentially, the book presents new perspectives on the enabling power of assessment. In so doing, the writers recognise that validity and reliability - the traditional canons of assessment – remain foundational and therefore necessary. However, they are not of themselves sufficient for quality education. The book argues that assessment needs to be radically reconsidered in the context of unprecedented societal change. Increasingly, communities are segregating more by wealth, with clear signs of social, political, economic and environmental instability. These changes raise important issues relating to ethics and equity, taken to be core dimensions in enabling the power of assessment to contribute to quality learning for all. This book offers readers new knowledge about how assessment can be used to re/engage learners across all phases of education.

Corroborative studies on the effects of sample variation on three macroinvertebrate biological assessment methods at different water quality states

In recent collaborative biological sampling exercises organised by the Nottingham Regional Laboratory of the Severn-Trent Water Authority, the effect of handnet sampling variation on the quality and usefulness of the data obtained has been questioned, especially when this data is transcribed into one or more of the commonly used biological methods of water quality assessment. This study investigates if this effect is constant at sites with similar typography but differing water quality states when the sampling method is standardized and carried out by a single operator. An argument is made for the use of a lowest common denominator approach to give a more consistent result and obviate the effect of sampling variation on these biological assessment methods.

Automated Assessment of Image Quality and Dose Attributes in Clinical CT Images

Computed tomography (CT) is a valuable technology to the healthcare enterprise as evidenced by the more than 70 million CT exams performed every year. As a result, CT has become the largest contributor to population doses amongst all medical imaging modalities that utilize man-made ionizing radiation. Acknowledging the fact that ionizing radiation poses a health risk, there exists the need to strike a balance between diagnostic benefit and radiation dose. Thus, to ensure that CT scanners are optimally used in the clinic, an understanding and characterization of image quality and radiation dose are essential.

The state-of-the-art in both image quality characterization and radiation dose estimation in CT are dependent on phantom based measurements reflective of systems and protocols. For image quality characterization, measurements are performed on inserts imbedded in static phantoms and the results are ascribed to clinical CT images. However, the key objective for image quality assessment should be its quantification in clinical images; that is the only characterization of image quality that clinically matters as it is most directly related to the actual quality of clinical images. Moreover, for dose estimation, phantom based dose metrics, such as CT dose index (CTDI) and size specific dose estimates (SSDE), are measured by the scanner and referenced as an indicator for radiation exposure. However, CTDI and SSDE are surrogates for dose, rather than dose per-se.

Currently there are several software packages that track the CTDI and SSDE associated with individual CT examinations. This is primarily the result of two causes. The first is due to bureaucracies and governments pressuring clinics and hospitals to monitor the radiation exposure to individuals in our society. The second is due to the personal concerns of patients who are curious about the health risks associated with the ionizing radiation exposure they receive as a result of their diagnostic procedures.

An idea that resonates with clinical imaging physicists is that patients come to the clinic to acquire quality images so they can receive a proper diagnosis, not to be exposed to ionizing radiation. Thus, while it is important to monitor the dose to patients undergoing CT examinations, it is equally, if not more important to monitor the image quality of the clinical images generated by the CT scanners throughout the hospital.

The purposes of the work presented in this thesis are threefold: (1) to develop and validate a fully automated technique to measure spatial resolution in clinical CT images, (2) to develop and validate a fully automated technique to measure image contrast in clinical CT images, and (3) to develop a fully automated technique to estimate radiation dose (not surrogates for dose) from a variety of clinical CT protocols.

Measuring the quality of teaching in elementary school classrooms

This paper reviews the means by which teacher quality has been measured. It considers data sources such as students, peers, experts, and examines the psychometrics and scaleproperties of teacher quality assessment instruments with respect to reliability and validity. A list of items for possible inclusion in an elementary student focussed instrument is considered, together with the potential use of such an instrument in measuring teacher quality.

The relationship between clinical outcomes and quality of life for residents of aged care facilities

Objectives It is widely assumed improving care in residential facilities will improve quality of life (QoL), but little research has explored this relationship. The Clinical Care Indicators (CCI) Tool was developed to fill an existing gap in quality assessment within Australian residential aged care facilities and it was used to explore potential links between clinical outcomes and QoL. Design and Setting Clinical outcome and QoL data were collected within four residential facilities from the same aged care provider. Subjects Subjects were 82 residents of four facilities. Outcome Measures Clinical outcomes were measured using the CCI Tool and QoL data was obtained using the Australian WHOQOL‑100. Results Independent t‑test analyses were calculated to compare individual CCIs with each domain of the WHOQOL‑100, while Pearson’s product moment coefficients (r) were calculated between the total number of problem indicators and QoL scores. Significant results suggested poorer clinical outcomes adversely affected QoL. Social and spiritual QoL were particularly affected by clinical outcomes and poorer status in hydration, falls and depression were most strongly associated with lower QoL scores. Poorer clinical status as a whole was also significantly correlated with poorer QoL. Conclusions Hydration, falls and depression were most often associated with poorer resident QoL and as such appear to be key areas for clinical management in residential aged care. However, poor clinical outcomes overall also adversely affected QoL, which suggests maintaining optimum clinical status through high quality nursing care, would not only be important for resident health but also for enhancing general life quality.

Building the toolkit of quality monitoring clinical outcomes as a means of assessing quality in residential care

Residential aged care in Australia does not have a system of quality assessment related to clinical outcomes, creating a significant gap in quality monitoring. Clinical outcomes represent the results of all inputs into care, thus providing an indication of the success of those inputs. To fill this gap, an assessment tool based on resident outcomes (the ResCareQA) was developed and evaluated in collaboration with residential care providers. A useful output of the ResCareQA is a profile of resident clinical status, and this paper will use such outputs to present a snapshot of nine residential facilities. Such comprehensive data has not yet been available within Australia, so this will provide an important insight. ResCareQA data was collected from all residents (N=498) of nine aged care facilities from two major aged care providers. For each facility, numerator–denominator data were calculated to assess the degree of potential clinical problems. Results varied across clinical areas and across facilities, and rank-ordered facility results for selected clinical areas are reviewed and discussed. Use of the ResCareQA to generate clinical outcome data provides a concrete means of monitoring care quality within residential facilities; regular use of the ResCareQA could thus contribute to improved care outcomes within residential aged care.

Development of a Systematic Approach to Assessing Quality within Australian Residential Aged Care Facilities: The Clinical Care Indicators Tool

Recent years have seen the introduction of formalised accreditation processes in both community and residential aged care, but these only partially address quality assessment within this sector. Residential aged care in Australia does not yet have a standardised system of resident assessment related to clinical, rather than administrative, outcomes. This paper describes the development of a quality assessment tool aimed at addressing this gap. Utilising previous research and the results of nominal groups with experts in the field, the 21-item Clinical Care Indicators (CCI) Tool for residential aged care was developed and trialled nationally. The CCI Tool was found to be simple to use and an effective means of collecting data on the state of resident health and care, with potential benefits for resident care planning and continuous quality improvement within facilities and organisations. The CCI Tool was further refined through a small intervention study to assess its utility as a quality improvement instrument and to investigate its relationship with resident quality of life. The current version covers 23 clinical indicators, takes about 30 minutes to complete and is viewed favourably by nursing staff who use it. Current work focuses on psychometric analysis and benchmarking, which should enable the CCI Tool to make a positive contribution to the measurement of quality in aged care in Australia.

A dog of a QCAT : collateral effects of mandated English assessment in the Torres Strait

This paper critiques a 2008 Queensland Studies Authority (QSA) assessment initiative known as Queensland Comparable Assessment Tasks, or QCATs. The rhetoric is that these centrally devised assessment tasks will provide information about how well students can apply what they know, understand and can do in different contexts (QSA, 2009). The QCATs are described as ‘authentic, performance-based assessment’ that involves a ‘meaningful problem’, ‘emphasises critical thinking and reasoning’ and ‘provides students with every opportunity to do their best work’ (QSA, 2009). From my viewpoint as a teacher, I detail my professional concerns with implementing the 2008 middle primary English QCAT in one case study Torres Strait Island community. Specifically I ask ‘QCATs: Comparable with what?’ and ‘QCATs: Whose authentic assessment?’. I predict the possible collateral effects of implementing this English assessment in this remote Indigenous community, concluding, rather than being an example of quality assessment, colloquially speaking, it is nothing more than a ‘dog’.

Predicting dry eye using non-invasive techniques of tear film surface assessment

PURPOSE. To measure tear film surface quality in healthy and dry eye subjects using three noninvasive techniques of tear film quality assessment and to establish the ability of these noninvasive techniques to predict dry eye. METHODS. Thirty four subjects participated in the study, and were classified as dry eye or normal, based on standard clinical assessments. Three non-invasive techniques were applied for measurement of tear film surface quality: dynamic-area high-speed videokeratoscopy (HSV), wavefront sensing (DWS) and lateral shearing interferometry (LSI). The measurements were performed in both natural blinking conditions (NBC) and in suppressed blinking conditions (SBC). RESULTS. In order to investigate the capability of each method to discriminate dry eye subjects from normal subjects, the receiver operating curve (ROC) was calculated and then the area under the curve (AUC) was extracted. The best result was obtained for the LSI technique (AUC=0.80 in SBC and AUC=0.73 in NBC), which was followed by HSV (AUC=0.72 in SBC and AUC=0.71 in NBC). The best result for DWS was AUC=0.64 obtained for changes in vertical coma in suppressed blinking conditions, while for normal blinking conditions the results were poorer. CONCLUSIONS. Non-invasive techniques of tear film surface assessment can be used for predicting dry eye and this can be achieved in natural blinking as well as suppressed blinking conditions. In this study, LSI showed the best detection performance, closely followed by the dynamic-area HSV. The wavefront sensing technique was less powerful, particularly in natural blinking conditions.

Benchmarking clinical indicators of quality for Australian residential aged care facilities

To undertake exploratory benchmarking of a set of clinical indicators of quality care in residential care in Australia, data were collected from 107 residents within four medium-sized facilities (40–80 beds) in Brisbane, Australia. The proportion of residents in each sample facility with a particular clinical problem was compared with US Minimum Data Set quality indicator thresholds. Results demonstrated variability within and between clinical indicators, suggesting breadth of assessment using various clinical indicators of quality is an important factor when monitoring quality of care. More comprehensive and objective measures of quality of care would be of great assistance in determining and monitoring the effectiveness of residential aged care provision in Australia, particularly as demands for accountability by consumers and their families increase. What is known about the topic? The key to quality improvement is effective quality assessment, and one means of evaluating quality of care is through clinical outcomes. The Minimum Data Set quality indicators have been credited with improving quality in United States nursing homes. What does this paper add? The Clinical Care Indicators Tool was used to collect data on clinical outcomes, enabling comparison of data from a small Australian sample with American quality benchmarks to illustrate the utility of providing guidelines for interpretation. What are the implications for practitioners? Collecting and comparing clinical outcome data would enable practitioners to better understand the quality of care being provided and whether practices required review. The Clinical Care Indicator Tool could provide a comprehensive and systematic means of doing this, thus filling a gap in quality monitoring within Australian residential aged care.

Clinical outcomes in residential care : setting benchmarks for quality

Aim Australian residential aged care does not have a system of quality assessment related to clinical outcomes, or comprehensive quality benchmarking. The Residential Care Quality Assessment was developed to fill this gap; and this paper discusses the process by which preliminary benchmarks representing high and low quality were developed for it. Methods Data were collected from all residents (n = 498) of nine facilities. Numerator–denominator analysis of clinical outcomes occurred at a facility-level, with rank-ordered results circulated to an expert panel. The panel identified threshold scores to indicate excellent and questionable care quality, and refined these through Delphi process. Results Clinical outcomes varied both within and between facilities; agreed thresholds for excellent and poor outcomes were finalised after three Delphi rounds. Conclusion Use of the Residential Care Quality Assessment provides a concrete means of monitoring care quality and allows benchmarking across facilities; its regular use could contribute to improved care outcomes within residential aged care in Australia.

Clinical indicators of quality for Australian residential aged care facilities : establishing reliability, validity, and quality thresholds

Background: In response to the need for more comprehensive quality assessment within Australian residential aged care facilities, the Clinical Care Indicator (CCI) Tool was developed to collect outcome data as a means of making inferences about quality. A national trial of its effectiveness and a Brisbane-based trial of its use within the quality improvement context determined the CCI Tool represented a potentially valuable addition to the Australian aged care system. This document describes the next phase in the CCI Tool.s development; the aims of which were to establish validity and reliability of the CCI Tool, and to develop quality indicator thresholds (benchmarks) for use in Australia. The CCI Tool is now known as the ResCareQA (Residential Care Quality Assessment). Methods: The study aims were achieved through a combination of quantitative data analysis, and expert panel consultations using modified Delphi process. The expert panel consisted of experienced aged care clinicians, managers, and academics; they were initially consulted to determine face and content validity of the ResCareQA, and later to develop thresholds of quality. To analyse its psychometric properties, ResCareQA forms were completed for all residents (N=498) of nine aged care facilities throughout Queensland. Kappa statistics were used to assess inter-rater and test-retest reliability, and Cronbach.s alpha coefficient calculated to determine internal consistency. For concurrent validity, equivalent items on the ResCareQA and the Resident Classification Scales (RCS) were compared using Spearman.s rank order correlations, while discriminative validity was assessed using known-groups technique, comparing ResCareQA results between groups with differing care needs, as well as between male and female residents. Rank-ordered facility results for each clinical care indicator (CCI) were circulated to the panel; upper and lower thresholds for each CCI were nominated by panel members and refined through a Delphi process. These thresholds indicate excellent care at one extreme and questionable care at the other. Results: Minor modifications were made to the assessment, and it was renamed the ResCareQA. Agreement on its content was reached after two Delphi rounds; the final version contains 24 questions across four domains, enabling generation of 36 CCIs. Both test-retest and inter-rater reliability were sound with median kappa values of 0.74 (test-retest) and 0.91 (inter-rater); internal consistency was not as strong, with a Chronbach.s alpha of 0.46. Because the ResCareQA does not provide a single combined score, comparisons for concurrent validity were made with the RCS on an item by item basis, with most resultant correlations being quite low. Discriminative validity analyses, however, revealed highly significant differences in total number of CCIs between high care and low care groups (t199=10.77, p=0.000), while the differences between male and female residents were not significant (t414=0.56, p=0.58). Clinical outcomes varied both within and between facilities; agreed upper and lower thresholds were finalised after three Delphi rounds. Conclusions: The ResCareQA provides a comprehensive, easily administered means of monitoring quality in residential aged care facilities that can be reliably used on multiple occasions. The relatively modest internal consistency score was likely due to the multi-factorial nature of quality, and the absence of an aggregate result for the assessment. Measurement of concurrent validity proved difficult in the absence of a gold standard, but the sound discriminative validity results suggest that the ResCareQA has acceptable validity and could be confidently used as an indication of care quality within Australian residential aged care facilities. The thresholds, while preliminary due to small sample size, enable users to make judgements about quality within and between facilities. Thus it is recommended the ResCareQA be adopted for wider use.

A data mining approach to improve the automated quality of data

This thesis describes the development of a robust and novel prototype to address the data quality problems that relate to the dimension of outlier data. It thoroughly investigates the associated problems with regards to detecting, assessing and determining the severity of the problem of outlier data; and proposes granule-mining based alternative techniques to significantly improve the effectiveness of mining and assessing outlier data.

Assessment for Education : Standards, Judgement and Moderation

Do you need a practical guide to assessment, curriculum and policy? Are you also looking for a book that is firmly grounded in the theory of this subject? Assessment for Education combines both theory and practice, making it the perfect guide for students, researchers, academics and teachers. This book makes assessment processes transparent for practitioners, and shows how assessment should relate to education. It looks at evidence-informed decision-making and the interrelationships between standards, judgment and moderation practice for improved assessment, teacher quality, schools and systems. The book will provide you with: ' Knowledge about quality assessment and judgement practice ' Understanding of relationships across curriculum, assessment, teaching and learning ' Knowledge of the concept of front-ending assessment based on the learner's needs ' An analysis of practitioner judgement approaches ' Understanding of the conditions under which teacher assessment can be valid ' Principles derived from research of social moderation practices Whether you are studying and researching assessment or working in curriculum and assessment policy, this book will show you how practitioner use of achievement standards can improve learning, equity, social justice and accountability.

The Mobile Application Rating Scale (MARS): A new tool to measure the quality of health mobile applications

Smartphone technology provides free or inexpensive access to mental health and wellbeing resources. As a result the use of mobile applications for these purposes has increased significantly in recent years. Yet, there is currently no app quality assessment alternative to the popular ‘star’-ratings, which are often unreliable. This presentation describes the development of the Mobile Application Rating Scale (MARS) a new measure for classifying and rating the quality of mobile applications. A review of existing literature on app and web quality identified 25 published papers, conference proceedings, and online resources (published since 1999), which identified 372 explicit quality criteria. Qualitative analysis identified five broad categories of app quality rating criteria: engagement, functionality, aesthetics, information quality, and overall satisfaction, which were refined into the 23-item MARS. Independent ratings of 50 randomly selected mental health and wellbeing mobile apps indicated the MARS had excellent levels of internal consistency (α = 0.92) and inter-rater reliability (ICC = 0.85). The MARS provides practitioners and researchers with an easy-to-use, simple, objective and reliable tool for assessing mobile app quality. It also provides mHealth professionals with a checklist for the design and development of high quality apps.