Occasional paper : Towards the development of an online learning environment (OLE) quality management framework

The project is committed to understanding, recognising and developing various forms of institutionally relevant distributed leadership in developing and trialling various components of a quality management framework for online learning environments in Australian higher education. This paper provides an overview of issues related to the management and improvement of quality, including in the context of higher education. In response to the complex and multi-dimensional nature of both quality and online learning environments (OLEs), the concept of a framework for organising policies, procedures and actions relating to the good governance of OLEs can be found in the literature. Such frameworks vary in scope, format and title, and a (non-exhaustive) sample is presented in summary here. Key learnings that can be drawn from the exemplars frameworks and the related literature include:
- the processes for the design of such frameworks;
- the components of such frameworks;
- the measurement mechanisms and metrics employed in such frameworks; and
- the validation of such frameworks.

Performance of the assessment of quality of life measure in people with hip and knee joint disease and implications for research and clinical use

Objective
To comprehensively evaluate the performance of the Assessment of Quality of Life (AQoL) instrument for measuring health-related quality of life (HRQOL) in people with hip and knee joint disease (arthritis or osteoarthritis).

Methods
Data from 237 individuals were available for analysis from a national cross-sectional, population-based study of hip and knee joint disease in Australia. AQoL-4D data were evaluated using Rasch analysis. A range of measurement properties was explored, including model and item fit, threshold ordering, differential item functioning, and targeting.

Results
Good overall fit of the AQoL with the Rasch model was demonstrated across a range of tests, supporting internal validity. Only 1 item (relating to hearing) showed evidence of misfit. Most AQoL items showed logical sequencing of response option categories, with threshold disordering evident for only 2 of the 12 items (items 4 and 9). Minor issues with potential clinical and research implications include limited options for reporting pain and some evidence of measurement bias between demographic subgroups (including age and sex). Participants' HRQOL was generally better than that represented by the AQoL items (mean ± SD for person abilities −2.15 ± 1.39, mean ± SD for item difficulties 0.00 ± 0.67), indicating ceiling effects that could impact the instrument's ability to detect HRQOL improvement in population-based studies.

Conclusion
The AQoL is a competent tool for assessing HRQOL in people with hip and knee joint disease, although researchers and clinicians should consider the caveats identified when selecting appropriate HRQOL measures for future outcome assessment involving this patient group.

Composite yarns fabricated from continuous needleless electrospun nanofibers

In this work, a spinning metal wire collector was employed to continuously collect polyacrylonitrile (PAN) nanofibers produced by a disc fiber generator and coil them around a polyethylene terephthalate (PET) yarn. The obtained composite yarns exhibited a core/shell structure (PET yarn/PAN nanofibers) with nanofibers orderly arranged on the surface of the PET yarn. The electric field analysis showed that the position of metal wire had insignificant effect on the formed electric field and high intensity electric field was formed at the disc circumferential area, which provided a constant electric field for the production of uniform nanofibers. The spinning solution, spinning speed of metal wire, and winding speed were found to play an important role in producing good quality nanofiber yarns, in terms of morphology, strength, and productivity. Pure nanofiber yarns were obtained after dissolving the core yarns in a proper solvent. This method has shown potential for the mass production of nanofiber yarns for industrial applications.

Using hospital standardised mortality ratios to assess quality of care--proceed with extreme caution

Australian Health Ministers have endorsed the hospital standardised mortality ratio (HSMR) as a key indicator of quality and safety, and efforts are currently underway towards its national implementation. In the United Kingdom, Canada, the Netherlands and the United States, the HSMR has been used for several years within organisations to monitor performance and response to various quality and safety programs. In the UK and Canada, the HSMR is also publicly reported and used to compare performance between hospitals. The validity and reliability of the HSMR as a screening tool for distinguishing low-quality from high-quality hospitals remain in doubt, and it has not yet been proven that HSMR reporting necessarily leads to worthwhile improvement in quality of care and patient outcomes.

Ustekinumab for the treatment of moderate to severe psoriasis

This paper presents a summary of the evidence review group (ERG) report into the clinical effectiveness and cost-effectiveness of ustekinumab for the treatment of moderate to severe psoriasis based upon a review of the manufacturer's submission to the National Institute for Health and Clinical Excellence (NICE) as part of the single technology appraisal (STA) process. The submission's main evidence came from three randomised controlled trials (RCTs), of reasonable methodological quality and measuring a range of clinically relevant outcomes. Higher proportions of participants treated with ustekinumab (45 mg and 90 mg) than with placebo or etanercept achieved an improvement on the Psoriasis Area and Severity Index (PASI) of at least 75% (PASI 75) after 12 weeks. There were also statistically significant differences in favour of ustekinumab over placebo for PASI 50 and PASI 90 results, and for ustekinumab over etanercept for PASI 90 results. A weight-based subgroup dosing analysis for each trial was presented, but the methodology was poorly described and no statistical analysis to support the chosen weight threshold was presented. The manufacturer carried out a mixed treatment comparison (MTC); however, the appropriateness of some of the methodological aspects of the MTC is uncertain. The incidence of adverse events was similar between groups at 12 weeks and withdrawals due to adverse events were low and less frequent in the ustekinumab than in the placebo or etanercept groups; however, statistical comparisons were not reported. The manufacturer's economic model of treatments for psoriasis compared ustekinumab with other biological therapies. The model used a reasonable approach; however, it is not clear whether the clinical effectiveness estimates from the subgroup analysis, used in the base-case analysis, were methodologically appropriate. The base-case incremental cost-effectiveness ratio for ustekinumab versus supportive care was 29,587 pounds per quality-adjusted life-year (QALY). In one-way sensitivity analysis the model was most sensitive to the number of hospital days associated with supportive care, the cost estimate for intermittent etanercept 25 mg and the utility scores used. In the ERG's scenario analysis the model was most sensitive to the price of ustekinumab 90 mg, the proportion of patients with baseline weight > 100 kg and the relative risk of intermittent versus continuous etanercept 25 mg. In the ERG's probabilistic sensitivity analysis ustekinumab had the highest probability of being cost-effective at conventional NICE thresholds, assuming the same price for the 45-mg and 90-mg doses; however, doubling the price of ustekinumab 90 mg resulted in ustekinumab no longer dominating the comparators. In conclusion, the clinical effectiveness and cost-effectiveness of ustekinumab in relation to other drugs in this class is uncertain. Provisional NICE guidance issued as a result of the STA states that ustekinumab is recommended as a treatment option for adults with plaque psoriasis when a number of criteria are met. Final guidance is anticipated in September 2009.

Principal led school improvement and teacher capacity building in the Barwon South Network : final report, May 2013

The period of interest for this report is the beginning of 2011 to the end of 2012. The period commenced when the Regional Network Leader of the Barwon South Network of schools in the Barwon South Region of the Department of Education and Early Childhood contacted the School of Education at Deakin University, Waurn Ponds Campus Geelong. The Regional Network Leader outlined a desire to engage with Deakin University to research a short-term-cycle model of school improvement to be implemented in the region. While the model was expected to be taken on by all schools in the region the research was limited to the 23 schools in the Barwon South Network with four schools to be investigated more closely for each of two years (2001 & 2012) – eight focus schools in total.

Many positive outcomes flowed from the implementation of short-term-cycle school improvement plans and their associated practices but there was wide variation in the nature and degrees of success and of the perception of the process. The research team asked the following questions of the data:

1. What aspects of the School Improvement Plan (SIP) approach were important for initiating and supporting worthwhile change?
2. What might we take from this, to provide guidance on how best to support change in teaching and learning processes in schools?

The School Improvement Plan (SIP) worked in a range of ways. At one level it was strongly focused on school leadership, and a need to improve principals’ capacity to initiate worthwhile teaching and learning processes in their schools. Underlying this intent, one might think an assumption is operation is that the leadership process involves top down decision-making and a willingness to hold staff accountable for the quality of their practice.

The second strong focus was on the translation into practice and the consequent effect on student learning, involving an emphasis on data and evidence led practice. Hence, along with the leadership focus there was a demand for the process of school improvement to reach down into students and classrooms. Thus, the SIP process inevitably involved a chain of decision-making by which student learning quality drove the intervention, and teachers responsible for this had a common view. The model therefore should not be seen as an intervention only on the principal, but rather on the school decision-making system and focus. Even though it was the principal receiving the SIP planning template, and reporting to the network, the reporting was required to include description of the operation of the school processes, of classroom processes, and of student learning. This of course placed significant constraints on principals, which may help explain the variation in responses and outcomes described above.

The findings from this study are based on multiple data sources: analysis of both open and closed survey questions which all teachers in the 23 schools in the network were invited to complete; interviews with principals, teachers and leaders in the eight case study schools; some interviews with students in the case study schools; and interviews with leaders who worked in the regional network office; and field notes from network meetings including the celebrations days. Celebrations days occurred each school term when groups of principals came together to share and celebrate the improvements and processes happening in their schools. Many of the themes emerging from the analysis of the different data sources were similar or overlapping, providing some confidence in the evidence-base for the findings.

The study, conducted over two years of data collection and analysis, has demonstrated a range of positive outcomes in at the case study schools relating to school communication and collaboration processes, professional learning of principals, leadership teams and classroom teachers. There was evidence in the survey responses and field notes from ‘celebration days’ that these outcomes were also represented in other schools in the network. The key points of change concerned the leadership processes of planning for improvement, and the rigorous attention to student data in framing teaching and learning processes. This latter point of change had the effect of basing SIP processes on a platform of evidence-based change. The research uncovered considerable anecdotal and observational evidence of improvements in student learning, in teacher accounts in interview, and presentations of student work. Interviews with students, although not as representative as the team would have liked, showed evidence of student awareness of learning goals, a key driver in the SIP improvement model. It was, however, not possible over this timescale to collect objective comparative evidence of enhanced learning outcomes.

A number of features of the short-term-cycle SIP were identified that supported positive change across the network. These were: 1) the support structures represented by the network leader and support personnel within schools, 2) the nature of the SIP model – focusing strongly on change leadership but within a collaborative structure that combined top-down and bottom-up elements, 3) the focus on data-led planning and implementation that helped drill down to explicit elements of classroom practice, and 4) the accountability regimes represented by network leader presence, and the celebration days in which principals became effectively accountable to their peers. We found that in the second year of the project, momentum was lost in the case study schools, as the network was dismantled. This raised issues also for the conduct of research in situations of systemic change.

Alongside the finding of evidence of positive outcomes in the case study schools overall, was the finding that the SIP processes and outcomes varied considerably across schools. A number of contextual factors were identified that led to this variation, including school histories of reform, principal management style, and school size and structure that made the short-term-cycle model unmanageable. In some cases there was overt resistance to the SIP model, at least in some part, and this led to an element of performativity in which the language of the SIP was conscripted to other purposes. The study found that even with functioning schools the SIP was understood differently and the processes performed differently, raising the question of whether in the study we are dealing with one SIP or many. The final take home message from the research is that schools are complex institutions, and models of school improvement need to involve both strong principled features, and flexibility in local application, if all schools’ interests in improving teaching and learning processes and outcomes are to be served.

Improving the quality of prediction intervals through optimal aggregation

Neural networks (NNs) are an effective tool to model nonlinear systems. However, their forecasting performance significantly drops in the presence of process uncertainties and disturbances. NN-based prediction intervals (PIs) offer an alternative solution to appropriately quantify uncertainties and disturbances associated with point forecasts. In this paper, an NN ensemble procedure is proposed to construct quality PIs. A recently developed lower-upper bound estimation method is applied to develop NN-based PIs. Then, constructed PIs from the NN ensemble members are combined using a weighted averaging mechanism. Simulated annealing and a genetic algorithm are used to optimally adjust the weights for the aggregation mechanism. The proposed method is examined for three different case studies. Simulation results reveal that the proposed method improves the average PI quality of individual NNs by 22%, 18%, and 78% for the first, second, and third case studies, respectively. The simulation study also demonstrates that a 3%-4% improvement in the quality of PIs can be achieved using the proposed method compared to the simple averaging aggregation method.