A known-groups evaluation of the Response Bias Scale in a neuropsychological setting

We evaluated the Minnesota Multiphasic Personality Inventory-Second Edition (MMPI-2) Response Bias Scale (RBS). Archival data from 83 individuals who were referred for neuropsychological assessment with no formal diagnosis (n = 10), following a known or suspected traumatic brain injury (n = 36), with a psychiatric diagnosis (n = 20), or with a history of both trauma and a psychiatric condition (n = 17) were retrieved. The criteria for malingered neurocognitive dysfunction (MNCD) were applied, and two groups of participants were formed: poor effort (n = 15) and genuine responders (n = 68). Consistent with previous studies, the difference in scores between groups was greatest for the RBS (d = 2.44), followed by two established MMPI-2 validity scales, F (d = 0.25) and K (d = 0.23), and strong significant correlations were found between RBS and F (rs = .48) and RBS and K (r = −.41). When MNCD group membership was predicted using logistic regression, the RBS failed to add incrementally to F. In a separate regression to predict group membership, K added significantly to the RBS. Receiver-operating curve analysis revealed a nonsignificant area under the curve statistic, and at the ideal cutoff in this sample of >12, specificity was moderate (.79), sensitivity was low (.47), and positive and negative predictive power values at a 13% base rate were .25 and .91, respectively. Although the results of this study require replication because of a number of limitations, this study has made an important first attempt to report RBS classification accuracy statistics for predicting poor effort at a range of base rates.

A fuzzy TOPSIS based multi criteria performance measurement model for lean supply chain

Increasing global competition, rapid technological changes, advances in manufacturing and information technology and discerning customers are forcing supply chains to adopt improvement practices that enable them to deliver high quality products at a lower cost and in a shorter period of time. A lean initiative is one of the most effective approaches toward achieving this goal. In the lean improvement process, it is critical to measure current and desired performance level in order to clearly evaluate the lean implementation efforts. Many attempts have tried to measure supply chain performance incorporating both quantitative and qualitative measures but failed to provide an effective method of measuring improvements in performances for dynamic lean supply chain situations. Therefore, the necessity of appropriate measurement of lean supply chain performance has become imperative. There are many lean tools available for supply chains; however, effectiveness of a lean tool depends on the type of the product and supply chain. One tool may be highly effective for a supply chain involved in high volume products but may not be effective for low volume products. There is currently no systematic methodology available for selecting appropriate lean strategies based on the type of supply chain and market strategy This thesis develops an effective method to measure the performance of supply chain consisting of both quantitative and qualitative metrics and investigates the effects of product types and lean tool selection on the supply chain performance Supply chain performance matrices and the effects of various lean tools over performance metrics mentioned in the SCOR framework have been investigated. A lean supply chain model based on the SCOR metric framework is then developed where non- lean and lean as well as quantitative and qualitative metrics are incorporated in appropriate metrics. The values of appropriate metrics are converted into triangular fuzzy numbers using similarity rules and heuristic methods. Data have been collected from an apparel manufacturing company for multiple supply chain products and then a fuzzy based method is applied to measure the performance improvements in supply chains. Using the fuzzy TOPSIS method, which chooses an optimum alternative to maximise similarities with positive ideal solutions and to minimise similarities with negative ideal solutions, the performances of lean and non- lean supply chain situations for three different apparel products have been evaluated. To address the research questions related to effective performance evaluation method and the effects of lean tools over different types of supply chains; a conceptual framework and two hypotheses are investigated. Empirical results show that implementation of lean tools have significant effects over performance improvements in terms of time, quality and flexibility. Fuzzy TOPSIS based method developed is able to integrate multiple supply chain matrices onto a single performance measure while lean supply chain model incorporates qualitative and quantitative metrics. It can therefore effectively measure the improvements for supply chain after implementing lean tools. It is demonstrated that product types involved in the supply chain and ability to select right lean tools have significant effect on lean supply chain performance. Future study can conduct multiple case studies in different contexts.

Design, implementation and multisite evaluation of a system suitability protocol for the quantitative assessment of instrument performance in liquid chromatography-multiple reaction monitoring-MS (LC-MRM-MS)

Multiple reaction monitoring (MRM) mass spectrometry coupled with stable isotope dilution (SID) and liquid chromatography (LC) is increasingly used in biological and clinical studies for precise and reproducible quantification of peptides and proteins in complex sample matrices. Robust LC-SID-MRM-MS-based assays that can be replicated across laboratories and ultimately in clinical laboratory settings require standardized protocols to demonstrate that the analysis platforms are performing adequately. We developed a system suitability protocol (SSP), which employs a predigested mixture of six proteins, to facilitate performance evaluation of LC-SID-MRM-MS instrument platforms, configured with nanoflow-LC systems interfaced to triple quadrupole mass spectrometers. The SSP was designed for use with low multiplex analyses as well as high multiplex approaches when software-driven scheduling of data acquisition is required. Performance was assessed by monitoring of a range of chromatographic and mass spectrometric metrics including peak width, chromatographic resolution, peak capacity, and the variability in peak area and analyte retention time (RT) stability. The SSP, which was evaluated in 11 laboratories on a total of 15 different instruments, enabled early diagnoses of LC and MS anomalies that indicated suboptimal LC-MRM-MS performance. The observed range in variation of each of the metrics scrutinized serves to define the criteria for optimized LC-SID-MRM-MS platforms for routine use, with pass/fail criteria for system suitability performance measures defined as peak area coefficient of variation <0.15, peak width coefficient of variation <0.15, standard deviation of RT <0.15 min (9 s), and the RT drift <0.5min (30 s). The deleterious effect of a marginally performing LC-SID-MRM-MS system on the limit of quantification (LOQ) in targeted quantitative assays illustrates the use and need for a SSP to establish robust and reliable system performance. Use of a SSP helps to ensure that analyte quantification measurements can be replicated with good precision within and across multiple laboratories and should facilitate more widespread use of MRM-MS technology by the basic biomedical and clinical laboratory research communities.

On evaluation of robust autonomy for uninhabited vehicles and systems

This paper discusses a method to quantify robust autonomy of Uninhabited Vehicles and Systems (UVS) in aerospace, marine, or land applications. Based on mission-vehicle specific performance criteria, we define an system utility function that can be evaluated using simulation scenarios for an envelope of environmental conditions. The results of these evaluations are used to compute a figure of merit or measure for operational efectiveness (MOE). The procedure is then augmented to consider faults and the performance of mechanisms to handle these faulty operational modes. This leads to a measure of robust autonomy (MRA). The objective of the proposed figures of merit is to assist in decision making about vehicle performance and reliability at both vehicle development stage (using simulation models) and at certification stage (using hardware-in-the-loop testing). Performance indices based on dynamic and geometric tasks associated with vehicle manoeuvring problems are proposed, and an example of a two- dimensional y scenario is provided to illustrate the use of the proposed figures of merit.

Improving criteria and feedback in student assessment in law

Spurred on by both the 1987 Pearce Report1 and the general changes to higher education spawned by the “Dawkins revolution” from 1988, there has been much critical self-evaluation leading to profound improvements to the quality of teaching in Australian law schools.2 Despite the changes there are still areas of general law teaching practice which have lagged behind recent developments in our understanding of what constitutes high quality teaching. One such area is assessment criteria and feedback. The project Improving Feedback in Student Assessment in Law is an attempt to remedy this. It aims to produce a manual containing key principles for the design of assessment and the provision of feedback, with practical yet flexible ideas and illustrations which law teachers may adopt or modify. Most of the examples have been developed by teachers at the University of Melbourne Law School. The project was supported in 1996 by a Committee for the Advancement of University Teaching grant and the manual will be published late in 1997.3 This note summarises the core principles which are elaborated further in the manual.

Design, construction and evaluation of systems to predict risk in obstetrics

We present a systematic, practical approach to developing risk prediction systems, suitable for use with large databases of medical information. An important part of this approach is a novel feature selection algorithm which uses the area under the receiver operating characteristic (ROC) curve to measure the expected discriminative power of different sets of predictor variables. We describe this algorithm and use it to select variables to predict risk of a specific adverse pregnancy outcome: failure to progress in labour. Neural network, logistic regression and hierarchical Bayesian risk prediction models are constructed, all of which achieve close to the limit of performance attainable on this prediction task. We show that better prediction performance requires more discriminative clinical information rather than improved modelling techniques. It is also shown that better diagnostic criteria in clinical records would greatly assist the development of systems to predict risk in pregnancy. We present a systematic, practical approach to developing risk prediction systems, suitable for use with large databases of medical information. An important part of this approach is a novel feature selection algorithm which uses the area under the receiver operating characteristic (ROC) curve to measure the expected discriminative power of different sets of predictor variables. We describe this algorithm and use it to select variables to predict risk of a specific adverse pregnancy outcome: failure to progress in labour. Neural network, logistic regression and hierarchical Bayesian risk prediction models are constructed, all of which achieve close to the limit of performance attainable on this prediction task. We show that better prediction performance requires more discriminative clinical information rather than improved modelling techniques. It is also shown that better diagnostic criteria in clinical records would greatly assist the development of systems to predict risk in pregnancy.

A critical evaluation of written discharge advice for people with mild traumatic brain injury : what should we be looking for?

Objective: To formally evaluate the written discharge advice for people with mild traumatic brain injury (mTBI). Methods: Eleven publications met the inclusion criteria: (1) intended for adults; (2) ≤two A4 pages; (3) published in English; (4) freely accessible; and (5) currently used (or suitable for use) in Australian hospital emergency departments or similar settings. Two independent raters evaluated the content and style of each publication against established standards. The readability of the publication, the diagnostic term(s) contained in it and a modified Patient Literature Usefulness Index (mPLUI) were also evaluated. Results: The mean content score was 19.18 ± 8.53 (maximum = 31) and the mean style score was 6.8 ± 1.34 (maximum = 8). The mean Flesch-Kincaid reading ease score was 66.42 ± 4.3. The mean mPLUI score was 65.86 ± 14.97 (maximum = 100). Higher scores on these metrics indicate more desirable properties. Over 80% of the publications used mixed diagnostic terminology. One publication scored optimally on two of the four metrics and highly on the others. Discussion: The content, style, readability and usefulness of written mTBI discharge advice was highly variable. The provision of written information to patients with mTBI is advised, but this variability in materials highlights the need for evaluation before distribution. Areas are identified to guide the improvement of written mTBI discharge advice.

Infrastructure, investment and development evaluation: A Complex Stakeholder Perception Mapping approach for improving local and regional economic development outcomes

Partial evaluation of infrastructure investments have resulted in expensive mistakes, unsatisfactory outcomes and increased uncertainties for too many stakeholders, communities and economies in both developing and developed nations. "Complex Stakeholder Perception Mapping" (CSPM), is a novel approach that can address existing limitations by inclusively framing, capturing and mapping the spectrum of insights and perceptions using extended Geographic Information Systems. Maps generated in CSPM offer presentations of flexibly combined, complex perceptions of stakeholders on multiple aspects of development. CSPM extends the applications of GIS software in non-spatial mapping and of Multi-Criteria Analysis with a multidimensional evaluation platform and augments decision science capabilities in addressing complexities. Application of CSPM can improve local and regional economic gains from infrastructure projects and aid any multi-objective and multi-stakeholder decision situations.

Review and evaluation of mindfulness-based iPhone apps

Background There is growing evidence for the positive impact of mindfulness on wellbeing. Mindfulness-based mobile apps may have potential as an alternative delivery medium for training. While there are hundreds of such apps, there is little information on their quality. Objective This study aimed to conduct a systematic review of mindfulness-based iPhone mobile apps and to evaluate their quality using a recently-developed expert rating scale, the Mobile Application Rating Scale (MARS). It also aimed to describe features of selected high-quality mindfulness apps. Methods A search for “mindfulness” was conducted in iTunes and Google Apps Marketplace. Apps that provided mindfulness training and education were included. Those containing only reminders, timers or guided meditation tracks were excluded. An expert rater reviewed and rated app quality using the MARS engagement, functionality, visual aesthetics, information quality and subjective quality subscales. A second rater provided MARS ratings on 30% of the apps for inter-rater reliability purposes. Results The “mindfulness” search identified 700 apps. However, 94 were duplicates, 6 were not accessible and 40 were not in English. Of the remaining 560, 23 apps met inclusion criteria and were reviewed. The median MARS score was 3.2 (out of 5.0), which exceeded the minimum acceptable score (3.0). The Headspace app had the highest average score (4.0), followed by Smiling Mind (3.7), iMindfulness (3.5) and Mindfulness Daily (3.5). There was a high level of inter-rater reliability between the two MARS raters. Conclusions Though many apps claim to be mindfulness-related, most were guided meditation apps, timers, or reminders. Very few had high ratings on the MARS subscales of visual aesthetics, engagement, functionality or information quality. Little evidence is available on the efficacy of the apps in developing mindfulness.

Educational interventions for preventing vascular catheter bloodstream infections in critical care: Evidence map, systematic review and economic evaluation

Background Bloodstream infections resulting from intravascular catheters (catheter-BSI) in critical care increase patients' length of stay, morbidity and mortality, and the management of these infections and their complications has been estimated to cost the NHS annually £19.1–36.2M. Catheter-BSI are thought to be largely preventable using educational interventions, but guidance as to which types of intervention might be most clinically effective is lacking. Objective To assess the effectiveness and cost-effectiveness of educational interventions for preventing catheter-BSI in critical care units in England. Data sources Sixteen electronic bibliographic databases – including MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, Cumulative Index to Nursing and Allied Health Literature (CINAHL), NHS Economic Evaluation Database (NHS EED), EMBASE and The Cochrane Library databases – were searched from database inception to February 2011, with searches updated in March 2012. Bibliographies of systematic reviews and related papers were screened and experts contacted to identify any additional references. Review methods References were screened independently by two reviewers using a priori selection criteria. A descriptive map was created to summarise the characteristics of relevant studies. Further selection criteria developed in consultation with the project Advisory Group were used to prioritise a subset of studies relevant to NHS practice and policy for systematic review. A decision-analytic economic model was developed to investigate the cost-effectiveness of educational interventions for preventing catheter-BSI. Results Seventy-four studies were included in the descriptive map, of which 24 were prioritised for systematic review. Studies have predominantly been conducted in the USA, using single-cohort before-and-after study designs. Diverse types of educational intervention appear effective at reducing the incidence density of catheter-BSI (risk ratios statistically significantly < 1.0), but single lectures were not effective. The economic model showed that implementing an educational intervention in critical care units in England would be cost-effective and potentially cost-saving, with incremental cost-effectiveness ratios under worst-case sensitivity analyses of < £5000/quality-adjusted life-year. Limitations Low-quality primary studies cannot definitively prove that the planned interventions were responsible for observed changes in catheter-BSI incidence. Poor reporting gave unclear estimates of risk of bias. Some model parameters were sourced from other locations owing to a lack of UK data. Conclusions Our results suggest that it would be cost-effective and may be cost-saving for the NHS to implement educational interventions in critical care units. However, more robust primary studies are needed to exclude the possible influence of secular trends on observed reductions in catheter-BSI.

Manufacturing and use of human placenta-derived mesenchymal stromal cells for phase I clinical trials: Establishment and evaluation of a protocol

Background/Aim. Mesenchymal stromal cells (MSCs) have been utilised in many clinical trials as an experimental treatment in numerous clinical settings. Bone marrow remains the traditional source tissue for MSCs but is relatively hard to access in large volumes. Alternatively, MSCs may be derived from other tissues including the placenta and adipose tissue. In an initial study no obvious differences in parameters such as cell surface phenotype, chemokine receptor display, mesodermal differentiation capacity or immunosuppressive ability, were detected when we compared human marrow derived- MSCs to human placenta-derived MSCs. The aim of this study was to establish and evaluate a protocol and related processes for preparation placenta-derived MSCs for early phase clinical trials. Methods. A full-term placenta was taken after delivery of the baby as a source of MSCs. Isolation, seeding, incubation, cryopreservation of human placentaderived MSCs and used production release criteria were in accordance with the complex regulatory requirements applicable to Code of Good Manufacturing Practice manufacturing of ex vivo expanded cells. Results. We established and evaluated instructions for MSCs preparation protocol and gave an overview of the three clinical areas application. In the first trial, MSCs were co-transplanted iv to patient receiving an allogeneic cord blood transplant as therapy for treatmentrefractory acute myeloid leukemia. In the second trial, MSCs were administered iv in the treatment of idiopathic pulmonary fibrosis and without serious adverse effects. In the third trial, MSCs were injected directly into the site of tendon damage using ultrasound guidance in the treatment of chronic refractory tendinopathy. Conclusion. Clinical trials using both allogeneic and autologous cells demonstrated MSCs to be safe. A described protocol for human placenta-derived MSCs is appropriate for use in a clinical setting, relatively inexpensive and can be relatively easily adjusted to a different set of regulatory requirements, as applicable to early phase clinical trials.

Improving SEA procedures within Italian regions: Towards a more effective evaluation of urban sustainability

Sustainable urban development, a major issue at global scale, will become more relevant according to population growth predictions in developed and developing countries. Societal and international recognition of sustainability concerns led to the development of specific tools and procedures, known as sustainability assessments/appraisals (SA). Their effectiveness however, considering that global quality life indicators have worsened since their introduction, has promoted a re-thinking of SA instruments. More precisely, Strategic Environmental Assessment (SEA), – a tool introduced in the European context to evaluate policies, plans, and programmes (PPPs), – is being reconsidered because of several features that seem to limit its effectiveness. Over time, SEA has evolved in response to external and internal factors dealing with technical, procedural, planning and governance systems thus involving a shift of paradigm from EIA-based SEAs (first generation protocols) towards more integrated approaches (second generation ones). Changes affecting SEA are formalised through legislation in each Member State, to guide institutions at regional and local level. Defining SEA effectiveness is quite difficult. Its’ capacity-building process appears quite far from its conclusion, even if any definitive version can be conceptualized. In this paper, we consider some European nations with different planning systems and SA traditions. After the identification of some analytical criteria, a multi-dimensional cluster analysis is developed on some case studies, to outline current weaknesses.

Improving SEA prodedures within Italian regions: Towards a more effective evaluation of urban sustainability

Sustainable urban development, a major issue at global scale, will become more relevant according to population growth predictions in developed and developing countries. Societal and international recognition of sustainability concerns led to the development of specific tools and procedures, known as sustainability assessments/appraisals (SA). Their effectiveness however, considering that global quality life indicators have worsened since their introduction, has promoted a re-thinking of SA instruments. More precisely, Strategic Environmental Assessment (SEA), – a tool introduced in the European context to evaluate policies, plans, and programmes (PPPs), – is being reconsidered because of several features that seem to limit its effectiveness. Over time, SEA has evolved in response to external and internal factors dealing with technical, procedural, planning and governance systems thus involving a shift of paradigm from EIA-based SEAs (first generation protocols) towards more integrated approaches (second generation ones). Changes affecting SEA are formalised through legislation in each Member State, to guide institutions at regional and local level. Defining SEA effectiveness is quite difficult. Its’ capacity-building process appears quite far from its conclusion, even if any definitive version can be conceptualized. In this paper, we consider some European nations with different planning systems and SA traditions. After the identification of some analytical criteria, a multi-dimensional cluster analysis is developed on some case studies, to outline current weaknesses.

Evaluation of Two MCDM Methods for UAV Emergency Landing

One of the major impediments for the use of UAVs in civilian environment is the capability to replicate some of the functionality of safe manned aircraft operations. One critical aspect is emergency landing. Once the possible landing sites have been rated, a decision on the most suitable choice to land is required. This is a multi-criteria decision making (MCDM) problem which needs to take into account various factors in its selection of landing site. This report summarises relevant literature in MCDM in the context of emergency forced landing and proposes and compares two algorithms and methods for this task.

The clinical effectiveness and cost-effectiveness of exercise referral schemes: A systematic review and economic evaluation

- Background Exercise referral schemes (ERS) aim to identify inactive adults in the primary-care setting. The GP or health-care professional then refers the patient to a third-party service, with this service taking responsibility for prescribing and monitoring an exercise programme tailored to the needs of the individual. - Objective To assess the clinical effectiveness and cost-effectiveness of ERS for people with a diagnosed medical condition known to benefit from physical activity (PA). The scope of this report was broadened to consider individuals without a diagnosed condition who are sedentary. - Data sources MEDLINE; EMBASE; PsycINFO; The Cochrane Library, ISI Web of Science; SPORTDiscus and ongoing trial registries were searched (from 1990 to October 2009) and included study references were checked. - Methods Systematic reviews: the effectiveness of ERS, predictors of ERS uptake and adherence, and the cost-effectiveness of ERS; and the development of a decision-analytic economic model to assess cost-effectiveness of ERS. - Results Seven randomised controlled trials (UK, n = 5; non-UK, n = 2) met the effectiveness inclusion criteria, five comparing ERS with usual care, two compared ERS with an alternative PA intervention, and one to an ERS plus a self-determination theory (SDT) intervention. In intention-to-treat analysis, compared with usual care, there was weak evidence of an increase in the number of ERS participants who achieved a self-reported 90-150 minutes of at least moderate-intensity PA per week at 6-12 months' follow-up [pooled relative risk (RR) 1.11, 95% confidence interval 0.99 to 1.25]. There was no consistent evidence of a difference between ERS and usual care in the duration of moderate/vigorous intensity and total PA or other outcomes, for example physical fitness, serum lipids, health-related quality of life (HRQoL). There was no between-group difference in outcomes between ERS and alternative PA interventions or ERS plus a SDT intervention. None of the included trials separately reported outcomes in individuals with medical diagnoses. Fourteen observational studies and five randomised controlled trials provided a numerical assessment of ERS uptake and adherence (UK, n = 16; non-UK, n = 3). Women and older people were more likely to take up ERS but women, when compared with men, were less likely to adhere. The four previous economic evaluations identified suggest ERS to be a cost-effective intervention. Indicative incremental cost per quality-adjusted life-year (QALY) estimates for ERS for various scenarios were based on a de novo model-based economic evaluation. Compared with usual care, the mean incremental cost for ERS was £169 and the mean incremental QALY was 0.008, with the base-case incremental cost-effectiveness ratio at £20,876 per QALY in sedentary people without a medical condition and a cost per QALY of £14,618 in sedentary obese individuals, £12,834 in sedentary hypertensive patients, and £8414 for sedentary individuals with depression. Estimates of cost-effectiveness were highly sensitive to plausible variations in the RR for change in PA and cost of ERS. - Limitations We found very limited evidence of the effectiveness of ERS. The estimates of the cost-effectiveness of ERS are based on a simple analytical framework. The economic evaluation reports small differences in costs and effects, and findings highlight the wide range of uncertainty associated with the estimates of effectiveness and the impact of effectiveness on HRQoL. No data were identified as part of the effectiveness review to allow for adjustment of the effect of ERS in different populations. - Conclusions There remains considerable uncertainty as to the effectiveness of ERS for increasing activity, fitness or health indicators or whether they are an efficient use of resources in sedentary people without a medical diagnosis. We failed to identify any trial-based evidence of the effectiveness of ERS in those with a medical diagnosis. Future work should include randomised controlled trials assessing the cinical effectiveness and cost-effectivenesss of ERS in disease groups that may benefit from PA. - Funding The National Institute for Health Research Health Technology Assessment programme.