Design, implementation and multisite evaluation of a system suitability protocol for the quantitative assessment of instrument performance in liquid chromatography-multiple reaction monitoring-MS (LC-MRM-MS)

Multiple reaction monitoring (MRM) mass spectrometry coupled with stable isotope dilution (SID) and liquid chromatography (LC) is increasingly used in biological and clinical studies for precise and reproducible quantification of peptides and proteins in complex sample matrices. Robust LC-SID-MRM-MS-based assays that can be replicated across laboratories and ultimately in clinical laboratory settings require standardized protocols to demonstrate that the analysis platforms are performing adequately. We developed a system suitability protocol (SSP), which employs a predigested mixture of six proteins, to facilitate performance evaluation of LC-SID-MRM-MS instrument platforms, configured with nanoflow-LC systems interfaced to triple quadrupole mass spectrometers. The SSP was designed for use with low multiplex analyses as well as high multiplex approaches when software-driven scheduling of data acquisition is required. Performance was assessed by monitoring of a range of chromatographic and mass spectrometric metrics including peak width, chromatographic resolution, peak capacity, and the variability in peak area and analyte retention time (RT) stability. The SSP, which was evaluated in 11 laboratories on a total of 15 different instruments, enabled early diagnoses of LC and MS anomalies that indicated suboptimal LC-MRM-MS performance. The observed range in variation of each of the metrics scrutinized serves to define the criteria for optimized LC-SID-MRM-MS platforms for routine use, with pass/fail criteria for system suitability performance measures defined as peak area coefficient of variation <0.15, peak width coefficient of variation <0.15, standard deviation of RT <0.15 min (9 s), and the RT drift <0.5min (30 s). The deleterious effect of a marginally performing LC-SID-MRM-MS system on the limit of quantification (LOQ) in targeted quantitative assays illustrates the use and need for a SSP to establish robust and reliable system performance. Use of a SSP helps to ensure that analyte quantification measurements can be replicated with good precision within and across multiple laboratories and should facilitate more widespread use of MRM-MS technology by the basic biomedical and clinical laboratory research communities.

Percutaneous coronary intervention with drug-eluting stents or coronary artery bypass surgery in subjects with type 2 diabetes : evaluation of: Farkouth ME, Domanski M, Sleeper LA, et al. Strategies for multivessel revascularization in patients with diabetes. N Engl J Med 2012;367(25):2375-84, and Contini GA, Nicolini F, Fortuna D, et al. Five-year outcomes of surgical or percutaneous myocardial revascularization in diabetic patients. Int J Cardiol 2012. [Epub ahead of print]

There is debate as to whether percutaneous coronary intervention (PCI) with drug-eluting stents or coronary artery bypass surgery (CABG) is the best procedure for subjects with type 2 diabetes and coronary artery disease requiring revascularization. There is some evidence that following these procedures there is less further revascularization with CABG than PCI in subjects with diabetes. Two recent studies; the FREEDOM (Future Revascularization Evaluation in patients with Diabetes mellitus: Optimal Management of Multivessel Disease) trial, and a trial using a real world diabetic population from a Registry, have shown that the benefits of CABG over PCI in subjects with type 2 diabetes extends to lower rates of death and myocardial infarct, in addition to lower rates of revascularization. However, the rates of stroke may be higher with CABG than PCI with drug-eluting stents in this population. Thus, if CABG is going to be preferred to PCI in subjects with type 2 diabetes and multivessel coronary disease, consideration should be given to how to reduce the rates of stroke with CABG.

Exploring the Possibilities and Methodological Challenges of Evaluation Practice : A Democratic Question

In our exploration of the methodological possibilities and challenges of evaluation practice that foregrounds a democratic approach, we argue that having sensitivity to participants’ different positions, ideological perspectives, and values, and an understanding of the existing power relationships are paramount for the evaluators if the evaluation is to achieve its goals. This chapter draws on our experiences as evaluators in Australia, where school leaders and teachers are currently experiencing significant curriculum and assessment changes, and now work in a context of heightened accountability. Our evaluation practices with schools are used to illustrate how democratic practices can be incorporated into evaluation in ways that provide opportunities and support for school self-evaluation. It is paramount to recognise that evaluation is a political and ideological practice, and therefore is not a neutral process. Our argument is that it remains the responsibility of evaluators to design spaces that build sustainable relationships with participants and the targets of the programmes being evaluated; ensure that dialogue and deliberation are valued between all stakeholders; represent a range of interests; and give a service back to the evaluated communities by offering understandings and promoting reflective practice and informed decision-making throughout programme implementation.

Creating a Sustainability Scorecard as a predictive tool for measuring the complex social, economic and environmental impacts of industries, a case study : assessing the viability and sustainability of the dairy industry

Sustainability is a key driver for decisions in the management and future development of industries. The World Commission on Environment and Development (WCED, 1987) outlined imperatives which need to be met for environmental, economic and social sustainability. Development of strategies for measuring and improving sustainability in and across these domains, however, has been hindered by intense debate between advocates for one approach fearing that efforts by those who advocate for another could have unintended adverse impacts. Studies attempting to compare the sustainability performance of countries and industries have also found ratings of performance quite variable depending on the sustainability indices used. Quantifying and comparing the sustainability of industries across the triple bottom line of economy, environment and social impact continues to be problematic. Using the Australian dairy industry as a case study, a Sustainability Scorecard, developed as a Bayesian network model, is proposed as an adaptable tool to enable informed assessment, dialogue and negotiation of strategies at a global level as well as being suitable for developing local solutions.

Evaluation of brief dietary questions to estimate vegetable and fruit consumption – using serum carotenoids and red-cell folate

Objective: To evaluate responses to self-administered brief questions regarding consumption of vegetables and fruit by comparison with blood levels of serum carotenoids and red-cell folate. Design: A cross-sectional study in which participants reported their usual intake of fruit and vegetables in servings per day, and serum levels of five carotenoids (α-carotene, β-carotene, β-cryptoxanthin, lutein/zeaxanthin and lycopene) and red-cell folate were measured. Serum carotenoid levels were determined by high-performance liquid chromatography, and red-cell folate by an automated immunoassay system. Settings and subjects: Between October and December 2000, a sample of 1598 adults aged 25 years and over, from six randomly selected urban centres in Queensland, Australia, were examined as part of a national study conducted to determine the prevalence of diabetes and associated cardiovascular risk factors. Results: Statistically significant (P<0.01) associations with vegetable and fruit intake (categorised into groups: ≤1 serving, 2–3 servings and ≥4 servings per day) were observed for α-carotene, β-carotene, β-cryptoxanthin, lutein/zeaxanthin and red-cell folate. The mean level of these carotenoids and of red-cell folate increased with increasing frequency of reported servings of vegetables and fruit, both before and after adjusting for potential confounding factors. A significant association with lycopene was observed only for vegetable intake before adjusting for confounders. Conclusions: These data indicate that brief questions may be a simple and valuable tool for monitoring vegetable and fruit intake in this population.

An evaluation of different features and learning models for anomalous event detection

The huge amount of CCTV footage available makes it very burdensome to process these videos manually through human operators. This has made automated processing of video footage through computer vision technologies necessary. During the past several years, there has been a large effort to detect abnormal activities through computer vision techniques. Typically, the problem is formulated as a novelty detection task where the system is trained on normal data and is required to detect events which do not fit the learned ‘normal’ model. There is no precise and exact definition for an abnormal activity; it is dependent on the context of the scene. Hence there is a requirement for different feature sets to detect different kinds of abnormal activities. In this work we evaluate the performance of different state of the art features to detect the presence of the abnormal objects in the scene. These include optical flow vectors to detect motion related anomalies, textures of optical flow and image textures to detect the presence of abnormal objects. These extracted features in different combinations are modeled using different state of the art models such as Gaussian mixture model(GMM) and Semi- 2D Hidden Markov model(HMM) to analyse the performances. Further we apply perspective normalization to the extracted features to compensate for perspective distortion due to the distance between the camera and objects of consideration. The proposed approach is evaluated using the publicly available UCSD datasets and we demonstrate improved performance compared to other state of the art methods.

Assessing land-use and transport integration via a spatial composite indexing model

Achieving sustainable urban development is identified as one ultimate goal of many contemporary planning endeavours and has become central to formulation of urban planning policies. Within this concept, land-use and transport integration is highlighted as one of the most important and attainable policy objectives. In many cities, integration is embraced as an integral part of local development plans, and a number of key integration principles are identified. However, the lack of available evaluation methods to measure extent of urban sustainability levels prevents successful implementation of these principles. This paper introduces a new indicator-based spatial composite indexing model developed to measure sustainability performance of urban settings by taking into account land-use and transport integration principles. Model indicators are chosen via a thorough selection process in line with key principles of land-use and transport integration. These indicators are grouped into categories and themes according to their topical relevance. These indicators are then aggregated to form a spatial composite index to portray an overview of the sustainability performance of the pilot study area used for model demonstration. The study results revealed that the model is a practical instrument for evaluating success of local integration policies and visualizing sustainability performance of built environments and useful in both identifying problematic areas as well as formulating policy interventions.

Training Researchers : Visits and Training Schools

COST IS0801, like all COST Actions, provided many opportunities for training of early career researchers, and initiation of new research projects. Some of these were supported by short-term visits of usually one or two weeks, up to a maximum of three months, by an Action member to another venue, for purposes that supported the overall aims of the Action. The first part of this chapter provides some description of these, illustrated by a number of case studies. The second part of the chapter overviews the organization and outcome of two Training Schools for early career researchers, one in Australia entitled Research to policy and practice: Innovation and sustainability in cyberbullying prevention, and one in Finland, entitled Adolescents and Social Media: Guidelines and Coping Strategies for Cyberbullying. The organization of these Training Schools, the educational approaches used, and their evaluation and impact, will be summarised.

Survival tucker : improved diet and health indicators in an Aboriginal community

The poor nutritional status of Aboriginal Australians is a serious and complex public health concern. We describe an unusually successful health and nutrition project initiated by the people of Minjilang, which was developed, implemented and evaluated with the community. Apparent community dietary intake, assessed by the ‘store-turnover’ method, and biochemical, anthropometric and haematological indicators of health and nutritional status were measured before intervention and at three-monthly intervals during the intervention year. Following intervention, there was a significant decrease in dietary intake of sugar and saturated fat, an increase in micronutrient density, corresponding improvements in biochemical indices (for example, a 12 per cent decrease in mean serum cholesterol, increases in serum and red cell folate, serum vitamin B6 and plasma ascorbic acid), decrease in mean systolic and diastolic blood pressures, a normalisation of body mass index, and a normalisation of haematologic indices. The success of this project demonstrates that Aboriginal communities can bring about improvements in their generally poor nutritional status, and that the store-turnover method provides a valid, inexpensive and noninvasive method for evaluating the resultant changes in community diet. Although the project was undoubtedly effective in the short term, further work is in progress to assess individual strategies with respect to sustainability, cost-effectiveness and generalisability.

A methodology for effective implementation of lean strategies and its performance evaluation in manufacturing organizations

Purpose – The purpose of this paper is to develop an effective methodology for implementing lean manufacturing strategies and a leanness evaluation metric using continuous performance measurement (CPM). Design/methodology/approach – Based on five lean principles, a systematic lean implementation methodology for manufacturing organizations has been proposed. A simplified leanness evaluation metric consisting of both efficiency and effectiveness attributes of manufacturing performance has been developed for continuous evaluation of lean implementation. A case study to validate the proposed methodology has been conducted and proposed CPM metric has been used to assess the manufacturing leanness. Findings – Proposed methodology is able to systematically identify manufacturing wastes, select appropriate lean tools, identify relevant performance indicators, achieve significant performance improvement and establish lean culture in the organization. Continuous performance measurement matrices in terms of efficiency and effectiveness are proved to be appropriate methods for continuous evaluation of lean performance. Research limitations/implications – Effectiveness of the method developed has been demonstrated by applying it in a real life assembly process. However, more tests/applications will be necessary to generalize the findings. Practical implications – Results show that applying the methods developed, managers can successfully identify and remove manufacturing wastes from their production processes. By improving process efficiency, they can optimize their resource allocations. Manufacturers now have a validated step by step methodology for successfully implementing lean strategies. Originality/value – According to the authors’ best knowledge, this is the first known study that proposed a systematic lean implementation methodology based on lean principles and continuous improvement techniques. Evaluation of performance improvement by lean strategies is a critical issue. This study develops a simplified leanness evaluation metric considering both efficiency and effectiveness attributes and integrates it with the lean implementation methodology.

Teacher Evidence Matrix : an evaluation using multiple lines of evidence

Welcome to the Teacher evidence matrix. This matrix is designed for highly qualified discipline experts to evaluate their teaching in a systematic manner. The primary purpose of the Teacher evidence matrix is to provide a tool that an academic staff member at university can annually review their teaching. The annual review will result in you being ready for performance, planning and review; promotion; awards; or employment application. This tool is designed for individual use and will lead to an action plan for implementation.

Facilitating local stories in post-disaster regional communities : evaluation in narrative-driven oral history projects

Cyclone Yasi struck the Cassowary Coast of Northern Queensland, Australia, in the early hours of February 3, 2011, destroying many homes and property, including the destruction of the Cardwell and district historical society’s premises. With their own homes flattened, many residents were forced to live in mobile accommodation, with extended family, or leave the area altogether. The historical society members seemed, however, particularly devastated by their flattened foreshore museum and loss of their precious collection of material. A call for assistance was made through the Oral History Association of Australia’s Queensland branch (OHAA Qld), which along with a Queensland University of Technology (QUT) research team sponsored a trip to best plan how they could start to pick up the pieces to rebuild the museum. This chapter highlights the need for communities to gather, preserve and present their own stories, in a way that is sustainable and meaningful to them – whether that be because of a disaster, or as they go about life in their contemporary communities – the key being that good advice, professional support and embedded evaluation practices at crucial moments along the way can be critically important.

Visually impaired drivers who use bioptic telescopes : self-assessed driving skills and agreement with on-road driving evaluation

Purpose. To compare self-assessed driving habits and skills of licensed drivers with central visual loss who use bioptic telescopes to those of age-matched normally sighted drivers, and to examine the association between bioptic drivers' impressions of the quality of their driving and ratings by a “backseat” evaluator. Methods. Participants were licensed bioptic drivers (n = 23) and age-matched normally sighted drivers (n = 23). A questionnaire was administered addressing driving difficulty, space, quality, exposure, and, for bioptic drivers, whether the telescope was helpful in on-road situations. Visual acuity and contrast sensitivity were assessed. Information on ocular diagnosis, telescope characteristics, and bioptic driving experience was collected from the medical record or in interview. On-road driving performance in regular traffic conditions was rated independently by two evaluators. Results. Like normally sighted drivers, bioptic drivers reported no or little difficulty in many driving situations (e.g., left turns, rush hour), but reported more difficulty under poor visibility conditions and in unfamiliar areas (P < 0.05). Driving exposure was reduced in bioptic drivers (driving 250 miles per week on average vs. 410 miles per week for normally sighted drivers, P = 0.02), but driving space was similar to that of normally sighted drivers (P = 0.29). All but one bioptic driver used the telescope in at least one driving task, and 56% used the telescope in three or more tasks. Bioptic drivers' judgments about the quality of their driving were very similar to backseat evaluators' ratings. Conclusions. Bioptic drivers show insight into the overall quality of their driving and areas in which they experience driving difficulty. They report using the bioptic telescope while driving, contrary to previous claims that it is primarily used to pass the vision screening test at licensure.

Building indicators in media assistance; four ways evaluation reports present “change”

The international aid and development community has supported programs that aim to build the capacity of media professionals or contribute to an enabling environment throughout the past 20 years. However, two decades on from the first modern media assistance programs, the sector is still struggling to identify, measure and understand the changes effected by their programs. There are questions raised as to whether it is even feasible to identify impacts on society and governance. This paper draws on some preliminary findings from a comparative thematic analysis of 47 evaluation documents of media assistance programs. The aim of this analysis is to identify trends in impact evaluation practice in the media assistance field, as well as the strengths and weaknesses of different evaluation approaches. This paper presents four types of social change claims commonly presented in reports; hypothetical changes, introduction of new opportunities, concrete examples of immediate impacts, and analysis of ongoing social and political changes. Although these types may appear as a spectrum from weak to strong, the interactions are perhaps more accurately understood using metaphors such as building blocks. This paper explores these types in more detail and suggests that a robust set of impacts-types could be useful in developing more grounded theories of change and indicators.

Social event detection at MediaEval 2013 : challenges, datasets and evaluation

In this paper, we provide an overview of the Social Event Detection (SED) task that is part of the MediaEval Bench mark for Multimedia Evaluation 2013. This task requires participants to discover social events and organize the re- lated media items in event-specific clusters within a collection of Web multimedia. Social events are events that are planned by people, attended by people and for which the social multimedia are also captured by people. We describe the challenges, datasets, and the evaluation methodology.