Measurement as legitimacy versus legitimacy of measures : Performance evaluation of social enterprise

Purpose The purpose of this paper is to review the growing emphasis on quantifiable performance measures such as social return on investment (SROI) in third sector organisations – specifically, social enterprise – through a legitimacy theory lens. It then examines what social enterprises value (i.e. consider important) in terms of performance evaluation, using a case study approach. Design/methodology/approach Case studies involving interviews, documentary analysis, and observation, of three social enterprises at different life-cycle stages with different funding structures, were constructed to consider “what measures matter” from a practitioner's perspective. Findings Findings highlight a priority on quality outcomes and impacts in primarily qualitative terms to evaluate performance. Further, there is a noticeable lack of emphasis on financial measures other than basic access to financial resources to continue pursuing social goals. Social implications The practical challenges faced by social enterprises – many of which are small to medium sized – in evaluating performance and by implication organisational legitimacy are contrasted with measures such as SROI which are resource intensive and have inherent methodological limitations. Hence, findings suggest the limited and valuable resources of social enterprises would be better allocated towards documenting the actual outcomes and impacts as a first step, in order to evaluate social and financial performance in terms appropriate to each objective, in order to demonstrate organisational legitimacy. Originality/value Findings distinguish between processes which may hold symbolic legitimacy for select stakeholder groups, and processes which hold substantive, cognitive legitimacy for stakeholders more broadly, in the under-researched context of social enterprise.

Design, implementation and multisite evaluation of a system suitability protocol for the quantitative assessment of instrument performance in liquid chromatography-multiple reaction monitoring-MS (LC-MRM-MS)

Multiple reaction monitoring (MRM) mass spectrometry coupled with stable isotope dilution (SID) and liquid chromatography (LC) is increasingly used in biological and clinical studies for precise and reproducible quantification of peptides and proteins in complex sample matrices. Robust LC-SID-MRM-MS-based assays that can be replicated across laboratories and ultimately in clinical laboratory settings require standardized protocols to demonstrate that the analysis platforms are performing adequately. We developed a system suitability protocol (SSP), which employs a predigested mixture of six proteins, to facilitate performance evaluation of LC-SID-MRM-MS instrument platforms, configured with nanoflow-LC systems interfaced to triple quadrupole mass spectrometers. The SSP was designed for use with low multiplex analyses as well as high multiplex approaches when software-driven scheduling of data acquisition is required. Performance was assessed by monitoring of a range of chromatographic and mass spectrometric metrics including peak width, chromatographic resolution, peak capacity, and the variability in peak area and analyte retention time (RT) stability. The SSP, which was evaluated in 11 laboratories on a total of 15 different instruments, enabled early diagnoses of LC and MS anomalies that indicated suboptimal LC-MRM-MS performance. The observed range in variation of each of the metrics scrutinized serves to define the criteria for optimized LC-SID-MRM-MS platforms for routine use, with pass/fail criteria for system suitability performance measures defined as peak area coefficient of variation <0.15, peak width coefficient of variation <0.15, standard deviation of RT <0.15 min (9 s), and the RT drift <0.5min (30 s). The deleterious effect of a marginally performing LC-SID-MRM-MS system on the limit of quantification (LOQ) in targeted quantitative assays illustrates the use and need for a SSP to establish robust and reliable system performance. Use of a SSP helps to ensure that analyte quantification measurements can be replicated with good precision within and across multiple laboratories and should facilitate more widespread use of MRM-MS technology by the basic biomedical and clinical laboratory research communities.

Visual metrics for the evaluation of sensor data quality in outdoor perception

This paper proposes an experimental study of quality metrics that can be applied to visual and infrared images acquired from cameras onboard an unmanned ground vehicle (UGV). The relevance of existing metrics in this context is discussed and a novel metric is introduced. Selected metrics are evaluated on data collected by a UGV in clear and challenging environmental conditions, represented in this paper by the presence of airborne dust or smoke. An example of application is given with monocular SLAM estimating the pose of the UGV while smoke is present in the environment. It is shown that the proposed novel quality metric can be used to anticipate situations where the quality of the pose estimate will be significantly degraded due to the input image data. This leads to decisions of advantageously switching between data sources (e.g. using infrared images instead of visual images).

High dietary diversity is associated with obesity in Sri Lankan adults : an evaluation of three dietary scores

Background Dietary diversity is recognized as a key element of a high quality diet. However, diets that offer a greater variety of energy-dense foods could increase food intake and body weight. The aim of this study was to explore association of diet diversity with obesity in Sri Lankan adults. Methods Six hundred adults aged > 18 years were randomly selected by using multi-stage stratified sample. Dietary intake assessment was undertaken by a 24 hour dietary recall. Three dietary scores, Dietary Diversity Score (DDS), Dietary Diversity Score with Portions (DDSP) and Food Variety Score (FVS) were calculated. Body mass index (BMI) ≥ 25 kg.m-2 is defined as obese and Asian waist circumference cut-offs were used diagnosed abdominal obesity. Results Mean of DDS for men and women were 6.23 and 6.50 (p=0.06), while DDSP was 3.26 and 3.17 respectively (p=0.24). FVS values were significantly different between men and women 9.55 and 10.24 (p=0.002). Dietary diversity among Sri Lankan adults was significantly associated with gender, residency, ethnicity, education level but not with diabetes status. As dietary scores increased, the percentage consumption was increased in most of food groups except starches. Obese and abdominal obese adults had the highest DDS compared to non obese groups (p<0.05). With increased dietary diversity the level of BMI, waist circumference and energy consumption was significantly increased in this population. Conclusion Our data suggests that dietary diversity is positively associated with several socio-demographic characteristics and obesity among Sri Lankan adults. Although high dietary diversity is widely recommended, public health messages should emphasize to improve dietary diversity in selective food items.

Diabetes mellitus in South Asia : scientific evaluation of the research output

BACKGROUND: Diabetes in South Asia represents a different disease entity in terms of its onset, progression, and complications. In the present study, we systematically analyzed the medical research output on diabetes in South Asia. METHODS: The online SciVerse Scopus database was searched using the search terms "diabetes" and "diabetes mellitus" in the article Title, Abstract or Keywords fields, in conjunction with the names of each regional country in the Author Affiliation field. RESULTS: In total, 8478 research articles were identified. Most were from India (85.1%) and Pakistan (9.6%) and the contribution to the global diabetes research output was 2.1%. Publications from South Asia increased markedly after 2007, with 58.7% of papers published between 2000 and 2010 being published after 2007. Most papers were Research Articles (75.9%) and Reviews (12.9%), with only 90 (1.1%) clinical trials. Publications predominantly appeared in local national journals. Indian authors and institutions had the most number of articles and the highest h-index. There were 136 (1.6%) intraregional collaborative studies. Only 39 articles (0.46%) had >100 citations. CONCLUSIONS: Regional research output on diabetes mellitus is unsatisfactory, with only a minimal contribution to global diabetes research. Publications are not highly cited and only a few randomized controlled trials have been performed. In the coming decades, scientists in the region must collaborate and focus on practical and culturally acceptable interventional studies on diabetes mellitus.

Retrospective evaluation of dosimetric quality for prostate carcinomas treated with 3D conformal, intensity-modulated and volumetric-modulated arc radiotherapy

Introduction This study examines and compares the dosimetric quality of radiotherapy treatment plans for prostate carcinoma across a cohort of 163 patients treated across 5 centres: 83 treated with three-dimensional conformal radiotherapy (3DCRT), 33 treated with intensity-modulated radiotherapy (IMRT) and 47 treated with volumetric-modulated arc therapy (VMAT). Methods Treatment plan quality was evaluated in terms of target dose homogeneity and organ-at-risk sparing, through the use of a set of dose metrics. These included the mean, maximum and minimum doses; the homogeneity and conformity indices for the target volumes; and a selection of dose coverage values that were relevant to each organ-at-risk. Statistical significance was evaluated using two-tailed Welch’s T-tests. The Monte Carlo DICOM ToolKit software was adapted to permit the evaluation of dose metrics from DICOM data exported from a commercial radiotherapy treatment planning system. Results The 3DCRT treatment plans offered greater planning target volume dose homogeneity than the other two treatment modalities. The IMRT and VMAT plans offered greater dose reduction in the organs-at-risk: with increased compliance with recommended organ-at-risk dose constraints, compared to conventional 3DCRT treatments. When compared to each other, IMRT and VMAT did not provide significantly different treatment plan quality for like-sized tumour volumes. Conclusions This study indicates that IMRT and VMAT have provided similar dosimetric quality, which is superior to the dosimetric quality achieved with 3DCRT.

Evaluation of a pilot-scale oil extraction from microalgae for biodiesel production

Biodiesel derived from microalgae is one of a suite of potential solutions to meet the increasing demand for a renewable, carbon-neutral energy source. However, there are numerous challenges that must be addressed before algae biodiesel can become commercially viable. These challenges include the economic feasibility of harvesting and dewatering the biomass and the extraction of lipids and their conversion into biodiesel. Therefore, it is essential to find a suitable extraction process given these processes presently contribute significantly to the total production costs which, at this stage, inhibit the ability of biodiesel to compete financially with petroleum diesel. This study focuses on pilot-scale (100 kg dried microalgae) solvent extraction of lipids from microalgae and subsequent transesterification to biodiesel. Three different solvents (hexane, isopropanol (IPA) and hexane + IPA (1:1)) were used with two different extraction methods (static and Soxhlet) at bench-scale to find the most suitable solvent extraction process for the pilot-scale. The Soxhlet method extracted only 4.2% more lipid compared to the static method. However, the fatty acid profiles of different extraction methods with different solvents are similar, suggesting that none of the solvents or extraction processes were biased for extraction of particular fatty acids. Considering the cost and availability of the solvents, hexane was chosen for pilot-scale extraction using static extraction. At pilot-scale the lipid yield was found to be 20.3% of total biomass which is 2.5% less than from bench scale. Extracted fatty acids were dominated by polyunsaturated fatty acids (PUFAs) (68.94±0.17%) including 47.7±0.43 and 17.86±0.42% being docosahexaenoic acid (DHA) (C22:6) and docosapentaenoic acid (DPA) (C22:5, ω-3), respectively. These high amounts of long chain poly unsaturated fatty acids are unique to some marine microalgae and protists and vary with environmental conditions, culture age and nutrient status, as well as with cultivation process. Calculated physical and chemical properties of density, viscosity of transesterified fatty acid methyl esters (FAMEs) were within the limits of the biodiesel standard specifications as per ASTM D6751-2012 and EN 14214. The calculated cetane number was, however, significantly lower (17.8~18.6) compared to ASTM D6751-2012 or EN 14214-specified minimal requirements. We conclude that the obtained microalgal biodiesel would likely only be suitable for blending with petroleum diesel to a maximum of 5 to 20%.

Calibration and evaluation of an objective measure of physical activity in pre-school children

Background The purposes of this study were 1) to establish accelerometer count cutoffs to categorize activity intensity of 3 to 5-y old-children and 2) to evaluate the accelerometer as a measure of children’s physical activity in preschool settings. Methods While wearing an ActiGraph accelerometer, 16 preschool children performed five, 3-min structured activities. Receiver Operating Characteristic (ROC) curve analyses identified count cutoffs for four physical activity intensities. In 9 preschools, 281 children wore an ActiGraph during observations performed by three trained observers (interobserver reli-ability = 0.91 to 0.98). Results Separate count cutoffs for 3, 4, and 5-y olds were established. Sensitivity and specificity for the count cutoffs ranged from 86.7% to 100.0% and 66.7% to 100.0%, respectively. ActiGraph counts/15 s were different among all activities (P < 0.05) except the two sitting activities. Correlations between observed and ActiGraph intensity categorizations at the preschools ranged from 0.46 to 0.70 (P < 0.001). Conclusions The ActiGraph count cutoffs established and validated in this study can be used to objectively categorize the time that preschool-age children spend in different physical activity intensity levels.

Integrating understandability in the evaluation of consumer health search engines

In this paper we propose a method that integrates the no- tion of understandability, as a factor of document relevance, into the evaluation of information retrieval systems for con- sumer health search. We consider the gain-discount evaluation framework (RBP, nDCG, ERR) and propose two understandability-based variants (uRBP) of rank biased precision, characterised by an estimation of understandability based on document readability and by different models of how readability influences user understanding of document content. The proposed uRBP measures are empirically contrasted to RBP by comparing system rankings obtained with each measure. The findings suggest that considering understandability along with topicality in the evaluation of in- formation retrieval systems lead to different claims about systems effectiveness than considering topicality alone.

Evaluation of resonant damping techniques for Z-source current-type inverter

For the renewable energy sources whose outputs vary continuously, a Z-source current-type inverter has been proposed as a possible buck-boost alternative for grid-interfacing. With a unique X-shaped LC network connected between its dc power source and inverter topology, Z-source current-type inverter is however expected to suffer from compounded resonant complications in addition to those associated with its second-order output filter. To improve its damping performance, this paper proposes the careful integration of Posicast or three-step compensators before the inverter pulse-width modulator for damping triggered resonant oscillations. In total, two compensators are needed for wave-shaping the inverter boost factor and modulation ratio, and they can conveniently be implemented using first-in first-out stacks and embedded timers of modern digital signal processors widely used in motion control applications. Both techniques are found to damp resonance of ac filter well, but for cases of transiting from current-buck to boost state, three-step technique is less effective due to the sudden intermediate discharging interval introduced by its non-monotonic stepping (unlike the monotonic stepping of Posicast damping). These findings have been confirmed both in simulations and experiments using an implemented laboratory prototype.

An evaluation of crowd counting methods, features and regression models

Existing crowd counting algorithms rely on holistic, local or histogram based features to capture crowd properties. Regression is then employed to estimate the crowd size. Insufficient testing across multiple datasets has made it difficult to compare and contrast different methodologies. This paper presents an evaluation across multiple datasets to compare holistic, local and histogram based methods, and to compare various image features and regression models. A K-fold cross validation protocol is followed to evaluate the performance across five public datasets: UCSD, PETS 2009, Fudan, Mall and Grand Central datasets. Image features are categorised into five types: size, shape, edges, keypoints and textures. The regression models evaluated are: Gaussian process regression (GPR), linear regression, K nearest neighbours (KNN) and neural networks (NN). The results demonstrate that local features outperform equivalent holistic and histogram based features; optimal performance is observed using all image features except for textures; and that GPR outperforms linear, KNN and NN regression

Brain networks subserving the evaluation of static and dynamic facial expressions

Because moving depictions of face emotion have greater ecological validity than their static counterparts, it has been suggested that still photographs may not engage ‘authentic’ mechanisms used to recognize facial expressions in everyday life. To date, however, no neuroimaging studies have adequately addressed the question of whether the processing of static and dynamic expressions rely upon different brain substrates. To address this, we performed an functional magnetic resonance imaging (fMRI) experiment wherein participants made emotional expression discrimination and Sex discrimination judgements to static and moving face images. Compared to Sex discrimination, Emotion discrimination was associated with widespread increased activation in regions of occipito-temporal, parietal and frontal cortex. These regions were activated both by moving and by static emotional stimuli, indicating a general role in the interpretation of emotion. However, portions of the inferior frontal gyri and supplementary/pre-supplementary motor area showed task by motion interaction. These regions were most active during emotion judgements to static faces. Our results demonstrate a common neural substrate for recognizing static and moving facial expressions, but suggest a role for the inferior frontal gyrus in supporting simulation processes that are invoked more strongly to disambiguate static emotional cues.

Combining passive sampling and toxicity testing for evaluation of mixtures of polar organic chemicals in sewage treatment plant effluent

Effluent from sewage treatment plants has been associated with a range of pollutant effects. Depending on the influent composition and treatment processes the effluent may contain a myriad of different chemicals which makes monitoring very complex. In this study we aimed to monitor relatively polar organic pollutant mixtures using a combination of passive sampling techniques and a set of biochemistry based assays covering acute bacterial toxicity (Microtox™), phytotoxicity (Max-I-PAM assay) and genotoxicity (umuC assay). The study showed that all of the assays were able to detect effects in the samples and allowed a comparison of the two plants as well as a comparison between the two sampling periods. Distinct improvements in water quality were observed in one of the plants as result of an upgrade to a UV disinfection system, which improved from 24× sample enrichment required to induce a 50% response in the Microtox™ assay to 84×, from 30× sample enrichment to induce a 50% reduction in photosynthetic yield to 125×, and the genotoxicity observed in the first sampling period was eliminated. Thus we propose that biochemical assay techniques in combination with time integrated passive sampling can substantially contribute to the monitoring of polar organic toxicants in STP effluents.

Evaluation of the Australian Performing Arts Market 2014-2018 - Year One Report

The Australia Council awarded the tender of APAMs 2014, 2016 and 2018 to the Brisbane Powerhouse. The Australia Council, in awarding the contract for the presentation of APAM by Brisbane Powerhouse, stipulated that a formal evaluation of the three iterations of APAM and activity in the intervening years be undertaken. Queensland University of Technology, Creative Industries Faculty, under the leadership of Associate Professor Sandra Gattenhof, were contracted to undertake the formal evaluation. This is the first year report on the Brisbane iteration of the Market. This report has drawn from data collected across a range of sources, drawing on the scoping study undertaken by Justin Macdonnell addressing the Market from 1994–2010; the tender document submitted by the Brisbane Powerhouse; in-person interviews with APAM staff, APAM Stakeholders, Vox Pops from delegates in response to individual sessions, producer company/artist case studies and, most significantly, responses from a detailed online survey sent to all delegates. The main body of the report is organised around three key research aims, as outlined in the Brisbane Powerhouse Tender document (2011). These have been articulated as: Evaluation of international market development outcomes through showcasing work to targeted international presenters and agents Evaluation of national market development outcomes through showcasing work to national presenters and producers Evaluation of the exchange ideas, dialogue, skill 
development, partnerships, collaborations and co- productions and networks with local and international peers. The culmination of the data analysis has been articulated through five key recommendations, which may assist the APAM delivery team for the next version, in 2016. In summary, the recommendations are described as: 1. Indigenous focus to remain central to the conception and delivery of APAM 2. Re-framing APAM’s function and its delivery 3. Logistics and communications in a multi-venue approach, including communications and housekeeping, volunteers, catering, re-calibrating the employment of Brisbane Powerhouse protocols and processes for APAM 4. Presentation and promotion for presenters 5. Strategic targeting of Asian producers.

Evaluation of Indigenous justice programs.: final report / Project D, Safe Aboriginal Youth Patrol Programs in New South Wales and Northbridge Policy and Juvenile Aid Group in Western Australia

The Australian Government has provided funding to evaluate the effectiveness of Indigenous law and justice programs across five subject areas to identify the best approaches to tackling crime and justice issues and better inform government funding decisions in the future. This report presents the findings of subject area "D", which examined two different approaches to delivering community and night patrol services for young people: the Safe Aboriginal Youth Patrol programs in New South Wales, and the Northbridge Policy project (the Young People in Northbridge project), in Western Australia. Night patrols can address crime either directly or indirectly, by prevention work or by addressing the social causes of crime through community development.