The efficacy of the Sortino ratio and other benchmarked performance measures under skewed return distributions

This paper will investigate the suitability of existing performance measures under the assumption of a clearly defined benchmark. A range of measures are examined including the Sortino Ratio, the Sharpe Selection ratio (SSR), the Student’s t-test and a decay rate measure. A simulation study is used to assess the power and bias of these measures based on variations in sample size and mean performance of two simulated funds. The Sortino Ratio is found to be the superior performance measure exhibiting more power and less bias than the SSR when the distribution of excess returns are skewed.

Dynamic Performance Measures for Object Tracking Systems

Performance evaluation of object tracking systems is typically performed after the data has been processed, by comparing tracking results to ground truth. Whilst this approach is fine when performing offline testing, it does not allow for real-time analysis of the systems performance, which may be of use for live systems to either automatically tune the system or report reliability. In this paper, we propose three metrics that can be used to dynamically asses the performance of an object tracking system. Outputs and results from various stages in the tracking system are used to obtain measures that indicate the performance of motion segmentation, object detection and object matching. The proposed dynamic metrics are shown to accurately indicate tracking errors when visually comparing metric results to tracking output, and are shown to display similar trends to the ETISEO metrics when comparing different tracking configurations.

Performance measures for Australian laboratories reporting cervical cytology : a decade of data 1998–2008

Aim Performance measures for Australian laboratories reporting cervical cytology are a set of quantifiable measures relating to the profile and accuracy of reporting. This study reviews aggregate data collected over the ten years in which participation in the performance measures has been mandatory. Methods Laboratories submit annual data on performance measures relating to the profile of reporting, including reporting rates for technically unsatisfactory specimens, high grade or possible high grade abnormalities and abnormal reports. Cytology-histology correlation data and review findings of negative smears reported from women with histological high grade disease are also collected. Suggested acceptable standards are set for each measure. This study reviews the aggregate data submitted by all laboratories for the years 1998-2008 and examines trends in reporting and the performance of laboratories against the suggested standards. Results The performance of Australian laboratories has shown continued improvement over the study period. There has been a fall in the proportion of laboratories with data outside the acceptable standard range in all performance measures. Laboratories are reporting a greater proportion of specimens as definite or possible high grade abnormality. This is partly attributable to an increase in the proportion of abnormal results classified as high grade or possible high grade abnormality. Despite this, the positive predictive value for high grade and possible high grade abnormalities has continued to rise. Conclusion Performance measures for cervical cytology have provided a valuable addition to external quality assurance procedures in Australia. They have documented continued improvements in the aggregate performance, as well as providing benchmarking data and goals for acceptable performance for individual laboratories.

Measurement as legitimacy versus legitimacy of measures : Performance evaluation of social enterprise

Purpose The purpose of this paper is to review the growing emphasis on quantifiable performance measures such as social return on investment (SROI) in third sector organisations – specifically, social enterprise – through a legitimacy theory lens. It then examines what social enterprises value (i.e. consider important) in terms of performance evaluation, using a case study approach. Design/methodology/approach Case studies involving interviews, documentary analysis, and observation, of three social enterprises at different life-cycle stages with different funding structures, were constructed to consider “what measures matter” from a practitioner's perspective. Findings Findings highlight a priority on quality outcomes and impacts in primarily qualitative terms to evaluate performance. Further, there is a noticeable lack of emphasis on financial measures other than basic access to financial resources to continue pursuing social goals. Social implications The practical challenges faced by social enterprises – many of which are small to medium sized – in evaluating performance and by implication organisational legitimacy are contrasted with measures such as SROI which are resource intensive and have inherent methodological limitations. Hence, findings suggest the limited and valuable resources of social enterprises would be better allocated towards documenting the actual outcomes and impacts as a first step, in order to evaluate social and financial performance in terms appropriate to each objective, in order to demonstrate organisational legitimacy. Originality/value Findings distinguish between processes which may hold symbolic legitimacy for select stakeholder groups, and processes which hold substantive, cognitive legitimacy for stakeholders more broadly, in the under-researched context of social enterprise.

Performance measurement and feedback in a public sector program

Establishing a framework for measuring the performance of public sector programs is fraught with dangers. Many public sector organisations are satisfied with measuring activity in programs and fail to see the need for establishing a framework that will meet the needs of participants and measure outcomes as well as activities. This paper explores how a government department in Queensland went about establishing a performance management framework to measure the outcomes and activities in a program that was a public private partnership. Findings indicate that using an iterative consultative approach performance measure can be put in place that are meaningful and assist the participants to review the program.

The effect of prolonged reading on visual functions and reading performance in students with low vision

This study is the first to investigate the effect of prolonged reading on reading performance and visual functions in students with low vision. The study focuses on one of the most common modes of achieving adequate magnification for reading by students with low vision, their close reading distance (proximal or relative distance magnification). Close reading distances impose high demands on near visual functions, such as accommodation and convergence. Previous research on accommodation in children with low vision shows that their accommodative responses are reduced compared to normal vision. In addition, there is an increased lag of accommodation for higher stimulus levels as may occur at close reading distance. Reduced accommodative responses in low vision and higher lag of accommodation at close reading distances together could impact on reading performance of students with low vision especially during prolonged reading tasks. The presence of convergence anomalies could further affect reading performance. Therefore, the aims of the present study were 1) To investigate the effect of prolonged reading on reading performance in students with low vision 2) To investigate the effect of prolonged reading on visual functions in students with low vision. This study was conducted as cross-sectional research on 42 students with low vision and a comparison group of 20 students with normal vision, aged 7 to 20 years. The students with low vision had vision impairments arising from a range of causes and represented a typical group of students with low vision, with no significant developmental delays, attending school in Brisbane, Australia. All participants underwent a battery of clinical tests before and after a prolonged reading task. An initial reading-specific history and pre-task measurements that included Bailey-Lovie distance and near visual acuities, Pelli-Robson contrast sensitivity, ocular deviations, sensory fusion, ocular motility, near point of accommodation (pull-away method), accuracy of accommodation (Monocular Estimation Method (MEM)) retinoscopy and Near Point of Convergence (NPC) (push-up method) were recorded for all participants. Reading performance measures were Maximum Oral Reading Rates (MORR), Near Text Visual Acuity (NTVA) and acuity reserves using Bailey-Lovie text charts. Symptoms of visual fatigue were assessed using the Convergence Insufficiency Symptom Survey (CISS) for all participants. Pre-task measurements of reading performance and accuracy of accommodation and NPC were compared with post-task measurements, to test for any effects of prolonged reading. The prolonged reading task involved reading a storybook silently for at least 30 minutes. The task was controlled for print size, contrast, difficulty level and content of the reading material. Silent Reading Rate (SRR) was recorded every 2 minutes during prolonged reading. Symptom scores and visual fatigue scores were also obtained for all participants. A visual fatigue analogue scale (VAS) was used to assess visual fatigue during the task, once at the beginning, once at the middle and once at the end of the task. In addition to the subjective assessments of visual fatigue, tonic accommodation was monitored using a photorefractor (PlusoptiX CR03™) every 6 minutes during the task, as an objective assessment of visual fatigue. Reading measures were done at the habitual reading distance of students with low vision and at 25 cms for students with normal vision. The initial history showed that the students with low vision read for significantly shorter periods at home compared to the students with normal vision. The working distances of participants with low vision ranged from 3-25 cms and half of them were not using any optical devices for magnification. Nearly half of the participants with low vision were able to resolve 8-point print (1M) at 25 cms. Half of the participants in the low vision group had ocular deviations and suppression at near. Reading rates were significantly reduced in students with low vision compared to those of students with normal vision. In addition, there were a significantly larger number of participants in the low vision group who could not sustain the 30-minute task compared to the normal vision group. However, there were no significant changes in reading rates during or following prolonged reading in either the low vision or normal vision groups. Individual changes in reading rates were independent of their baseline reading rates, indicating that the changes in reading rates during prolonged reading cannot be predicted from a typical clinical assessment of reading using brief reading tasks. Contrary to previous reports the silent reading rates of the students with low vision were significantly lower than their oral reading rates, although oral and silent reading was assessed using different methods. Although the visual acuity, contrast sensitivity, near point of convergence and accuracy of accommodation were significantly poorer for the low vision group compared to those of the normal vision group, there were no significant changes in any of these visual functions following prolonged reading in either group. Interestingly, a few students with low vision (n =10) were found to be reading at a distance closer than their near point of accommodation. This suggests a decreased sensitivity to blur. Further evaluation revealed that the equivalent intrinsic refractive errors (an estimate of the spherical dioptirc defocus which would be expected to yield a patient’s visual acuity in normal subjects) were significantly larger for the low vision group compared to those of the normal vision group. As expected, accommodative responses were significantly reduced for the low vision group compared to the expected norms, which is consistent with their close reading distances, reduced visual acuity and contrast sensitivity. For those in the low vision group who had an accommodative error exceeding their equivalent intrinsic refractive errors, a significant decrease in MORR was found following prolonged reading. The silent reading rates however were not significantly affected by accommodative errors in the present study. Suppression also had a significant impact on the changes in reading rates during prolonged reading. The participants who did not have suppression at near showed significant decreases in silent reading rates during and following prolonged reading. This impact of binocular vision at near on prolonged reading was possibly due to the high demands on convergence. The significant predictors of MORR in the low vision group were age, NTVA, reading interest and reading comprehension, accounting for 61.7% of the variances in MORR. SRR was not significantly influenced by any factors, except for the duration of the reading task sustained; participants with higher reading rates were able to sustain a longer reading duration. In students with normal vision, age was the only predictor of MORR. Participants with low vision also reported significantly greater visual fatigue compared to the normal vision group. Measures of tonic accommodation however were little influenced by visual fatigue in the present study. Visual fatigue analogue scores were found to be significantly associated with reading rates in students with low vision and normal vision. However, the patterns of association between visual fatigue and reading rates were different for SRR and MORR. The participants with low vision with higher symptom scores had lower SRRs and participants with higher visual fatigue had lower MORRs. As hypothesized, visual functions such as accuracy of accommodation and convergence did have an impact on prolonged reading in students with low vision, for students whose accommodative errors were greater than their equivalent intrinsic refractive errors, and for those who did not suppress one eye. Those students with low vision who have accommodative errors higher than their equivalent intrinsic refractive errors might significantly benefit from reading glasses. Similarly, considering prisms or occlusion for those without suppression might reduce the convergence demands in these students while using their close reading distances. The impact of these prescriptions on reading rates, reading interest and visual fatigue is an area of promising future research. Most importantly, it is evident from the present study that a combination of factors such as accommodative errors, near point of convergence and suppression should be considered when prescribing reading devices for students with low vision. Considering these factors would also assist rehabilitation specialists in identifying those students who are likely to experience difficulty in prolonged reading, which is otherwise not reflected during typical clinical reading assessments.

Performance pressure an empirical analysis using world and Euro cups data

There is a notable shortage of empirical research directed at measuring the magnitude and direction of stress effects on performance in a controlled environment. One reason for this is the inherent difficulties in identifying and isolating direct performance measures for individuals. Additionally most traditional work environments contain a multitude of exogenous factors impacting individual performance, but controlling for all such factors is generally unfeasible (omitted variable bias). Moreover, instead of asking individuals about their self-reported stress levels we observe workers' behavior in situations that can be classified as stressful. For this reason we have stepped outside the traditional workplace in an attempt to gain greater controllability of these factors using the sports environment as our experimental space. We empirically investigate the relationship between stress and performance, in an extreme pressure situation (football penalty kicks) in a winner take all sporting environment (FIFA World Cup and UEFA European Cup competitions). Specifically, we examine all the penalty shootouts between 1976 and 2008 covering in total 16 events. The results indicate that extreme stressors can have a positive or negative impact on Individuals' performance. On the other hand, more commonly experienced stressors do not affect professionals' performances.

Investigating the performance of automatic new topic identification across multiple datasets 1

Recent studies on automatic new topic identification in Web search engine user sessions demonstrated that neural networks are successful in automatic new topic identification. However most of this work applied their new topic identification algorithms on data logs from a single search engine. In this study, we investigate whether the application of neural networks for automatic new topic identification are more successful on some search engines than others. Sample data logs from the Norwegian search engine FAST (currently owned by Overture) and Excite are used in this study. Findings of this study suggest that query logs with more topic shifts tend to provide more successful results on shift-based performance measures, whereas logs with more topic continuations tend to provide better results on continuation-based performance measures.

Clinical performance of a new silicone hydrogel contact lens over 6 months of extended wear

Purpose: Silicone hydrogel contact lenses (CLs) are becoming increasingly popular for daily wear (DW), extended wear (EW) and continuous wear (CW), due to their higher oxygen transmissibility compared to hydrogel CLs. The aim of this study was to investigate the clinical and subjective performance of asmofilcon A (Menicon Co., Ltd), a new surface treated silicone hydrogel CL, during 6-night EW over 6 months (M). Methods: A prospective, randomised, single-masked, monadic study was conducted. N=60 experienced DW soft CL wearers were randomly assigned to wear either asmofilcon A (test: Dk=129, water content (WC)=40%, Nanogloss surface treatment) or senofilcon A (control: Dk=103, WC=38%, PVP internal wetting agent, Vistakon, Johnson & Johnson Vision Care) CLs bilaterally for 6 M on an EW basis. A PHMB-preserved solution (Menicon Co., Ltd) was dispensed for CL care. Evaluations were conducted at CL delivery and after 1 week (W), 4 W, 3 M and 6 M of EW. At each visit, a range of objective and subjective clinical performance measures were assessed. Results: N=50 subjects (83%) successfully completed the study, with the majority of discontinuations due to loss to follow-up (n=3) or moving away/travel (n=5). N=2 subjects experienced adverse events; n=1 unilateral red eye with asmofilcon A and n=1 asymptomatic infiltrate with senofilcon A. There were no significant differences in high or low contrast distance visual acuity (HCDVA or LCDVA) between asmofilcon A and senofilcon A; however, LCDVA decreased significantly over time with both CL types (p<0.05). The two CL types did not vary significantly with respect to any of the objective and subjective measures assessed (p>0.05); CL fitting characteristics and CL surface measurements were very similar and mean bulbar and limbal redness measures were always less than grade 1.0. Superior palpebral conjunctival injection showed a statistically, but not clinically, significant increase over time with both CL types (p<0.05). Corneal staining did not vary significantly between asmofilcon A and senofilcon A (p>0.05), with low median gradings of less than 0.5 observed for all areas assessed. There were no solution-related staining reactions observed with either CL type. The asmofilcon A and senofilcon A CLs were both rated highly with respect to overall comfort, with medians of 14 or 15 hours of comfortable lens wearing time per day reported at each of the study visits (p>0.05). Conclusions: Over 6 months of EW, the asmofilcon A and senofilcon A CLs performed in a similar manner with respect to visual acuity, ocular health and CL performance measures. Some changes over time were observed with both CL types, including reduced LCDVA and increased superior palpebral injection, which warrant further investigation in longer-term EW studies. Asmofilcon A appeared to be equivalent in performance to senofilcon A.

The role of organisational concern for workplace fairness in the choice of a performance mesaurement system

Prior studies linking performance management systems (PMS) and organisational justice have examined how PMS influence procedural fairness. Our investigation differs from these studies. First, it examines fairness as an antecedent (instead of as a consequence) of the choice of PMS. Second, instead of conceptualising organisational fairness as procedural fairness, it relies on the impression management interpretation of organisational fairness. Hence, the study investigates how the need of senior managers to cultivate an impression of being fair is related to the choice of PMS systems and employee outcomes. Based on a sample of 276 employees, the results indicate that the need of senior management to cultivate an impression of being fair is associated with employee performance. They also indicate that a substantial component of these effects is indirect through the choice of comprehensive performance measures (CPM) and employee job satisfaction. These findings highlight the importance of organisational concern for workplace fairness as an antecedent of choice of CPM. From a theoretical perspective, the adoption of the impression management interpretation of organisational fairness contributes by providing new insights into the relationship between fairness and choice of PMS from a perspective that is different from those used in prior management accounting research.

Nerves of steel? Stress, work performance and elite athletes

There is a notable shortage of empirical research directed at measuring the magnitude and direction of stress effects on performance in a controlled environment. One reason for this is the inherent difficulties in identifying and isolating direct performance measures for individuals. Additionally, most traditional work environments contain a multitude of exogenous factors impacting individual performance, but controlling for all such factors is generally unfeasible (omitted variable bias). Moreover, instead of asking individuals about their self-reported stress levels, we observe workers’ behaviour in situations that can be classified as stressful. For this reason, we have stepped outside the traditional workplace in an attempt to gain greater controllability of these factors using the sports environment as our experimental space. We empirically investigate the relationship between stress and performance, in an extreme pressure situation (football penalty kicks) in a winner take all sporting environment (FIFA World Cup and UEFA European Cup competitions). Specifically, we examine all the penalty shootouts between 1976 and 2008 covering in total 16 events. The results indicate that extreme stressors can have a positive or negative impact on individuals’ performance. On the other hand, more commonly experienced stressors do not affect professionals’ performances.

Planning for transit system reliability using productive performance and risk assessment

Urban transit system performance may be quantified and assessed using transit capacity and productive capacity for planning, design and operational management. Bunker (4) defines important productive performance measures of an individual transit service and transit line. Transit work (p-km) captures transit task performed over distance. Transit productiveness (p-km/h) captures transit work performed over time. This paper applies productive performance with risk assessment to quantify transit system reliability. Theory is developed to monetize transit segment reliability risk on the basis of demonstration Annual Reliability Event rates by transit facility type, segment productiveness, and unit-event severity. A comparative example of peak hour performance of a transit sub-system containing bus-on-street, busway, and rail components in Brisbane, Australia demonstrates through practical application the importance of valuing reliability. Comparison reveals the highest risk segments to be long, highly productive on street bus segments followed by busway (BRT) segments and then rail segments. A transit reliability risk reduction treatment example demonstrates that benefits can be significant and should be incorporated into project evaluation in addition to those of regular travel time savings, reduced emissions and safety improvements. Reliability can be used to identify high risk components of the transit system and draw comparisons between modes both in planning and operations settings, and value improvement scenarios in a project evaluation setting. The methodology can also be applied to inform daily transit system operational management.

Planning for transit system reliability using productive performance and risk assessment

Urban transit system performance may be quantified and assessed using transit capacity and productive capacity for planning, design and operational management. Bunker (4) defines important productive performance measures of an individual transit service and transit line. Transit work (p-km) captures transit task performed over distance. Transit productiveness (p-km/h) captures transit work performed over time. This paper applies productive performance with risk assessment to quantify transit system reliability. Theory is developed to monetize transit segment reliability risk on the basis of demonstration Annual Reliability Event rates by transit facility type, segment productiveness, and unit-event severity. A comparative example of peak hour performance of a transit sub-system containing bus-on-street, busway, and rail components in Brisbane, Australia demonstrates through practical application the importance of valuing reliability. Comparison reveals the highest risk segments to be long, highly productive on street bus segments followed by busway (BRT) segments and then rail segments. A transit reliability risk reduction treatment example demonstrates that benefits can be significant and should be incorporated into project evaluation in addition to those of regular travel time savings, reduced emissions and safety improvements. Reliability can be used to identify high risk components of the transit system and draw comparisons between modes both in planning and operations settings, and value improvement scenarios in a project evaluation setting. The methodology can also be applied to inform daily transit system operational management.

Contribution of knowledge management activities to organisational business performance

Purpose – Within the construction industry there is a growing awareness of the need for linking knowledge management (KM) to business strategy, organisational objectives and existing performance measures. This study was undertaken within the context of construction organisations, and attempts to provide the empirical evidence about the relationships between KM activities and organisational business performance. Design/methodology/approach – A questionnaire survey was administered to a sample of construction contractors operating in Hong Kong to investigate the opinions of construction professionals regarding the intensity of KM activities and business performance within their organisations. In parallel to the survey, semi-structured interviews were undertaken to provide qualitative insights that helped to clarify and deepen understanding of the KM process within the context of the research target. Findings – The investigation shows that knowledge utilisation is the strongest contributor to general business performance. In addition, the impact of KM activities on the lagging performance indicators of the Balanced Scorecard (BSC), such as “financial performance”, is in an indirect manner, and through the leading indicators such as performance from “internal process” as well as “learning and growth” perspectives. Originality/value – The study empirically establishes the linkage between intensity of KM activities and business performance, and demonstrates that KM strategies need to be explicitly formulated and measured according to organisational business objectives.

Practical framework for measuring performance of international construction firms

The internationalization of construction companies has become of significant interest as the global construction market continues to be integrated into a more competitive and turbulent business environment. However, due to the complicated and multifaceted nature of international business and performance, there is as yet no consensus on how to evaluate the performance of international construction firms (ICFs). The purpose of this paper, therefore, is to develop a practical framework for measuring the performance of ICFs. Based on the balanced scorecard (BSC), a framework with detailed measures is developed, investigated, and tested using a three-step research design. In the first step, 27 measures under six dimensions (financial, market, customer, internal business processes, stakeholders, and learning and growth) are determined by literature review, interviews with academics, and seminar discussions. Subsequently, a questionnaire survey is conducted to investigate weights of these 27 performance measures. The questionnaire survey also supports the importance of measuring intangible aspects of international construction performance from the practitioner’s viewpoint. Additionally, a case study is described to test the framework’s robustness and usefulness. This is achieved by benchmarking the performance of a Chinese ICF with nine other counterparts worldwide. It is found that the framework provides an effective basis for benchmarking ICFs to effectively monitor their performance and support the development of strategies for improved competitiveness in the international arena. This paper is the first attempt to present a balanced and practically tested framework for evaluating the performance of ICFs. It contributes to the practice of performance measurement and related internationalization in the construction industry in general.