A combined score test for binary and ordinal endpoints from clinical trials

There is growing interest, especially for trials in stroke, in combining multiple endpoints in a single clinical evaluation of an experimental treatment. The endpoints might be repeated evaluations of the same characteristic or alternative measures of progress on different scales. Often they will be binary or ordinal, and those are the cases studied here. In this paper we take a direct approach to combining the univariate score statistics for comparing treatments with respect to each endpoint. The correlations between the score statistics are derived and used to allow a valid combined score test to be applied. A sample size formula is deduced and application in sequential designs is discussed. The method is compared with an alternative approach based on generalized estimating equations in an illustrative analysis and replicated simulations, and the advantages and disadvantages of the two approaches are discussed.

Equitability revisited: why the “equitable threat score” is not equitable

In the forecasting of binary events, verification measures that are “equitable” were defined by Gandin and Murphy to satisfy two requirements: 1) they award all random forecasting systems, including those that always issue the same forecast, the same expected score (typically zero), and 2) they are expressible as the linear weighted sum of the elements of the contingency table, where the weights are independent of the entries in the table, apart from the base rate. The authors demonstrate that the widely used “equitable threat score” (ETS), as well as numerous others, satisfies neither of these requirements and only satisfies the first requirement in the limit of an infinite sample size. Such measures are referred to as “asymptotically equitable.” In the case of ETS, the expected score of a random forecasting system is always positive and only falls below 0.01 when the number of samples is greater than around 30. Two other asymptotically equitable measures are the odds ratio skill score and the symmetric extreme dependency score, which are more strongly inequitable than ETS, particularly for rare events; for example, when the base rate is 2% and the sample size is 1000, random but unbiased forecasting systems yield an expected score of around −0.5, reducing in magnitude to −0.01 or smaller only for sample sizes exceeding 25 000. This presents a problem since these nonlinear measures have other desirable properties, in particular being reliable indicators of skill for rare events (provided that the sample size is large enough). A potential way to reconcile these properties with equitability is to recognize that Gandin and Murphy’s two requirements are independent, and the second can be safely discarded without losing the key advantages of equitability that are embodied in the first. This enables inequitable and asymptotically equitable measures to be scaled to make them equitable, while retaining their nonlinearity and other properties such as being reliable indicators of skill for rare events. It also opens up the possibility of designing new equitable verification measures.

A method for the economic valuation of animal welfare benefits using a single welfare score

Unless the benefits to society of measures to protect and improve the welfare of animals are made transparent by means of their valuation they are likely to go unrecognised and cannot easily be weighed against the costs of such measures as required, for example, by policy-makers. A simple single measure scoring system, based on the Welfare Quality® index, is used, together with a choice experiment economic valuation method, to estimate the value that people place on improvements to the welfare of different farm animal species measured on a continuous (0-100) scale. Results from using the method on a survey sample of some 300 people show that it is able to elicit apparently credible values. The survey found that 96% of respondents thought that we have a moral obligation to safeguard the welfare of animals and that over 72% were concerned about the way farm animals are treated. Estimated mean annual willingness to pay for meat from animals with improved welfare of just one point on the scale was £5.24 for beef cattle, £4.57 for pigs and £5.10 for meat chickens. Further development of the method is required to capture the total economic value of animal welfare benefits. Despite this, the method is considered a practical means for obtaining economic values that can be used in the cost-benefit appraisal of policy measures intended to improve the welfare of animals.

Estimating reliability and resolution of probability forecasts through decomposition of the empirical score

References (20)Cited By (1)Export CitationAboutAbstract Proper scoring rules provide a useful means to evaluate probabilistic forecasts. Independent from scoring rules, it has been argued that reliability and resolution are desirable forecast attributes. The mathematical expectation value of the score allows for a decomposition into reliability and resolution related terms, demonstrating a relationship between scoring rules and reliability/resolution. A similar decomposition holds for the empirical (i.e. sample average) score over an archive of forecast–observation pairs. This empirical decomposition though provides a too optimistic estimate of the potential score (i.e. the optimum score which could be obtained through recalibration), showing that a forecast assessment based solely on the empirical resolution and reliability terms will be misleading. The differences between the theoretical and empirical decomposition are investigated, and specific recommendations are given how to obtain better estimators of reliability and resolution in the case of the Brier and Ignorance scoring rule.

Evaluating raw ensembles with the continuous ranked probability score

The continuous ranked probability score (CRPS) is a frequently used scoring rule. In contrast with many other scoring rules, the CRPS evaluates cumulative distribution functions. An ensemble of forecasts can easily be converted into a piecewise constant cumulative distribution function with steps at the ensemble members. This renders the CRPS a convenient scoring rule for the evaluation of ‘raw’ ensembles, obviating the need for sophisticated ensemble model output statistics or dressing methods prior to evaluation. In this article, a relation between the CRPS score and the quantile score is established. The evaluation of ‘raw’ ensembles using the CRPS is discussed in this light. It is shown that latent in this evaluation is an interpretation of the ensemble as quantiles but with non-uniform levels. This needs to be taken into account if the ensemble is evaluated further, for example with rank histograms.

'Score it upon my taille': the use (and abuse) of tallies by the medieval Exchequer

The paper traces the evolution of the tally from a receipt for cash payments into the treasury, to proof of payments made by royal officials outside of the treasury and finally to an assignment of revenue to be paid out by royal officials. Each of these processes is illustrated by examples drawn from the Exchequer records and explains their significance for royal finance and for historians working on the Exchequer records.

Forecast convergence score: a forecaster's approach to analysing hydro-meteorological forecast systems.

In this paper the properties of a hydro-meteorological forecasting system for forecasting river flows have been analysed using a probabilistic forecast convergence score (FCS). The focus on fixed event forecasts provides a forecaster's approach to system behaviour and adds an important perspective to the suite of forecast verification tools commonly used in this field. A low FCS indicates a more consistent forecast. It can be demonstrated that the FCS annual maximum decreases over the last 10 years. With lead time, the FCS of the ensemble forecast decreases whereas the control and high resolution forecast increase. The FCS is influenced by the lead time, threshold and catchment size and location. It indicates that one should use seasonality based decision rules to issue flood warnings.

Cocoa flavanol intake improves endothelial function and Framingham Risk Score in healthy men and women: a randomised, controlled, double-masked trial: the Flaviola Health Study

Cocoa flavanol (CF) intake improves endothelial function in patients with cardiovascular risk factors and disease. We investigated the effects of CF on surrogate markers of cardiovascular health in low risk, healthy, middle-aged individuals without history, signs or symptoms of CVD. In a 1-month, open-label, one-armed pilot study, bi-daily ingestion of 450 mg of CF led to a time-dependent increase in endothelial function (measured as flow-mediated vasodilation (FMD)) that plateaued after 2 weeks. Subsequently, in a randomised, controlled, double-masked, parallel-group dietary intervention trial (Clinicaltrials.gov: NCT01799005), 100 healthy, middle-aged (35–60 years) men and women consumed either the CF-containing drink (450 mg) or a nutrient-matched CF-free control bi-daily for 1 month. The primary end point was FMD. Secondary end points included plasma lipids and blood pressure, thus enabling the calculation of Framingham Risk Scores and pulse wave velocity. At 1 month, CF increased FMD over control by 1·2 % (95 % CI 1·0, 1·4 %). CF decreased systolic and diastolic blood pressure by 4·4 mmHg (95 % CI 7·9, 0·9 mmHg) and 3·9 mmHg (95 % CI 6·7, 0·9 mmHg), pulse wave velocity by 0·4 m/s (95 % CI 0·8, 0·04 m/s), total cholesterol by 0·20 mmol/l (95 % CI 0·39, 0·01 mmol/l) and LDL-cholesterol by 0·17 mmol/l (95 % CI 0·32, 0·02 mmol/l), whereas HDL-cholesterol increased by 0·10 mmol/l (95 % CI 0·04, 0·17 mmol/l). By applying the Framingham Risk Score, CF predicted a significant lowering of 10-year risk for CHD, myocardial infarction, CVD, death from CHD and CVD. In healthy individuals, regular CF intake improved accredited cardiovascular surrogates of cardiovascular risk, demonstrating that dietary flavanols have the potential to maintain cardiovascular health even in low-risk subjects.

The iso-Score Curve Graph. A new tool for competitive bidding

The present work describes a new tool that helps bidders improve their competitive bidding strategies. This new tool consists of an easy-to-use graphical tool that allows the use of more complex decision analysis tools in the field of Competitive Bidding. The graphic tool described here tries to move away from previous bidding models which attempt to describe the result of an auction or a tender process by means of studying each possible bidder with probability density functions. As an illustration, the tool is applied to three practical cases. Theoretical and practical conclusions on the great potential breadth of application of the tool are also presented.

Fatigue and workability in Brazilian textile companies in different corporate social responsibility score groups

The aim of this study was to evaluate working conditions in the textile industry for different stages of Corporate Social Responsibility (CSR) development, and workers` perception of fatigue and workability. A cross-sectional study was undertaken with 126 workers in the production areas of five Brazilian textile plants. The corporate executive officers and managers of each company provided their personal evaluations of CSR. Companies were divided into 2 groups (higher and lower) of CSR scores. Workers completed questionnaires on fatigue, workability and working conditions. Ergonomic job analysis showed similar results for working conditions, independent of their CSR score. Multivariate analysis models were developed for fatigue and workability, indicating that they are both associated to factors related to working conditions and individual workers` characteristics and life styles. Work organization, (what, how, when, where and for how long the work is done), is also an associated factor for fatigue. This study suggests that workers` opinions should be taken into greater consideration when companies develop their CSR programs, in particular for those relating to working conditions. Relevance to industry: This paper underlines the importance of considering working conditions and workers` opinions of them, work organization and individual workers` characteristics and life styles in order to restore or to maintain workability and to reduce fatigue, independently of how developed a company may be in the field of Corporate Social Responsibility. (C) 2010 Elsevier B.V. All rights reserved.

Improved score tests in symmetric linear regression models

The class of symmetric linear regression models has the normal linear regression model as a special case and includes several models that assume that the errors follow a symmetric distribution with longer-than-normal tails. An important member of this class is the t linear regression model, which is commonly used as an alternative to the usual normal regression model when the data contain extreme or outlying observations. In this article, we develop second-order asymptotic theory for score tests in this class of models. We obtain Bartlett-corrected score statistics for testing hypotheses on the regression and the dispersion parameters. The corrected statistics have chi-squared distributions with errors of order O(n(-3/2)), n being the sample size. The corrections represent an improvement over the corresponding original Rao`s score statistics, which are chi-squared distributed up to errors of order O(n(-1)). Simulation results show that the corrected score tests perform much better than their uncorrected counterparts in samples of small or moderate size.

Small-Sample Corrections for Score Tests in Birnbaum-Saunders Regressions

In this article, we deal with the issue of performing accurate small-sample inference in the Birnbaum-Saunders regression model, which can be useful for modeling lifetime or reliability data. We derive a Bartlett-type correction for the score test and numerically compare the corrected test with the usual score test and some other competitors.

Three Bartlett-type corrections for score statistics in symmetric nonlinear regression models

We present simple matrix formulae for corrected score statistics in symmetric nonlinear regression models. The corrected score statistics follow more closely a chi (2) distribution than the classical score statistic. Our simulation results indicate that the corrected score tests display smaller size distortions than the original score test. We also compare the sizes and the powers of the corrected score tests with bootstrap-based score tests.