A combined score test for binary and ordinal endpoints from clinical trials

There is growing interest, especially for trials in stroke, in combining multiple endpoints in a single clinical evaluation of an experimental treatment. The endpoints might be repeated evaluations of the same characteristic or alternative measures of progress on different scales. Often they will be binary or ordinal, and those are the cases studied here. In this paper we take a direct approach to combining the univariate score statistics for comparing treatments with respect to each endpoint. The correlations between the score statistics are derived and used to allow a valid combined score test to be applied. A sample size formula is deduced and application in sequential designs is discussed. The method is compared with an alternative approach based on generalized estimating equations in an illustrative analysis and replicated simulations, and the advantages and disadvantages of the two approaches are discussed.

Understanding of auditory discourse in older adults: the effects of syntax and working memory

Background: As people age, language-processing ability changes. While several factors modify discourse comprehension ability in older adults, syntactic complexity of auditory discourse has received scant attention. This is despite the widely researched domain of syntactic processing of single sentences in older adults. Aims: The aims of this study were to investigate the ability of healthy older adults to understand stories that differed in syntactic complexity, and its relation to working memory. Methods & Procedures: A total of 51 healthy adults (divided into three age groups) took part. They listened to brief stories (syntactically simple and syntactically complex) and had to respond to false/true comprehension probes following each story. Working memory capacity (digit span, forward and backward) was also measured. Outcomes & Results: Differences were found in the ability of healthy older adults to understand simple and complex discourse. The complex discourse in particular was more sensitive in discerning age-related language patterns. Only the complex discourse task correlated moderately with age. There was no correlation between age and simple discourse. As far as working memory is concerned, moderate correlations were found between working memory and complex discourse. Education did not correlate with discourse, neither simple, nor complex. Conclusions: Older adults may be less efficient in forming syntactically complex representations and this may be influenced by limitations in working memory.

Equitability revisited: why the “equitable threat score” is not equitable

In the forecasting of binary events, verification measures that are “equitable” were defined by Gandin and Murphy to satisfy two requirements: 1) they award all random forecasting systems, including those that always issue the same forecast, the same expected score (typically zero), and 2) they are expressible as the linear weighted sum of the elements of the contingency table, where the weights are independent of the entries in the table, apart from the base rate. The authors demonstrate that the widely used “equitable threat score” (ETS), as well as numerous others, satisfies neither of these requirements and only satisfies the first requirement in the limit of an infinite sample size. Such measures are referred to as “asymptotically equitable.” In the case of ETS, the expected score of a random forecasting system is always positive and only falls below 0.01 when the number of samples is greater than around 30. Two other asymptotically equitable measures are the odds ratio skill score and the symmetric extreme dependency score, which are more strongly inequitable than ETS, particularly for rare events; for example, when the base rate is 2% and the sample size is 1000, random but unbiased forecasting systems yield an expected score of around −0.5, reducing in magnitude to −0.01 or smaller only for sample sizes exceeding 25 000. This presents a problem since these nonlinear measures have other desirable properties, in particular being reliable indicators of skill for rare events (provided that the sample size is large enough). A potential way to reconcile these properties with equitability is to recognize that Gandin and Murphy’s two requirements are independent, and the second can be safely discarded without losing the key advantages of equitability that are embodied in the first. This enables inequitable and asymptotically equitable measures to be scaled to make them equitable, while retaining their nonlinearity and other properties such as being reliable indicators of skill for rare events. It also opens up the possibility of designing new equitable verification measures.

Sentence repetition in adolescents with specific language impairments and autism: an investigation of complex syntax

Background: Recent studies have indicated that many children with autism spectrum disorders present with language difficulties that are similar to those of children with specific language impairments, leading some to argue for similar structural deficits in these two disorders. Aims: Repetition of sentences involving long-distance dependencies was used to investigate complex syntax in these groups. Methods & Procedures: Adolescents with specific language impairments (mean age = 15;3, n = 14) and autism spectrum disorders plus language impairment (autism plus language impairment; mean age = 14;8, n = 16) were recruited alongside typically developing adolescents (mean age = 14;4, n = 17). They were required to repeat sentences containing relative clauses that varied in syntactic complexity. Outcomes & Results: The adolescents with specific language impairments presented with greater syntactic difficulties than the adolescents with autism plus language impairment, as manifested by higher error rates on the more complex object relative clauses, and a greater tendency to make syntactic changes during repetition. Conclusions & Implications: Adolescents with specific language impairments may have more severe syntactic difficulties than adolescents with autism plus language impairment, possibly due to their short-term memory limitations.

A method for the economic valuation of animal welfare benefits using a single welfare score

Unless the benefits to society of measures to protect and improve the welfare of animals are made transparent by means of their valuation they are likely to go unrecognised and cannot easily be weighed against the costs of such measures as required, for example, by policy-makers. A simple single measure scoring system, based on the Welfare Quality® index, is used, together with a choice experiment economic valuation method, to estimate the value that people place on improvements to the welfare of different farm animal species measured on a continuous (0-100) scale. Results from using the method on a survey sample of some 300 people show that it is able to elicit apparently credible values. The survey found that 96% of respondents thought that we have a moral obligation to safeguard the welfare of animals and that over 72% were concerned about the way farm animals are treated. Estimated mean annual willingness to pay for meat from animals with improved welfare of just one point on the scale was £5.24 for beef cattle, £4.57 for pigs and £5.10 for meat chickens. Further development of the method is required to capture the total economic value of animal welfare benefits. Despite this, the method is considered a practical means for obtaining economic values that can be used in the cost-benefit appraisal of policy measures intended to improve the welfare of animals.

Differential effects of internal and external factors on the development of vocabulary, tense morphology and morpho-syntax in successive bilingual children

The present study investigates the effects of child internal (age/time) and child external/environmental factors on the development of a wide range of language domains in successive bilingual (L2) Turkish-English children of homogeneously low SES. Forty-three L2 children were tested on standardized assessments examining the acquisition of vocabulary and morpho-syntax. The L2 children exhibited a differential acquisition of the various domains: they were better on the general comprehension of grammar and tense morphology and less accurate on the acquisition of vocabulary and (complex) morpho-syntax. Profile effects were confirmed by the differential effects of internal and external factors on the language domains. The development of vocabulary and complex syntax were affected by internal and external factors, whereas external factors had no contribution to the development of tense morphology. These results are discussed in light of previous studies on the impact of internal and external factors in child L2 acquisition.

Estimating reliability and resolution of probability forecasts through decomposition of the empirical score

References (20)Cited By (1)Export CitationAboutAbstract Proper scoring rules provide a useful means to evaluate probabilistic forecasts. Independent from scoring rules, it has been argued that reliability and resolution are desirable forecast attributes. The mathematical expectation value of the score allows for a decomposition into reliability and resolution related terms, demonstrating a relationship between scoring rules and reliability/resolution. A similar decomposition holds for the empirical (i.e. sample average) score over an archive of forecast–observation pairs. This empirical decomposition though provides a too optimistic estimate of the potential score (i.e. the optimum score which could be obtained through recalibration), showing that a forecast assessment based solely on the empirical resolution and reliability terms will be misleading. The differences between the theoretical and empirical decomposition are investigated, and specific recommendations are given how to obtain better estimators of reliability and resolution in the case of the Brier and Ignorance scoring rule.

Evaluating raw ensembles with the continuous ranked probability score

The continuous ranked probability score (CRPS) is a frequently used scoring rule. In contrast with many other scoring rules, the CRPS evaluates cumulative distribution functions. An ensemble of forecasts can easily be converted into a piecewise constant cumulative distribution function with steps at the ensemble members. This renders the CRPS a convenient scoring rule for the evaluation of ‘raw’ ensembles, obviating the need for sophisticated ensemble model output statistics or dressing methods prior to evaluation. In this article, a relation between the CRPS score and the quantile score is established. The evaluation of ‘raw’ ensembles using the CRPS is discussed in this light. It is shown that latent in this evaluation is an interpretation of the ensemble as quantiles but with non-uniform levels. This needs to be taken into account if the ensemble is evaluated further, for example with rank histograms.

Subject-to-subject raising and the syntax of tense in L2 Spanish: a full access approach

This paper investigates the acquisition of syntax in L2 grammars. We tested adult L2 speakers of Spanish (English L1) on the feature specification of T(ense), which is different in English and Spanish in so-called subject-to-subject raising structures. We present experimental results with the verb parecer “to seem/to appear” in different tenses, with and without experiencers, and with Tense Phrase (TP), verb phrase (vP) and Adjectival Phrase (AP) complements. The results show that advanced L2 learners can perform just like native Spanish speakers regarding grammatical knowledge in this domain, although the subtle differences between both languages are not explicitly taught. We argue that these results support Full Access approaches to Universal Grammar (UG) in L2 acquisition, by providing evidence that uninterpretable syntactic features can be learned in adult L2, even when such features are not directly instantiated in the same grammatical domain in the L1 grammar.

Clitic-doubled left dislocation and focus fronting in L2 Spanish: a case of successful acquisition at the syntax-discourse interface

This experimental study tests the Interface Hypothesis by looking into processes at the syntax– discourse interface, teasing apart acquisition of syntactic, semantic and discourse knowledge. Adopting López’s (2009) pragmatic features [±a(naphor)] and [±c(ontrast)], which in combination account for the constructions of dislocation and fronting, we tested clitic left dislocation and fronted focus in the comprehension of English native speakers learning Spanish. Furthermore, we tested knowledge of an additional semantic property: the relationship between the discourse anaphor and the antecedent in clitic left dislocation (CLLD). This relationship is free: it can be subset, superset, part/whole. Syntactic knowledge of clitics was a condition for inclusion in the main test. Our findings indicate that all learners are sensitive to the semantic constraints. While the near-native speakers display native-like discourse knowledge, the advanced speakers demonstrated some discourse knowledge, and intermediate learners did not display any discourse knowledge. The findings support as well as challenge the Interface Hypothesis.

The syntax-semantics of bare and definite plural subjects in the L2 Spanish of English natives

This study investigates the extent to which advanced native-English L2 learners of Spanish come to acquire restrictions on bare plural preverbal subjects in L2 Spanish (e.g. gatos “cats” vs. definite plurals such as los gatos “the cats”). It tests L2 knowledge of available semantic readings of bare plurals and definite plurals in Spanish, where [+specific] and [+generic] interpretations are syntactically represented differently from English. Assuming L1 transfer, and in view of a potential subset/superset relationship of the two grammars, the learning task in this domain is not a straightforward one. Target acquisition requires both grammatical expansion and retraction; Spanish definite plural subjects require the addition of an L1-unavailable [+generic] reading, while a loss of an L1-available [+generic] reading for preverbal subject bare plurals is required. The results and analysis show that advanced L2 learners of Spanish (English L1) can circumvent a superficial subset/superset learnability problem by means of feature resetting in line with the Nominal Mapping Parameter.