The binding site distance test score: a robust method for the assessment of predicted protein binding sites

We propose a novel method for scoring the accuracy of protein binding site predictions – the Binding-site Distance Test (BDT) score. Recently, the Matthews Correlation Coefficient (MCC) has been used to evaluate binding site predictions, both by developers of new methods and by the assessors for the community wide prediction experiment – CASP8. Whilst being a rigorous scoring method, the MCC does not take into account the actual 3D location of the predicted residues from the observed binding site. Thus, an incorrectly predicted site that is nevertheless close to the observed binding site will obtain an identical score to the same number of nonbinding residues predicted at random. The MCC is somewhat affected by the subjectivity of determining observed binding residues and the ambiguity of choosing distance cutoffs. By contrast the BDT method produces continuous scores ranging between 0 and 1, relating to the distance between the predicted and observed residues. Residues predicted close to the binding site will score higher than those more distant, providing a better reflection of the true accuracy of predictions. The CASP8 function predictions were evaluated using both the MCC and BDT methods and the scores were compared. The BDT was found to strongly correlate with the MCC scores whilst also being less susceptible to the subjectivity of defining binding residues. We therefore suggest that this new simple score is a potentially more robust method for future evaluations of protein-ligand binding site predictions.

The sequential analysis of repeated binary responses: a score test for the case of three time points

In this paper a robust method is developed for the analysis of data consisting of repeated binary observations taken at up to three fixed time points on each subject. The primary objective is to compare outcomes at the last time point, using earlier observations to predict this for subjects with incomplete records. A score test is derived. The method is developed for application to sequential clinical trials, as at interim analyses there will be many incomplete records occurring in non-informative patterns. Motivation for the methodology comes from experience with clinical trials in stroke and head injury, and data from one such trial is used to illustrate the approach. Extensions to more than three time points and to allow for stratification are discussed. Copyright © 2005 John Wiley & Sons, Ltd.

A score test for binary data with patient non-compliance

A score test is developed for binary clinical trial data, which incorporates patient non-compliance while respecting randomization. It is assumed in this paper that compliance is all-or-nothing, in the sense that a patient either accepts all of the treatment assigned as specified in the protocol, or none of it. Direct analytic comparisons of the adjusted test statistic for both the score test and the likelihood ratio test are made with the corresponding test statistics that adhere to the intention-to-treat principle. It is shown that no gain in power is possible over the intention-to-treat analysis, by adjusting for patient non-compliance. Sample size formulae are derived and simulation studies are used to demonstrate that the sample size approximation holds. Copyright © 2003 John Wiley & Sons, Ltd.

Variation in the ovarian reserve Is linked to alterations in intrafollicular estradiol production and ovarian biomarkers of follicular differentiation and oocyte quality in cattle

The mechanisms whereby the high variation in numbers of morphologically healthy oocytes and follicles in ovaries (ovarian reserve) may have an impact onovarian function, oocyte quality, and fertility are poorly understood. The objective was to determine whether previously validated biomarkers for follicular differentiation and function, as well as oocyte quality differed between cattle with low versus a high antral follicle count (AFC). Ovaries were removed (n = 5 per group) near the beginning of the nonovulatory follicular wave, before follicles could be identified via ultrasonography as being dominant, from heifers with high versus a low AFC. The F1, F2, and F3 follicles were dissected and diameters determined. Follicular fluid and thecal, granulosal, and cumulus cells and the oocyte were isolated and subjected to biomarker analyses. Although the size and numerous biomarkers of differentiation, such as mRNAs for the gonadotropin receptors, were similar, intrafollicular concentrations of estradiol and the abundance of mRNAs for CYP19A1 in granulosal cells and ESR1, ESR2, and CTSB in cumulus cells were greater, whereas mRNAs for AMH in granulosal cells and TBC1D1 in thecal cells were lower for animals with low versus a high AFC during follicle waves. Hence, variation in the ovarian reserve may have an impact on follicular function and oocyte quality via alterations in intrafollicular estradiol production and expression of key genes involved in follicle-stimulating hormone action (AMH) and estradiol (CYP19A1) production by granulosal cells, function and survival of thecal cells (TBC1D1), responsiveness of cumulus cells to estradiol (ESR1, ESR2), and cumulus cell determinants of oocyte quality (CTSB).

Evidence for an inhibitory role of bone morphogenetic protein(s) in the follicular-luteal transition in cattle

Bone morphogenetic proteins (BMPs) and their receptors are expressed in ovarian theca cells (TC) and granulosa cells (GC) and BMPs have been implicated in the regulation of several aspects of follicle development including thecal androgen production and granulosal oestrogen production. Their potential involvement in luteal function has received less attention. in this study, we first compared relative abundance of mRNA transcripts for BMPs, activin-beta A and BMP/activin receptors in bovine corpus luteum (CL) and follicular theca and granulosa layers before undertaking functional in vitro experiments to test the effect of selected ligands (BMP6 and activin A) on luteinizing bovine TC and GC. Relative to P-actin transcript abundance, CL tissue contained more BMP4 and -6 mRNA than granulosa, more BMP2 mRNA than theca but much less activin-beta A mRNA than both granulosa and theca. Transcripts for all seven BMP/activin receptors were readily detected in each tissue and two transcripts (BMPRII, ActRIIA) were more abundant in CL than either theca or granulosa, consistent with tissue responsiveness. In vitro luteinization of TC and GC from antral follicles (4-6 mm) was achieved by culturing with 5% serum for 6 days. Treatment with BMP6 (0, 2, 10, and 50 ng/ml) and activin A (0, 2, 10 and 50 ng/ml) under these conditions dose-dependently suppressed forskolin-induced progesterone (P-4) secretion from both cell types without affecting cell number. BMP6 reduced forskolin-stimulated upregulation of STAR mRNA and raised 'basal' CYP17A1 mRNA level in theca-lutein cells without affecting expression of CYP11A1 or hydroxy-Delta-5-steroid dehydrogenase, 3 beta- and steroid Delta-isomerase 1 (HSD3B1). In granulosa-lutein cells, STAR transcript abundance was not affected by BMP6, whereas forskolin-induced expression of CYP11A1, HSD3B1, CYP19A1 and oxytocin transcripts was reduced. In both cell types, follistatin attenuated the suppressive effect of activin A and BMP6 on forskolin-induced P4 secretion but had no effect alone. Granulosa-lutein cells secreted low levels of endogenous activin A (similar to 1 ng/ml); BMP6 reduced this, while raising follistatin secretion thus decreasing activin A:follistatin ratio. Collectively, these findings support inhibitory roles for BMP/activin signalling in luteinization and steroidogenesis in both TC and GC.

Bovine follicle development is associated with divergent changes in activin-A, inhibin-A and follistatin and the relative abundance of different follistatin isoforms in follicular fluid

The aim was to determine whether follicle growth in cattle is accompanied by changes in levels of inhibin-A (inh-A), activin-A (act-A) and different Mr isofomus of follistatin (FS) in bovine follicular fluid (bFF), reflecting differential roles of these proteins during folliculogenesis. Follcles (n= 146) from 2-20 min diameter were dissected from ovaries of similar to 40 cattle. Immunoassays were used to measure total FS, act-A, inh-A, oestradiol (E) and progesterone (P) levels; immunoblotting was used to quanti, the relative abundance of different FS isoforms. Follicle growth from 2-6 mm was associated with a 6-fold increase in inh-A and 30-fold increase in act-A; FS remained uniformly high from 2-10 turn. From 6-2 min, inh-A remained high while act-A and FS fell 3-fold and 2-fold, respectively. Act-A/FS ratio increased 20-fold from 2-6 mm before falling slightly through to 20 mm. Act-A/inh-A ratio increased 6-fold from 2-6 nun before falling 2-fold from 6 to 17-20 mm. These findings imply a marked increase in relative activin 'tone' around the stage at which dominant follicle,;election occurs. When larger follicles (13-20 mm) were subdivided according to E/P ratio, those with high (> 5) E/P ratio had lower (2-fold; P < 0(.)001) levels of inh-A and act-A in comparison to follicles with low (< 5) E/P ratio, but there were no significant diffierences in FS, act-A/inh-A ratio or act-A/FS ratio. Thus follicle size, but not oestrogenic status, has a major influence on the intrafollicular balance between act-A and its opposing factors, inh-A and FS. Six FS isoforms were detected in bFF (apparent Mr: 65, 41, 37, 35, 33 and 31 kDa) averaging 6, 13, 24, 26 13 and 17% respectively of total FS. During growth from 2-20 mm the proportion of total FS represented by 605, 41 and 37 kDa isoforms increased similar to 2-fold while the proportion represented by the 33 and 31 kDa isoforms decreased by 3-fold and 1(.)6-fold, respectively. Treatment of bovine granulosa cells in vitro with FSH and IGF alone or in combination increased total FS secretion up to 12-fold but did not affect the relative abundance of the five different FS isoforms detected. While the functional significance of the intriguing shift in FS isoform abundance in bFF during follicle development remains to be established, we have shown that a marked increase in intrafollicular activin 'tone' accompanies bovine follicle growth from 3-6 min, corresponding to the stage at which the FSH-dependent follicle selection mechanism operates in this species.

Numbers of antral follicles during follicular waves in cattle: Evidence for high variation among animals, very high repeatability in individuals, and an inverse association with serum follicle-stimulating hormone concentrations

The extent, causes, and physiological significance of the variation in number of follicles growing during ovarian follicular waves in human beings and cattle are unknown. Therefore, the present study examined the variability and repeatability in numbers of follicles 3 mm or greater in diameter during the follicular waves in bovine estrous cycles, and we determined if the variation in number of follicles during waves was associated with alterations in secretion of FSH, estradiol, inhibin, and insulin-like growth factor I (IGF-I). Dairy cattle were subjected to twice-daily ultrasound analysis to count total number of antral follicles 3 mm or greater in diameter throughout 138 different follicular waves. In another study, blood samples were taken at frequent intervals from cows that consistently had low or very high numbers of follicles during waves and were subjected to immunoassays. Results indicate the following: First, despite an approximately sevenfold variation in number of follicles during waves among animals and marked differences in age, stage of lactation, and season of the year, a very highly repeatable (0.95) number of follicles 3 mm or greater in diameter is maintained during the ovulatory and nonovulatory follicular waves of individuals. Second, variation in number of follicles 3 mm or greater in diameter during waves and the inverse association of number of follicles during waves with FSH are not directly explained by alterations in the patterns of secretion of estradiol, inhibin, or IGF-I. Third, ovarian ultrasound analysis can be used reliably by investigators to identify cattle that consistently have low or high numbers of follicles during waves, thus providing a novel experimental model to determine the causes and physiological significance of the high variation in antral follicle number during follicular waves among single-ovulating species, such as cattle or humans.

Sparse kernel density construction using orthogonal forward regression with leave-one-out test score and local regularization

This paper presents an efficient construction algorithm for obtaining sparse kernel density estimates based on a regression approach that directly optimizes model generalization capability. Computational efficiency of the density construction is ensured using an orthogonal forward regression, and the algorithm incrementally minimizes the leave-one-out test score. A local regularization method is incorporated naturally into the density construction process to further enforce sparsity. An additional advantage of the proposed algorithm is that it is fully automatic and the user is not required to specify any criterion to terminate the density construction procedure. This is in contrast to an existing state-of-art kernel density estimation method using the support vector machine (SVM), where the user is required to specify some critical algorithm parameter. Several examples are included to demonstrate the ability of the proposed algorithm to effectively construct a very sparse kernel density estimate with comparable accuracy to that of the full sample optimized Parzen window density estimate. Our experimental results also demonstrate that the proposed algorithm compares favorably with the SVM method, in terms of both test accuracy and sparsity, for constructing kernel density estimates.

A combined score test for binary and ordinal endpoints from clinical trials

There is growing interest, especially for trials in stroke, in combining multiple endpoints in a single clinical evaluation of an experimental treatment. The endpoints might be repeated evaluations of the same characteristic or alternative measures of progress on different scales. Often they will be binary or ordinal, and those are the cases studied here. In this paper we take a direct approach to combining the univariate score statistics for comparing treatments with respect to each endpoint. The correlations between the score statistics are derived and used to allow a valid combined score test to be applied. A sample size formula is deduced and application in sequential designs is discussed. The method is compared with an alternative approach based on generalized estimating equations in an illustrative analysis and replicated simulations, and the advantages and disadvantages of the two approaches are discussed.

Equitability revisited: why the “equitable threat score” is not equitable

In the forecasting of binary events, verification measures that are “equitable” were defined by Gandin and Murphy to satisfy two requirements: 1) they award all random forecasting systems, including those that always issue the same forecast, the same expected score (typically zero), and 2) they are expressible as the linear weighted sum of the elements of the contingency table, where the weights are independent of the entries in the table, apart from the base rate. The authors demonstrate that the widely used “equitable threat score” (ETS), as well as numerous others, satisfies neither of these requirements and only satisfies the first requirement in the limit of an infinite sample size. Such measures are referred to as “asymptotically equitable.” In the case of ETS, the expected score of a random forecasting system is always positive and only falls below 0.01 when the number of samples is greater than around 30. Two other asymptotically equitable measures are the odds ratio skill score and the symmetric extreme dependency score, which are more strongly inequitable than ETS, particularly for rare events; for example, when the base rate is 2% and the sample size is 1000, random but unbiased forecasting systems yield an expected score of around −0.5, reducing in magnitude to −0.01 or smaller only for sample sizes exceeding 25 000. This presents a problem since these nonlinear measures have other desirable properties, in particular being reliable indicators of skill for rare events (provided that the sample size is large enough). A potential way to reconcile these properties with equitability is to recognize that Gandin and Murphy’s two requirements are independent, and the second can be safely discarded without losing the key advantages of equitability that are embodied in the first. This enables inequitable and asymptotically equitable measures to be scaled to make them equitable, while retaining their nonlinearity and other properties such as being reliable indicators of skill for rare events. It also opens up the possibility of designing new equitable verification measures.

A method for the economic valuation of animal welfare benefits using a single welfare score

Unless the benefits to society of measures to protect and improve the welfare of animals are made transparent by means of their valuation they are likely to go unrecognised and cannot easily be weighed against the costs of such measures as required, for example, by policy-makers. A simple single measure scoring system, based on the Welfare Quality® index, is used, together with a choice experiment economic valuation method, to estimate the value that people place on improvements to the welfare of different farm animal species measured on a continuous (0-100) scale. Results from using the method on a survey sample of some 300 people show that it is able to elicit apparently credible values. The survey found that 96% of respondents thought that we have a moral obligation to safeguard the welfare of animals and that over 72% were concerned about the way farm animals are treated. Estimated mean annual willingness to pay for meat from animals with improved welfare of just one point on the scale was £5.24 for beef cattle, £4.57 for pigs and £5.10 for meat chickens. Further development of the method is required to capture the total economic value of animal welfare benefits. Despite this, the method is considered a practical means for obtaining economic values that can be used in the cost-benefit appraisal of policy measures intended to improve the welfare of animals.