Statnote 8: statistical power and sample size

Statistical software is now commonly available to calculate Power (P') and sample size (N) for most experimental designs. In many circumstances, however, sample size is constrained by lack of time, cost, and in research involving human subjects, the problems of recruiting suitable individuals. In addition, the calculation of N is often based on erroneous assumptions about variability and therefore such estimates are often inaccurate. At best, we would suggest that such calculations provide only a very rough guide of how to proceed in an experiment. Nevertheless, calculation of P' is very useful especially in experiments that have failed to detect a difference which the experimenter thought was present. We would recommend that P' should always be calculated in these circumstances to determine whether the experiment was actually too small to test null hypotheses adequately.

Sample size estimation and statistical power analyses

The concept of sample size and statistical power estimation is now something that Optometrists that want to perform research, whether it be in practice or in an academic institution, cannot simply hide away from. Ethics committees, journal editors and grant awarding bodies are now increasingly requesting that all research be backed up with sample size and statistical power estimation in order to justify any study and its findings. This article presents a step-by-step guide of the process for determining sample sizeand statistical power. It builds on statistical concepts presented in earlier articles in Optometry Today by Richard Armstrong and Frank Eperjesi.

The Effect of nutritional supplementation on subjective and objective measures of visual and retinal function:a random controlled trial

In industrialised countries age-related macular disease (ARMD) is the leading cause of visual loss in older people. Because oxidative stress is purported to be associated with an increased risk of disease development the role of antioxidant supplementation is of interest. Lutein is a carotenoid antioxidant that accumulates within the retina and is thought to filter blue light. Increased levels of lutein have been associated with reduced risk of developing ARMD and improvements in visual and retinal function in eyes with ARMD. The aim of this randomised controlled trial (RCT) was to investigate the effect of a lutein-based nutritional supplement on subjective and objective measures of visual function in healthy eyes and in eyes with age-related maculopathy (ARM) – an early form of ARMD. Supplement withdrawal effects were also investigated. A sample size of 66 healthy older (HO), healthy younger (HY), and ARM eyes were randomly allocated to receive a lutein-based supplement or no treatment for 40 weeks. The supplemented group then stopped supplementation to look at the effects of withdrawal over a further 20 weeks. The primary outcome measure was multifocal electroretinogram (mfERG) N1P1 amplitude. Secondary outcome measures were mfERG N1, P1 and N2 latency, contrast sensitivity (CS), Visual acuity (VA) and macular pigment optical density (MPOD). Sample sizes were sufficient for the RCT to have an 80% power to detect a significant clinical effect at the 5% significance level for all outcome measures when the healthy eye groups were combined, and CS, VA and mfERG in the ARM group. This RCT demonstrates significant improvements in MPOD in HY and HO supplemented eyes. When HY and HO supplemented groups were combined, MPOD improvements were maintained, and mfERG ring 2 P1 latency became shorter. On withdrawal of the supplement mfERG ring 1 N1P1 amplitude reduced in HO eyes. When HO and HY groups were combined, mfERG ring 1 and ring 2 N1P1 amplitudes were reduced. In ARM eyes, ring 3 N2 latency and ring 4 P1 latency became longer. These statistically significant changes may not be clinically significant. The finding that a lutein-based supplement increases MPOD in healthy eyes, but does not increase mfERG amplitudes contrasts with the CARMIS study and contributes to the debate on the use of nutritional supplementation in ARM.

What determines the size frequency distribution of β-amyloid (Aβ) deposits in Alzheimer's disease patients?

The factors determining the size of individual β-amyloid (A,8) deposits and their size frequency distribution in tissue from Alzheimer's disease (AD) patients have not been established. In 23/25 cortical tissues from 10 AD patients, the frequency of Aβ deposits declined exponentially with increasing size. In a random sample of 400 Aβ deposits, 88% were closely associated with one or more neuronal cell bodies. The frequency distribution of (Aβ) deposits which were associated with 0,1,2,...,n neuronal cell bodies deviated significantly from a Poisson distribution, suggesting a degree of clustering of the neuronal cell bodies. In addition, the frequency of Aβ deposits declined exponentially as the number of associated neuronal cell bodies increased. Aβ deposit area was positively correlated with the frequency of associated neuronal cell bodies, the degree of correlation being greater for pyramidal cells than smaller neurons. These data suggested: (1) the number of closely adjacent neuronal cell bodies which simultaneously secrete Aβ was an important factor determining the size of an Aβ deposit and (2) the exponential decline in larger Aβ deposits reflects the low probability that larger numbers of adjacent neurons will secrete Aβ simultaneously to form a deposit. © 1995.

The exact sample complexity of PAC-learning problems with unit VC dimension

The Vapnik-Chervonenkis (VC) dimension is a combinatorial measure of a certain class of machine learning problems, which may be used to obtain upper and lower bounds on the number of training examples needed to learn to prescribed levels of accuracy. Most of the known bounds apply to the Probably Approximately Correct (PAC) framework, which is the framework within which we work in this paper. For a learning problem with some known VC dimension, much is known about the order of growth of the sample-size requirement of the problem, as a function of the PAC parameters. The exact value of sample-size requirement is however less well-known, and depends heavily on the particular learning algorithm being used. This is a major obstacle to the practical application of the VC dimension. Hence it is important to know exactly how the sample-size requirement depends on VC dimension, and with that in mind, we describe a general algorithm for learning problems having VC dimension 1. Its sample-size requirement is minimal (as a function of the PAC parameters), and turns out to be the same for all non-trivial learning problems having VC dimension 1. While the method used cannot be naively generalised to higher VC dimension, it suggests that optimal algorithm-dependent bounds may improve substantially on current upper bounds.

Sampling and measurement methods for a study of childhood refractive error in a UK population

Background There is a paucity of data describing the prevalence of childhood refractive error in the United Kingdom. The Northern Ireland Childhood Errors of Refraction study, along with its sister study the Aston Eye Study, are the first population-based surveys of children using both random cluster sampling and cycloplegic autorefraction to quantify levels of refractive error in the United Kingdom. Methods Children aged 6–7 years and 12–13 years were recruited from a stratified random sample of primary and post-primary schools, representative of the population of Northern Ireland as a whole. Measurements included assessment of visual acuity, oculomotor balance, ocular biometry and cycloplegic binocular open-field autorefraction. Questionnaires were used to identify putative risk factors for refractive error. Results 399 (57%) of 6–7 years and 669 (60%) of 12–13 years participated. School participation rates did not vary statistically significantly with the size of the school, whether the school is urban or rural, or whether it is in a deprived/non-deprived area. The gender balance, ethnicity and type of schooling of participants are reflective of the Northern Ireland population. Conclusions The study design, sample size and methodology will ensure accurate measures of the prevalence of refractive errors in the target population and will facilitate comparisons with other population-based refractive data.

Statnote 39: The binomial distribution: comparing two proportions

Many populations consist of two classes only, e.g., alive or dead, present or absent, clean or dirty, infected or non-infected, and it is the proportion or percentage of observations that fall into one of these classes that is of interest to an investigator. An observation that falls into one of the two classes is considered a ‘success’ (S), and ‘p’ is defined as the proportion of observations falling into that class. If a random sample of size ‘n’ is obtained from a population, the probability of obtaining 0, 1, 2, 3, etc., successes is then given by the binomial distribution. The binomial distribution can be used as the basis of a number of statistical tests but is most useful when comparing two proportions. This statnote describes two such scenarios in which the binomial distribution is used to compare: (1) two proportions when the samples are independent and (2) two proportions when the samples are paired.

Pharmacogenetics and the print media:what is the public told?

Background: Pharmacogenetics is a rapidly growing field that aims to identify the genes that influence drug response. This science can be used as a powerful tool to tailor drug treatment to the genetic makeup of individuals. The present study explores the coverage of the topic of pharmacogenetics and its potential benefit in personalised medicine by the UK newsprint media. Methods: The LexisNexis database was used to identify and retrieve full text articles from the 10 highest circulation national daily newspapers and their Sunday equivalents in the UK. Content analysis of newspaper articles which referenced pharmacogenetic testing was carried out. A second researcher coded a random sample (21%) of newspaper articles to establish the inter-rater reliability of coding. Results: Of the 256 articles captured by the search terms, 96 articles (with pharmacogenetics as a major component) met the study inclusion criteria. The majority of articles over-stated the benefits of pharmacogenetic testing while paying less attention to the associated risks. Overall beneficial effects were mentioned 5.3 times more frequently than risks (p < 0.001). The most common illnesses for which pharmacogenetically based personalised medicine was discussed were cancer, cardiovascular disease and CNS diseases. Only 13% of newspaper articles that cited a specific scientific study mentioned this link in the article. There was a positive correlation between the size of the article and both the number of benefits and risks stated (P < 0.01). Conclusion: More comprehensive coverage of the area of personalised medicine within the print media is needed to inform public debate on the inclusion of pharmacogentic testing in routine practice.

Optimising the diagnostic performance of the Geriatric Depression Scale

The aim of this work is to empirically generate a shortened version of the Geriatric Depression Scale (GDS), with the intention of maximising the diagnostic performance in the detection of depression compared with previously GDS validated versions, while optimizing the size of the instrument. A total of 233 individuals (128 from a Day Hospital, 105 randomly selected from the community) aged 60 or over completed the GDS and other measures. The 30 GDS items were entered in the Day Hospital sample as independent variables in a stepwise logistic regression analysis predicting diagnosis of Major Depression. A final solution of 10 items was retained, which correctly classified 97.4% of cases. The diagnostic performance of these 10 GDS items was analysed in the random sample with a receiver operating characteristic (ROC) curve. Sensitivity (100%), specificity (97.2%), positive (81.8%) and negative (100%) predictive power, and the area under the curve (0.994) were comparable with values for GDS-30 and higher compared with GDS-15, GDS-10 and GDS-5. In addition, the new scale proposed had excellent fit when testing its unidimensionality with CFA for categorical outcomes (e.g., CFI=0.99). The 10-item version of the GDS proposed here, the GDS-R, seems to retain the diagnostic performance for detecting depression in older adults of the GDS-30 items, while increasing the sensitivity and predictive values relative to other shortened versions.