Towards accurate estimation of the proportion of true null hypotheses in multiple testing

Background

Biomedical researchers are now often faced with situations where it is necessary to test a large number of hypotheses simultaneously, eg, in comparative gene expression studies using high-throughput microarray technology. To properly control false positive errors the FDR (false discovery rate) approach has become widely used in multiple testing. The accurate estimation of FDR requires the proportion of true null hypotheses being accurately estimated. To date many methods for estimating this quantity have been proposed. Typically when a new method is introduced, some simulations are carried out to show the improved accuracy of the new method. However, the simulations are often very limited to covering only a few points in the parameter space.

Results

Here I have carried out extensive in silico experiments to compare some commonly used methods for estimating the proportion of true null hypotheses. The coverage of these simulations is unprecedented thorough over the parameter space compared to typical simulation studies in the literature. Thus this work enables us to draw conclusions globally as to the performance of these different methods. It was found that a very simple method gives the most accurate estimation in a dominantly large area of the parameter space. Given its simplicity and its overall superior accuracy I recommend its use as the first choice for estimating the proportion of true null hypotheses in multiple testing.

A feasibility study testing four hypotheses with phase II outcomes in advanced colorectal cancer (MRC FOCUS3): a model for randomised controlled trials in the era of personalised medicine?

Background: Molecular characteristics of cancer vary between individuals. In future, most trials will require assessment of biomarkers to allocate patients into enriched populations in which targeted therapies are more likely to be effective. The MRC FOCUS3 trial is a feasibility study to assess key elements in the planning of such studies.

Patients and methods: Patients with advanced colorectal cancer were registered from 24 centres between February 2010 and April 2011. With their consent, patients' tumour samples were analysed for KRAS/BRAF oncogene mutation status and topoisomerase 1 (topo-1) immunohistochemistry. Patients were then classified into one of four molecular strata; within each strata patients were randomised to one of two hypothesis-driven experimental therapies or a common control arm (FOLFIRI chemotherapy). A 4-stage suite of patient information sheets (PISs) was developed to avoid patient overload.

Results: A total of 332 patients were registered, 244 randomised. Among randomised patients, biomarker results were provided within 10 working days (w.d.) in 71%, 15 w.d. in 91% and 20 w.d. in 99%. DNA mutation analysis was 100% concordant between two laboratories. Over 90% of participants reported excellent understanding of all aspects of the trial. In this randomised phase II setting, omission of irinotecan in the low topo-1 group was associated with increased response rate and addition of cetuximab in the KRAS, BRAF wild-type cohort was associated with longer progression-free survival.

Conclusions: Patient samples can be collected and analysed within workable time frames and with reproducible mutation results. Complex multi-arm designs are acceptable to patients with good PIS. Randomisation within each cohort provides outcome data that can inform clinical practice.

Substance use and misuse in burn patients: testing the classical hypotheses of the interaction between posttraumatic symptomatology and substance use

Background: We aimed to test whether the three classical hypotheses of the interaction between posttraumatic symptomatology and substance use (high risk of trauma exposure, susceptibility for posttraumatic symptomatology, and self-medication of symptoms), may be useful in the understanding of substance use among burn patients. Methods: We analysed substance use data (nicotine, alcohol, cannabis, amphetamines, cocaine, opiates, and tranquilizers) and psychopathology measures among burn patients admitted to a Burns Unit and enrolled in a longitudinal observational study. Lifetime substance use information (n = 246) was incorporated to analyses aiming to test the high risk hypothesis. Only patients assessed for psychopathology in a six months follow-up (n = 183) were included in prospective analyses testing the susceptibility and self-medication hypotheses. Results: Regarding the high risk hypothesis, results show a higher proportion of heroin and tranquilizer users compared to the general population. Furthermore, in line with the susceptibility hypothesis, higher levels of symptomatology were found in lifetime alcohol, tobacco and drug users during recovery. The self-medication hypothesis could be tested partially due to the hospital stay “cleaning” effect, but severity of symptoms was linked to caffeine, nicotine, alcohol and cannabis use after discharge. Conclusions: We found that the three classical hypotheses could be used to understand the link between traumatic experiences and substance use explaining different patterns of burn patient’s risk for trauma exposure and emergence of symptomatology.

Logique et tests d'hypotheses: reflexions sur les problemes mal poses en econometrie

Dans ce texte, nous analysons les développements récents de l’économétrie à la lumière de la théorie des tests statistiques. Nous revoyons d’abord quelques principes fondamentaux de philosophie des sciences et de théorie statistique, en mettant l’accent sur la parcimonie et la falsifiabilité comme critères d’évaluation des modèles, sur le rôle de la théorie des tests comme formalisation du principe de falsification de modèles probabilistes, ainsi que sur la justification logique des notions de base de la théorie des tests (tel le niveau d’un test). Nous montrons ensuite que certaines des méthodes statistiques et économétriques les plus utilisées sont fondamentalement inappropriées pour les problèmes et modèles considérés, tandis que de nombreuses hypothèses, pour lesquelles des procédures de test sont communément proposées, ne sont en fait pas du tout testables. De telles situations conduisent à des problèmes statistiques mal posés. Nous analysons quelques cas particuliers de tels problèmes : (1) la construction d’intervalles de confiance dans le cadre de modèles structurels qui posent des problèmes d’identification; (2) la construction de tests pour des hypothèses non paramétriques, incluant la construction de procédures robustes à l’hétéroscédasticité, à la non-normalité ou à la spécification dynamique. Nous indiquons que ces difficultés proviennent souvent de l’ambition d’affaiblir les conditions de régularité nécessaires à toute analyse statistique ainsi que d’une utilisation inappropriée de résultats de théorie distributionnelle asymptotique. Enfin, nous soulignons l’importance de formuler des hypothèses et modèles testables, et de proposer des techniques économétriques dont les propriétés sont démontrables dans les échantillons finis.

Tests of Joint Hypotheses for Time Series Regression with a Unit Root

This Paper Studies Tests of Joint Hypotheses in Time Series Regression with a Unit Root in Which Weakly Dependent and Heterogeneously Distributed Innovations Are Allowed. We Consider Two Types of Regression: One with a Constant and Lagged Dependent Variable, and the Other with a Trend Added. the Statistics Studied Are the Regression \"F-Test\" Originally Analysed by Dickey and Fuller (1981) in a Less General Framework. the Limiting Distributions Are Found Using Functinal Central Limit Theory. New Test Statistics Are Proposed Which Require Only Already Tabulated Critical Values But Which Are Valid in a Quite General Framework (Including Finite Order Arma Models Generated by Gaussian Errors). This Study Extends the Results on Single Coefficients Derived in Phillips (1986A) and Phillips and Perron (1986).

Les hypotheses cosmogoniques : Examen des théories scientifiques modernes sur l'origine des mondes, suivi de la traduction de la Théoriedu ciel de Kant

UANL

Compositional hypotheses of subcompositional stability and specific perturbation change and their testing

In standard multivariate statistical analysis common hypotheses of interest concern changes in mean vectors and subvectors. In compositional data analysis it is now well established that compositional change is most readily described in terms of the simplicial operation of perturbation and that subcompositions replace the marginal concept of subvectors. To motivate the statistical developments of this paper we present two challenging compositional problems from food production processes. Against this background the relevance of perturbations and subcompositions can be clearly seen. Moreover we can identify a number of hypotheses of interest involving the specification of particular perturbations or differences between perturbations and also hypotheses of subcompositional stability. We identify the two problems as being the counterpart of the analysis of paired comparison or split plot experiments and of separate sample comparative experiments in the jargon of standard multivariate analysis. We then develop appropriate estimation and testing procedures for a complete lattice of relevant compositional hypotheses