A practical comparison of group-sequential and adaptive designs

Sequential methods provide a formal framework by which clinical trial data can be monitored as they accumulate. The results from interim analyses can be used either to modify the design of the remainder of the trial or to stop the trial as soon as sufficient evidence of either the presence or absence of a treatment effect is available. The circumstances under which the trial will be stopped with a claim of superiority for the experimental treatment, must, however, be determined in advance so as to control the overall type I error rate. One approach to calculating the stopping rule is the group-sequential method. A relatively recent alternative to group-sequential approaches is the adaptive design method. This latter approach provides considerable flexibility in changes to the design of a clinical trial at an interim point. However, a criticism is that the method by which evidence from different parts of the trial is combined means that a final comparison of treatments is not based on a sufficient statistic for the treatment difference, suggesting that the method may lack power. The aim of this paper is to compare two adaptive design approaches with the group-sequential approach. We first compare the form of the stopping boundaries obtained using the different methods. We then focus on a comparison of the power of the different trials when they are designed so as to be as similar as possible. We conclude that all methods acceptably control type I error rate and power when the sample size is modified based on a variance estimate, provided no interim analysis is so small that the asymptotic properties of the test statistic no longer hold. In the latter case, the group-sequential approach is to be preferred. Provided that asymptotic assumptions hold, the adaptive design approaches control the type I error rate even if the sample size is adjusted on the basis of an estimate of the treatment effect, showing that the adaptive designs allow more modifications than the group-sequential method.

Sequentially testing for a gene-drug interaction in a genomewide analysis

Assaying a large number of genetic markers from patients in clinical trials is now possible in order to tailor drugs with respect to efficacy. The statistical methodology for analysing such massive data sets is challenging. The most popular type of statistical analysis is to use a univariate test for each genetic marker, once all the data from a clinical study have been collected. This paper presents a sequential method for conducting an omnibus test for detecting gene-drug interactions across the genome, thus allowing informed decisions at the earliest opportunity and overcoming the multiple testing problems from conducting many univariate tests. We first propose an omnibus test for a fixed sample size. This test is based on combining F-statistics that test for an interaction between treatment and the individual single nucleotide polymorphism (SNP). As SNPs tend to be correlated, we use permutations to calculate a global p-value. We extend our omnibus test to the sequential case. In order to control the type I error rate, we propose a sequential method that uses permutations to obtain the stopping boundaries. The results of a simulation study show that the sequential permutation method is more powerful than alternative sequential methods that control the type I error rate, such as the inverse-normal method. The proposed method is flexible as we do not need to assume a mode of inheritance and can also adjust for confounding factors. An application to real clinical data illustrates that the method is computationally feasible for a large number of SNPs. Copyright (c) 2007 John Wiley & Sons, Ltd.

Detecting the node-density artifact in phylogeny reconstruction

The node-density effect is an artifact of phylogeny reconstruction that can cause branch lengths to be underestimated in areas of the tree with fewer taxa. Webster, Payne, and Pagel (2003, Science 301:478) introduced a statistical procedure (the "delta" test) to detect this artifact, and here we report the results of computer simulations that examine the test's performance. In a sample of 50,000 random data sets, we find that the delta test detects the artifact in 94.4% of cases in which it is present. When the artifact is not present (n = 10,000 simulated data sets) the test showed a type I error rate of approximately 1.69%, incorrectly reporting the artifact in 169 data sets. Three measures of tree shape or "balance" failed to predict the size of the node-density effect. This may reflect the relative homogeneity of our randomly generated topologies, but emphasizes that nearly any topology can suffer from the artifact, the effect not being confined only to highly unevenly sampled or otherwise imbalanced trees. The ability to screen phylogenies for the node-density artifact is important for phylogenetic inference and for researchers using phylogenetic trees to infer evolutionary processes, including their use in molecular clock dating. [Delta test; molecular clock; molecular evolution; node-density effect; phylogenetic reconstruction; speciation; simulation.]

Improvement of classification accuracy for stochastic discrimination - Multi-class classification

In this paper, an improved stochastic discrimination (SD) is introduced to reduce the error rate of the standard SD in the context of multi-class classification problem. The learning procedure of the improved SD consists of two stages. In the first stage, a standard SD, but with shorter learning period is carried out to identify an important space where all the misclassified samples are located. In the second stage, the standard SD is modified by (i) restricting sampling in the important space; and (ii) introducing a new discriminant function for samples in the important space. It is shown by mathematical derivation that the new discriminant function has the same mean, but smaller variance than that of standard SD for samples in the important space. It is also analyzed that the smaller the variance of the discriminant function, the lower the error rate of the classifier. Consequently, the proposed improved SD improves standard SD by its capability of achieving higher classification accuracy. Illustrative examples axe provided to demonstrate the effectiveness of the proposed improved SD.

On improvement of classification accuracy for stochastic discrimination

Stochastic discrimination (SD) depends on a discriminant function for classification. In this paper, an improved SD is introduced to reduce the error rate of the standard SD in the context of a two-class classification problem. The learning procedure of the improved SD consists of two stages. Initially a standard SD, but with shorter learning period is carried out to identify an important space where all the misclassified samples are located. Then the standard SD is modified by 1) restricting sampling in the important space, and 2) introducing a new discriminant function for samples in the important space. It is shown by mathematical derivation that the new discriminant function has the same mean, but with a smaller variance than that of the standard SD for samples in the important space. It is also analyzed that the smaller the variance of the discriminant function, the lower the error rate of the classifier. Consequently, the proposed improved SD improves standard SD by its capability of achieving higher classification accuracy. Illustrative examples are provided to demonstrate the effectiveness of the proposed improved SD.

Component processes of early reading, spelling, and narrative writing skills in Turkish: a longitudinal study

The study examined: (a) the role of phonological, grammatical, and rapid automatized naming (RAN) skills in reading and spelling development; and (b) the component processes of early narrative writing skills. Fifty-seven Turkish-speaking children were followed from Grade 1 to Grade 2. RAN was the most powerful longitudinal predictor of reading speed and its effect was evident even when previous reading skills were taken into account. Broadly, the phonological and grammatical skills made reliable contributions to spelling performance but their effects were completely mediated by previous spelling skills. Different aspects of the narrative writing skills were related to different processing skills. While handwriting speed predicted writing fluency, spelling accuracy predicted spelling error rate. Vocabulary and working memory were the only reliable longitudinal predictors of the quality of composition content. The overall model, however, failed to explain any reliable variance in the structural quality of the compositions

Improvement of the GenTHREADER method for genomic fold recognition

Motivation: In order to enhance genome annotation, the fully automatic fold recognition method GenTHREADER has been improved and benchmarked. The previous version of GenTHREADER consisted of a simple neural network which was trained to combine sequence alignment score, length information and energy potentials derived from threading into a single score representing the relationship between two proteins, as designated by CATH. The improved version incorporates PSI-BLAST searches, which have been jumpstarted with structural alignment profiles from FSSP, and now also makes use of PSIPRED predicted secondary structure and bi-directional scoring in order to calculate the final alignment score. Pairwise potentials and solvation potentials are calculated from the given sequence alignment which are then used as inputs to a multi-layer, feed-forward neural network, along with the alignment score, alignment length and sequence length. The neural network has also been expanded to accommodate the secondary structure element alignment (SSEA) score as an extra input and it is now trained to learn the FSSP Z-score as a measurement of similarity between two proteins. Results: The improvements made to GenTHREADER increase the number of remote homologues that can be detected with a low error rate, implying higher reliability of score, whilst also increasing the quality of the models produced. We find that up to five times as many true positives can be detected with low error rate per query. Total MaxSub score is doubled at low false positive rates using the improved method.

Latent learning in end stage renal disease (ESRD)

Cognitive functions such as attention and memory are known to be impaired in End Stage Renal Disease (ESRD), but the sites of the neural changes underlying these impairments are uncertain. Patients and controls took part in a latent learning task, which had previously shown a dissociation between patients with Parkinson’s disease and those with medial temporal damage. ESRD patients (n=24) and age and education-matched controls (n=24) were randomly assigned to either an exposed or unexposed condition. In Phase 1 of the task, participants learned that a cue (word) on the back of a schematic head predicted that the subsequently seen face would be smiling. For the exposed (but not unexposed) condition, an additional (irrelevant) colour cue was shown during presentation. In Phase 2, a different association, between colour and facial expression, was learned. Instructions were the same for each phase: participants had to predict whether the subsequently viewed face was going to be happy or sad. No difference in error rate between the groups was found in Phase 1, suggesting that patients and controls learned at a similar rate. However, in Phase 2, a significant interaction was found between group and condition, with exposed controls performing significantly worse than unexposed (therefore demonstrating learned irrelevance). In contrast, exposed patients made a similar number of errors to unexposed in Phase 2. The pattern of results in ESRD was different from that previously found in Parkinson’s disease, suggesting a different neural origin.

Flexible selection of a single treatment incorporating short-term endpoint information in a phase II/III clinical trial

Seamless phase II/III clinical trials in which an experimental treatment is selected at an interim analysis have been the focus of much recent research interest. Many of the methods proposed are based on the group sequential approach. This paper considers designs of this type in which the treatment selection can be based on short-term endpoint information for more patients than have primary endpoint data available. We show that in such a case, the familywise type I error rate may be inflated if previously proposed group sequential methods are used and the treatment selection rule is not specified in advance. A method is proposed to avoid this inflation by considering the treatment selection that maximises the conditional error given the data available at the interim analysis. A simulation study is reported that illustrates the type I error rate inflation and compares the power of the new approach with two other methods: a combination testing approach and a group sequential method that does not use the short-term endpoint data, both of which also strongly control the type I error rate. The new method is also illustrated through application to a study in Alzheimer's disease. © 2015 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

A Survey on Graphical Methods for Classification Predictive Performance Evaluation

Predictive performance evaluation is a fundamental issue in design, development, and deployment of classification systems. As predictive performance evaluation is a multidimensional problem, single scalar summaries such as error rate, although quite convenient due to its simplicity, can seldom evaluate all the aspects that a complete and reliable evaluation must consider. Due to this, various graphical performance evaluation methods are increasingly drawing the attention of machine learning, data mining, and pattern recognition communities. The main advantage of these types of methods resides in their ability to depict the trade-offs between evaluation aspects in a multidimensional space rather than reducing these aspects to an arbitrarily chosen (and often biased) single scalar measure. Furthermore, to appropriately select a suitable graphical method for a given task, it is crucial to identify its strengths and weaknesses. This paper surveys various graphical methods often used for predictive performance evaluation. By presenting these methods in the same framework, we hope this paper may shed some light on deciding which methods are more suitable to use in different situations.

Package models and the information crisis of prebiotic evolution

The coexistence between different types of templates has been the choice solution to the information crisis of prebiotic evolution, triggered by the finding that a single RNA-like template cannot carry enough information to code for any useful replicase. In principle, confining d distinct templates of length L in a package or protocell, whose Survival depends on the coexistence of the templates it holds in, could resolve this crisis provided that d is made sufficiently large. Here we review the prototypical package model of Niesert et al. [1981. Origin of life between Scylla and Charybdis. J. Mol. Evol. 17, 348-353] which guarantees the greatest possible region of viability of the protocell population, and show that this model, and hence the entire package approach, does not resolve the information crisis. In particular, we show that the total information stored in a viable protocell (Ld) tends to a constant value that depends only on the spontaneous error rate per nucleotide of the template replication mechanism. As a result, an increase of d must be followed by a decrease of L, so that the net information gain is null. (C) 2008 Elsevier Ltd. All rights reserved.

Legislação de compras e desempenho em instituições pública de pesquisa: um estudo de caso na Embrapa Semiárido

O objetivo desta pesquisa foi identificar a percepção dos empregados sobre a relação da legislação de compras com o desempenho da Embrapa Semiárido com base no critério de eficiência após a implantação do Pregão como uma nova modalidade licitatória. Foram realizadas pesquisas bibliográfica, documental, de campo, e para assegurar a validade das informações foi utilizada uma triangulação de técnicas de coleta de dados de análise documental, observação direta e entrevistas semiabertas, realizadas com empregados que atuam ou atuaram no Setor de Compras a mais de 10 anos e que fossem pregoeiros, e pesquisadores da Unidade, também com mais de 10 anos de experiência, que tivessem projetos aprovados com orçamento do Tesouro Nacional, com execução nos períodos anteriores e após à implantação do Pregão. De acordo com a documentação analisada e na ótica dos empregados do Setor, foi identificado que após a implantação do Pregão, a unidade tem conseguido economia de recursos nas contratações em média de 20% abaixo do valor de referência estabelecido nos Editais, o que poderia indicar eficiência. Entretanto, ficou evidente que o índice de erro nos processos passou a ser muito maior em relação aos processos realizados antes do pregão, e que apesar de tal modalidade ser apontada como benéfica, tem-se a impressão de que está sendo utilizada de maneira inadequada generalizadamente para toda e qualquer contratação sem uma análise prévia da modalidade licitatória mais adequada. Nessa mesma linha, no tocante aos relatórios de pesquisa e à opinião dos pesquisadores, foi evidenciado que tem havido perdas orçamentárias por não entrega de processos feitos em fim de ano, implicando em não cumprimento de metas, etapas ou tarefas dos experimentos, bem como, muitos atrasos nas entregas, e desperdícios de recursos por conta de aquisições de má qualidade que não servem para o trabalho de pesquisa. Por fim, a título de contribuição, este estudo apontou que a legislação de compras públicas atual influencia diretamente no desempenho da Embrapa Semiárido, e não atende às suas necessidades enquanto instituição de pesquisa, indicando que a instituição não pode seguir as mesmas regras, normas e legislação dos processos de aquisições dos demais órgãos da administração pública, e precisa de um novo aparato legal, que lhe proporcione maior flexibilidade para tocar seus projetos e cumprir sua missão institucional

Equipamento de biotelemetria para monitoramento em tempo real do teste da caminhada de seis minutos

Este estudo visa desenvolver um sistema portátil de radiocomunicação de radiação restrita, indicado para biotelemetria digital de curta distância aplicada ao Teste da Caminhada de Seis Minutos (TC6M) em pacientes com doença pulmonar obstrutiva crônica ou hipertensão pulmonar. A saturação periférica da hemoglobina (SpO2) e a freqüência cardíaca (FC) são monitoradas em tempo real. É utilizada a banda destinada a aplicações médicas, industriais e científicas (ISM), com freqüência de portadora em 916MHz e potência de transmissão de 0,75mW. Este sistema foi projetado para operar através de um enlace half duplex e codificação Manchester NRZ incorporando um protocolo para correção de erros do tipo automatic repeat request error com utilização de um código CRC-16 para detecção de erros. A velocidade máxima de transmissão de dados é de 115.2 kbps. O sistema é constituído de três partes: unidade portátil (Master), unidade estacionária (Slave) e software de visualização em tempo real. A unidade portátil recebe do oxímetro os parâmetros de monitorização que são transmitidos via enlace de rádio-freqüência. A interface da unidade estacionária com o software é feita através da porta de comunicação serial padrão RS-232. Os testes de laboratório e de campo demonstraram que o sistema de biotelemetria é adequado a realizar o TC6M com precisão de SpO2 de ±3 dígitos (com ±1 desvio padrão) e FC de ±3% ambos com taxa de Frame Error Rate < 10-4 (0,01%), sem restrigir os movimentos do usuário durante o processo de monitorização.

Análise de desempenho baseada em simulação de redes WirelessHART

This dissertation describes the implementation of a WirelessHART networks simulation module for the Network Simulator 3, aiming for the acceptance of both on the present context of networks research and industry. For validating the module were imeplemented tests for attenuation, packet error rate, information transfer success rate and battery duration per station

Aplicação de técnicas de aprendizado de máquina no reconhecimento de classes estruturais de proteínas

Nowadays, classifying proteins in structural classes, which concerns the inference of patterns in their 3D conformation, is one of the most important open problems in Molecular Biology. The main reason for this is that the function of a protein is intrinsically related to its spatial conformation. However, such conformations are very difficult to be obtained experimentally in laboratory. Thus, this problem has drawn the attention of many researchers in Bioinformatics. Considering the great difference between the number of protein sequences already known and the number of three-dimensional structures determined experimentally, the demand of automated techniques for structural classification of proteins is very high. In this context, computational tools, especially Machine Learning (ML) techniques, have become essential to deal with this problem. In this work, ML techniques are used in the recognition of protein structural classes: Decision Trees, k-Nearest Neighbor, Naive Bayes, Support Vector Machine and Neural Networks. These methods have been chosen because they represent different paradigms of learning and have been widely used in the Bioinfornmatics literature. Aiming to obtain an improvment in the performance of these techniques (individual classifiers), homogeneous (Bagging and Boosting) and heterogeneous (Voting, Stacking and StackingC) multiclassification systems are used. Moreover, since the protein database used in this work presents the problem of imbalanced classes, artificial techniques for class balance (Undersampling Random, Tomek Links, CNN, NCL and OSS) are used to minimize such a problem. In order to evaluate the ML methods, a cross-validation procedure is applied, where the accuracy of the classifiers is measured using the mean of classification error rate, on independent test sets. These means are compared, two by two, by the hypothesis test aiming to evaluate if there is, statistically, a significant difference between them. With respect to the results obtained with the individual classifiers, Support Vector Machine presented the best accuracy. In terms of the multi-classification systems (homogeneous and heterogeneous), they showed, in general, a superior or similar performance when compared to the one achieved by the individual classifiers used - especially Boosting with Decision Tree and the StackingC with Linear Regression as meta classifier. The Voting method, despite of its simplicity, has shown to be adequate for solving the problem presented in this work. The techniques for class balance, on the other hand, have not produced a significant improvement in the global classification error. Nevertheless, the use of such techniques did improve the classification error for the minority class. In this context, the NCL technique has shown to be more appropriated