Statistical racing techniques for improved empirical evaluation of evolutionary algorithms

In empirical studies of Evolutionary Algorithms, it is usually desirable to evaluate and compare algorithms using as many different parameter settings and test problems as possible, in border to have a clear and detailed picture of their performance. Unfortunately, the total number of experiments required may be very large, which often makes such research work computationally prohibitive. In this paper, the application of a statistical method called racing is proposed as a general-purpose tool to reduce the computational requirements of large-scale experimental studies in evolutionary algorithms. Experimental results are presented that show that racing typically requires only a small fraction of the cost of an exhaustive experimental study.

Partitioned likelihood support and the evaluation of data set conflict

In simultaneous analyses of multiple data partitions, the trees relevant when measuring support for a clade are the optimal tree, and the best tree lacking the clade (i.e., the most reasonable alternative). The parsimony-based method of partitioned branch support (PBS) forces each data set to arbitrate between the two relevant trees. This value is the amount each data set contributes to clade support in the combined analysis, and can be very different to support apparent in separate analyses. The approach used in PBS can also be employed in likelihood: a simultaneous analysis of all data retrieves the maximum likelihood tree, and the best tree without the clade of interest is also found. Each data set is fitted to the two trees and the log-likelihood difference calculated, giving partitioned likelihood support (PLS) for each data set. These calculations can be performed regardless of the complexity of the ML model adopted. The significance of PLS can be evaluated using a variety of resampling methods, such as the Kishino-Hasegawa test, the Shimodiara-Hasegawa test, or likelihood weights, although the appropriateness and assumptions of these tests remains debated.

Prospective evaluation of a D-optimal designed population pharmacokinetic study

Recently, methods for computing D-optimal designs for population pharmacokinetic studies have become available. However there are few publications that have prospectively evaluated the benefits of D-optimality in population or single-subject settings. This study compared a population optimal design with an empirical design for estimating the base pharmacokinetic model for enoxaparin in a stratified randomized setting. The population pharmacokinetic D-optimal design for enoxaparin was estimated using the PFIM function (MATLAB version 6.0.0.88). The optimal design was based on a one-compartment model with lognormal between subject variability and proportional residual variability and consisted of a single design with three sampling windows (0-30 min, 1.5-5 hr and 11 - 12 hr post-dose) for all patients. The empirical design consisted of three sample time windows per patient from a total of nine windows that collectively represented the entire dose interval. Each patient was assigned to have one blood sample taken from three different windows. Windows for blood sampling times were also provided for the optimal design. Ninety six patients were recruited into the study who were currently receiving enoxaparin therapy. Patients were randomly assigned to either the optimal or empirical sampling design, stratified for body mass index. The exact times of blood samples and doses were recorded. Analysis was undertaken using NONMEM (version 5). The empirical design supported a one compartment linear model with additive residual error, while the optimal design supported a two compartment linear model with additive residual error as did the model derived from the full data set. A posterior predictive check was performed where the models arising from the empirical and optimal designs were used to predict into the full data set. This revealed the optimal'' design derived model was superior to the empirical design model in terms of precision and was similar to the model developed from the full dataset. This study suggests optimal design techniques may be useful, even when the optimized design was based on a model that was misspecified in terms of the structural and statistical models and when the implementation of the optimal designed study deviated from the nominal design.

Evaluation of unsupervised semantic mapping of natural language with Leximancer concept mapping

The Leximancer system is a relatively new method for transforming lexical co-occurrence information from natural language into semantic patterns in an unsupervised manner. It employs two stages of co-occurrence information extraction-semantic and relational-using a different algorithm for each stage. The algorithms used are statistical, but they employ nonlinear dynamics and machine learning. This article is an attempt to validate the output of Leximancer, using a set of evaluation criteria taken from content analysis that are appropriate for knowledge discovery tasks.

Evaluation of the indirect measures of rock brittleness and fracture toughness in rock cutting

Performance prediction models for partial face mechanical excavators, when developed in laboratory conditions, depend on relating the results of a set of rock property tests and indices to specific cutting energy (SE) for various rock types. There exist some studies in the literature aiming to correlate the geotechnical properties of intact rocks with the SE, especially for massive and widely jointed rock environments. However, those including direct and/or indirect measures of rock fracture parameters such as rock brittleness and fracture toughness, along with the other rock parameters expressing different aspects of rock behavior under drag tools (picks), are rather limited. With this study, it was aimed to investigate the relationships between the indirect measures of rock brittleness and fracture toughness and the SE depending on the results of a new and two previous linear rock cutting programmes. Relationships between the SE, rock strength parameters, and the rock index tests have also been investigated in this study. Sandstone samples taken from the different fields around Ankara, Turkey were used in the new testing programme. Detailed mineralogical analyses, petrographic studies, and rock mechanics and rock cutting tests were performed on these selected sandstone specimens. The assessment of rock cuttability was based on the SE. Three different brittleness indices (B1, B2, and B4) were calculated for sandstones samples, whereas a toughness index (T-i), being developed by Atkinson et al.(1), was employed to represent the indirect rock fracture toughness. The relationships between the SE and the large amounts of new data obtained from the mineralogical analyses, petrographic studies, rock mechanics, and linear rock cutting tests were evaluated by using bivariate correlation and curve fitting techniques, variance analysis, and Student's t-test. Rock cutting and rock property testing data that came from well-known studies of McFeat-Smith and Fowell(2) and Roxborough and Philips(3) have also been employed in statistical analyses together with the new data. Laboratory tests and subsequent analyses revealed that there were close correlations between the SE and B4 whereas no statistically significant correlation has been found between the SE and T-i. Uniaxial compressive and Brazilian tensile strengths and Shore scleroscope hardness of sandstones also exhibited strong relationships with the SE. NCB cone indenter test had the greatest influence on the SE among the other engineering properties of rocks, confirming the previous studies in rock cutting and mechanical excavation. Therefore, it was recommended to employ easy-to-use index tests of NCB cone indenter and Shore scleroscope in the estimation of laboratory SE of sandstones ranging from very low to high strengths in the absence of a rock cutting rig to measure it until the easy-to-use universal measures of the rock brittleness and especially the rock fracture toughness, being an intrinsic rock property, are developed.

A learning approach for performance evaluation of local network

In this paper, a novel approach is developed to evaluate the overall performance of a local area network as well as to monitor some possible intrusion detections. The data is obtained via system utility 'ping' and huge data is analyzed via statistical methods. Finally, an overall performance index is defined and simulation experiments in three months proved the effectiveness of the proposed performance index. A software package is developed based on these ideas.