Discriminating DDoS attacks from flash crowds using flow correlation coefficient

Distributed Denial of Service (DDoS) attack is a critical threat to the Internet, and botnets are usually the engines behind them. Sophisticated botmasters attempt to disable detectors by mimicking the traffic patterns of flash crowds. This poses a critical challenge to those who defend against DDoS attacks. In our deep study of the size and organization of current botnets, we found that the current attack flows are usually more similar to each other compared to the flows of flash crowds. Based on this, we proposed a discrimination algorithm using the flow correlation coefficient as a similarity metric among suspicious flows. We formulated the problem, and presented theoretical proofs for the feasibility of the proposed discrimination method in theory. Our extensive experiments confirmed the theoretical analysis and demonstrated the effectiveness of the proposed method in practice.

The electron antineutrino angular correlation coefficient a in free neutron decay: Testing the standard model with the aSPECT-spectrometer

The beta-decay of free neutrons is a strongly over-determined process in the Standard Model (SM) of Particle Physics and is described by a multitude of observables. Some of those observables are sensitive to physics beyond the SM. For example, the correlation coefficients of the involved particles belong to them. The spectrometer aSPECT was designed to measure precisely the shape of the proton energy spectrum and to extract from it the electron anti-neutrino angular correlation coefficient "a". A first test period (2005/ 2006) showed the “proof-of-principles”. The limiting influence of uncontrollable background conditions in the spectrometer made it impossible to extract a reliable value for the coefficient "a" (publication: Baessler et al., 2008, Europhys. Journ. A, 38, p.17-26). A second measurement cycle (2007/ 2008) aimed to under-run the relative accuracy of previous experiments (Stratowa et al. (1978), Byrne et al. (2002)) da/a =5%. I performed the analysis of the data taken there which is the emphasis of this doctoral thesis. A central point are background studies. The systematic impact of background on a was reduced to da/a(syst.)=0.61 %. The statistical accuracy of the analyzed measurements is da/a(stat.)=1.4 %. Besides, saturation effects of the detector electronics were investigated which were initially observed. These turned out not to be correctable on a sufficient level. An applicable idea how to avoid the saturation effects will be discussed in the last chapter.

A Bayesian approach to estimating the intraclass correlation coefficient

A Bayesian approach to estimating the intraclass correlation coefficient was used for this research project. The background of the intraclass correlation coefficient, a summary of its standard estimators, and a review of basic Bayesian terminology and methodology were presented. The conditional posterior density of the intraclass correlation coefficient was then derived and estimation procedures related to this derivation were shown in detail. Three examples of applications of the conditional posterior density to specific data sets were also included. Two sets of simulation experiments were performed to compare the mean and mode of the conditional posterior density of the intraclass correlation coefficient to more traditional estimators. Non-Bayesian methods of estimation used were: the methods of analysis of variance and maximum likelihood for balanced data; and the methods of MIVQUE (Minimum Variance Quadratic Unbiased Estimation) and maximum likelihood for unbalanced data. The overall conclusion of this research project was that Bayesian estimates of the intraclass correlation coefficient can be appropriate, useful and practical alternatives to traditional methods of estimation. ^

Statnote 32 : the partial correlation coefficient

In previous statnotes, the application of correlation and regression methods to the analysis of two variables (X,Y) was described. The most important statistic used to measure the degree of correlation between two variables is Pearson’s ‘product moment correlation coefficient’ (‘r’). The correlation between two variables may be due to their common relation to other variables. Hence, investigators using correlation studies need to be alert to the possibilities of spurious correlation and the methods of ‘partial correlation’ are one method of taking this into account. This statnote applies the methods of partial correlation to three scenarios. First, to a fairly obvious example of a spurious correlation resulting from the ‘size effect’ involving the relationship between the number of general practitioners (GP) and the number of deaths of patients in a town. Second, to the relationship between the abundance of the nitrogen-fixing bacterium Azotobacter in soil and three soil variables, and finally, to a more complex scenario, first introduced in Statnote 24involving the relationship between the growth of lichens in the field and climate.

Sparse Correlation Kernel Analysis and Reconstruction

This paper presents a new paradigm for signal reconstruction and superresolution, Correlation Kernel Analysis (CKA), that is based on the selection of a sparse set of bases from a large dictionary of class- specific basis functions. The basis functions that we use are the correlation functions of the class of signals we are analyzing. To choose the appropriate features from this large dictionary, we use Support Vector Machine (SVM) regression and compare this to traditional Principal Component Analysis (PCA) for the tasks of signal reconstruction, superresolution, and compression. The testbed we use in this paper is a set of images of pedestrians. This paper also presents results of experiments in which we use a dictionary of multiscale basis functions and then use Basis Pursuit De-Noising to obtain a sparse, multiscale approximation of a signal. The results are analyzed and we conclude that 1) when used with a sparse representation technique, the correlation function is an effective kernel for image reconstruction and superresolution, 2) for image compression, PCA and SVM have different tradeoffs, depending on the particular metric that is used to evaluate the results, 3) in sparse representation techniques, L_1 is not a good proxy for the true measure of sparsity, L_0, and 4) the L_epsilon norm may be a better error metric for image reconstruction and compression than the L_2 norm, though the exact psychophysical metric should take into account high order structure in images.

High-precision measurements of the co-polar correlation coefficient: non-Gaussian errors and retrieval of the dispersion parameter µ in rainfall

The co-polar correlation coefficient (ρhv) has many applications, including hydrometeor classification, ground clutter and melting layer identification, interpretation of ice microphysics and the retrieval of rain drop size distributions (DSDs). However, we currently lack the quantitative error estimates that are necessary if these applications are to be fully exploited. Previous error estimates of ρhv rely on knowledge of the unknown "true" ρhv and implicitly assume a Gaussian probability distribution function of ρhv samples. We show that frequency distributions of ρhv estimates are in fact highly negatively skewed. A new variable: L = -log10(1 - ρhv) is defined, which does have Gaussian error statistics, and a standard deviation depending only on the number of independent radar pulses. This is verified using observations of spherical drizzle drops, allowing, for the first time, the construction of rigorous confidence intervals in estimates of ρhv. In addition, we demonstrate how the imperfect co-location of the horizontal and vertical polarisation sample volumes may be accounted for. The possibility of using L to estimate the dispersion parameter (µ) in the gamma drop size distribution is investigated. We find that including drop oscillations is essential for this application, otherwise there could be biases in retrieved µ of up to ~8. Preliminary results in rainfall are presented. In a convective rain case study, our estimates show µ to be substantially larger than 0 (an exponential DSD). In this particular rain event, rain rate would be overestimated by up to 50% if a simple exponential DSD is assumed.

Generalised 2D-correlation NMR analysis of a wine fermentation

A wine fermentation has been monitored on a daily basis by 1H NMR spectroscopy. Following data pre-processing that includes synthesis of the spectra to ensure all peaks are of constant half-width, the series of spectra were examined using generalised two-dimensional correlation techniques. Synchronous and asynchronous data maps have been generated and employed to interpret the changes in the fermentation process as a function of time. The results illustrate the potential of high resolution NMR with multivariate data analysis as a tool for process monitoring and the manner in which two-dimensional correlation mapping can aid in data interpretation.

Comparative Analysis of 2-Flap Designs for Extraction of Mandibular Third Molar

Objective: The objective of the study was to analyze 2-flap designs for surgical extraction of third molar, evaluating the periodontal status of the second lower molar.Study Design: Forty-five lower third molars were extracted from 24 patients. In 23 teeth, a vertical incision to the mandibular ramus was used (technique A), whereas 22 teeth were submitted to classic L-shaped flap (technique B) with controls at 60 and 90 days postoperatively.Results: Pearson correlation coefficient analysis showed a significant correlation only between immediate preoperative probing depth variables from techniques A and B in the studied surfaces. Statistical significances in the preoperative (vestibular) and postoperative day 60 (distovestibular and vestibular) were noted. In contrast, Student t-test showed no statistical difference in probing depths between preoperative and postoperative values, as well as no statistically significant difference regarding the type of incision alone.Conclusions: Technique A allowed a less traumatic surgery, guaranteeing a more comfortable postoperative period.

(Table 2) Similarity Indexes (Pearson's correlation coefficient) of the silicoflagellate assemblages of IODP Exp302

The silicoflagellate and ebridian assemblages in early middle Eocene Arctic cores obtained by IODP Expedition 302 (ACEX) were studied in order to decipher the paleoceanography of the upper water column. The assemblages in Lithologic Unit 2 (49.7-45.1 Ma), one of the biosiliceous intervals, were usually endemic as compared to the assemblages that occurred outside of the Arctic Ocean. The presence of these endemic assemblages is probably due to a unique environmental setting, controlled by the degree of mixing between the low-salinity Arctic waters and relatively high salinity waters supplied from outside the Arctic Ocean, such as the Atlantic and possibly the Western Siberian Sea. Using the basin-to-basin fractionation model, the early middle Eocene Arctic Ocean corresponds to an estuarine circulation type, which includes the modern-day Black Sea. The abundant down-core occurrence of ebridians strongly suggests the past presence of low-salinity waters, and may indicate that low oxygen concentrations prevailed in the euphotic layer, on the basis of the ecology of the modern ebridian Hermesinum adriaticum.

Statnote 33:the intra-class correlation coefficient

The intra-class correlation coefficient (ICC or ri) is a method of measuring correlation when the data are paired and therefore, should be used when experimental units are organised into groups. A useful analogy is with the unpaired or paired ‘t’ test to compare the differences between the means of two groups. In studies of reproducibility, there may actually be little difference between the ICC and Pearson’s ‘r’ for ‘true’ repeated measurements. If, however, there is a systematic change in the measurements made on the first compared with the second occasion, then the ICC will be significantly less than ‘r’, and less confidence would be placed in the reproducibility of the results.

Diversity and path coefficient analysis of Southern African maize hybrids

Detailed knowledge on genetic diversity among germplasm is important for hybrid maize ( Zea mays L.) breeding. The objective of the study was to determine genetic diversity in widely grown hybrids in Southern Africa, and compare effectiveness of phenotypic analysis models for determining genetic distances between hybrids. Fifty hybrids were evaluated at one site with two replicates. The experiment was a randomized complete block design. Phenotypic and genotypic data were analyzed using SAS and Power Marker respectively. There was significant (p < 0.01) variation and diversity among hybrid brands but small within brand clusters. Polymorphic Information Content (PIC) ranged from 0.07 to 0.38 with an average of 0.34 and genetic distance ranged from 0.08 to 0.50 with an average of 0.43. SAH23 and SAH21 (0.48) and SAH33 and SAH3 (0.47) were the most distantly related hybrids. Both single nucleotide polymorphism (SNP) markers and phenotypic data models were effective for discriminating genotypes according to genetic distance. SNP markers revealed nine clusters of hybrids. The 12-trait phenotypic analysis model, revealed eight clusters at 85%, while the five-trait model revealed six clusters. Path analysis revealed significant direct and indirect effects of secondary traits on yield. Plant height and ear height were negatively correlated with grain yield meaning shorter hybrids gave high yield. Ear weight, days to anthesis, and number of ears had highest positive direct effects on yield. These traits can provide good selection index for high yielding maize hybrids. Results confirmed that diversity of hybrids is small within brands and also confirm that phenotypic trait models are effective for discriminating hybrids.

Inter-Area Power Oscillation Frequency Mode with Wind Turbine Generator in Irish Power System using PMU Data

Two case studies are presented in this paper to demonstrate the impact of different power system operation conditions on the power oscillation frequency modes in the Irish power system. A simplified 2 area equivalent of the Irish power system has been used in this paper, where area 1 represents the Republic of Ireland power system and area 2 represents the Northern Ireland power system.

The potential power oscillation frequency modes on the interconnector during different operation conditions have been analysed in this paper. The main objective of this paper is to analyse the influence of different operation conditions involving wind turbine generator (WTG) penetration on power oscillation frequency modes using phasor measurement unit (PMU) data.

Fast Fourier transform (FFT) analysis was performed to identify the frequency oscillation mode while correlation coefficient analysis was used to determine the source of the frequency oscillation. The results show that WTG, particularly fixed speed induction generation (FSIG), gives significant contribution to inter-area power oscillation frequency modes during high WTG operation.