Evolving features-algorithms knowledge map to support NIDS data intelligence and learning loop architecting – a generalised approach to NIDS pattern feature space knowledge processing

This paper describes a proposed new approach to the Computer Network Security Intrusion Detection Systems (NIDS) application domain knowledge processing focused on a topic map technology-enabled representation of features of the threat pattern space as well as the knowledge of situated efficacy of alternative candidate algorithms for pattern recognition within the NIDS domain. Thus an integrative knowledge representation framework for virtualisation, data intelligence and learning loop architecting in the NIDS domain is described together with specific aspects of its deployment.

The Knowledge: Its Presentation and Role in Recognition Systems

The concept of knowledge is the central one used when solving the various problems of data mining and pattern recognition in finite spaces of Boolean or multi-valued attributes. A special form of knowledge representation, called implicative regularities, is proposed for applying in two powerful tools of modern logic: the inductive inference and the deductive inference. The first one is used for extracting the knowledge from the data. The second is applied when the knowledge is used for calculation of the goal attribute values. A set of efficient algorithms was developed for that, dealing with Boolean functions and finite predicates represented by logical vectors and matrices.

Derivation of base-line geochemistry, petrography, and isotopic data for the host rocks to the Lucky Strike deposit and comparison with data from other alteration zones, Buchans Mining Camp, Newfoundland

The Buchans ore bodies of central Newfoundland represent some of the highest grade VMS deposits ever mined. These Kuroko-type deposits are also known for the well developed and preserved nature of the mechanically transported deposits. The deposits are hosted in Cambro-Ordovician, dominantly calc-alkaline, bimodal volcanic and epiclastic sequences of the Notre Dame Subzone, Newfoundland Appalachians. Stratigraphic relationships in this zone are complicated by extensively developed, brittledominated Silurian thrust faulting. Hydrothermal alteration of host rocks is a common feature of nearly all VMS deposits, and the recognition of these zones has been a key exploration tool. Alteration of host rocks has long been described to be spatially associated with the Buchans ore bodies, most notably with the larger in-situ deposits. This report represents a base-line study in which a complete documentation of the geochemical variance, in terms of both primary (igneous) and alteration effects, is presented from altered volcanic rocks in the vicinity of the Lucky Strike deposit (LSZ), the largest in-situ deposit in the Buchans camp. Packages of altered rocks also occur away from the immediate mining areas and constitute new targets for exploration. These zones, identified mostly by recent and previous drilling, represent untested targets and include the Powerhouse (PHZ), Woodmans Brook (WBZ) and Airport (APZ) alteration zones, as well as the Middle Branch alteration zone (MBZ), which represents a more distal alteration facies related to Buchans ore-formation. Data from each of these zones were compared to those from the LSZ in order to evaluate their relative propectivity. Derived litho geochemical data served two functions: (i) to define primary (igneous) trends and (ii) secondary alteration trends. Primary trends were established using immobile, or conservative, elements (i. e., HFSE, REE, Th, Ti0₂, Al₂0₃, P₂0₅). From these, altered volcanic rocks were interpreted in terms of composition (e.g., basalt - rhyodacite) and magmatic affinity (e.g., calc-alkaline vs. tholeiitic). The information suggests that bimodality is a common feature of all zones, with most rocks plotting as either basalt/andesite or dacite (or rhyodacite); andesitic senso stricto compositions are rare. Magmatic affinities are more varied and complex, but indicate that all units are arc volcanic sequences. Rocks from the LSZ/MBZ represent a transitional to calc-alkalic sequence, however, a slight shift in key geochemical discriminants occurs between the foot-wall to the hanging-wall. Specifically, mafic and felsic lavas of the foot-wall are of transitional (or mildly calc-alkaline) affinity whereas the hanging-wall rocks are relatively more strongly calc-alkaline as indicated by enriched LREE/HREE and higher ZrN, NbN and other ratios in the latter. The geochemical variations also serve as a means to separate the units (at least the felsic rocks) into hanging-wall and foot-wall sequences, therefore providing a valuable exploration tool. Volcanic rocks from the WBZ/PHZ (and probably the APZ) are more typical of tholeiitic to transitional suites, yielding flatter mantlenormalized REE patterns and lower ZrN ratios. Thus, the relationships between the immediate mining area (represented by LSZ/MBZ) and the Buchans East (PHZ/WBZ) and the APZ are uncertain. Host rocks for all zones consist of mafic to felsic volcanic rocks, though the proportion of pyroclastic and epiclastic rocks, is greatest at the LSZ. Phenocryst assemblages and textures are common in all zones, with minor exceptions, and are not useful for discrimination purposes. Felsic rocks from all zones are dominated by sericiteclay+/- silica alteration, whereas mafic rocks are dominated by chlorite- quartz- sericite alteration. Pyrite is ubiquitous in all moderately altered rocks and minor associated base metal sulphides occur locally. The exception is at Lucky Strike, where stockwork quartzveining contains abundant base-metal mineralization and barite. Rocks completely comprised of chlorite (chloritite) also occur in the LSZ foot-wall. In addition, K-feldspar alteration occurs in felsic volcanic rocks at the MBZ associated with Zn-Pb-Ba and, notably, without chlorite. This zone represents a peripheral, but proximal, zone of alteration induced by lower temperature hydrothermal fluids, presumably with little influence from seawater. Alteration geochemistry was interpreted from raw data as well as from mass balanced (recalculated) data derived from immobile element pairs. The data from the LSZ/MBZ indicate a range in the degree of alteration from only minor to severe modification of precursor compositions. Ba tends to show a strong positive correlation with K₂0, although most Ba occurs as barite. With respect to mass changes, Al₂0₃, Ti0₂ and P₂0₅ were shown to be immobile. Nearly all rocks display mass loss of Na₂O, CaO, and Sr reflecting feldspar destruction. These trends are usually mirrored by K₂0-Rb and MgO addition, indicating sericitic and chloritic alteration, respectively. More substantial gains ofK₂0 often occur in rocks with K-feldspar alteration, whereas a few samples also displayed excessive MgO enrichment and represent chloritites. Fe₂0₃ indicates both chlorite and sulphide formation. Si0₂ addition is almost always the case for the altered mafic rocks as silica often infills amygdules and replaces the finer tuffaceous material. The felsic rocks display more variability in Si0₂. Silicic, sericitic and chloritic alteration trends were observed from the other zones, but not K-feldspar, chloritite, or barite. Microprobe analysis of chlorites, sericites and carbonates indicate: (i) sericites from all zones are defined as muscovite and are not phengitic; (ii) at the LSZ, chlorites ranged from Fe-Mg chlorites (pycnochlorite) to Mg-rich chlorite (penninite), with the latter occurring in the stockwork zone and more proximal alteration facies; (iii) chlorites from the WBZ were typical of those from the more distal alteration facies of the LSZ, plotting as ripidolite to pycnochlorite; (iv) conversely, chlorite from the PHZ plot with Mg-Al-rich compositions (chlinochlore to penninite); and (v) carbonate species from each zone are also varied, with calcite occurring in each zone, in addition to dolomite and ankerite in the PHZ and WBZ, respectively. Lead isotope ratios for galena separates from the different various zones, when combined with data from older studies, tend to cluster into four distinctive fields. Overall, the data plot on a broad mixing line and indicate evolution in a relatively low-μ environment. Data from sulphide stringers in altered MBZ rocks, as well as from clastic sulphides (Sandfill prospect), plot in the Buchans ore field, as do the data for galena from altered rocks in the APZ. Samples from the Buchans East area are even more primitive than the Buchans ores, with lead from the PHZ plotting with the Connel Option prospect and data from the WBZ matching that of the Skidder prospect. A sample from a newly discovered debris flow-type sulphide occurrence (Middle Branch East) yields lead isotope ratios that are slightly more radiogenic than Buchans and plot with the Mary March alteration zone. Data within each cluster are interpreted to represent derivation from individual hydrothermal systems in which metals were derived from a common source.

Forecasting human entrances at a commercial store using facial recognition data

Dissertação de mestrado integrado em Engenharia e Gestão de Sistemas de Informação

The evolution of unicoloniality in ant societies : recognition, dispersal and population genetic structure

Introduction Societies of ants, bees, wasps and termites dominate many terrestrial ecosystems (Wilson 1971). Their evolutionary and ecological success is based upon the regulation of internal conflicts (e.g. Ratnieks et al. 2006), control of diseases (e.g. Schmid-Hempel 1998) and individual skills and collective intelligence in resource acquisition, nest building and defence (e.g. Camazine 2001). Individuals in social species can pass on their genes not only directly trough their own offspring, but also indirectly by favouring the reproduction of relatives. The inclusive fitness theory of Hamilton (1963; 1964) provides a powerful explanation for the evolution of reproductive altruism and cooperation in groups with related individuals. The same theory also led to the realization that insect societies are subject to internal conflicts over reproduction. Relatedness of less-than-one is not sufficient to eliminate all incentive for individual selfishness. This would indeed require a relatedness of one, as found among cells of an organism (Hardin 1968; Keller 1999). The challenge for evolutionary biology is to understand how groups can prevent or reduce the selfish exploitation of resources by group members, and how societies with low relatedness are maintained. In social insects the evolutionary shift from single- to multiple queens colonies modified the relatedness structure, the dispersal, and the mode of colony founding (e.g. (Crozier & Pamilo 1996). In ants, the most common, and presumably ancestral mode of reproduction is the emission of winged males and females, which found a new colony independently after mating and dispersal flights (Hölldobler & Wilson 1990). The alternative reproductive tactic for ant queens in multiple-queen colonies (polygyne) is to seek to be re-accepted in their natal colonies, where they may remain as additional reproductives or subsequently disperse on foot with part of the colony (budding) (Bourke & Franks 1995; Crozier & Pamilo 1996; Hölldobler & Wilson 1990). Such ant colonies can contain up to several hundred reproductive queens with an even more numerous workforce (Cherix 1980; Cherix 1983). As a consequence in polygynous ants the relatedness among nestmates is very low, and workers raise brood of queens to which they are only distantly related (Crozier & Pamilo 1996; Queller & Strassmann 1998). Therefore workers could increase their inclusive fitness by preferentially caring for their closest relatives and discriminate against less related or foreign individuals (Keller 1997; Queller & Strassmann 2002; Tarpy et al. 2004). However, the bulk of the evidence suggests that social insects do not behave nepotistically, probably because of the costs entailed by decreased colony efficiency or discrimination errors (Keller 1997). Recently, the consensus that nepotistic behaviour does not occur in insect colonies was challenged by a study in the ant Formica fusca (Hannonen & Sundström 2003b) showing that the reproductive share of queens more closely related to workers increases during brood development. However, this pattern can be explained either by nepotism with workers preferentially rearing the brood of more closely related queens or intrinsic differences in the viability of eggs laid by queens. In the first chapter, we designed an experiment to disentangle nepotism and differences in brood viability. We tested if workers prefer to rear their kin when given the choice between highly related and unrelated brood in the ant F. exsecta. We also looked for differences in egg viability among queens and simulated if such differences in egg viability may mistakenly lead to the conclusion that workers behave nepotistically. The acceptance of queens in polygnous ants raises the question whether the varying degree of relatedness affects their share in reproduction. In such colonies workers should favour nestmate queens over foreign queens. Numerous studies have investigated reproductive skew and partitioning of reproduction among queens (Bourke et al. 1997; Fournier et al. 2004; Fournier & Keller 2001; Hammond et al. 2006; Hannonen & Sundström 2003a; Heinze et al. 2001; Kümmerli & Keller 2007; Langer et al. 2004; Pamilo & Seppä 1994; Ross 1988; Ross 1993; Rüppell et al. 2002), yet almost no information is available on whether differences among queens in their relatedness to other colony members affects their share in reproduction. Such data are necessary to compare the relative reproductive success of dispersing and non-dispersing individuals. Moreover, information on whether there is a difference in reproductive success between resident and dispersing queens is also important for our understanding of the genetic structure of ant colonies and the dynamics of within group conflicts. In chapter two, we created single-queen colonies and then introduced a foreign queens originating from another colony kept under similar conditions in order to estimate the rate of queen acceptance into foreign established colonies, and to quantify the reproductive share of resident and introduced queens. An increasing number of studies have investigated the discrimination ability between ant workers (e.g. Holzer et al. 2006; Pedersen et al. 2006), but few have addressed the recognition and discrimination behaviour of workers towards reproductive individuals entering colonies (Bennett 1988; Brown et al. 2003; Evans 1996; Fortelius et al. 1993; Kikuchi et al. 2007; Rosengren & Pamilo 1986; Stuart et al. 1993; Sundström 1997; Vásquez & Silverman in press). These studies are important, because accepting new queens will generally have a large impact on colony kin structure and inclusive fitness of workers (Heinze & Keller 2000). In chapter three, we examined whether resident workers reject young foreign queens that enter into their nest. We introduced mated queens into their natal nest, a foreign-female producing nest, or a foreign male-producing nest and measured their survival. In addition, we also introduced young virgin and mated queens into their natal nest to examine whether the mating status of the queens influences their survival and acceptance by workers. On top of polgyny, some ant species have evolved an extraordinary social organization called 'unicoloniality' (Hölldobler & Wilson 1977; Pedersen et al. 2006). In unicolonial ants, intercolony borders are absent and workers and queens mix among the physically separated nests, such that nests form one large supercolony. Super-colonies can become very large, so that direct cooperative interactions are impossible between individuals of distant nests. Unicoloniality is an evolutionary paradox and a potential problem for kin selection theory because the mixing of queens and workers between nests leads to extremely low relatedness among nestmates (Bourke & Franks 1995; Crozier & Pamilo 1996; Keller 1995). A better understanding of the evolution and maintenance of unicoloniality requests detailed information on the discrimination behavior, dispersal, population structure, and the scale of competition. Cryptic genetic population structure may provide important information on the relevant scale to be considered when measuring relatedness and the role of kin selection. Theoretical studies have shown that relatedness should be measured at the level of the `economic neighborhood', which is the scale at which intraspecific competition generally takes place (Griffin & West 2002; Kelly 1994; Queller 1994; Taylor 1992). In chapter four, we conducted alarge-scale study to determine whether the unicolonial ant Formica paralugubris forms populations that are organised in discrete supercolonies or whether there is a continuous gradation in the level of aggression that may correlate with genetic isolation by distance and/or spatial distance between nests. In chapter five, we investigated the fine-scale population structure in three populations of F. paralugubris. We have developed mitochondria) markers, which together with the nuclear markers allowed us to detect cryptic genetic clusters of nests, to obtain more precise information on the genetic differentiation within populations, and to separate male and female gene flow. These new data provide important information on the scale to be considered when measuring relatedness in native unicolonial populations.

Self-Organising Maps and Correlation Analysis as a Tool to Explore Patterns in Excitation-Emission Matrix Data Sets and to Discriminate Dissolved Organic Matter Fluorescence Components

Dissolved organic matter (DOM) is a complex mixture of organic compounds, ubiquitous in marine and freshwater systems. Fluorescence spectroscopy, by means of Excitation-Emission Matrices (EEM), has become an indispensable tool to study DOM sources, transport and fate in aquatic ecosystems. However the statistical treatment of large and heterogeneous EEM data sets still represents an important challenge for biogeochemists. Recently, Self-Organising Maps (SOM) has been proposed as a tool to explore patterns in large EEM data sets. SOM is a pattern recognition method which clusterizes and reduces the dimensionality of input EEMs without relying on any assumption about the data structure. In this paper, we show how SOM, coupled with a correlation analysis of the component planes, can be used both to explore patterns among samples, as well as to identify individual fluorescence components. We analysed a large and heterogeneous EEM data set, including samples from a river catchment collected under a range of hydrological conditions, along a 60-km downstream gradient, and under the influence of different degrees of anthropogenic impact. According to our results, chemical industry effluents appeared to have unique and distinctive spectral characteristics. On the other hand, river samples collected under flash flood conditions showed homogeneous EEM shapes. The correlation analysis of the component planes suggested the presence of four fluorescence components, consistent with DOM components previously described in the literature. A remarkable strength of this methodology was that outlier samples appeared naturally integrated in the analysis. We conclude that SOM coupled with a correlation analysis procedure is a promising tool for studying large and heterogeneous EEM data sets.

Self-Organising Maps and Correlation Analysis as a Tool to Explore Patterns in Excitation-Emission Matrix Data Sets and to Discriminate Dissolved Organic Matter Fluorescence Components

Dissolved organic matter (DOM) is a complex mixture of organic compounds, ubiquitous in marine and freshwater systems. Fluorescence spectroscopy, by means of Excitation-Emission Matrices (EEM), has become an indispensable tool to study DOM sources, transport and fate in aquatic ecosystems. However the statistical treatment of large and heterogeneous EEM data sets still represents an important challenge for biogeochemists. Recently, Self-Organising Maps (SOM) has been proposed as a tool to explore patterns in large EEM data sets. SOM is a pattern recognition method which clusterizes and reduces the dimensionality of input EEMs without relying on any assumption about the data structure. In this paper, we show how SOM, coupled with a correlation analysis of the component planes, can be used both to explore patterns among samples, as well as to identify individual fluorescence components. We analysed a large and heterogeneous EEM data set, including samples from a river catchment collected under a range of hydrological conditions, along a 60-km downstream gradient, and under the influence of different degrees of anthropogenic impact. According to our results, chemical industry effluents appeared to have unique and distinctive spectral characteristics. On the other hand, river samples collected under flash flood conditions showed homogeneous EEM shapes. The correlation analysis of the component planes suggested the presence of four fluorescence components, consistent with DOM components previously described in the literature. A remarkable strength of this methodology was that outlier samples appeared naturally integrated in the analysis. We conclude that SOM coupled with a correlation analysis procedure is a promising tool for studying large and heterogeneous EEM data sets.

Use of fused airborne scanning laser altimetry and digital map data for urban flood modelling

Flood modelling of urban areas is still at an early stage, partly because until recently topographic data of sufficiently high resolution and accuracy have been lacking in urban areas. However, Digital Surface Models (DSMs) generated from airborne scanning laser altimetry (LiDAR) having sub-metre spatial resolution have now become available, and these are able to represent the complexities of urban topography. The paper describes the development of a LiDAR post-processor for urban flood modelling based on the fusion of LiDAR and digital map data. The map data are used in conjunction with LiDAR data to identify different object types in urban areas, though pattern recognition techniques are also employed. Post-processing produces a Digital Terrain Model (DTM) for use as model bathymetry, and also a friction parameter map for use in estimating spatially-distributed friction coefficients. In vegetated areas, friction is estimated from LiDAR-derived vegetation height, and (unlike most vegetation removal software) the method copes with short vegetation less than ~1m high, which may occupy a substantial fraction of even an urban floodplain. The DTM and friction parameter map may also be used to help to generate an unstructured mesh of a vegetated urban floodplain for use by a 2D finite element model. The mesh is decomposed to reflect floodplain features having different frictional properties to their surroundings, including urban features such as buildings and roads as well as taller vegetation features such as trees and hedges. This allows a more accurate estimation of local friction. The method produces a substantial node density due to the small dimensions of many urban features.

Unsupervised segmentation using Gabor wavelets and statistical features in LIDAR data analysis

In this paper, we address issues in segmentation Of remotely sensed LIDAR (LIght Detection And Ranging) data. The LIDAR data, which were captured by airborne laser scanner, contain 2.5 dimensional (2.5D) terrain surface height information, e.g. houses, vegetation, flat field, river, basin, etc. Our aim in this paper is to segment ground (flat field)from non-ground (houses and high vegetation) in hilly urban areas. By projecting the 2.5D data onto a surface, we obtain a texture map as a grey-level image. Based on the image, Gabor wavelet filters are applied to generate Gabor wavelet features. These features are then grouped into various windows. Among these windows, a combination of their first and second order of statistics is used as a measure to determine the surface properties. The test results have shown that ground areas can successfully be segmented from LIDAR data. Most buildings and high vegetation can be detected. In addition, Gabor wavelet transform can partially remove hill or slope effects in the original data by tuning Gabor parameters.

Optimization algorithms in FMRF model-based segmentation for LIDAR data and co-registered bands

In this paper, a fuzzy Markov random field (FMRF) model is used to segment land-objects into free, grass, building, and road regions by fusing remotely, sensed LIDAR data and co-registered color bands, i.e. scanned aerial color (RGB) photo and near infra-red (NIR) photo. An FMRF model is defined as a Markov random field (MRF) model in a fuzzy domain. Three optimization algorithms in the FMRF model, i.e. Lagrange multiplier (LM), iterated conditional mode (ICM), and simulated annealing (SA), are compared with respect to the computational cost and segmentation accuracy. The results have shown that the FMRF model-based ICM algorithm balances the computational cost and segmentation accuracy in land-cover segmentation from LIDAR data and co-registered bands.

Threshold-free object and ground point separation in LIDAR data

Light Detection And Ranging (LIDAR) is an important modality in terrain and land surveying for many environmental, engineering and civil applications. This paper presents the framework for a recently developed unsupervised classification algorithm called Skewness Balancing for object and ground point separation in airborne LIDAR data. The main advantages of the algorithm are threshold-freedom and independence from LIDAR data format and resolution, while preserving object and terrain details. The framework for Skewness Balancing has been built in this contribution with a prediction model in which unknown LIDAR tiles can be categorised as “hilly” or “moderate” terrains. Accuracy assessment of the model is carried out using cross-validation with an overall accuracy of 95%. An extension to the algorithm is developed to address the overclassification issue for hilly terrain. For moderate terrain, the results show that from the classified tiles detached objects (buildings and vegetation) and attached objects (bridges and motorway junctions) are separated from bare earth (ground, roads and yards) which makes Skewness Balancing ideal to be integrated into geographic information system (GIS) software packages.