Fast unsupervised technique for extraction of endmembers spectra from hyperspectral data

One of the most challenging task underlying many hyperspectral imagery applications is the linear unmixing. The key to linear unmixing is to find the set of reference substances, also called endmembers, that are representative of a given scene. This paper presents the vertex component analysis (VCA) a new method to unmix linear mixtures of hyperspectral sources. The algorithm is unsupervised and exploits a simple geometric fact: endmembers are vertices of a simplex. The algorithm complexity, measured in floating points operations, is O (n), where n is the sample size. The effectiveness of the proposed scheme is illustrated using simulated data.

Vertex component analysis: a geometric-based approach to unmix hyperspectral data

Hyperspectral remote sensing exploits the electromagnetic scattering patterns of the different materials at specific wavelengths [2, 3]. Hyperspectral sensors have been developed to sample the scattered portion of the electromagnetic spectrum extending from the visible region through the near-infrared and mid-infrared, in hundreds of narrow contiguous bands [4, 5]. The number and variety of potential civilian and military applications of hyperspectral remote sensing is enormous [6, 7]. Very often, the resolution cell corresponding to a single pixel in an image contains several substances (endmembers) [4]. In this situation, the scattered energy is a mixing of the endmember spectra. A challenging task underlying many hyperspectral imagery applications is then decomposing a mixed pixel into a collection of reflectance spectra, called endmember signatures, and the corresponding abundance fractions [8–10]. Depending on the mixing scales at each pixel, the observed mixture is either linear or nonlinear [11, 12]. Linear mixing model holds approximately when the mixing scale is macroscopic [13] and there is negligible interaction among distinct endmembers [3, 14]. If, however, the mixing scale is microscopic (or intimate mixtures) [15, 16] and the incident solar radiation is scattered by the scene through multiple bounces involving several endmembers [17], the linear model is no longer accurate. Linear spectral unmixing has been intensively researched in the last years [9, 10, 12, 18–21]. It considers that a mixed pixel is a linear combination of endmember signatures weighted by the correspondent abundance fractions. Under this model, and assuming that the number of substances and their reflectance spectra are known, hyperspectral unmixing is a linear problem for which many solutions have been proposed (e.g., maximum likelihood estimation [8], spectral signature matching [22], spectral angle mapper [23], subspace projection methods [24,25], and constrained least squares [26]). In most cases, the number of substances and their reflectances are not known and, then, hyperspectral unmixing falls into the class of blind source separation problems [27]. Independent component analysis (ICA) has recently been proposed as a tool to blindly unmix hyperspectral data [28–31]. ICA is based on the assumption of mutually independent sources (abundance fractions), which is not the case of hyperspectral data, since the sum of abundance fractions is constant, implying statistical dependence among them. This dependence compromises ICA applicability to hyperspectral images as shown in Refs. [21, 32]. In fact, ICA finds the endmember signatures by multiplying the spectral vectors with an unmixing matrix, which minimizes the mutual information among sources. If sources are independent, ICA provides the correct unmixing, since the minimum of the mutual information is obtained only when sources are independent. This is no longer true for dependent abundance fractions. Nevertheless, some endmembers may be approximately unmixed. These aspects are addressed in Ref. [33]. Under the linear mixing model, the observations from a scene are in a simplex whose vertices correspond to the endmembers. Several approaches [34–36] have exploited this geometric feature of hyperspectral mixtures [35]. Minimum volume transform (MVT) algorithm [36] determines the simplex of minimum volume containing the data. The method presented in Ref. [37] is also of MVT type but, by introducing the notion of bundles, it takes into account the endmember variability usually present in hyperspectral mixtures. The MVT type approaches are complex from the computational point of view. Usually, these algorithms find in the first place the convex hull defined by the observed data and then fit a minimum volume simplex to it. For example, the gift wrapping algorithm [38] computes the convex hull of n data points in a d-dimensional space with a computational complexity of O(nbd=2cþ1), where bxc is the highest integer lower or equal than x and n is the number of samples. The complexity of the method presented in Ref. [37] is even higher, since the temperature of the simulated annealing algorithm used shall follow a log( ) law [39] to assure convergence (in probability) to the desired solution. Aiming at a lower computational complexity, some algorithms such as the pixel purity index (PPI) [35] and the N-FINDR [40] still find the minimum volume simplex containing the data cloud, but they assume the presence of at least one pure pixel of each endmember in the data. This is a strong requisite that may not hold in some data sets. In any case, these algorithms find the set of most pure pixels in the data. PPI algorithm uses the minimum noise fraction (MNF) [41] as a preprocessing step to reduce dimensionality and to improve the signal-to-noise ratio (SNR). The algorithm then projects every spectral vector onto skewers (large number of random vectors) [35, 42,43]. The points corresponding to extremes, for each skewer direction, are stored. A cumulative account records the number of times each pixel (i.e., a given spectral vector) is found to be an extreme. The pixels with the highest scores are the purest ones. N-FINDR algorithm [40] is based on the fact that in p spectral dimensions, the p-volume defined by a simplex formed by the purest pixels is larger than any other volume defined by any other combination of pixels. This algorithm finds the set of pixels defining the largest volume by inflating a simplex inside the data. ORA SIS [44, 45] is a hyperspectral framework developed by the U.S. Naval Research Laboratory consisting of several algorithms organized in six modules: exemplar selector, adaptative learner, demixer, knowledge base or spectral library, and spatial postrocessor. The first step consists in flat-fielding the spectra. Next, the exemplar selection module is used to select spectral vectors that best represent the smaller convex cone containing the data. The other pixels are rejected when the spectral angle distance (SAD) is less than a given thresh old. The procedure finds the basis for a subspace of a lower dimension using a modified Gram–Schmidt orthogonalizati on. The selected vectors are then projected onto this subspace and a simplex is found by an MV T pro cess. ORA SIS is oriented to real-time target detection from uncrewed air vehicles using hyperspectral data [46]. In this chapter we develop a new algorithm to unmix linear mixtures of endmember spectra. First, the algorithm determines the number of endmembers and the signal subspace using a newly developed concept [47, 48]. Second, the algorithm extracts the most pure pixels present in the data. Unlike other methods, this algorithm is completely automatic and unsupervised. To estimate the number of endmembers and the signal subspace in hyperspectral linear mixtures, the proposed scheme begins by estimating sign al and noise correlation matrices. The latter is based on multiple regression theory. The signal subspace is then identified by selectin g the set of signal eigenvalue s that best represents the data, in the least-square sense [48,49 ], we note, however, that VCA works with projected and with unprojected data. The extraction of the end members exploits two facts: (1) the endmembers are the vertices of a simplex and (2) the affine transformation of a simplex is also a simplex. As PPI and N-FIND R algorithms, VCA also assumes the presence of pure pixels in the data. The algorithm iteratively projects data on to a direction orthogonal to the subspace spanned by the endmembers already determined. The new end member signature corresponds to the extreme of the projection. The algorithm iterates until all end members are exhausted. VCA performs much better than PPI and better than or comparable to N-FI NDR; yet it has a computational complexity between on e and two orders of magnitude lower than N-FINDR. The chapter is structure d as follows. Section 19.2 describes the fundamentals of the proposed method. Section 19.3 and Section 19.4 evaluate the proposed algorithm using simulated and real data, respectively. Section 19.5 presents some concluding remarks.

Fast unsupervised extraction of endmembers spectra from hyperspectral data

Linear unmixing decomposes an hyperspectral image into a collection of re ectance spectra, called endmember signatures, and a set corresponding abundance fractions from the respective spatial coverage. This paper introduces vertex component analysis, an unsupervised algorithm to unmix linear mixtures of hyperpsectral data. VCA exploits the fact that endmembers occupy vertices of a simplex, and assumes the presence of pure pixels in data. VCA performance is illustrated using simulated and real data. VCA competes with state-of-the-art methods with much lower computational complexity.

Comparison of Disposable Pipette Extraction and Dispersive Solid-Phase Extraction in the QuEChERS Method for Analysis of Pesticides in Strawberries

In this study, we sought to assess the applicability of GC–MS/MS for the identification and quantification of 36 pesticides in strawberry from integrated pest management (IPM) and organic farming (OF). Citrate versions of QuEChERS (quick, easy, cheap, effective, rugged and safe) using dispersive solid-phase extraction (d-SPE) and disposable pipette extraction (DPX) for cleanup were compared for pesticide extraction. For cleanup, a combination of MgSO4, primary secondary amine and C18 was used for both the versions. Significant differences were observed in recovery results between the two sample preparation versions (DPX and d-SPE). Overall, 86% of the pesticides achieved recoveries (three spiking levels 10, 50 and 200 µg/kg) in the range of 70–120%, with <13% RSD. The matrix effects were also evaluated in both the versions and in strawberries from different crop types. Although not evidencing significant differences between the two methodologies were observed, however, the DPX cleanup proved to be a faster technique and easy to execute. The results indicate that QuEChERS with d-SPE and DPX and GC–MS/MS analysis achieved reliable quantification and identification of 36 pesticide residues in strawberries from OF and IPM.

Long term follow-up and patterns of response of ALT in patients with chronic hepatitis NANB/C treated with recombinant interferon-alpha

The response to interferon treatment in chronic hepatitis NANB/C has usually been classified as complete, partial or absent, according to the behavior of serum alanine aminotransferase (ALT). However, a more detailed observation of the enzymatic activity has shown that the patterns may be more complex. The aim of this study was to describe the long term follow-up and patterns of ALT response in patients with chronic hepatitis NANB/C treated with recombinant interferon-alpha. A follow-up of 6 months or more after interferon-a was achieved in 44 patients. We have classified the serum ALT responses into six patterns and the observed frequencies were as follows: I. Long term response = 9 (20.5%); II. Normalization followed by persistent relapse after IFN = 7 (15.9%); III. Normalization with transient relapse = 5 (11.9%); IV. Temporary normalization and relapse during IFN = 4 (9.1%); V. Partial response (more than 50% of ALT decrease) = 7 (15.9%); VI. No response = 12 (27.3%). In conclusion, ALT patterns vary widely during and after IFN treatment and can be classified in at least 6 types.

Using data mining techniques to support breast cancer diagnosis

More than ever, there is an increase of the number of decision support methods and computer aided diagnostic systems applied to various areas of medicine. In breast cancer research, many works have been done in order to reduce false-positives when used as a double reading method. In this study, we aimed to present a set of data mining techniques that were applied to approach a decision support system in the area of breast cancer diagnosis. This method is geared to assist clinical practice in identifying mammographic findings such as microcalcifications, masses and even normal tissues, in order to avoid misdiagnosis. In this work a reliable database was used, with 410 images from about 115 patients, containing previous reviews performed by radiologists as microcalcifications, masses and also normal tissue findings. Throughout this work, two feature extraction techniques were used: the gray level co-occurrence matrix and the gray level run length matrix. For classification purposes, we considered various scenarios according to different distinct patterns of injuries and several classifiers in order to distinguish the best performance in each case described. The many classifiers used were Naïve Bayes, Support Vector Machines, k-nearest Neighbors and Decision Trees (J48 and Random Forests). The results in distinguishing mammographic findings revealed great percentages of PPV and very good accuracy values. Furthermore, it also presented other related results of classification of breast density and BI-RADS® scale. The best predictive method found for all tested groups was the Random Forest classifier, and the best performance has been achieved through the distinction of microcalcifications. The conclusions based on the several tested scenarios represent a new perspective in breast cancer diagnosis using data mining techniques.

Trypanosoma cruzi strains: behavior after passage into authoctonous or foreign species of triatomine (biological and biochemical patterns)

The behavior of T. cruzi strains from S. Felipe - BA (19 SF, 21 SF and 22 SF) classified as Type II Zymodeme 2, was investigated after passage through the authoctonous (P. megistus) and foreign vectors (T. infestans and R. prolixus). For each strain Swiss mice were infected: I - with blood forms (control); II - with metacyclic forms (MF) from P. megistus; III - with MF from T. infestans; IV - with MF from R. prolixus. Inocula: MF from the three species of triatomine, 60 to 120 days after feeding in infected mice, adjusted to 10 4. Biological behavior in mice (parasitemia, morphology, mortality, virulence and pathogenicity) after passage through triatomine was compared with data from the same strain in control mice. Isoenzymic electrophoresis (ASAT, ALAT, PGM, GPI) were also performed after culture into Warren medium. The three strains maintained the isoenzyme profiles (zymodeme 2), in the control groups and after passages through different species of triatomine. Biological characterization disclosed Type II strains patterns for all groups. An increased virulence was observed with the 22 SF strain isolated from P. megistus and T. infestans and higher levels of parasitemia and predominance of slender forms in mice inoculated with the 19 SF and 21 SF from these same species. Results indicate that the passage through the two species T. infestans and P. megistus had a positive influence on the virulence of the regional strains of S. Felipe, regardless of being autocthonous (P. megistus) or foreign to the area (T. infestans).

An integer programming framework for sequencing cutting patterns based on interval graph completion

We derived a framework in integer programming, based on the properties of a linear ordering of the vertices in interval graphs, that acts as an edge completion model for obtaining interval graphs. This model can be applied to problems of sequencing cutting patterns, namely the minimization of open stacks problem (MOSP). By making small modifications in the objective function and using only some of the inequalities, the MOSP model is applied to another pattern sequencing problem that aims to minimize, not only the number of stacks, but also the order spread (the minimization of the stack occupation problem), and the model is tested.

Sequencing cutting patterns with colored interval graphs

The problem addressed here originates in the industry of flat glass cutting and wood panel sawing, where smaller items are cut from larger items accordingly to predefined cutting patterns. In this type of industry the smaller pieces that are cut from the patterns are piled around the machine in stacks according to the size of the pieces, which are moved to the warehouse only when all items of the same size have been cut. If the cutting machine can process only one pattern at a time, and the workspace is limited, it is desirable to set the sequence in which the cutting patterns are processed in a way to minimize the maximum number of open stacks around the machine. This problem is known in literature as the minimization of open stacks (MOSP). To find the best sequence of the cutting patterns, we propose an integer programming model, based on interval graphs, that searches for an appropriate edge completion of the given graph of the problem, while defining a suitable coloring of its vertices.

Data Extraction Tool to Analyse, Transform and Store Real Data from Electricity Markets

The study of electricity markets operation has been gaining an increasing importance in the last years, as result of the new challenges that the restructuring process produced. Currently, lots of information concerning electricity markets is available, as market operators provide, after a period of confidentiality, data regarding market proposals and transactions. These data can be used as source of knowledge to define realistic scenarios, which are essential for understanding and forecast electricity markets behavior. The development of tools able to extract, transform, store and dynamically update data, is of great importance to go a step further into the comprehension of electricity markets and of the behaviour of the involved entities. In this paper an adaptable tool capable of downloading, parsing and storing data from market operators’ websites is presented, assuring constant updating and reliability of the stored data.

Automatic Electricity Markets Data Extraction for Realistic Multi-agent Simulations

Electricity markets worldwide suffered profound transformations. The privatization of previously nationally owned systems; the deregulation of privately owned systems that were regulated; and the strong interconnection of national systems, are some examples of such transformations [1, 2]. In general, competitive environments, as is the case of electricity markets, require good decision-support tools to assist players in their decisions. Relevant research is being undertaken in this field, namely concerning player modeling and simulation, strategic bidding and decision-support.

Changing patterns of Salmonella serovars: increase of Salmonella Enteritidis in São Paulo, Brazil

Serovars of a total of 5,490 Salmonella strains isolated during the period of 1991-95, from human infections (2,254 strains) and from non-human materials (3,236 strains) were evaluated. In the studied period, 81 different serovars were determined among human isolates. Salmonella Enteritidis corresponded to 1.2% in 1991, 2% in 1992, 10.1% in 1993, 43.3% in 1994, and 64.9% in 1995 of all isolates. A significant rise on the isolation of this serovar was seen since 1993 linked to food poisoning outbreaks. It is reported also an increase on the isolation of S. Enteritidis from blood cultures, associated mainly with patients with immunodeficiency syndrome. S. Enteritidis was prevalent among one hundred and thirty different serovars isolated from non-human sources. Increasing number of isolation of this serovar was seen from shell eggs, breeding flocks and from environmental samples. It is also reported a contamination of commercial feed stuffs by S. Enteritidis which represents a major concern for Brazilian poultry industry.

Engineering Temporal and Spatial Aspects in OWL using Patterns

WWW is a huge, open, heterogeneous system, however its contents data is mainly human oriented. The Semantic Web needs to assure that data is readable and “understandable” to intelligent software agents, though the use of explicit and formal semantics. Ontologies constitute a privileged artifact for capturing the semantic of the WWW data. Temporal and spatial dimensions are transversal to the generality of knowledge domains and therefore are fundamental for the reasoning process of software agents. Representing temporal/spatial evolution of concepts and their relations in OWL (W3C standard for ontologies) it is not straightforward. Although proposed several strategies to tackle this problem but there is still no formal and standard approach. This work main goal consists of development of methods/tools to support the engineering of temporal and spatial aspects in intelligent systems through the use of OWL ontologies. An existing method for ontology engineering, Fonte was used as framework for the development of this work. As main contributions of this work Fonte was re-engineered in order to: i) support the spatial dimension; ii) work with OWL Ontologies; iii) and support the application of Ontology Design Patterns. Finally, the capabilities of the proposed approach were demonstrated by engineering time and space in a demo ontology about football.

Duality of patterns in hepatitis a epidemiology: A study involving two socioeconomically distinct populations in Campinas, São Paulo state, Brazil

To evaluate the prevalence of antibodies against hepatitis A in two socioeconomically distinct populations, 101 and 82 serum samples from high and low socioeconomic groups, respectively, were analysed for the presence of IgG anti-HAV using a commercial ELISA. The prevalence in low socioeconomic level subjects was 95.0%, whereas in high socioeconomic subjects was only 19.6% (p<0.001). These data show a duality in Brazil: anti-HAV prevalence in low socioeconomic subjects is similar to that of developing countries, while in high socioeconomic subjects, a pattern typical of developed countries is found. The control of this infection in our country is primarily related to the improvement of sanitation, but especially for high socioeconomic level populations, the use of vaccination against hepatitis A is strongly advisable to avoid the occasional appearance of this disease in adults.

Declarative approach to data extraction of web pages

Thesis submitted to Faculdade de Ciências e Tecnologia of the Universidade Nova de Lisboa, in partial fulfilment of the requirements for the degree of Master in Computer Science