Exploiting the bin-class histograms for feature selection on discrete data

In machine learning and pattern recognition tasks, the use of feature discretization techniques may have several advantages. The discretized features may hold enough information for the learning task at hand, while ignoring minor fluctuations that are irrelevant or harmful for that task. The discretized features have more compact representations that may yield both better accuracy and lower training time, as compared to the use of the original features. However, in many cases, mainly with medium and high-dimensional data, the large number of features usually implies that there is some redundancy among them. Thus, we may further apply feature selection (FS) techniques on the discrete data, keeping the most relevant features, while discarding the irrelevant and redundant ones. In this paper, we propose relevance and redundancy criteria for supervised feature selection techniques on discrete data. These criteria are applied to the bin-class histograms of the discrete features. The experimental results, on public benchmark data, show that the proposed criteria can achieve better accuracy than widely used relevance and redundancy criteria, such as mutual information and the Fisher ratio.

GPU implementation of the simplex identification via split augmented Lagrangian

Hyperspectral imaging can be used for object detection and for discriminating between different objects based on their spectral characteristics. One of the main problems of hyperspectral data analysis is the presence of mixed pixels, due to the low spatial resolution of such images. This means that several spectrally pure signatures (endmembers) are combined into the same mixed pixel. Linear spectral unmixing follows an unsupervised approach which aims at inferring pure spectral signatures and their material fractions at each pixel of the scene. The huge data volumes acquired by such sensors put stringent requirements on processing and unmixing methods. This paper proposes an efficient implementation of a unsupervised linear unmixing method on GPUs using CUDA. The method finds the smallest simplex by solving a sequence of nonsmooth convex subproblems using variable splitting to obtain a constraint formulation, and then applying an augmented Lagrangian technique. The parallel implementation of SISAL presented in this work exploits the GPU architecture at low level, using shared memory and coalesced accesses to memory. The results herein presented indicate that the GPU implementation can significantly accelerate the method's execution over big datasets while maintaining the methods accuracy.

Fast unsupervised technique for extraction of endmembers spectra from hyperspectral data

One of the most challenging task underlying many hyperspectral imagery applications is the linear unmixing. The key to linear unmixing is to find the set of reference substances, also called endmembers, that are representative of a given scene. This paper presents the vertex component analysis (VCA) a new method to unmix linear mixtures of hyperspectral sources. The algorithm is unsupervised and exploits a simple geometric fact: endmembers are vertices of a simplex. The algorithm complexity, measured in floating points operations, is O (n), where n is the sample size. The effectiveness of the proposed scheme is illustrated using simulated data.

The web feature service: transaction standard: an example of prototype implementation

Trabalho de Projecto apresentado como requisito parcial para obtenção do grau de Mestre em Ciência e Sistemas de Informação Geográfica

Parallel hyperspectral unmixing method via split augmented lagrangian on GPU

One of the main problems of hyperspectral data analysis is the presence of mixed pixels due to the low spatial resolution of such images. Linear spectral unmixing aims at inferring pure spectral signatures and their fractions at each pixel of the scene. The huge data volumes acquired by hyperspectral sensors put stringent requirements on processing and unmixing methods. This letter proposes an efficient implementation of the method called simplex identification via split augmented Lagrangian (SISAL) which exploits the graphics processing unit (GPU) architecture at low level using Compute Unified Device Architecture. SISAL aims to identify the endmembers of a scene, i.e., is able to unmix hyperspectral data sets in which the pure pixel assumption is violated. The proposed implementation is performed in a pixel-by-pixel fashion using coalesced accesses to memory and exploiting shared memory to store temporary data. Furthermore, the kernels have been optimized to minimize the threads divergence, therefore achieving high GPU occupancy. The experimental results obtained for the simulated and real hyperspectral data sets reveal speedups up to 49 times, which demonstrates that the GPU implementation can significantly accelerate the method's execution over big data sets while maintaining the methods accuracy.

Parallel method for sparse semisupervised hyperspectral unmixing

Parallel hyperspectral unmixing problem is considered in this paper. A semisupervised approach is developed under the linear mixture model, where the abundance's physical constraints are taken into account. The proposed approach relies on the increasing availability of spectral libraries of materials measured on the ground instead of resorting to endmember extraction methods. Since Libraries are potentially very large and hyperspectral datasets are of high dimensionality a parallel implementation in a pixel-by-pixel fashion is derived to properly exploits the graphics processing units (GPU) architecture at low level, thus taking full advantage of the computational power of GPUs. Experimental results obtained for real hyperspectral datasets reveal significant speedup factors, up to 164 times, with regards to optimized serial implementation.

Parallel sparse unmixing of hyperspectral data

In this paper, a new parallel method for sparse spectral unmixing of remotely sensed hyperspectral data on commodity graphics processing units (GPUs) is presented. A semi-supervised approach is adopted, which relies on the increasing availability of spectral libraries of materials measured on the ground instead of resorting to endmember extraction methods. This method is based on the spectral unmixing by splitting and augmented Lagrangian (SUNSAL) that estimates the material's abundance fractions. The parallel method is performed in a pixel-by-pixel fashion and its implementation properly exploits the GPU architecture at low level, thus taking full advantage of the computational power of GPUs. Experimental results obtained for simulated and real hyperspectral datasets reveal significant speedup factors, up to 1 64 times, with regards to optimized serial implementation.

Spectral and dynamical invariants in a complete clustered network

The main result of this work is a new criterion for the formation of good clusters in a graph. This criterion uses a new dynamical invariant, the performance of a clustering, that characterizes the quality of the formation of clusters. We prove that the growth of the dynamical invariant, the network topological entropy, has the effect of worsening the quality of a clustering, in a process of cluster formation by the successive removal of edges. Several examples of clustering on the same network are presented to compare the behavior of other parameters such as network topological entropy, conductance, coefficient of clustering and performance of a clustering with the number of edges in a process of clustering by successive removal.

Comparison of Disposable Pipette Extraction and Dispersive Solid-Phase Extraction in the QuEChERS Method for Analysis of Pesticides in Strawberries

In this study, we sought to assess the applicability of GC–MS/MS for the identification and quantification of 36 pesticides in strawberry from integrated pest management (IPM) and organic farming (OF). Citrate versions of QuEChERS (quick, easy, cheap, effective, rugged and safe) using dispersive solid-phase extraction (d-SPE) and disposable pipette extraction (DPX) for cleanup were compared for pesticide extraction. For cleanup, a combination of MgSO4, primary secondary amine and C18 was used for both the versions. Significant differences were observed in recovery results between the two sample preparation versions (DPX and d-SPE). Overall, 86% of the pesticides achieved recoveries (three spiking levels 10, 50 and 200 µg/kg) in the range of 70–120%, with <13% RSD. The matrix effects were also evaluated in both the versions and in strawberries from different crop types. Although not evidencing significant differences between the two methodologies were observed, however, the DPX cleanup proved to be a faster technique and easy to execute. The results indicate that QuEChERS with d-SPE and DPX and GC–MS/MS analysis achieved reliable quantification and identification of 36 pesticide residues in strawberries from OF and IPM.

Data Extraction Tool to Analyse, Transform and Store Real Data from Electricity Markets

The study of electricity markets operation has been gaining an increasing importance in the last years, as result of the new challenges that the restructuring process produced. Currently, lots of information concerning electricity markets is available, as market operators provide, after a period of confidentiality, data regarding market proposals and transactions. These data can be used as source of knowledge to define realistic scenarios, which are essential for understanding and forecast electricity markets behavior. The development of tools able to extract, transform, store and dynamically update data, is of great importance to go a step further into the comprehension of electricity markets and of the behaviour of the involved entities. In this paper an adaptable tool capable of downloading, parsing and storing data from market operators’ websites is presented, assuring constant updating and reliability of the stored data.

Automatic Electricity Markets Data Extraction for Realistic Multi-agent Simulations

Electricity markets worldwide suffered profound transformations. The privatization of previously nationally owned systems; the deregulation of privately owned systems that were regulated; and the strong interconnection of national systems, are some examples of such transformations [1, 2]. In general, competitive environments, as is the case of electricity markets, require good decision-support tools to assist players in their decisions. Relevant research is being undertaken in this field, namely concerning player modeling and simulation, strategic bidding and decision-support.

Declarative approach to data extraction of web pages

Thesis submitted to Faculdade de Ciências e Tecnologia of the Universidade Nova de Lisboa, in partial fulfilment of the requirements for the degree of Master in Computer Science

Automated image tagging through tag propagation

Trabalho apresentado no âmbito do Mestrado em Engenharia Informática, como requisito parcial Para obtenção do grau de Mestre em Engenharia Informática

Natural antioxidants extraction and their incorporation into model pharmaceutical systems

Dissertation to obtain a Master Degree in Biotechnology

Analysis of UV spectral bands using multidimensional scaling

This study describes the change of the ultraviolet spectral bands starting from 0.1 to 5.0 nm slit width in the spectral range of 200–400 nm. The analysis of the spectral bands is carried out by using the multidimensional scaling (MDS) approach to reach the latent spectral background. This approach indicates that 0.1 nm slit width gives higher-order noise together with better spectral details. Thus, 5.0 nm slit width possesses the higher peak amplitude and lower-order noise together with poor spectral details. In the above-mentioned conditions, the main problem is to find the relationship between the spectral band properties and the slit width. For this aim, the MDS tool is to used recognize the hidden information of the ultraviolet spectra of sildenafil citrate by using a Shimadzu UV–VIS 2550, which is in the world the best double monochromator instrument. In this study, the proposed mathematical approach gives the rich findings for the efficient use of the spectrophotometer in the qualitative and quantitative studies.