Gesture based interface for image annotation

Dissertação apresentada para obtenção do Grau de Mestre em Engenharia Informática pela Universidade Nova de Lisboa, Faculdade de Ciências e Tecnologia

Pesquisa de clip arts combinando imagens raster e vectoriais

Existe actualmente um crescente desenvolvimento de sistemas de armazenamento e pesquisa de imagens. Uma aproximação adoptada nesses sistemas é a recuperação de imagens baseada em conteúdo (CBIR, Content-Based Image Retrieval). No âmbito destas aplicações existem utilizadores que pretendem utilizar imagens clip art para os seus trabalhos e apresentações. Existem muitas imagens clip art espalhadas por diversas bases de dados em sítios na Internet ou em colecções vendidas em dispositivos ópticos. A pesquisa de imagens nestas bases de dados leva os utilizadores a percorrem várias listas de imagens manualmente ou por métodos de pesquisa por texto, muitas vezes ineficientes. Essas bases de dados de clip arts são representadas por imagens vectoriais e imagens raster. Existem várias tecnologias de pesquisa e recuperação de ambos os tipos de imagens clip art, raster e vectoriais, contudo, a investigação tem sido realizada em separado sem retirar partido das duas áreas de investigação em conjunto, no problema de recuperar e explorar colecções de clip arts. O objectivo deste trabalho é implementar um motor de busca para encontrar clip arts em base de dados compostas por imagens vectoriais e imagens raster. O trabalho envolve um conversor de imagens raster em vectoriais, a extracção de características das imagens raster e vectoriais e a avaliação do sistema de recuperação de clip arts.

Linked Data based Health Information Representation, Visualization and Retrieval System on the Semantic Web

Dissertation submitted in partial fulfillment of the requirements for the Degree of Master of Science in Geospatial Technologies.

Semantic enrichment of knowledge sources supported by domain ontologies

This thesis introduces a novel conceptual framework to support the creation of knowledge representations based on enriched Semantic Vectors, using the classical vector space model approach extended with ontological support. One of the primary research challenges addressed here relates to the process of formalization and representation of document contents, where most existing approaches are limited and only take into account the explicit, word-based information in the document. This research explores how traditional knowledge representations can be enriched through incorporation of implicit information derived from the complex relationships (semantic associations) modelled by domain ontologies with the addition of information presented in documents. The relevant achievements pursued by this thesis are the following: (i) conceptualization of a model that enables the semantic enrichment of knowledge sources supported by domain experts; (ii) development of a method for extending the traditional vector space, using domain ontologies; (iii) development of a method to support ontology learning, based on the discovery of new ontological relations expressed in non-structured information sources; (iv) development of a process to evaluate the semantic enrichment; (v) implementation of a proof-of-concept, named SENSE (Semantic Enrichment kNowledge SourcEs), which enables to validate the ideas established under the scope of this thesis; (vi) publication of several scientific articles and the support to 4 master dissertations carried out by the department of Electrical and Computer Engineering from FCT/UNL. It is worth mentioning that the work developed under the semantic referential covered by this thesis has reused relevant achievements within the scope of research European projects, in order to address approaches which are considered scientifically sound and coherent and avoid “reinventing the wheel”.

Enhancing information retrieval in folksonomies using ontology of place constructed from Gazetteer information

Dissertation submitted in partial fulfilment of the requirements for the Degree of Master of Science in Geospatial Technologies

The visual image in Epiphanic Short History

The visual image is a fundamental component of epiphany, stressing its immediacy and vividness, corresponding to the enargeia of the traditional ekphrasis and also playing with cultural and social meanings. Morris Beja in his seminal book Epiphany in the Modern Novel, draws our attention to the distinction made by Joyce between the epiphany originated in a common object, in a discourse or gesture and the one arising in “a memorable phase of the mind itself”. This type materializes in the “dream-epiphany” and in the epiphany based in memory. On the other hand, Robert Langbaum in his study of the epiphanic mode, suggests that the category of “visionary epiphany” could account for the modern effect of an internally glowing vision like Blake’s “The Tyger”, which projects the vitality of a real tyger. The short story, whose length renders it a fitting genre for the use of different types of epiphany, has dealt with the impact of the visual image in this technique, to convey different effects and different aesthetic aims. This paper will present some examples of this occurrence in short stories of authors in whose work epiphany is a fundamental concept and literary technique: Walter Pater, Joseph Conrad, K. Mansfield, Clarice Lispector. Pater’s “imaginary portraits” concentrate on “priviledged moments” of the lives of the characters depicting their impressions through pictorial language; Conrad tries to show “moments of awakening” that can be remembered by the eye; Mansfield suggests that epiphany, the “glimpse”, should replace plot as an internal ordering principle of her impressionist short-stories; in C. Lispector the visualization of some situations is so aggressive that it causes nausea and a radical revelation on the protagonist’s.

Geospatial database generation from digital newspapers: use case for risk and disaster domains.

Dissertation submitted in partial fulfilment of the requirements for the Degree of Master of Science in Geospatial Technologies.

Automated image tagging through tag propagation

Trabalho apresentado no âmbito do Mestrado em Engenharia Informática, como requisito parcial Para obtenção do grau de Mestre em Engenharia Informática

A framework for supporting knowledge representation – an ontological based approach

Dissertação para obtenção do Grau de Mestre em Engenharia Electrotécnica e de Computadores

Mobile image processing to support educational geometry games

The mobile IT era is here, it is still growing and expanding at a steady rate and, most of all, it is entertaining. Mobile devices are used for entertainment, whether social through the so-called social networks, or private through web browsing, video watching or gaming. Youngsters make heavy use of these devices, and even small children show impressive adaptability and skill. However not much attention is directed towards education, especially in the case of young children. Too much time is usually spent in games which only purpose is to keep children entertained, time that could be put to better use such as developing elementary geometric notions. Taking advantage of this pocket computer scenario, it is proposed an application geared towards small children in the 6 – 9 age group that allows them to consolidate knowledge regarding geometric shapes, forming a stepping stone that leads to some fundamental mathematical knowledge to be exercised later on. To achieve this goal, the application will detect simple geometric shapes like squares, circles and triangles using the device’s camera. The novelty of this application will be a core real-time detection system designed and developed from the ground up for mobile devices, taking into account their characteristic limitations such as reduced processing power, memory and battery. User feedback was be gathered, aggregated and studied to assess the educational factor of the application.

Video metadata extraction in a videoMail system

Currently the world swiftly adapts to visual communication. Online services like YouTube and Vine show that video is no longer the domain of broadcast television only. Video is used for different purposes like entertainment, information, education or communication. The rapid growth of today’s video archives with sparsely available editorial data creates a big problem of its retrieval. The humans see a video like a complex interplay of cognitive concepts. As a result there is a need to build a bridge between numeric values and semantic concepts. This establishes a connection that will facilitate videos’ retrieval by humans. The critical aspect of this bridge is video annotation. The process could be done manually or automatically. Manual annotation is very tedious, subjective and expensive. Therefore automatic annotation is being actively studied. In this thesis we focus on the multimedia content automatic annotation. Namely the use of analysis techniques for information retrieval allowing to automatically extract metadata from video in a videomail system. Furthermore the identification of text, people, actions, spaces, objects, including animals and plants. Hence it will be possible to align multimedia content with the text presented in the email message and the creation of applications for semantic video database indexing and retrieving.

Semantic relations extraction from unstructured information for domain ontologies enrichment

Based in internet growth, through semantic web, together with communication speed improvement and fast development of storage device sizes, data and information volume rises considerably every day. Because of this, in the last few years there has been a growing interest in structures for formal representation with suitable characteristics, such as the possibility to organize data and information, as well as the reuse of its contents aimed for the generation of new knowledge. Controlled Vocabulary, specifically Ontologies, present themselves in the lead as one of such structures of representation with high potential. Not only allow for data representation, as well as the reuse of such data for knowledge extraction, coupled with its subsequent storage through not so complex formalisms. However, for the purpose of assuring that ontology knowledge is always up to date, they need maintenance. Ontology Learning is an area which studies the details of update and maintenance of ontologies. It is worth noting that relevant literature already presents first results on automatic maintenance of ontologies, but still in a very early stage. Human-based processes are still the current way to update and maintain an ontology, which turns this into a cumbersome task. The generation of new knowledge aimed for ontology growth can be done based in Data Mining techniques, which is an area that studies techniques for data processing, pattern discovery and knowledge extraction in IT systems. This work aims at proposing a novel semi-automatic method for knowledge extraction from unstructured data sources, using Data Mining techniques, namely through pattern discovery, focused in improving the precision of concept and its semantic relations present in an ontology. In order to verify the applicability of the proposed method, a proof of concept was developed, presenting its results, which were applied in building and construction sector.

Cognitive models for the concept of angle

Tese de doutoramento em Filosofia

Concept of Customer Relationship Management as an example of innovation in banking sector

Banks that has introduced CRM system, had to make some difficult changes in their organization in order to become more customer oriented. Beside the pure CRM banks try to adopt other innovative tools related with the core CRM. Some of these solutions are constructed in such a way so that ensured could be also access to the information beside to bank’s organization.

Automatic learning for the classification of chemical reactions and in statistical thermodynamics

This Thesis describes the application of automatic learning methods for a) the classification of organic and metabolic reactions, and b) the mapping of Potential Energy Surfaces(PES). The classification of reactions was approached with two distinct methodologies: a representation of chemical reactions based on NMR data, and a representation of chemical reactions from the reaction equation based on the physico-chemical and topological features of chemical bonds. NMR-based classification of photochemical and enzymatic reactions. Photochemical and metabolic reactions were classified by Kohonen Self-Organizing Maps (Kohonen SOMs) and Random Forests (RFs) taking as input the difference between the 1H NMR spectra of the products and the reactants. The development of such a representation can be applied in automatic analysis of changes in the 1H NMR spectrum of a mixture and their interpretation in terms of the chemical reactions taking place. Examples of possible applications are the monitoring of reaction processes, evaluation of the stability of chemicals, or even the interpretation of metabonomic data. A Kohonen SOM trained with a data set of metabolic reactions catalysed by transferases was able to correctly classify 75% of an independent test set in terms of the EC number subclass. Random Forests improved the correct predictions to 79%. With photochemical reactions classified into 7 groups, an independent test set was classified with 86-93% accuracy. The data set of photochemical reactions was also used to simulate mixtures with two reactions occurring simultaneously. Kohonen SOMs and Feed-Forward Neural Networks (FFNNs) were trained to classify the reactions occurring in a mixture based on the 1H NMR spectra of the products and reactants. Kohonen SOMs allowed the correct assignment of 53-63% of the mixtures (in a test set). Counter-Propagation Neural Networks (CPNNs) gave origin to similar results. The use of supervised learning techniques allowed an improvement in the results. They were improved to 77% of correct assignments when an ensemble of ten FFNNs were used and to 80% when Random Forests were used. This study was performed with NMR data simulated from the molecular structure by the SPINUS program. In the design of one test set, simulated data was combined with experimental data. The results support the proposal of linking databases of chemical reactions to experimental or simulated NMR data for automatic classification of reactions and mixtures of reactions. Genome-scale classification of enzymatic reactions from their reaction equation. The MOLMAP descriptor relies on a Kohonen SOM that defines types of bonds on the basis of their physico-chemical and topological properties. The MOLMAP descriptor of a molecule represents the types of bonds available in that molecule. The MOLMAP descriptor of a reaction is defined as the difference between the MOLMAPs of the products and the reactants, and numerically encodes the pattern of bonds that are broken, changed, and made during a chemical reaction. The automatic perception of chemical similarities between metabolic reactions is required for a variety of applications ranging from the computer validation of classification systems, genome-scale reconstruction (or comparison) of metabolic pathways, to the classification of enzymatic mechanisms. Catalytic functions of proteins are generally described by the EC numbers that are simultaneously employed as identifiers of reactions, enzymes, and enzyme genes, thus linking metabolic and genomic information. Different methods should be available to automatically compare metabolic reactions and for the automatic assignment of EC numbers to reactions still not officially classified. In this study, the genome-scale data set of enzymatic reactions available in the KEGG database was encoded by the MOLMAP descriptors, and was submitted to Kohonen SOMs to compare the resulting map with the official EC number classification, to explore the possibility of predicting EC numbers from the reaction equation, and to assess the internal consistency of the EC classification at the class level. A general agreement with the EC classification was observed, i.e. a relationship between the similarity of MOLMAPs and the similarity of EC numbers. At the same time, MOLMAPs were able to discriminate between EC sub-subclasses. EC numbers could be assigned at the class, subclass, and sub-subclass levels with accuracies up to 92%, 80%, and 70% for independent test sets. The correspondence between chemical similarity of metabolic reactions and their MOLMAP descriptors was applied to the identification of a number of reactions mapped into the same neuron but belonging to different EC classes, which demonstrated the ability of the MOLMAP/SOM approach to verify the internal consistency of classifications in databases of metabolic reactions. RFs were also used to assign the four levels of the EC hierarchy from the reaction equation. EC numbers were correctly assigned in 95%, 90%, 85% and 86% of the cases (for independent test sets) at the class, subclass, sub-subclass and full EC number level,respectively. Experiments for the classification of reactions from the main reactants and products were performed with RFs - EC numbers were assigned at the class, subclass and sub-subclass level with accuracies of 78%, 74% and 63%, respectively. In the course of the experiments with metabolic reactions we suggested that the MOLMAP / SOM concept could be extended to the representation of other levels of metabolic information such as metabolic pathways. Following the MOLMAP idea, the pattern of neurons activated by the reactions of a metabolic pathway is a representation of the reactions involved in that pathway - a descriptor of the metabolic pathway. This reasoning enabled the comparison of different pathways, the automatic classification of pathways, and a classification of organisms based on their biochemical machinery. The three levels of classification (from bonds to metabolic pathways) allowed to map and perceive chemical similarities between metabolic pathways even for pathways of different types of metabolism and pathways that do not share similarities in terms of EC numbers. Mapping of PES by neural networks (NNs). In a first series of experiments, ensembles of Feed-Forward NNs (EnsFFNNs) and Associative Neural Networks (ASNNs) were trained to reproduce PES represented by the Lennard-Jones (LJ) analytical potential function. The accuracy of the method was assessed by comparing the results of molecular dynamics simulations (thermal, structural, and dynamic properties) obtained from the NNs-PES and from the LJ function. The results indicated that for LJ-type potentials, NNs can be trained to generate accurate PES to be used in molecular simulations. EnsFFNNs and ASNNs gave better results than single FFNNs. A remarkable ability of the NNs models to interpolate between distant curves and accurately reproduce potentials to be used in molecular simulations is shown. The purpose of the first study was to systematically analyse the accuracy of different NNs. Our main motivation, however, is reflected in the next study: the mapping of multidimensional PES by NNs to simulate, by Molecular Dynamics or Monte Carlo, the adsorption and self-assembly of solvated organic molecules on noble-metal electrodes. Indeed, for such complex and heterogeneous systems the development of suitable analytical functions that fit quantum mechanical interaction energies is a non-trivial or even impossible task. The data consisted of energy values, from Density Functional Theory (DFT) calculations, at different distances, for several molecular orientations and three electrode adsorption sites. The results indicate that NNs require a data set large enough to cover well the diversity of possible interaction sites, distances, and orientations. NNs trained with such data sets can perform equally well or even better than analytical functions. Therefore, they can be used in molecular simulations, particularly for the ethanol/Au (111) interface which is the case studied in the present Thesis. Once properly trained, the networks are able to produce, as output, any required number of energy points for accurate interpolations.