Gesture based interface for image annotation

Dissertação apresentada para obtenção do Grau de Mestre em Engenharia Informática pela Universidade Nova de Lisboa, Faculdade de Ciências e Tecnologia

Interface Humana Músculo-Facial

Esta dissertação tem como principal objetivo a criação de uma interface humana, baseada na eletromiografia dos músculos orbicular do olho e frontalis. O algoritmo de programação do microcontrolador ATmega2560 deteta o piscar de olhos voluntário, conta o número de vezes que este acontece e verifica se preenche os requisitos necessários à execução de um comando. Para este efeito foram utilizados elétrodos para a captação do sinal eletromiográfico. O sinal analógico é condicionado pela Shield ECG/EMG da Olimex sendo enviado para o arduíno ATmega2560. Este microcontrolador administra todos os atuadores, dos quais o mais importante é um painel de comandos (quatro comandos diferentes), no qual existe um ponteiro motorizado que indica qual a ação a realizar. O código de execução é extremamente simples:  se o utilizador piscar os olhos três vezes, o ponteiro movimenta-se para a secção do painel imediatamente à direita; e  se o utilizador piscar os olhos quatro vezes, o ponteiro movimenta-se para a secção do painel imediatamente à esquerda. Os testes realizados com este dispositivo indicam que os utilizadores demoram menos de 10 minutos a aprender a utilizar e executar todos os comandos do painel. Apenas num dos testes realizados o dispositivo não funcionou. Dos utilizadores que realizaram o teste:  vários usam óculos;  um idoso com graves problemas auditivos, cegueira parcial e dificuldades locomotoras;  nenhum foi incapaz de piscar, pelo menos, um dos olhos voluntariamente; e  a maioria referiu que, com alguma concentração e principalmente se ouvirem o bip sonoro, a aprendizagem de utilização torna-se muito fácil. Apesar dos limites impostos à concretização de um projeto deste tipo (dos quais se evidenciam as dificuldades em conseguir voluntários com paralisia medular, bem como os limites orçamentais), pode-se afirmar que este dispositivo é eficaz e seria uma mais valia quando implementado num cenário de paralisia medular (total ou parcial). A melhoria de qualidade de vida de um utilizador com estes problemas físicos, ou outros que lhe comprometam a locomoção é garantida. O cenário em que vivem é tremendamente limitado sendo urgente criar soluções para tornar estas vidas mais cómodas. Com os devidos aplicativos, o utilizador poderia abrir portas ou janelas, acender ou apagar luzes, pedir ajuda, ajustar a posição da cama, controlar cadeiras de rodas, entre outros. É neste sentido que surge a minha motivação de criar algo que ajude estas pessoas.

Development of advanced human robot interaction systems

O trabalho aqui apresentado é a Dissertação da minha Tese do curso de Mestrado em Engenharia Eletrotécnica e de Computadores do ISEP, realizada em parceria com o INESC TEC. O trabalho consiste no desenvolvimento de um sistema avançado de interação entre homem-robô, usando ferramentas de software livres e de domínio público e hardware pouco dispendioso e facilmente acessível. Pretende-se que o sistema desenvolvido possa ser adotado por pequenas ou micro empresas, daí a restrição monetária. Este tipo de empresas tem, por norma, uma capacidade de investimento pequena, e ficam impossibilitadas de aceder a este tipo de sistemas automatizados se estes forem caros. No entanto, o robô continua a ser um componente fundamental, sendo dispendioso. Os trabalhos realizados pelos sistemas robóticos podem por um lado, ser repetitivos sem necessidade de grandes ajustes; por outro lado, o trabalho a realizar pode ser bastante diverso, sendo necessários bastantes ajustes com (possivelmente) programação do robô. As empresas podem não ter disponível mão-de-obra qualificada para realização da programação do robô. Pretende-se então um sistema de “ensino” que seja simples e rápido. Este trabalho pretende satisfazer as necessidades de um sistema de interação homem-robô intuitivo mesmo para operadores que não estejam familiarizados com a robótica. Para simplificar a transferência de informação da tarefa a desempenhar pelo sistema robótico é usado um sistema de infravermelhos para delinear a operação a desempenhar, neste caso concreto uma operação de soldadura. O operador usa um apontador com marcadores, a posição destes marcadores é detetada usando duas câmaras para permitir o posicionamento tridimensional no espaço. As câmaras possuem filtros infravermelhos para separar o espectro de luz. Para o controlo do sistema e interface com o robô é usado um computador de baixos recursos computacionais e energéticos, e também de baixo custo. O sistema desenvolvido é portanto computacionalmente leve para poder ser executado neste computador.

Linked Data based Health Information Representation, Visualization and Retrieval System on the Semantic Web

Dissertation submitted in partial fulfillment of the requirements for the Degree of Master of Science in Geospatial Technologies.

Brain-computer interface (BCI): a methodological proposal to assess the impacts of medical applications in 2022

Based on the report for “Project IV” unit of the PhD programme on Technology Assessment (Doctoral Conference) at Universidade Nova de Lisboa (December 2011). This thesis research has the supervision of António Moniz (FCT-UNL and ITAS-KIT) and Armin Grunwald (Karlsruhe Institute of Technology-ITAS, Germany). Other members of the thesis committee are Mário Forjaz Secca (FCT-UNL) and Femke Nijboer (University of Twente, Netherlands).

Avatar modeling: a telepresence study with natural user interface

Dissertação para obtenção do Grau de Mestre em Engenharia Electrotécnica e de Computadores

Human activity recognition with accelerometry: novel time and frequency features

Human Activity Recognition systems require objective and reliable methods that can be used in the daily routine and must offer consistent results according with the performed activities. These systems are under development and offer objective and personalized support for several applications such as the healthcare area. This thesis aims to create a framework for human activities recognition based on accelerometry signals. Some new features and techniques inspired in the audio recognition methodology are introduced in this work, namely Log Scale Power Bandwidth and the Markov Models application. The Forward Feature Selection was adopted as the feature selection algorithm in order to improve the clustering performances and limit the computational demands. This method selects the most suitable set of features for activities recognition in accelerometry from a 423th dimensional feature vector. Several Machine Learning algorithms were applied to the used accelerometry databases – FCHA and PAMAP databases - and these showed promising results in activities recognition. The developed algorithm set constitutes a mighty contribution for the development of reliable evaluation methods of movement disorders for diagnosis and treatment applications.

Hand gesture recognition for human computer interaction : a comparative study of different image features

Hand gesture recognition for human computer interaction, being a natural way of human computer interaction, is an area of active research in computer vision and machine learning. This is an area with many different possible applications, giving users a simpler and more natural way to communicate with robots/systems interfaces, without the need for extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them to convey information or for device control. For that, vision-based hand gesture interfaces require fast and extremely robust hand detection, and gesture recognition in real time. In this study we try to identify hand features that, isolated, respond better in various situations in human-computer interaction. The extracted features are used to train a set of classifiers with the help of RapidMiner in order to find the best learner. A dataset with our own gesture vocabulary consisted of 10 gestures, recorded from 20 users was created for later processing. Experimental results show that the radial signature and the centroid distance are the features that when used separately obtain better results, with an accuracy of 91% and 90,1% respectively obtained with a Neural Network classifier. These to methods have also the advantage of being simple in terms of computational complexity, which make them good candidates for real-time hand gesture recognition.

Vision-based portuguese sign language recognition system

Vision-based hand gesture recognition is an area of active current research in computer vision and machine learning. Being a natural way of human interaction, it is an area where many researchers are working on, with the goal of making human computer interaction (HCI) easier and natural, without the need for any extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them, for example, to convey information. For that, vision-based hand gesture interfaces require fast and extremely robust hand detection, and gesture recognition in real time. Hand gestures are a powerful human communication modality with lots of potential applications and in this context we have sign language recognition, the communication method of deaf people. Sign lan- guages are not standard and universal and the grammars differ from country to coun- try. In this paper, a real-time system able to interpret the Portuguese Sign Language is presented and described. Experiments showed that the system was able to reliably recognize the vowels in real-time, with an accuracy of 99.4% with one dataset of fea- tures and an accuracy of 99.6% with a second dataset of features. Although the im- plemented solution was only trained to recognize the vowels, it is easily extended to recognize the rest of the alphabet, being a solid foundation for the development of any vision-based sign language recognition user interface system.

Discrimination of Brazilian propolis according to the seasoning using chemometrics and machine learning based on UV-Vis scanning data

Propolis is a chemically complex biomass produced by honeybees (Apis mellifera) from plant resins added of salivary enzymes, beeswax, and pollen. The biological activities described for propolis were also identified for donor plants resin, but a big challenge for the standardization of the chemical composition and biological effects of propolis remains on a better understanding of the influence of seasonality on the chemical constituents of that raw material. Since propolis quality depends, among other variables, on the local flora which is strongly influenced by (a)biotic factors over the seasons, to unravel the harvest season effect on the propolis chemical profile is an issue of recognized importance. For that, fast, cheap, and robust analytical techniques seem to be the best choice for large scale quality control processes in the most demanding markets, e.g., human health applications. For that, UV-Visible (UV-Vis) scanning spectrophotometry of hydroalcoholic extracts (HE) of seventy-three propolis samples, collected over the seasons in 2014 (summer, spring, autumn, and winter) and 2015 (summer and autumn) in Southern Brazil was adopted. Further machine learning and chemometrics techniques were applied to the UV-Vis dataset aiming to gain insights as to the seasonality effect on the claimed chemical heterogeneity of propolis samples determined by changes in the flora of the geographic region under study. Descriptive and classification models were built following a chemometric approach, i.e. principal component analysis (PCA) and hierarchical clustering analysis (HCA) supported by scripts written in the R language. The UV-Vis profiles associated with chemometric analysis allowed identifying a typical pattern in propolis samples collected in the summer. Importantly, the discrimination based on PCA could be improved by using the dataset of the fingerprint region of phenolic compounds ( = 280-400m), suggesting that besides the biological activities of those secondary metabolites, they also play a relevant role for the discrimination and classification of that complex matrix through bioinformatics tools. Finally, a series of machine learning approaches, e.g., partial least square-discriminant analysis (PLS-DA), k-Nearest Neighbors (kNN), and Decision Trees showed to be complementary to PCA and HCA, allowing to obtain relevant information as to the sample discrimination.

Regulation of human mesenchymal stem cell osteogenesis by specific surface density of fibronectin: a gradient study

 The success of synthetic bone implants requires good interface between the material and the host tissue. To study the biological relevance of fi bronectin (FN) density on the osteogenic commitment of human bone marrow mesenchymal stem cells (hBMMSCs), human FN was adsorbed in a linear density gradient on the surface of PCL. The evolution of the osteogenic markers alkaline phosphatase and collagen 1 alpha 1 was monitored by immunohistochemistry, and the cytoskeletal organization and the cell-derived FN were assessed. The functional analysis of the gradient revealed that the lower FN-density elicited stronger osteogenic expression and higher cytoskeleton spreading, hallmarks of the stem cell commitment to the osteoblastic lineage. The identifi cation of the optimal FN density regime for the osteogenic commitment of hBM-MSCs presents a simple and versatile strategy to signifi cantly enhance the surface properties of polycaprolactone as a paradigm for other synthetic polymers intended for bone-related applications.

Forecasting human entrances at a commercial store using facial recognition data

Dissertação de mestrado integrado em Engenharia e Gestão de Sistemas de Informação

Dried blood spots collected on filter paper: an international resource for the diagnosis and genetic characterization of human immunodeficiency virus Type-1

The collection of dried blood spots (DBS) on filter paper provides a powerful approach for the development of large-scale, population-based screening programs. DBS methods are particularly valuable in developing countries and isolated rural regions where resources are limited. Large numbers of field specimens can be economically collected and shipped to centralized reference laboratories for genetic and (or) serological analysis. Alternatively, the dried blood can be stored and used as an archival resource to rapidly establish the frequency and distribution of newly recognized mutations, confirm patient identity or track the origins and emergence of newly identified pathogens. In this report, we describe how PCR-based technologies are beginning to interface with international screening programmes for the diagnosis and genetic characterization of human immunodeficiency virus type 1 (HIV-1). In particular, we review recent progress using DBS specimens to resolve the HIV-1 infection status of neonates, monitor the genetic evolution of HIV-1 during early infancy and establish a sentinel surveillance system for the systematic monitoring of HIV-1 genetic variation in Asia.

Oxidative stress and inflammation response after nanoparticle exposure : differences between human lung cell monocultures and an advanced three-dimensional model of the human epithelial airways

Combustion-derived and manufactured nanoparticles (NPs) are known to provoke oxidative stress and inflammatory responses in human lung cells; therefore, they play an important role during the development of adverse health effects. As the lungs are composed of more than 40 different cell types, it is of particular interest to perform toxicological studies with co-cultures systems, rather than with monocultures of only one cell type, to gain a better understanding of complex cellular reactions upon exposure to toxic substances. Monocultures of A549 human epithelial lung cells, human monocyte-derived macrophages and monocyte-derived dendritic cells (MDDCs) as well as triple cell co-cultures consisting of all three cell types were exposed to combustion-derived NPs (diesel exhaust particles) and to manufactured NPs (titanium dioxide and single-walled carbon nanotubes). The penetration of particles into cells was analysed by transmission electron microscopy. The amount of intracellular reactive oxygen species (ROS), the total antioxidant capacity (TAC) and the production of tumour necrosis factor (TNF)-a and interleukin (IL)-8 were quantified. The results of the monocultures were summed with an adjustment for the number of each single cell type in the triple cell co-culture. All three particle types were found in all cell and culture types. The production of ROS was induced by all particle types in all cell cultures except in monocultures of MDDCs. The TAC and the (pro-)inflammatory reactions were not statistically significantly increased by particle exposure in any of the cell cultures. Interestingly, in the triple cell co-cultures, the TAC and IL-8 concentrations were lower and the TNF-a concentrations were higher than the expected values calculated from the monocultures. The interplay of different lung cell types seems to substantially modulate the oxidative stress and the inflammatory responses after NP exposure. [Authors]

Functional analysis and structural modeling of human APOBEC3G reveal the role of evolutionarily conserved elements in the inhibition of human immunodeficiency virus type 1 infection and Alu transposition.

Retroelements are important evolutionary forces but can be deleterious if left uncontrolled. Members of the human APOBEC3 family of cytidine deaminases can inhibit a wide range of endogenous, as well as exogenous, retroelements. These enzymes are structurally organized in one or two domains comprising a zinc-coordinating motif. APOBEC3G contains two such domains, only the C terminal of which is endowed with editing activity, while its N-terminal counterpart binds RNA, promotes homo-oligomerization, and is necessary for packaging into human immunodeficiency virus type 1 (HIV-1) virions. Here, we performed a large-scale mutagenesis-based analysis of the APOBEC3G N terminus, testing mutants for (i) inhibition of vif-defective HIV-1 infection and Alu retrotransposition, (ii) RNA binding, and (iii) oligomerization. Furthermore, in the absence of structural information on this domain, we used homology modeling to examine the positions of functionally important residues and of residues found to be under positive selection by phylogenetic analyses of primate APOBEC3G genes. Our results reveal the importance of a predicted RNA binding dimerization interface both for packaging into HIV-1 virions and inhibition of both HIV-1 infection and Alu transposition. We further found that the HIV-1-blocking activity of APOBEC3G N-terminal mutants defective for packaging can be almost entirely rescued if their virion incorporation is forced by fusion with Vpr, indicating that the corresponding region of APOBEC3G plays little role in other aspects of its action against this pathogen. Interestingly, residues forming the APOBEC3G dimer interface are highly conserved, contrasting with the rapid evolution of two neighboring surface-exposed amino acid patches, one targeted by the Vif protein of primate lentiviruses and the other of yet-undefined function.