Human activity recognition with accelerometry: novel time and frequency features

Human Activity Recognition systems require objective and reliable methods that can be used in the daily routine and must offer consistent results according with the performed activities. These systems are under development and offer objective and personalized support for several applications such as the healthcare area. This thesis aims to create a framework for human activities recognition based on accelerometry signals. Some new features and techniques inspired in the audio recognition methodology are introduced in this work, namely Log Scale Power Bandwidth and the Markov Models application. The Forward Feature Selection was adopted as the feature selection algorithm in order to improve the clustering performances and limit the computational demands. This method selects the most suitable set of features for activities recognition in accelerometry from a 423th dimensional feature vector. Several Machine Learning algorithms were applied to the used accelerometry databases – FCHA and PAMAP databases - and these showed promising results in activities recognition. The developed algorithm set constitutes a mighty contribution for the development of reliable evaluation methods of movement disorders for diagnosis and treatment applications.

Botnet cluster identification

Botnets are a group of computers infected with a specific sub-set of a malware family and controlled by one individual, called botmaster. This kind of networks are used not only, but also for virtual extorsion, spam campaigns and identity theft. They implement different types of evasion techniques that make it harder for one to group and detect botnet traffic. This thesis introduces one methodology, called CONDENSER, that outputs clusters through a self-organizing map and that identify domain names generated by an unknown pseudo-random seed that is known by the botnet herder(s). Aditionally DNS Crawler is proposed, this system saves historic DNS data for fast-flux and double fastflux detection, and is used to identify live C&Cs IPs used by real botnets. A program, called CHEWER, was developed to automate the calculation of the SVM parameters and features that better perform against the available domain names associated with DGAs. CONDENSER and DNS Crawler were developed with scalability in mind so the detection of fast-flux and double fast-flux networks become faster. We used a SVM for the DGA classififer, selecting a total of 11 attributes and achieving a Precision of 77,9% and a F-Measure of 83,2%. The feature selection method identified the 3 most significant attributes of the total set of attributes. For clustering, a Self-Organizing Map was used on a total of 81 attributes. The conclusions of this thesis were accepted in Botconf through a submited article. Botconf is known conferênce for research, mitigation and discovery of botnets tailled for the industry, where is presented current work and research. This conference is known for having security and anti-virus companies, law enforcement agencies and researchers.

A era de um mercado social

O crescimento e a expansão das redes sociais trouxe novas formas de interação entre os seres humanos que se repercutem na vida real. Os textos partilhados nas redes sociais e as interações resultantes de todas as atividades virtuais têm vindo a ganhar um grande impacto no quotidiano da sociedade e no âmbito económico e financeiro, as redes sociais tem sido alvo de diversos estudos, particularmente em termos de previsão e descrição do mercado acionista (Zhang, Fuehres, & Gloor, 2011) (Bollen, Mao & Zheng, 2010). Nesta investigação percebemos se o sentimento do Twitter, rede social de microblogging, se relaciona diretamente com o mercado acionista, querendo assim compreender qual o impacto das redes sociais no mercado financeiro. Tentámos assim relacionar duas dimensões, social e financeira, de forma a conseguirmos compreender de que forma poderemos utilizar os valores de uma para prever a outra. É um tópico especialmente interessante para empresas e investidores na medida em que se tenta compreender se o que se diz de determinada empresa no Twitter pode ter relação com o valor de mercado dessa empresa. Usámos duas técnicas de análise de sentimentos, uma de comparação léxica de palavras e outra de machine learning para compreender qual das duas tinha uma melhor precisão na classificação dos tweets em três atributos, positivo, negativo ou neutro. O modelo de machine learning foi o modelo escolhido e relacionámos esses dados com os dados do mercado acionista através de um teste de causalidade de Granger. Descobrimos que para certas empresas existe uma relação entre as duas variáveis, sentimento do Twitter e alteração da posição da ação entre dois períodos de tempo no mercado acionista, esta última variável estando dependente da dimensão temporal em que agrupamos o nosso sentimento do Twitter. Este estudo pretendeu assim dar seguimento ao trabalho desenvolvido por Bollen, Mao e Zheng (2010) que descobriram que uma dimensão de sentimento (calma) consegue ser usada para prever a direção das ações do mercado acionista, apesar de terem rejeitado que o sentimento geral (positivo, negativo ou neutro) não se relacionava de modo global com o mercado acionista. No seu trabalho compararam o sentimento de todos os tweets de um determinado período sem exclusão com o índice geral de ações no mercado enquanto a metodologia adotada nesta investigação foi realizada por empresa e apenas nos interessaram tweets que se relacionavam com aquela empresa em específico. Com esta diferença obtemos resultados diferentes e certas empresas demonstravam que existia relação entre várias combinações, principalmente para empresas tecnológicas. Testamos o agrupamento do sentimento do Twitter em 3 minutos, 1 hora e 1 dia, sendo que certas empresas só demonstravam relação quando aumentávamos a nossa dimensão temporal. Isto leva-nos a querer que o sentimento geral da empresa, e se a mesma for uma empresa tecnológica, está ligado ao mercado acionista estando condicionada esta relação à dimensão temporal que possamos estar a analisar.

Analysis and visualization of energy use for university campus

The reduction of greenhouse gas emissions is one of the big global challenges for the next decades due to its severe impact on the atmosphere that leads to a change in the climate and other environmental factors. One of the main sources of greenhouse gas is energy consumption, therefore a number of initiatives and calls for awareness and sustainability in energy use are issued among different types of institutional and organizations. The European Council adopted in 2007 energy and climate change objectives for 20% improvement until 2020. All European countries are required to use energy with more efficiency. Several steps could be conducted for energy reduction: understanding the buildings behavior through time, revealing the factors that influence the consumption, applying the right measurement for reduction and sustainability, visualizing the hidden connection between our daily habits impacts on the natural world and promoting to more sustainable life. Researchers have suggested that feedback visualization can effectively encourage conservation with energy reduction rate of 18%. Furthermore, researchers have contributed to the identification process of a set of factors which are very likely to influence consumption. Such as occupancy level, occupants behavior, environmental conditions, building thermal envelope, climate zones, etc. Nowadays, the amount of energy consumption at the university campuses are huge and it needs great effort to meet the reduction requested by European Council as well as the cost reduction. Thus, the present study was performed on the university buildings as a use case to: a. Investigate the most dynamic influence factors on energy consumption in campus; b. Implement prediction model for electricity consumption using different techniques, such as the traditional regression way and the alternative machine learning techniques; and c. Assist energy management by providing a real time energy feedback and visualization in campus for more awareness and better decision making. This methodology is implemented to the use case of University Jaume I (UJI), located in Castellon, Spain.

Context classification for service robots

This dissertation presents a solution for environment sensing using sensor fusion techniques and a context/environment classification of the surroundings in a service robot, so it could change his behavior according to the different rea-soning outputs. As an example, if a robot knows he is outdoors, in a field environment, there can be a sandy ground, in which it should slow down. Contrariwise in indoor environments, that situation is statistically unlikely to happen (sandy ground). This simple assumption denotes the importance of context-aware in automated guided vehicles.

Métodos de aprendizagem automática: um estudo baseado na avaliação e previsão de clientes bancários

Data Mining surge, hoje em dia, como uma ferramenta importante e crucial para o sucesso de um negócio. O considerável volume de dados que atualmente se encontra disponível, por si só, não traz valor acrescentado. No entanto, as ferramentas de Data Mining, capazes de transformar dados e mais dados em conhecimento, vêm colmatar esta lacuna, constituindo, assim, um trunfo que ninguém quer perder. O presente trabalho foca-se na utilização das técnicas de Data Mining no âmbito da atividade bancária, mais concretamente na sua atividade de telemarketing. Neste trabalho são aplicados catorze algoritmos a uma base de dados proveniente do call center de um banco português, resultante de uma campanha para a angariação de clientes para depósitos a prazo com taxas de juro favoráveis. Os catorze algoritmos aplicados no caso prático deste projeto podem ser agrupados em sete grupos: Árvores de Decisão, Redes Neuronais, Support Vector Machine, Voted Perceptron, métodos Ensemble, aprendizagem Bayesiana e Regressões. De forma a beneficiar, ainda mais, do que a área de Data Mining tem para oferecer, este trabalho incide ainda sobre o redimensionamento da base de dados em questão, através da aplicação de duas estratégias de seleção de atributos: Best First e Genetic Search. Um dos objetivos deste trabalho prende-se com a comparação dos resultados obtidos com os resultados presentes no estudo dos autores Sérgio Moro, Raul Laureano e Paulo Cortez (Sérgio Moro, Laureano, & Cortez, 2011). Adicionalmente, pretende-se identificar as variáveis mais relevantes aquando da identificação do potencial cliente deste produto financeiro. Como principais conclusões, depreende-se que os resultados obtidos são comparáveis com os resultados publicados pelos autores mencionados, sendo os mesmos de qualidade e consistentes. O algoritmo Bagging é o que apresenta melhores resultados e a variável referente à duração da chamada telefónica é a que mais influencia o sucesso de campanhas similares.

From user browsing behaviour to user demographics

A Internet conta hoje com mais de 3 mil milhões de utilizadores e esse valor não para de aumentar. Desta forma, proporcionar uma experiência online agradável aos seus utilizadores é cada vez mais importante para as empresas. De modo a tirar partido dos benefícios deste crescimento, as empresas devem ser capazes de identificar os seus clientes-alvo dentro do total de utilizadores; e, subsequentemente, personalizar a sua experiência online. Existem diversas formas de estudar o comportamento online dos utilizadores; no entanto, estas não são ideais e existe uma ampla margem para melhoria. A inovação nesta área pode comportar um grande potencial comercial e até ser disruptiva. Com isto em mente, proponho-me a estudar a possível criacão de um sistema de aprendizagem automática (machine learning) que permita prever informa ações demográficas dos utilizadores estritamente com base no seu comportamento online. Tal sistema poderia constituir uma alternativa às atuais opções, que são mais invasivas; mitigando assim preocupações ao nível da proteção de dados pessoais. No primeiro capítulo (Introdução) explico a motivação para o estudo do comportamento dos utilizadores online por parte de empresas, e descrevo as opções disponíveis atualmente. Apresento também a minha proposta e o contexto em que assenta. O capítulo termina com a identicação de limitações que possam existir a priori. O segundo capítulo (Machine Learning) fornece uma introdução sobre machine learning, com o estudo dos algoritmos que vão ser utilizados e explicando como analisar os resultados. O terceiro capítulo (Implementação) explica a implementação do sistema proposto e descreve o sistema que desenvolvi no decorrer deste estudo, e como integra-lo em sistemas já existentes. No quarto capítulo (Análise e manipulação dos dados), mostro os dados compilados e explico como os recolhi e manipulei para testar a hipótese. No quinto capítulo (Análise de dados e discussão) vemos como e que os dados recolhidos foram usados pelos vários algoritmos para descobrir como se correlacionam com dados dos utilizadores e analiso e discuto os resultados observados. Por fim, o sexto e último capítulo apresenta as conclusões. Dependendo dos resultados, mostro como a hipótese poderia ser melhor testada, ou então discuto os próximos passos para tornar o sistema realidade.

Hand gesture recognition for human computer interaction : a comparative study of different image features

Hand gesture recognition for human computer interaction, being a natural way of human computer interaction, is an area of active research in computer vision and machine learning. This is an area with many different possible applications, giving users a simpler and more natural way to communicate with robots/systems interfaces, without the need for extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them to convey information or for device control. For that, vision-based hand gesture interfaces require fast and extremely robust hand detection, and gesture recognition in real time. In this study we try to identify hand features that, isolated, respond better in various situations in human-computer interaction. The extracted features are used to train a set of classifiers with the help of RapidMiner in order to find the best learner. A dataset with our own gesture vocabulary consisted of 10 gestures, recorded from 20 users was created for later processing. Experimental results show that the radial signature and the centroid distance are the features that when used separately obtain better results, with an accuracy of 91% and 90,1% respectively obtained with a Neural Network classifier. These to methods have also the advantage of being simple in terms of computational complexity, which make them good candidates for real-time hand gesture recognition.

Generic system for human-computer gesture interaction: applications on sign language recognition and robotic soccer refereeing

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for human-computer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of visionbased interaction systems could be the same for all applications and thus facilitate the implementation. For hand posture recognition, a SVM (Support Vector Machine) model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM (Hidden Markov Model) model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications. To validate the proposed framework two applications were implemented. The first one is a real-time system able to interpret the Portuguese Sign Language. The second one is an online system able to help a robotic soccer game referee judge a game in real time.

Vision-based portuguese sign language recognition system

Vision-based hand gesture recognition is an area of active current research in computer vision and machine learning. Being a natural way of human interaction, it is an area where many researchers are working on, with the goal of making human computer interaction (HCI) easier and natural, without the need for any extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them, for example, to convey information. For that, vision-based hand gesture interfaces require fast and extremely robust hand detection, and gesture recognition in real time. Hand gestures are a powerful human communication modality with lots of potential applications and in this context we have sign language recognition, the communication method of deaf people. Sign lan- guages are not standard and universal and the grammars differ from country to coun- try. In this paper, a real-time system able to interpret the Portuguese Sign Language is presented and described. Experiments showed that the system was able to reliably recognize the vowels in real-time, with an accuracy of 99.4% with one dataset of fea- tures and an accuracy of 99.6% with a second dataset of features. Although the im- plemented solution was only trained to recognize the vowels, it is easily extended to recognize the rest of the alphabet, being a solid foundation for the development of any vision-based sign language recognition user interface system.

Generic system for human-computer gesture interaction

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for humancomputer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of vision-based interaction systems can be the same for all applications and thus facilitate the implementation. In order to test the proposed solutions, three prototypes were implemented. For hand posture recognition, a SVM model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications.

Development of computational tools for the integrated analysis of DNA microarray data with applications in cancer research

The MAP-i Doctoral Program of the Universities of Minho, Aveiro and Porto

Benchmarking sobre técnicas de otimização para modelos de apoio à decisão na medicina intensiva

Dissertação de mestrado integrado em Engenharia e Gestão de Sistemas de Informação

Automatic eye blink detection using consumer web cameras

This research aims to advance blinking detection in the context of work activity. Rather than patients having to attend a clinic, blinking videos can be acquired in a work environment, and further automatically analyzed. Therefore, this paper presents a methodology to perform the automatic detection of eye blink using consumer videos acquired with low-cost web cameras. This methodology includes the detection of the face and eyes of the recorded person, and then it analyzes the low-level features of the eye region to create a quantitative vector. Finally, this vector is classified into one of the two categories considered —open and closed eyes— by using machine learning algorithms. The effectiveness of the proposed methodology was demonstrated since it provides unbiased results with classification errors under 5%

Forecasting human entrances at a commercial store using facial recognition data

Dissertação de mestrado integrado em Engenharia e Gestão de Sistemas de Informação