Recognition of Facial Expressions with Autoencoders and Convolutional-Nets

Les humains communiquent via différents types de canaux: les mots, la voix, les gestes du corps, des émotions, etc. Pour cette raison, un ordinateur doit percevoir ces divers canaux de communication pour pouvoir interagir intelligemment avec les humains, par exemple en faisant usage de microphones et de webcams. Dans cette thèse, nous nous intéressons à déterminer les émotions humaines à partir d’images ou de vidéo de visages afin d’ensuite utiliser ces informations dans différents domaines d’applications. Ce mémoire débute par une brève introduction à l'apprentissage machine en s’attardant aux modèles et algorithmes que nous avons utilisés tels que les perceptrons multicouches, réseaux de neurones à convolution et autoencodeurs. Elle présente ensuite les résultats de l'application de ces modèles sur plusieurs ensembles de données d'expressions et émotions faciales. Nous nous concentrons sur l'étude des différents types d’autoencodeurs (autoencodeur débruitant, autoencodeur contractant, etc) afin de révéler certaines de leurs limitations, comme la possibilité d'obtenir de la coadaptation entre les filtres ou encore d’obtenir une courbe spectrale trop lisse, et étudions de nouvelles idées pour répondre à ces problèmes. Nous proposons également une nouvelle approche pour surmonter une limite des autoencodeurs traditionnellement entrainés de façon purement non-supervisée, c'est-à-dire sans utiliser aucune connaissance de la tâche que nous voulons finalement résoudre (comme la prévision des étiquettes de classe) en développant un nouveau critère d'apprentissage semi-supervisé qui exploite un faible nombre de données étiquetées en combinaison avec une grande quantité de données non-étiquetées afin d'apprendre une représentation adaptée à la tâche de classification, et d'obtenir une meilleure performance de classification. Finalement, nous décrivons le fonctionnement général de notre système de détection d'émotions et proposons de nouvelles idées pouvant mener à de futurs travaux.

Evaluación continua para alumnos enfermos por medio de internet.

Facilitar la escolarización de un alumno enfermo, que no puede asistir al centro educativo desde primero de ESO, mediante asistencia educativa virtual, desarrollando materiales en soporte digital, estableciendo la comunicación virtual alumno-profesorado y alumno-compañeros y compañeras del grupo de referencia y de otros de niveles superiores, a través de la red, mediante webcam y además haciendo intervenciones domiciliarias.

Internet y los programas informáticos educativos como herramientas fundamentales en el aula.

Este proyecto de innovación educativa se presentó en el Congreso Internacional de Innovación en la Educación celebrado en Valladolid los dias 26, 27 y 28 de abril de 2005

Low cost eye tracking for human-machine interfacing

One of the main challenges for developers of new human-computer interfaces is to provide a more natural way of interacting with computer systems, avoiding excessive use of hand and finger movements. In this way, also a valuable alternative communication pathway is provided to people suffering from motor disabilities. This paper describes the construction of a low cost eye tracker using a fixed head setup. Therefore a webcam, laptop and an infrared lighting source were used together with a simple frame to fix the head of the user. Furthermore, detailed information on the various image processing techniques used for filtering the centre of the pupil and different methods to calculate the point of gaze are discussed. An overall accuracy of 1.5 degrees was obtained while keeping the hardware cost of the device below 100 euros.

MusiCam: an instrument to demonstrate chromaphonic synesthesia

Inspired by a type of synesthesia where colour typically induces musical notes the MusiCam project investigates this unusual condition, particularly the transition from colour to sound. MusiCam explores the potential benefits of this idiosyncrasy as a mode of human computer interaction (1-10), providing a host of meaningful applications spanning control, communication and composition. Colour data is interpreted by means of an off-the-shelf webcam, and music is generated in real-time through regular speakers. By making colour-based gestures users can actively control the parameters of sounds, compose melodies and motifs or mix multiple tracks on the fly. The system shows great potential as an interactive medium and as a musical controller. The trials conducted to date have produced encouraging results, and only hint at the new possibilities achievable by such a device.

Motion Cues Analysis for Parkinson Gait Recognition

Background: Previous assessment methods for PG recognition used sensor mechanisms for PG that may cause discomfort. In order to avoid stress of applying wearable sensors, computer vision (CV) based diagnostic systems for PG recognition have been proposed. Main constraints in these methods are the laboratory setup procedures: Novel colored dresses for the patients were specifically designed to segment the test body from a specific colored background. Objective: To develop an image processing tool for home-assessment of Parkinson Gait(PG) by analyzing motion cues extracted during the gait cycles. Methods: The system is based on the idea that a normal body attains equilibrium during the gait by aligning the body posture with the axis of gravity. Due to the rigidity in muscular tone, persons with PD fail to align their bodies with the axis of gravity. The leaned posture of PD patients appears to fall forward. Whereas a normal posture exhibits a constant erect posture throughout the gait. Patients with PD walk with shortened stride angle (less than 15 degrees on average) with high variability in the stride frequency. Whereas a normal gait exhibits a constant stride frequency with an average stride angle of 45 degrees. In order to analyze PG, levodopa-responsive patients and normal controls were videotaped with several gait cycles. First, the test body is segmented in each frame of the gait video based on the pixel contrast from the background to form a silhouette. Next, the center of gravity of this silhouette is calculated. This silhouette is further skeletonized from the video frames to extract the motion cues. Two motion cues were stride frequency based on the cyclic leg motion and the lean frequency based on the angle between the leaned torso tangent and the axis of gravity. The differences in the peaks in stride and lean frequencies between PG and normal gait are calculated using Cosine Similarity measurements. Results: High cosine dissimilarity was observed in the stride and lean frequencies between PG and normal gait. High variations are found in the stride intervals of PG whereas constant stride intervals are found in the normal gait. Conclusions: We propose an algorithm as a source to eliminate laboratory constraints and discomfort during PG analysis. Installing this tool in a home computer with a webcam allows assessment of gait in the home environment.

Sistemas dopaminérgicos e ação antipsicótica : abordagens experimentais e teóricas

Os objetivos da presente tese de doutorado foram os de buscar novos antipsicóticos atípicos de baixo preço comercial e também procurar entender o mecanismo de ação que leva a um perfil antipsicótico atípico. Os resultados da tese são divididos em duas partes, de acordo com sua natureza, em experimentais (primeira parte) e teóricos (segunda parte). Para o desenvolvimento da primeira parte, foi necessária primeiramente a programação de um software para medir locomoção em roedores após filmagem com webcam. A seguir, foram investigados os efeitos da guanosina, flunarizina e cinarizina em modelos animais de psicose, bem como em outros paradigmas comportamentais. A guanosina foi escolhida para estudo uma vez que tem se mostrado que ela interage com o sistema glutamatérgico – que sabidamente está envolvido na fisiopatologia da esquizofrenia – promovendo a captação astrocitária de glutamato. Já a flunarizina e a cinarizina, dois bloqueadores de canal de cálcio empregados para tratar enxaqueca e vertigem foram escolhidas pelo fato delas produzirem sinais e sintomas extrapiramidais em pacientes idosos, o que posteriormente foi relacionado às suas propriedades como antagonistas moderados dos receptores dopaminérgicos do tipo D2 A guanosina diminuiu o aumento de locomoção induzido por um antagonista NMDA (MK-801), enquanto que não apresentou efeito sobre o aumento de locomoção induzido por anfetamina, de forma que sua utilidade como potencial antipsicótico deve ser ainda melhor estudada. Tanto a flunarizina quanto a cinarizina foram capazes de diminuir o aumento de locomoção induzido por MK-801 e por anfetamina em doses que não causam efeitos catalépticos importantes. Portanto, foi concluído que estes dois compostos apresentam um potencial perfil de antipsicótico atípico, com as vantagens de já estarem disponíveis para uso comercial, boa tolerabilidade e baixo custo quando comparados com os antipsicóticos atípicos disponíveis comercial. A segunda parte da tese apresenta alguns resultados teóricos matemáticos que podem ser derivados da teoria da lei de ação das massas aplicada ao binding de receptores, utilizando também resultados experimentais já conhecidos de PET Estes resultados apresentam insights ao entendimento das diferenças entre os perfis antipsicóticos atípicos e típicos em relação à geração de sinais extrapiramidais. É discutido que fatores culturais e comerciais relacionados à posologia atual empregada no tratamento com antipsicóticos típicos podem ser os responsáveis pelas diferenças de perfis, uma vez que alguns deles são prescritos em doses proporcionalmente maiores em relação à sua afinidade, atingindo assim maiores níveis de bloqueio dopaminérgico no estriado. Uma curta meia-vida plasmática também é apontada como um possível parâmetro importante na geração de um perfil atípico. É mostrado ainda alguns erros de concepção relacionados ao curso temporal da ocupação dopaminérgica que tem sido atualmente cometidos na literatura científica, como o conceito de meia-vida de ocupação de receptores. Como um último resultado teórico, é proposto um algoritmo para a redução de dose em pacientes tratados com antipsicóticos apresentando sinais e sintomas extrapiramidais.

Como usar o Youtube para gravação de videoaulas

Tutorial sobre gravação de vídeos para Youtube.

Inexpensive solution for real-time video and image stitching

Image stitching is the process of joining several images to obtain a bigger view of a scene. It is used, for example, in tourism to transmit to the viewer the sensation of being in another place. I am presenting an inexpensive solution for automatic real time video and image stitching with two web cameras as the video/image sources. The proposed solution relies on the usage of several markers in the scene as reference points for the stitching algorithm. The implemented algorithm is divided in four main steps, the marker detection, camera pose determination (in reference to the markers), video/image size and 3d transformation, and image translation. Wii remote controllers are used to support several steps in the process. The built‐in IR camera provides clean marker detection, which facilitates the camera pose determination. The only restriction in the algorithm is that markers have to be in the field of view when capturing the scene. Several tests where made to evaluate the final algorithm. The algorithm is able to perform video stitching with a frame rate between 8 and 13 fps. The joining of the two videos/images is good with minor misalignments in objects at the same depth of the marker,misalignments in the background and foreground are bigger. The capture process is simple enough so anyone can perform a stitching with a very short explanation. Although real‐time video stitching can be achieved by this affordable approach, there are few shortcomings in current version. For example, contrast inconsistency along the stitching line could be reduced by applying a color correction algorithm to every source videos. In addition, the misalignments in stitched images due to camera lens distortion could be eased by optical correction algorithm. The work was developed in Apple’s Quartz Composer, a visual programming environment. A library of extended functions was developed using Xcode tools also from Apple.

Wow!Table: framework de visão computacional

Orientador: Pedro Filipe Pereira Campos

Controle feedback de nível baseado em sensor de imagem aplicado ao equipamento misturador-decantador à inversão de fases (MDIF®)

The treatment of wastewaters contaminated with oil is of great practical interest and it is fundamental in environmental issues. A relevant process, which has been studied on continuous treatment of contaminated water with oil, is the equipment denominated MDIF® (a mixer-settler based on phase inversion). An important variable during the operation of MDIF® is the water-solvent interface level in the separation section. The control of this level is essential both to avoid the dragging of the solvent during the water removal and improve the extraction efficiency of the oil by the solvent. The measurement of oil-water interface level (in line) is still a hard task. There are few sensors able to measure oil-water interface level in a reliable way. In the case of lab scale systems, there are no interface sensors with compatible dimensions. The objective of this work was to implement a level control system to the organic solvent/water interface level on the equipment MDIF®. The detection of the interface level is based on the acquisition and treatment of images obtained dynamically through a standard camera (webcam). The control strategy was developed to operate in feedback mode, where the level measure obtained by image detection is compared to the desired level and an action is taken on a control valve according to an implemented PID law. A control and data acquisition program was developed in Fortran to accomplish the following tasks: image acquisition; water-solvent interface identification; to perform decisions and send control signals; and to record data in files. Some experimental runs in open-loop were carried out using the MDIF® and random pulse disturbances were applied on the input variable (water outlet flow). The responses of interface level permitted the process identification by transfer models. From these models, the parameters for a PID controller were tuned by direct synthesis and tests in closed-loop were performed. Preliminary results for the feedback loop demonstrated that the sensor and the control strategy developed in this work were suitable for the control of organic solvent-water interface level

A ferramenta VoiceThread em uma abordagem hibrida: o desenvolvimento da produção oral e da habilidade de noticing na aprendizagem de inglês como L2

VoiceThread (VT) is a collaborative and asynchronous web 2.0 tool, which permits the creation of oral presentations with the help of images, documents, texts and voice, allowing groups of people to browse and contribute with comments using several options: voice (microphone or cell phone), text and audio-file or video (webcam) (BOTTENTUIT JUNIOR, LISBÔA E COUTINHO, 2009). The hybrid experience with VoiceThread allows learners to plan their speech before recording it, without the pressure often existent in the classroom. Furthermore, the presentations can be recorded several times, enabling students to listen to them, notice the gaps in their oral production (noticing) and edit innumerous times before publishing them online. In this perspective, oral production is seen as a process of L2 acquisition, not only as practice of already existent knowledge, because it can stimulate the learner to process the language syntactically (SWAIN, 1985; 1995). In this context, this study aims to verify if there is a relation between the oral production of the learners more specifically the grammatical accuracy and the global oral grade and their noticing capacity, how the systematic practice with VoiceThread, in a hybrid approach, can impact the learners global oral development, their oral production in terms of fluency (number of words per minute), accuracy (number of errors in hundred words), and complexity (number of dependent clauses per minute), and on their noticing capacity (SCHMIDT, 1990; 1995; 2001), that is, the learner s capacity of noticing the gaps existent in their oral production. In order to answer these research questions, 49 L2 learners of English were divided into an experimental group (25 students) and a control group (24 students). The experimental group was exposed to the hybrid approach with VT during two months and, through a pre- and post-test, we verified if this systematic practice would positively influence these participants oral production and noticing capacity. These results were compared to the pre- and post-test scores from the control group, which was not exposed to VT. Finally, learners impressions in relation to the use of this tool were also sought through a questionnaire applied after the post-test. The results indicate that there is a statistically significant correlation between the learners speech production (accuracy and global oral grade) and their noticing capacity. Besides, it was verified a positive impact of VoiceThread on the learners speech production variables and on their noticing capacity. They also reveal a positive reaction by the learners in relation to the hybrid experience with this web tool

Sistema não-Intrusivo para Estimação da Direção do olhar utilizando redes neurais artificiais

The fundamental senses of the human body are: vision, hearing, touch, taste and smell. These senses are the functions that provide our relationship with the environment. The vision serves as a sensory receptor responsible for obtaining information from the outside world that will be sent to the brain. The gaze reflects its attention, intention and interest. Therefore, the estimation of gaze direction, using computer tools, provides a promising alternative to improve the capacity of human-computer interaction, mainly with respect to those people who suffer from motor deficiencies. Thus, the objective of this work is to present a non-intrusive system that basically uses a personal computer and a low cost webcam, combined with the use of digital image processing techniques, Wavelets transforms and pattern recognition, such as artificial neural network models, resulting in a complete system that performs since the image acquisition (including face detection and eye tracking) to the estimation of gaze direction. The obtained results show the feasibility of the proposed system, as well as several feature advantages.

Estratégias de comunicação na aprendizagem de português / espanhol por teletandem

O contexto de interação de protótipo teletandem, por meio do aplicativo MSN Messenger, permite o intercâmbio de informação de linguagem em tempo real, pelo uso de voz, texto e imagens (webcam). Foi observado que, por se tratar de um ambiente de troca entre línguas próximas, alguns dos processos de aquisição demonstrados pelos aprendizes se assemelham. No entanto, a língua materna foi o fator de influência para particularidades em relação ao uso de estratégias de comunicação.

A video-based biometric authentication for e-learning web applications

In the last years there was an exponential growth in the offering of Web-enabled distance courses and in the number of enrolments in corporate and higher education using this modality. However, the lack of efficient mechanisms that assures user authentication in this sort of environment, in the system login as well as throughout his session, has been pointed out as a serious deficiency. Some studies have been led about possible biometric applications for web authentication. However, password based authentication still prevails. With the popularization of biometric enabled devices and resultant fall of prices for the collection of biometric traits, biometrics is reconsidered as a secure remote authentication form for web applications. In this work, the face recognition accuracy, captured on-line by a webcam in Internet environment, is investigated, simulating the natural interaction of a person in the context of a distance course environment. Partial results show that this technique can be successfully applied to confirm the presence of users throughout the course attendance in an educational distance course. An efficient client/server architecture is also proposed. © 2009 Springer Berlin Heidelberg.