Deep learning architecture for gesture recognition

Pós-graduação em Engenharia Mecânica - FEG

Realidade misturada: conceitos, ferramentas e aplicações

The Mixed Reality proposes scenes combining between virtual and real worlds offering to the user an intuitive way of interaction according to a specific application. This tutorial paper aims at presenting the fundamentals concepts of this emergent kind of human-computer interface.

LeeMed

[ES] La irrupción actual de los teléfonos inteligentes (smartphones) equipados con diversos sensores y herramientas nativas, propicia la posibilidad de crear una gran gama de aplicaciones para mejorar la vida de personas con discapacidades. Con este proyecto se pretende cubrir estos objetivos: explorar las distintas posibilidades que ofrece la plataforma Android para implementar métodos de interacción hombre-máquina adaptados a personas con discapacidad visual. Identificar las problemáticas que afectan a las personas con discapacidad visual en el ámbito sociosanitario. Desarrollar una aplicación de carácter social que contribuya a mejorar la calidad de vida de estas personas. Como resultado del trabajo, se ha desarrollado una aplicación software llamada LeeMed, que consiste en una app para la plataforma Android, dirigida a personas con discapacidad visual, para la consulta de prospectos de medicamentos a través de múltiples interfaces humanas. El trabajo ha abordado tres tipos de interfaces: la oral (órdenes de voz), la gestual y la convencional de menús y opciones (GUI)

Smile detection for user interfaces

[EN]Perceptual User Interfaces (PUIs) aim at facilitating human-computer interaction with the aid of human-like capacities (computer vision, speech recognition, etc.). In PUIs, the human face is a central element, since it conveys not only identity but also other important information, particularly with respect to the user’s mood or emotional state. This paper describes both a face detector and a smile detector for PUIs. Both are suitable for real-time interaction.

Entorno de simulación aplicado al estudio, comprensión y control del fenómeno de la realimentación acústica

[ES]El diseño de entornos de simulación aplicados al estudio, aprendizaje y adquisición de competencias es un campo de trabajo activo en todos los niveles educativos. Los sucesivos avances experimentados por las Tecnologías de la Información y Comunicaciones (TIC), las plataformas que las sustentan y los estudios e investigaciones llevadas a cabo en el campo de la Interacción Persona Ordenador (Human Computer Interaction, HCI), soportan y hacen posible la realización de herramientas como la que aquí proponemos. Se trata de un entorno de simulación que ha sido diseñado con dos grandes objetivos: fomentar el aprendizaje activo y mejorar el rendimiento académico, como los más destacados. Para ello hemos elegido la Realimentación Acústica en los sistemas de refuerzo sonoro, fenómeno que exige un estudio detallado de los parámetros que lo controlan, que presentado de la forma que aquí proponemos permitirá un mejor aprovechamiento del escaso tiempo disponible para la experimentación en situaciones reales.

Eyewear selector

[EN]The widespread availability of portable computing power and inexpensive digital cameras is opening up new possibilities for retailers. One example is in optical shops, where a number of systems exist that facilitate eyeglasses selection. These systems are now more necessary as the market is saturated with an increasingly complex array of lenses, frames, coatings, tints, photochromic and polarizing treatments, etc. Research challenges encompass Computer Vision, Multimedia and Human-Computer Interaction. Cost factors are also of importance for widespread product acceptance. This paper describes a low-cost system that allows the user to visualize di erent spectacle models in live video. The user can also move the spectacles to adjust its position on the face. Experiments show the potential of the system.

Registrazione e replicazione di procedure Web con una Firefox Extension

In questo lavoro di tesi è stata sviluppata una Firefox Extension per la registrazione e la replicazione di procedure sul Web. Si tratterà a fondo l’ambiente tecnologico nel quale è stata sviluppata l’applicazione e il contesto in cui si inserisce una Firefox Extension. Illustreremo il problema che intendiamo risolvere con la nostra estensione,il contesto applicativo in cui si inserisce e riporteremo una serie di lavori correlati che cercano, con diversi approcci, di risolvere il nostro stesso problema. Illustreremo il lavoro trattando approfonditamente l’approccio da noi utilizzato, mostrandone i vantaggi e i limiti.

Semantic Service Architectures for Smart Environments

Many industries and academic institutions share the vision that an appropriate use of information originated from the environment may add value to services in multiple domains and may help humans in dealing with the growing information overload which often seems to jeopardize our life. It is also clear that information sharing and mutual understanding between software agents may impact complex processes where many actors (humans and machines) are involved, leading to relevant socioeconomic benefits. Starting from these two input, architectural and technological solutions to enable “environment-related cooperative digital services” are here explored. The proposed analysis starts from the consideration that our environment is physical space and here diversity is a major value. On the other side diversity is detrimental to common technological solutions, and it is an obstacle to mutual understanding. An appropriate environment abstraction and a shared information model are needed to provide the required levels of interoperability in our heterogeneous habitat. This thesis reviews several approaches to support environment related applications and intends to demonstrate that smart-space-based, ontology-driven, information-sharing platforms may become a flexible and powerful solution to support interoperable services in virtually any domain and even in cross-domain scenarios. It also shows that semantic technologies can be fruitfully applied not only to represent application domain knowledge. For example semantic modeling of Human-Computer Interaction may support interaction interoperability and transformation of interaction primitives into actions, and the thesis shows how smart-space-based platforms driven by an interaction ontology may enable natural ad flexible ways of accessing resources and services, e.g, with gestures. An ontology for computational flow execution has also been built to represent abstract computation, with the goal of exploring new ways of scheduling computation flows with smart-space-based semantic platforms.

Evoluzione delle user interface: dalla metafora del desktop al Wearable Computing

Negli ultimi decenni abbiamo assistito ad una graduale evoluzione delle interfacce utente e della tecnologia. Sono stati introdotti nuovi dispositivi mobile e wearable che negli ultimi anni hanno subito un incremento tecnologico esponenziale arrivando a fondersi con la vita di tutti i giorni. Le classiche interfacce grafiche WIMP, la metafora del desktop e le linee guida di progettazione fino ad ora sviluppate non risultano ideali per la nuova tecnologia di wearable computing. Il proposito che la tesi vuole andare ad affrontare è proprio quello di indagare lo sviluppo di nuove user inteface basate sulla tecnologia wearable ed in particolare per smart glasses.

Driver Safety in Far-side and Far-oblique Crashes: A Study of Patrol Vehicles in the United States of America

The dissertation titled "Driver Safety in Far-side and Far-oblique Crashes" presents a novel approach to assessing vehicle cockpit safety by integrating Human Factors and Applied Mechanics. The methodology of this approach is aimed at improving safety in compact mobile workspaces such as patrol vehicle cockpits. A statistical analysis performed using Michigan state's traffic crash data to assess various contributing factors that affect the risk of severe driver injuries showed that the risk was greater for unrestrained drivers (OR=3.38, p<0.0001) and for incidents involving front and far-side crashes without seatbelts (OR=8.0 and 23.0 respectively, p<0.005). Statistics also showed that near-side and far-side crashes pose similar threat to driver injury severity. A Human Factor survey was conducted to assess various Human-Machine/Human-Computer Interaction aspects in patrol vehicle cockpits. Results showed that tasks requiring manual operation, especially the usage of laptop, would require more attention and potentially cause more distraction. A vehicle survey conducted to evaluate ergonomics-related issues revealed that some of the equipment was in airbag deployment zones. In addition, experiments were conducted to assess the effects on driver distraction caused by changing the position of in-car accessories. A driving simulator study was conducted to mimic HMI/HCI in a patrol vehicle cockpit (20 subjects, average driving experience = 5.35 years, s.d. = 1.8). It was found that the mounting locations of manual tasks did not result in a significant change in response times. Visual displays resulted in response times less than 1.5sec. It can also be concluded that the manual task was equally distracting regardless of mounting positions (average response time was 15 secs). Average speeds and lane deviations did not show any significant results. Data from 13 full-scale sled tests conducted to simulate far-side impacts at 70 PDOF and 40 PDOF was used to analyze head injuries and HIC/AIS values. It was found that accelerations generated by the vehicle deceleration alone were high enough to cause AIS 3 - AIS 6 injuries. Pretensioners could mitigated injuries only in 40 PDOF (oblique) impacts but are useless in 70 PDOF impacts. Seat belts were ineffective in protecting the driver's head from injuries. Head would come in contact with the laptop during a far-oblique (40 PDOF) crash and far-side door for an angle-type crash (70 PDOF). Finite Element analysis head-laptop impact interaction showed that the contact velocity was the most crucial factor in causing a severe (and potentially fatal) head injury. Results indicate that no equipment may be mounted in driver trajectory envelopes. A very narrow band of space is left in patrol vehicles for installation of manual-task equipment to be both safe and ergonomic. In case of a contact, the material stiffness and damping properties play a very significant role in determining the injury outcome. Future work may be done on improving the interiors' material properties to better absorb and dissipate kinetic energy of the head. The design of seat belts and pretensioners may also be seen as an essential aspect to be further improved.

Playing with the Real World

In this paper we provide a framework that enables the rapid development of applications using non-standard input devices. Flash is chosen as programming language since it can be used for quickly assembling applications. We overcome the difficulties of Flash to access external devices by introducing a very generic concept: The state information generated by input devices is transferred to a PC where a program collects them, interprets them and makes them available on a web server. Application developers can now integrate a Flash component that accesses the data stored in XML format and directly use it in their application.

Exploring Urban Environments Using Virtual and Augmented Reality

In this paper, we propose the use of specific system architecture, based on mobile device, for navigation in urban environments. The aim of this work is to assess how virtual and augmented reality interface paradigms can provide enhanced location based services using real-time techniques in the context of these two different technologies. The virtual reality interface is based on faithful graphical representation of the localities of interest, coupled with sensory information on the location and orientation of the user, while the augmented reality interface uses computer vision techniques to capture patterns from the real environment and overlay additional way-finding information, aligned with real imagery, in real-time. The knowledge obtained from the evaluation of the virtual reality navigational experience has been used to inform the design of the augmented reality interface. Initial results of the user testing of the experimental augmented reality system for navigation are presented.

Augmenting a Laser Pointer with a Diffraction Grating for Monoscopic 6DOF Detection

This article illustrates the detection of 6 degrees of freedom (DOF) for Virtual Environment interactions using a modified simple laser pointer device and a camera. The laser pointer is combined with a diffraction rating to project a unique laser grid onto the projection planes used in projection-based immersive VR setups. The distortion of the projected grid is used to calculate the translational and rotational degrees of freedom required for human-computer interaction purposes.

Multi-Contact Grasp Interaction for Virtual Environments

The grasping of virtual objects has been an active research field for several years. Solutions providing realistic grasping rely on special hardware or require time-consuming parameterizations. Therefore, we introduce a flexible grasping algorithm enabling grasping without computational complex physics. Objects can be grasped and manipulated with multiple fingers. In addition, multiple objects can be manipulated simultaneously with our approach. Through the usage of contact sensors the technique is easily configurable and versatile enough to be used in different scenarios.