Gesture based interface for image annotation

Dissertação apresentada para obtenção do Grau de Mestre em Engenharia Informática pela Universidade Nova de Lisboa, Faculdade de Ciências e Tecnologia

A software development process model for gesture-based interface

Gesture-based applications have particularities, since users interact in a natural way, much as they interact in the non-digital world. Hence, new requirements are needed on the software design process. This paper shows a software development process model for these applications, including requirement specification, design, implementation, and testing procedures. The steps and activities of the proposed model were tested through a game case study, which is a puzzle game. The puzzle is completed when all pieces of a painting are correctly positioned by the drag and drop action of users hand gesture. It also shows the results obtained of applying a heuristic evaluation on this game. © 2012 IEEE.

Gesture based persuasive interfaces for public ambient displays

Dissertação de Mestrado em Engenharia Informática 2º Semestre, 2011/2012

Gesture-based teleoperation using a holonomic robot

This work presents the development and integration of an user interface (UI) framework based on various current input devices that take advantage of our ergonomics. The purpose is to teleoperate a holonomic robot using upper member gestures and postures for studying the suitable of such interfaces when programming and interacting with a mobile robot. As performance vary from UI to UI the framework is focused to be used as a complementary industrial or didactic tool thus, changing how inexperience users tackle their first impressions when working with mobile robots while performing simple gesture-based teleoperation tasks. © 2012 ICROS.

Gesture-Based, Touch-Free Multi-User Gaming on Wall-Sized, High-Resolution Tiled Displays

Having to carry input devices can be inconvenient when interacting with wall-sized, high-resolution tiled displays. Such displays are typically driven by a cluster of computers. Running existing games on a cluster is non-trivial, and the performance attained using software solutions like Chromium is not good enough. This paper presents a touch-free, multi-user, humancomputer interface for wall-sized displays that enables completely device-free interaction. The interface is built using 16 cameras and a cluster of computers, and is integrated with the games Quake 3 Arena (Q3A) and Homeworld. The two games were parallelized using two different approaches in order to run on a 7x4 tile, 21 megapixel display wall with good performance. The touch-free interface enables interaction with a latency of 116 ms, where 81 ms are due to the camera hardware. The rendering performance of the games is compared to their sequential counterparts running on the display wall using Chromium. Parallel Q3A’s framerate is an order of magnitude higher compared to using Chromium. The parallel version of Homeworld performed on par with the sequential, which did not run at all using Chromium. Informal use of the touch-free interface indicates that it works better for controlling Q3A than Homeworld.

Microcontroller-based interface circuit for inductive sensors

This work proposes a fully-digital interface circuit for the measurement of inductive sensors using a low-cost microcontroller (µC) and without any intermediate active circuit. Apart from the µC and the sensor, the circuit just requires an external resistor and a reference inductance so that two RL circuits with a high-pass filter (HPF) topology are formed. The µC appropriately excites such RL circuits in order to measure the discharging time of the voltage across each inductance (i.e. sensing and reference) and then it uses such discharging times to estimate the sensor inductance. Experimental tests using a commercial µC show a non-linearity error (NLE) lower than 0.5%FSS (Full-Scale Span) when measuring inductances from 1 mH to 10 mH, and from 10 mH to 100 mH.

Combating shoulder-surfing: a hidden button gesture based scheme

This project describes an authentication technique that is shoulder-surfing resistant. Shoulder surfing is an attack in which an attacker can get access to private information by observing the user’s interaction with a terminal, or by using recording tools to record the user interaction and study the obtained data, with the objective of obtaining unauthorized access to a target user’s personal information. The technique described here relies on gestural analysis coupled with a secondary channel of authentication that uses button pressing. The thesis presents and evaluates multiple alternative algorithms for gesture analysis, and furthermore assesses the effectiveness of the technique.

Progettazione di un'applicazione di tipo "gesture-based" per il controllo di velivoli senza pilota a bordo

Progettazione e programmazione di un interfaccia per il controllo col movimento della mano di un simulatore di UAV.

A multimodal tablet-based interface for designing and reviewing 3D engineering models

The ability to view and interact with 3D models has been happening for a long time. However, vision-based 3D modeling has only seen limited success in applications, as it faces many technical challenges. Hand-held mobile devices have changed the way we interact with virtual reality environments. Their high mobility and technical features, such as inertial sensors, cameras and fast processors, are especially attractive for advancing the state of the art in virtual reality systems. Also, their ubiquity and fast Internet connection open a path to distributed and collaborative development. However, such path has not been fully explored in many domains. VR systems for real world engineering contexts are still difficult to use, especially when geographically dispersed engineering teams need to collaboratively visualize and review 3D CAD models. Another challenge is the ability to rendering these environments at the required interactive rates and with high fidelity. In this document it is presented a virtual reality system mobile for visualization, navigation and reviewing large scale 3D CAD models, held under the CEDAR (Collaborative Engineering Design and Review) project. It’s focused on interaction using different navigation modes. The system uses the mobile device's inertial sensors and camera to allow users to navigate through large scale models. IT professionals, architects, civil engineers and oil industry experts were involved in a qualitative assessment of the CEDAR system, in the form of direct user interaction with the prototypes and audio-recorded interviews about the prototypes. The lessons learned are valuable and are presented on this document. Subsequently it was prepared a quantitative study on the different navigation modes to analyze the best mode to use it in a given situation.

Improving Hardware Reuse through XML-based Interface Encapsulation

This work proposes an encapsulation scheme aimed at simplifying the reuse process of hardware cores. This hardware encapsulation approach has been conceived with a twofold objective. First, we look for the improvement of the reuse interface associated with the hardware core description. This is carried out in a first encapsulation level by improving the limited types and configuration options available in the conventional HDLs interface, and also providing information related to the implementation itself. Second, we have devised a more generic interface focused on describing the function avoiding details from a particular implementation, what corresponds to a second encapsulation level. This encapsulation allows the designer to define how to configure and use the design to implement a given functionality. The proposed encapsulation schemes help improving the amount of information that can be supplied with the design, and also allow to automate the process of searching, configuring and implementing diverse alternatives.

Diseño e implementación de un controlador de ratón gestual para Microsoft Windows con Kinect - Touchless Gesture User Interface

El dispositivo Microsoft Kinect for Windows y similares, han introducido en el mundo del PC una nueva forma de interacción denominada “Touchless Gesture User Interface” o TGUI (Interfaz de Usuario por Gestos sin Contacto) [Gentile et al. 2011]. Se trata de una tecnología novedosa en proceso de evolución. La tecnología de Kinect detecta la presencia de un usuario y monitoriza la posición en el espacio de sus articulaciones principales. Esta información permite desarrollar aplicaciones que posibiliten interactuar al usuario con una computadora mediante gestos y sin la necesidad de estar en contacto con periférico alguno. Desde la invención del periférico ratón en los años 60, resulta curioso que con la frenética evolución que ha experimentado el mundo de la informática en todos estos años, este dispositivo no haya sufrido cambios significativos o no haya sido incluso sustituido por otro periférico. En este proyecto se ha abordado el reto de desarrollar un controlador de ratón gestual para Windows utilizando Microsoft Kinect, de tal forma que se sustituya el uso del típico ratón y sea el propio usuario el que actúe como controlador mediante gestos y movimientos de sus manos. El resultado es llamativo y aporta numerosas mejoras y novedades frente a aplicaciones similares, aunque deja en evidencia algunas de las limitaciones de la tecnología implementada por Kinect a día de hoy. Es de esperar que cuando evolucione su tecnología, su uso se convierta en cotidiano.

A multimodal tablet-based interface for designing and reviewing 3D engineering models

The ability to view and interact with 3D models has been happening for a long time. However, vision-based 3D modeling has only seen limited success in applications, as it faces many technical challenges. Hand-held mobile devices have changed the way we interact with virtual reality environments. Their high mobility and technical features, such as inertial sensors, cameras and fast processors, are especially attractive for advancing the state of the art in virtual reality systems. Also, their ubiquity and fast Internet connection open a path to distributed and collaborative development. However, such path has not been fully explored in many domains. VR systems for real world engineering contexts are still difficult to use, especially when geographically dispersed engineering teams need to collaboratively visualize and review 3D CAD models. Another challenge is the ability to rendering these environments at the required interactive rates and with high fidelity. In this document it is presented a virtual reality system mobile for visualization, navigation and reviewing large scale 3D CAD models, held under the CEDAR (Collaborative Engineering Design and Review) project. It’s focused on interaction using different navigation modes. The system uses the mobile device's inertial sensors and camera to allow users to navigate through large scale models. IT professionals, architects, civil engineers and oil industry experts were involved in a qualitative assessment of the CEDAR system, in the form of direct user interaction with the prototypes and audio-recorded interviews about the prototypes. The lessons learned are valuable and are presented on this document. Subsequently it was prepared a quantitative study on the different navigation modes to analyze the best mode to use it in a given situation.

Gesture based IoT light control for smart clothing

In this paper, a smart wireless wristband is proposed. The potential of innovative gesture based interactivity with connected lighting solutions is reviewed. The solution is intended to offer numerous benefits, in terms of ease of use, and enhanced dynamic interactive functionality. A comparative analysis will be carried out between this work and existing solutions. The evolution of lighting and gesture controls will be discussed and an overview of alternative applications will be provided, as part of the critical analysis.

Interface caligráfica de escrita no ar

Na atualidade, está a emergir um novo paradigma de interação, designado por Natural User Interface (NUI) para reconhecimento de gestos produzidos com o corpo do utilizador. O dispositivo de interação Microsoft Kinect foi inicialmente concebido para controlo de videojogos, para a consola Xbox360. Este dispositivo demonstra ser uma aposta viável para explorar outras áreas, como a do apoio ao processo de ensino e de aprendizagem para crianças do ensino básico. O protótipo desenvolvido visa definir um modo de interação baseado no desenho de letras no ar, e realizar a interpretação dos símbolos desenhados, usando os reconhecedores de padrões Kernel Discriminant Analysis (KDA), Support Vector Machines (SVM) e $N. O desenvolvimento deste projeto baseou-se no estudo dos diferentes dispositivos NUI disponíveis no mercado, bibliotecas de desenvolvimento NUI para este tipo de dispositivos e algoritmos de reconhecimento de padrões. Com base nos dois elementos iniciais, foi possível obter uma visão mais concreta de qual o hardware e software disponíveis indicados à persecução do objetivo pretendido. O reconhecimento de padrões constitui um tema bastante extenso e complexo, de modo que foi necessária a seleção de um conjunto limitado deste tipo de algoritmos, realizando os respetivos testes por forma a determinar qual o que melhor se adequava ao objetivo pretendido. Aplicando as mesmas condições aos três algoritmos de reconhecimento de padrões permitiu avaliar as suas capacidades e determinar o $N como o que apresentou maior eficácia no reconhecimento. Por último, tentou-se averiguar a viabilidade do protótipo desenvolvido, tendo sido testado num universo de elementos de duas faixas etárias para determinar a capacidade de adaptação e aprendizagem destes dois grupos. Neste estudo, constatou-se um melhor desempenho inicial ao modo de interação do grupo de idade mais avançada. Contudo, o grupo mais jovem foi revelando uma evolutiva capacidade de adaptação a este modo de interação melhorando progressivamente os resultados.