Kinect crowd interaction

Most of the state-of-the-art commercial simulation software mainly focuses on providing realistic animations and convincing artificial intelligence to avatars in the scenario. However, works on how to trigger the events and avatar reactions in the scenario in a natural and intuitive way are less noticed and developed. Typical events are usually triggered by predefined timestamps. Once the events are set, there is no easy way to interactively generate new events while the scene is running and therefore difficult to dynamically affect the avatar reactions. Based on this situation, we propose a framework to use human gesture as input to trigger events within a DI-Guy simulation scenario in real-time, which could greatly help users to control events and avatar reactions in the scenario. By implementing such a framework, we will be able to identify user’s intentions interactively and ensure that the avatars make corresponding reactions.

Integrating kinect and haptics for interactive STEM education in local and distributed environments

Skill shortage is a realistic social problem that Australia is currently facing, especially in the fields of Science, Technology, Engineering and Mathematics (STEM). Various approaches have been proposed to soften this issue. By now the most successful approach is to attract pre-university youth and university freshmen into those fields before they make a decision on future subjects by introducing them with interactive, modifiable and inspiring virtual environments, which incorporates most essential knowledge of STEM. We propose to design a comprehensive virtual reality platform with immersive interactions, pluggable components and flexible configurations. It also involves haptics, motion capture and gesture recognition, and could be deployed in both local and distributed environments. The platform utilizes off the shelf low cost haptics and motion capture products, however the fidelity can be maintained at a good level. The proposed platform has been implemented with different configurations and has been tested on a group of users. Preliminary test results show that the interactivity, flexibility and fidelity of the platform are highly appreciated by users. User surveys also indicate that the proposed platform could help pre-university students and university freshmen build an overview of various aspects of STEM education. Besides, users are also positive on the fact that the platform enabled them to identify the challenges for higher education in STEM by providing them opportunities to interactively modify system configurations and instantly experience the corresponding results both visually and haptically.

Multi-Kinect skeleton fusion for physical rehabilitation monitoring

 Kinect has been increasingly applied in rehabilitation as a motion capture device. However, the inherent limitations significantly hinder its further development in this important area. Although a number of Kinect fusion approaches have been proposed, only a few of them was actually considered for rehabilitation. In this paper, we propose to fuse information from multiple Kinects to achieve this. Given the specific scenario of users suffering from limited range of movements, we propose to calibrate depth cameras in multiple Kinects with 3D positions of joints on a human body rather than in a checkerboard pattern, so that patients are able to calibrate Kinects without extra support. Kalman filter is applied for skeleton-wise Kinect fusion since skeleton data (3D positions of joints) and its derivatives are preferred by physiotherapists to evaluate the exercise performance of patients. Various preliminary experiments were conducted to illustrate the accuracy of proposed calibration and fusion approach by comparing with a commercial Vicon system®, confirming the practical use of the system in rehabilitation exercise monitoring.

A general pose estimation algorithm in a multi-Kinect system

 Microsoft Kinect which has been primarily aimed at the computer gaming industry has been used in bio-kinematic research related implementations. A multi-Kinect system can be useful in exploiting spatial diversity to increase measurement accuracy. One of the main problems in deploying multi-Kinect systems is to estimate the pose, including the position and orientation of each Kinect. In this paper, a singular value decomposition (SVD) least-squares algorithm is extended to a more generic time-series based approach to solve this pose estimation problem utilising 3D positions of one or more joints in skeletons obtained from a multi-Kinect system. Additionally, computer simulations are performed to demonstrate the use and to evaluate the efficiency of the proposed algorithm. The former is further validated with a commercial Vicon system.

Safety applications using Kinect technology

 Microsoft Kinect sensor was introduced with the XBOX gaming console. It features a simple and portable motion capturing system. Kinect nowadays presents a point of interest in many fields of study and areas of research where its affordable price compared to its capabilities. The Kinect sensor has the capability to capture and track detected 3D objects with accuracy comparable to that captured by state of the art commercial systems. Human safety is considered one of the highest concerns, specially nowadays where the existence of machines and robots is widely used. In this paper we present using the Kinect technology for enhancing the safety of equipment and operations in seven different applications. These applications include 1) positioning of child’s car seat to optimise the child’s position in respected to front and side air-bags; 2) board positioning system to improve the teacher’s arm reach posture; 3) gas station safety to prevent children from accessing the gas pump; 4) indoor pool safety to avoid children access to deep pool area; 5) robot safety emergency stop; 6) Workplace safety; and 7) older adults fall prediction.

Navegação, reconhecimento de gestos e controle de interface no sistema ITV utilizando o dispositivo kinect

Esta dissertação apresenta a implementação de navegação no ambiente virtual, reconhecimento de gestos e controle de interface, feitos através do dispositivo Kinect, no Sistema ITV: um sistema de treinamento de operadores e mantenedores de usinas hidrelétricas e subestações elétricas. São mostrados, também, determinados aperfeiçoamentos recentes, como conversão em vídeo, telas de alarmes sonoros e visuais, ambientação sonora em três dimensões e narração do processo. Além da apresentação do Sistema ITV, são expostos o dispositivo Kinect e o algoritmo utilizado na comparação dos padrões de movimento, o DTW. Em seguida, são abordados em detalhes o projeto e a implementação da navegação, do reconhecimento de gestos e do controle de interface. Como estudo de caso, é exibida uma Instrução Técnica Virtual (ITV), elaborada especialmente para testar e avaliar a nova interface proposta. Posteriormente, são apresentados os resultados, considerados satisfatórios, obtidos através da análise de questionários qualitativos aplicados a estudantes da Universidade Federal do Pará. Por fim, são realizadas as considerações referentes a este trabalho e expostas idéias de trabalhos futuros.

Avaliação automatizada de uma habilidade motora fundamental com o kinect

Pós-graduação em Desenvolvimento Humano e Tecnologias - IBRC

Reconhecimento de faces 3D com Kinect

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Avaliação de desenpenho do método 3DLBP utilizando mapas de profundidade gerados pelo Microsoft Kinect

Problemas com pose e iluminação são desafios complexos para o reconhecimento de faces 2D. Devido a estes problemas vários métodos para reconhecimento de faces 3D têm sido propostos, principalmente pelo fato que os dados em 3D são mais resistentes a iluminação e são úteis para correção de pose. O maior problema com a utilização de métodos 3D é o custo elevado dos scanners 3D tradicionais. Uma alternativa é a utilização do Microsoft Kinect que, além de ser consideravelmente mais barato, é capaz de capturar os dados de profundidade com precisão necessária para discriminar sujeitos. O principal objetivo do presente trabalho é avaliar a performance do método 3DLBP para o reconhecimento de faces quando utilizando os mapas de profundidade gerados pelo Kinect. Outro objetivo é investigar quais regiões da face que desempenham melhor no reconhecimento de face.

Transformaciones basadas en el algoritmo Local Binary Pattern de imágenes capturadas con la Kinect para clasificación facial

Máster Universitario en Sistemas Inteligentes y Aplicaciones Numéricas en Ingeniería (SIANI)

Solución de bajo coste de captura de movimiento basada en Kinect

[ES] En este trabajo se estudia e implementa una solución de bajo coste basada en Kinect para la captura de movimiento y su integración con Blender, de manera que se cuente con una solución eficiente para la realización de captura de movimiento en animación. Dicho desarrollo está apoyado con la elaboración de un tutorial sobre el procedimiento necesario para realizar MOCAP con Kinect y Blender, facilitando así su uso con fines formativos.

On the use of a low-cost thermal sensor to improve kinect people detection in a mobile robot

[EN]Detecting people is a key capability for robots that operate in populated environments. In this paper, we have adopted a hierarchical approach that combines classifiers created using supervised learning in order to identify whether a person is in the view-scope of the robot or not. Our approach makes use of vision, depth and thermal sensors mounted on top of a mobile platform.