Low cost multimodal facial recognition via kinect sensors

This paper presents a novel low cost 3D facial recognition using gaming sensors such as Kinect™. The paper describes the hardware, calibration and infrared noise and pattern interference challenges of integrating multiple Kinect sensors. The preliminary results show a promising trend for low cost solutions that can be populated in crowded facilities such as malls and airports.

Extracting 3D mesh skeletons using antipodal points locations

Finding the skeleton of a 3D mesh is an essential task for many applications such as mesh animation, tracking, and 3D registeration. In recent years, new technologies in computer vision such as Microsoft Kinect have proven that a mesh skeleton can be useful such as in the case of human machine interactions. To calculate the 3D mesh skeleton, the mesh properties such as topology and its components relations are utilized. In this paper, we propose the usage of a novel algorithm that can efficiently calculate a vertex antipodal point. A vertex antipodal point is the diametrically opposite point that belongs to the same mesh. The set of centers of the connecting lines between each vertex and its antipodal point represents the 3D mesh desired skeleton. Post processing is completed for smoothing and fitting centers into optimized skeleton parts. The algorithm is tested on different classes of 3D objects and produced efficient results that are comparable with the literature. The algorithm has the advantages of producing high quality skeletons as it preserves details. This is suitable for applications where the mesh skeleton mapping is required to be kept as much as possible.

Multi-Kinect skeleton fusion for physical rehabilitation monitoring

 Kinect has been increasingly applied in rehabilitation as a motion capture device. However, the inherent limitations significantly hinder its further development in this important area. Although a number of Kinect fusion approaches have been proposed, only a few of them was actually considered for rehabilitation. In this paper, we propose to fuse information from multiple Kinects to achieve this. Given the specific scenario of users suffering from limited range of movements, we propose to calibrate depth cameras in multiple Kinects with 3D positions of joints on a human body rather than in a checkerboard pattern, so that patients are able to calibrate Kinects without extra support. Kalman filter is applied for skeleton-wise Kinect fusion since skeleton data (3D positions of joints) and its derivatives are preferred by physiotherapists to evaluate the exercise performance of patients. Various preliminary experiments were conducted to illustrate the accuracy of proposed calibration and fusion approach by comparing with a commercial Vicon system®, confirming the practical use of the system in rehabilitation exercise monitoring.

A general pose estimation algorithm in a multi-Kinect system

 Microsoft Kinect which has been primarily aimed at the computer gaming industry has been used in bio-kinematic research related implementations. A multi-Kinect system can be useful in exploiting spatial diversity to increase measurement accuracy. One of the main problems in deploying multi-Kinect systems is to estimate the pose, including the position and orientation of each Kinect. In this paper, a singular value decomposition (SVD) least-squares algorithm is extended to a more generic time-series based approach to solve this pose estimation problem utilising 3D positions of one or more joints in skeletons obtained from a multi-Kinect system. Additionally, computer simulations are performed to demonstrate the use and to evaluate the efficiency of the proposed algorithm. The former is further validated with a commercial Vicon system.

Safety applications using Kinect technology

 Microsoft Kinect sensor was introduced with the XBOX gaming console. It features a simple and portable motion capturing system. Kinect nowadays presents a point of interest in many fields of study and areas of research where its affordable price compared to its capabilities. The Kinect sensor has the capability to capture and track detected 3D objects with accuracy comparable to that captured by state of the art commercial systems. Human safety is considered one of the highest concerns, specially nowadays where the existence of machines and robots is widely used. In this paper we present using the Kinect technology for enhancing the safety of equipment and operations in seven different applications. These applications include 1) positioning of child’s car seat to optimise the child’s position in respected to front and side air-bags; 2) board positioning system to improve the teacher’s arm reach posture; 3) gas station safety to prevent children from accessing the gas pump; 4) indoor pool safety to avoid children access to deep pool area; 5) robot safety emergency stop; 6) Workplace safety; and 7) older adults fall prediction.

3D motion matching algorithm using signature feature descriptor

This paper introduces a basic frame for rehabilitation motion practice system which detects 3D motion trajectory with the Microsoft Kinect (MSK) sensor system and proposes a cost-effective 3D motion matching algorithm. The rehabilitation motion practice system displays a reference 3D motion in the database system that the player (patient) tries to follow. The player’s motion is traced by the MSK sensor system and then compared with the reference motion to evaluate how well the player follows the reference motion. In this system, 3D motion matching algorithm is a key feature for accurate evaluation for player’s performance. Even though similarity measurement of 3D trajectories is one of the most important tasks in 3D motion analysis, existing methods are still limited. Recent researches focus on the full length 3D trajectory data set. However, it is not true that every point on the trajectory plays the same role and has the same meaning. In this situation, we developed a new cost-effective method that only uses the less number of features called ‘signature’ which is a flexible descriptor computed from the region of ‘elbow points’. Therefore, our proposed method runs faster than other methods which use the full length trajectory information. The similarity of trajectories is measured based on the signature using an alignment method such as dynamic time warping (DTW), continuous dynamic time warping (CDTW) or longest common sub-sequence (LCSS) method. In the experimental studies, we applied the MSK sensor system to detect, trace and match the 3D motion of human body. This application was assumed as a system for guiding a rehabilitation practice which can evaluate how well the motion practice was performed based on comparison of the patient’s practice motion traced by the MSK system with the pre-defined reference motion in a database. In order to evaluate the accuracy of our 3D motion matching algorithm, we compared our method with two other methods using Australian sign word dataset. As a result, our matching algorithm outperforms in matching 3D motion, and it can be exploited for a base framework for various 3D motion-based applications at low cost with high accuracy.

Kinect with ROS, interact with oculus: towards dynamic user interfaces for robotic teleoperation

Teleoperation remains an important aspect for robotic systems especially when deployed in unstructured environments. While a range of research strives for robots that are completely autonomous, many robotic applications still require some level of human-in-The-loop control. In any situation where teleoperation is required an effective User Interface (UI) remains a key component within the systems design. Current advancements in Virtual Reality (VR) software and hardware such as the Oculus Rift, HTC Vive and Google Cardboard combined with greater transparency to robotic systems afforded by middleware such as the Robot Operating System (ROS) provides an opportunity to rapidly improve traditional teleoperation interfaces. This paper uses a System of System (SoS) approach to present the concept of a Virtual Reality Dynamic User Interface (VRDUI) for the teleoperation of heterogeneous robots. Different geometric virtual workspaces are discussed and a cylindrical workspace aligned with interactive displays is presented as a virtual control room. A presentation mode within the proposed VRDUI is also detailed, this shows how point cloud information obtained from the Microsoft Kinect can be incorporated within the proposed virtual workspace. This point cloud data is successfully processed into an OctoMap utilizing the octree data structure to create a voxelized representation of the 3D scanned environment. The resulting OctoMap is then displayed to an operator as a 3D point cloud using the Oculus Rift Head Mounted Display (HMD).