Physiological modelling for improved reliability in silhouette-driven gradient-based hand tracking

We present a gradient-based motion capture system that robustly tracks a human hand, based on abstracted visual information - silhouettes. Despite the ambiguity in the visual data and despite the vulnerability of gradient-based methods in the face of such ambiguity, we minimise problems related to misfit by using a model of the hand's physiology, which is entirely non-visual, subject-invariant, and assumed to be known a priori. By modelling seven distinct aspects of the hand's physiology we derive prior densities which are incorporated into the tracking system within a Bayesian framework. We demonstrate how the posterior is formed, and how our formulation leads to the extraction of the maximum a posteriori estimate using a gradient-based search. Our results demonstrate an enormous improvement in tracking precision and reliability, while also achieving near real-time performance. © 2009 IEEE.

Adaptive control of stiffness to stabilize hand position with large loads.

The goal of this work was to investigate stability in relation to the magnitude and direction of forces applied by the hand. The endpoint stiffness and joint stiffness of the arm were measured during a postural task in which subjects exerted up to 30% maximum voluntary force in each of four directions while controlling the position of the hand. All four coefficients of the joint stiffness matrix were found to vary linearly with both elbow and shoulder torque. This contrasts with the results of a previous study, which employed a force control task and concluded that the joint stiffness coefficients varied linearly with either shoulder or elbow torque but not both. Joint stiffness was transformed into endpoint stiffness to compare the effect on stability as endpoint force increased. When the joint stiffness coefficients were modeled as varying with the net torque at only one joint, as in the previous study, we found that hand position became unstable if endpoint force exceeded about 22 N in a specific direction. This did not occur when the joint stiffness coefficients were modeled as varying with the net torque at both joints, as in the present study. Rather, hand position became increasingly more stable as endpoint force increased for all directions of applied force. Our analysis suggests that co-contraction of biarticular muscles was primarily responsible for the increased stability. This clearly demonstrates how the central nervous system can selectively adapt the impedance of the arm in a specific direction to stabilize hand position when the force applied by the hand has a destabilizing effect in that direction.

Exploring the use of sensors to measure behavioral interactions: an experimental evaluation of using hand trajectories.

Humans appear to be sensitive to relative small changes in their surroundings. These changes are often initially perceived as irrelevant, but they can cause significant changes in behavior. However, how exactly people's behavior changes is often hard to quantify. A reliable and valid tool is needed in order to address such a question, ideally measuring an important point of interaction, such as the hand. Wearable-body-sensor systems can be used to obtain valuable, behavioral information. These systems are particularly useful for assessing functional interactions that occur between the endpoints of the upper limbs and our surroundings. A new method is explored that consists of computing hand position using a wearable sensor system and validating it against a gold standard reference measurement (optical tracking device). Initial outcomes related well to the gold standard measurements (r = 0.81) showing an acceptable average root mean square error of 0.09 meters. Subsequently, the use of this approach was further investigated by measuring differences in motor behavior, in response to a changing environment. Three subjects were asked to perform a water pouring task with three slightly different containers. Wavelet analysis was introduced to assess how motor consistency was affected by these small environmental changes. Results showed that the behavioral motor adjustments to a variable environment could be assessed by applying wavelet coherence techniques. Applying these procedures in everyday life, combined with correct research methodologies, can assist in quantifying how environmental changes can cause alterations in our motor behavior.

structuring and manipulating hand-drawn concept maps

Concept maps are an important tool to knowledge organization,representation, and sharing. Most current concept map tools do not provide full support for hand-drawn concept map creation and manipulation, largely due to the lack of methods to recognize hand-drawn concept maps. This paper proposes a structure recognition method. Our algorithm can extract node blocks and link blocks of a hand-drawn concept map by combining dynamic programming and graph partitioning and then build a concept-map structure by relating extracted nodes and links. We also introduce structure-based intelligent manipulation technique of hand-drawn concept maps. Evaluation shows that our method has high structure recognition accuracy in real time, and the intelligent manipulation technique is efficient and effective.

The Grasping Problem: Toward Task-Level Programming for an Articulated Hand

This report presents a system for generating a stable, feasible, and reachable grasp of a polyhedral object. A set of contact points on the object is found that can result in a stable grasp; a feasible grasp is found in which the robot contacts the object at those contact points; and a path is constructed from the initial configuration of the robot to the stable, feasible final grasp configuration. The algorithm described in the report is designed for the Salisbury hand mounted on a Puma 560 arm, but a similar approach could be used to develop grasping systems for other robots.

Automated cross-modal mapping in robotic eye/hand systems using plastic radial basis function networks

Q. Meng and M. H Lee, Automated cross-modal mapping in robotic eye/hand systems using plastic radial basis function networks, Connection Science, 19(1), pp 25-52, 2007.

Biologically inspired automatic construction of cross-modal mapping in robotic eye/hand systems

Q. Meng and M.H. Lee, 'Biologically inspired automatic construction of cross-modal mapping in robotic eye/hand systems', IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2006,) ,4742-49, Beijing, 2006.

Estimating 3D Hand Pose from a Cluttered Image

A method is proposed that can generate a ranked list of plausible three-dimensional hand configurations that best match an input image. Hand pose estimation is formulated as an image database indexing problem, where the closest matches for an input hand image are retrieved from a large database of synthetic hand images. In contrast to previous approaches, the system can function in the presence of clutter, thanks to two novel clutter-tolerant indexing methods. First, a computationally efficient approximation of the image-to-model chamfer distance is obtained by embedding binary edge images into a high-dimensional Euclide an space. Second, a general-purpose, probabilistic line matching method identifies those line segment correspondences between model and input images that are the least likely to have occurred by chance. The performance of this clutter-tolerant approach is demonstrated in quantitative experiments with hundreds of real hand images.

Database Indexing Methods for 3D Hand Pose Estimation

Estimation of 3D hand pose is useful in many gesture recognition applications, ranging from human-computer interaction to automated recognition of sign languages. In this paper, 3D hand pose estimation is treated as a database indexing problem. Given an input image of a hand, the most similar images in a large database of hand images are retrieved. The hand pose parameters of the retrieved images are used as estimates for the hand pose in the input image. Lipschitz embeddings of edge images into a Euclidean space are used to improve the efficiency of database retrieval. In order to achieve interactive retrieval times, similarity queries are initially performed in this Euclidean space. The paper describes ongoing work that focuses on how to best choose reference images, in order to improve retrieval accuracy.

Simultaneous Localization and Recognition of Dynamic Hand Gestures

A framework for the simultaneous localization and recognition of dynamic hand gestures is proposed. At the core of this framework is a dynamic space-time warping (DSTW) algorithm, that aligns a pair of query and model gestures in both space and time. For every frame of the query sequence, feature detectors generate multiple hand region candidates. Dynamic programming is then used to compute both a global matching cost, which is used to recognize the query gesture, and a warping path, which aligns the query and model sequences in time, and also finds the best hand candidate region in every query frame. The proposed framework includes translation invariant recognition of gestures, a desirable property for many HCI systems. The performance of the approach is evaluated on a dataset of hand signed digits gestured by people wearing short sleeve shirts, in front of a background containing other non-hand skin-colored objects. The algorithm simultaneously localizes the gesturing hand and recognizes the hand-signed digit. Although DSTW is illustrated in a gesture recognition setting, the proposed algorithm is a general method for matching time series, that allows for multiple candidate feature vectors to be extracted at each time step.