Efficient and precise interactive hand tracking through joint, continuous optimization of pose and correspondences

Fully articulated hand tracking promises to enable fundamentally new interactions with virtual and augmented worlds, but the limited accuracy and efficiency of current systems has prevented widespread adoption. Today's dominant paradigm uses machine learning for initialization and recovery followed by iterative model-fitting optimization to achieve a detailed pose fit. We follow this paradigm, but make several changes to the model-fitting, namely using: (1) a more discriminative objective function; (2) a smooth-surface model that provides gradients for non-linear optimization; and (3) joint optimization over both the model pose and the correspondences between observed data points and the model surface. While each of these changes may actually increase the cost per fitting iteration, we find a compensating decrease in the number of iterations. Further, the wide basin of convergence means that fewer starting points are needed for successful model fitting. Our system runs in real-time on CPU only, which frees up the commonly over-burdened GPU for experience designers. The hand tracker is efficient enough to run on low-power devices such as tablets. We can track up to several meters from the camera to provide a large working volume for interaction, even using the noisy data from current-generation depth cameras. Quantitative assessments on standard datasets show that the new approach exceeds the state of the art in accuracy. Qualitative results take the form of live recordings of a range of interactive experiences enabled by this new approach.

‘‘Model age-based” and ‘‘copy when uncertain” biases in children’s social learning of a novel task

Theoretical models of social learning predict that individuals can benefit from using strategies that specify when and whom to copy. Here the interaction of two social learning strategies, model age-based biased copying and copy when uncertain, was investigated. Uncertainty was created via a systematic manipulation of demonstration efficacy (completeness) and efficiency (causal relevance of some actions). The participants, 4- to 6-year-old children (N = 140), viewed both an adult model and a child model, each of whom used a different tool on a novel task. They did so in a complete condition, a near-complete condition, a partial demonstration condition, or a no-demonstration condition. Half of the demonstrations in each condition incorporated causally irrelevant actions by the models. Social transmission was assessed by first responses but also through children’s continued fidelity, the hallmark of social traditions. Results revealed a bias to copy the child model both on first response and in continued interactions. Demonstration efficacy and efficiency did not affect choice of model at first response but did influence solution exploration across trials, with demonstrations containing causally irrelevant actions decreasing exploration of alternative methods. These results imply that uncertain environments can result in canalized social learning from specific classes of mode