Naturalistic approaches to sensorimotor control.

Human sensorimotor control has been predominantly studied using fixed tasks performed under laboratory conditions. This approach has greatly advanced our understanding of the mechanisms that integrate sensory information and generate motor commands during voluntary movement. However, experimental tasks necessarily restrict the range of behaviors that are studied. Moreover, the processes studied in the laboratory may not be the same processes that subjects call upon during their everyday lives. Naturalistic approaches thus provide an important adjunct to traditional laboratory-based studies. For example, wearable self-contained tracking systems can allow subjects to be monitored outside the laboratory, where they engage spontaneously in natural everyday behavior. Similarly, advances in virtual reality technology allow laboratory-based tasks to be made more naturalistic. Here, we review naturalistic approaches, including perspectives from psychology and visual neuroscience, as well as studies and technological advances in the field of sensorimotor control.

Near optimal combination of sensory and motor uncertainty in time during a naturalistic perception-action task.

Most behavioral tasks have time constraints for successful completion, such as catching a ball in flight. Many of these tasks require trading off the time allocated to perception and action, especially when only one of the two is possible at any time. In general, the longer we perceive, the smaller the uncertainty in perceptual estimates. However, a longer perception phase leaves less time for action, which results in less precise movements. Here we examine subjects catching a virtual ball. Critically, as soon as subjects began to move, the ball became invisible. We study how subjects trade-off sensory and movement uncertainty by deciding when to initiate their actions. We formulate this task in a probabilistic framework and show that subjects' decisions when to start moving are statistically near optimal given their individual sensory and motor uncertainties. Moreover, we accurately predict individual subject's task performance. Thus we show that subjects in a natural task are quantitatively aware of how sensory and motor variability depend on time and act so as to minimize overall task variability.

Cognitive Tomography Reveals Complex, Task-Independent Mental Representations

Humans develop rich mental representations that guide their behavior in a variety of everyday tasks. However, it is unknown whether these representations, often formalized as priors in Bayesian inference, are specific for each task or subserve multiple tasks. Current approaches cannot distinguish between these two possibilities because they cannot extract comparable representations across different tasks [1-10]. Here, we develop a novel method, termed cognitive tomography, that can extract complex, multidimensional priors across tasks. We apply this method to human judgments in two qualitatively different tasks, "familiarity" and "odd one out," involving an ecologically relevant set of stimuli, human faces. We show that priors over faces are structurally complex and vary dramatically across subjects, but are invariant across the tasks within each subject. The priors we extract from each task allow us to predict with high precision the behavior of subjects for novel stimuli both in the same task as well as in the other task. Our results provide the first evidence for a single high-dimensional structured representation of a naturalistic stimulus set that guides behavior in multiple tasks. Moreover, the representations estimated by cognitive tomography can provide independent, behavior-based regressors for elucidating the neural correlates of complex naturalistic priors. © 2013 The Authors.