Keeping the world at hand: rapid visuomotor processing for hand-object interactions

The existence of hand-centred visual processing has long been established in the macaque premotor cortex. These hand-centred mechanisms have been thought to play some general role in the sensory guidance of movements towards objects, or, more recently, in the sensory guidance of object avoidance movements. We suggest that these hand-centred mechanisms play a specific and prominent role in the rapid selection and control of manual actions following sudden changes in the properties of the objects relevant for hand-object interactions. We discuss recent anatomical and physiological evidence from human and non-human primates, which indicates the existence of rapid processing of visual information for hand-object interactions. This new evidence demonstrates how several stages of the hierarchical visual processing system may be bypassed, feeding the motor system with hand-related visual inputs within just 70 ms following a sudden event. This time window is early enough, and this processing rapid enough, to allow the generation and control of rapid hand-centred avoidance and acquisitive actions, for aversive and desired objects, respectively

Visual Object Recognition Using Deformable Models of Vehicles

This workshop paper reports recent developments to a vision system for traffic interpretation which relies extensively on the use of geometrical and scene context. Firstly, a new approach to pose refinement is reported, based on forces derived from prominent image derivatives found close to an initial hypothesis. Secondly, a parameterised vehicle model is reported, able to represent different vehicle classes. This general vehicle model has been fitted to sample data, and subjected to a Principal Component Analysis to create a deformable model of common car types having 6 parameters. We show that the new pose recovery technique is also able to operate on the PCA model, to allow the structure of an initial vehicle hypothesis to be adapted to fit the prevailing context. We report initial experiments with the model, which demonstrate significant improvements to pose recovery.

Biologically motivated model for object detection and identification in real-world scenes

The classical computer vision methods can only weakly emulate some of the multi-level parallelisms in signal processing and information sharing that takes place in different parts of the primates’ visual system thus enabling it to accomplish many diverse functions of visual perception. One of the main functions of the primates’ vision is to detect and recognise objects in natural scenes despite all the linear and non-linear variations of the objects and their environment. The superior performance of the primates’ visual system compared to what machine vision systems have been able to achieve to date, motivates scientists and researchers to further explore this area in pursuit of more efficient vision systems inspired by natural models. In this paper building blocks for a hierarchical efficient object recognition model are proposed. Incorporating the attention-based processing would lead to a system that will process the visual data in a non-linear way focusing only on the regions of interest and hence reducing the time to achieve real-time performance. Further, it is suggested to modify the visual cortex model for recognizing objects by adding non-linearities in the ventral path consistent with earlier discoveries as reported by researchers in the neuro-physiology of vision.

Neural mechanisms of visual object priming: evidence for perceptual and semantic distinctions in fusiform cortex

Previous functional imaging studies have shown that facilitated processing of a visual object on repeated, relative to initial, presentation (i.e., repetition priming) is associated with reductions in neural activity in multiple regions, including fusiforin/lateral occipital cortex. Moreover, activity reductions have been found, at diminished levels, when a different exemplar of an object is presented on repetition. In one previous study, the magnitude of diminished priming across exemplars was greater in the right relative to the left fusiform, suggesting greater exemplar specificity in the right. Another previous study, however, observed fusiform lateralization modulated by object viewpoint, but not object exemplar. The present fMRI study sought to determine whether the result of differential fusiform responses for perceptually different exemplars could be replicated. Furthermore, the role of the left fusiform cortex in object recognition was investigated via the inclusion of a lexical/semantic manipulation. Right fusiform cortex showed a significantly greater effect of exemplar change than left fusiform, replicating the previous result of exemplar-specific fusiform lateralization. Right fusiform and lateral occipital cortex were not differentially engaged by the lexical/semantic manipulation, suggesting that their role in visual object recognition is predominantly in the. C visual discrimination of specific objects. Activation in left fusiform cortex, but not left lateral occipital cortex, was modulated by both exemplar change and lexical/semantic manipulation, with further analysis suggesting a posterior-to-anterior progression between regions involved in processing visuoperceptual and lexical/semantic information about objects. The results are consistent with the view that the right fusiform plays a greater role in processing specific visual form information about objects, whereas the left fusiform is also involved in lexical/semantic processing. (C) 2003 Elsevier Science (USA). All rights reserved.

Neural processing of imminent collision in humans

Detecting a looming object and its imminent collision is imperative to survival. For most humans, it is a fundamental aspect of daily activities such as driving, road crossing and participating in sport, yet little is known about how the brain both detects and responds to such stimuli. Here we use functional magnetic resonance imaging to assess neural response to looming stimuli in comparison with receding stimuli and motion-controlled static stimuli. We demonstrate for the first time that, in the human, the superior colliculus and the pulvinar nucleus of the thalamus respond to looming in addition to cortical regions associated with motor preparation. We also implicate the anterior insula in making timing computations for collision events.

Peripersonal space: a multisensory interface for body-object interactions

Research in the last four decades has brought a considerable advance in our understanding of how the brain synthesizes information arising from different sensory modalities. Indeed, many cortical and subcortical areas, beyond those traditionally considered to be ‘associative,’ have been shown to be involved in multisensory interaction and integration (Ghazanfar and Schroeder 2006). Visuo-tactile interaction is of particular interest, because of the prominent role played by vision in guiding our actions and anticipating their tactile consequences in everyday life. In this chapter, we focus on the functional role that visuo-tactile processing may play in driving two types of body-object interactions: avoidance and approach. We will first review some basic features of visuo-tactile interactions, as revealed by electrophysiological studies in monkeys. These will prove to be relevant for interpreting the subsequent evidence arising from human studies. A crucial point that will be stressed is that these visuo-tactile mechanisms have not only sensory, but also motor-related activity that qualifies them as multisensory-motor interfaces. Evidence will then be presented for the existence of functionally homologous processing in the human brain, both from neuropsychological research in brain-damaged patients and in healthy participants. The final part of the chapter will focus on some recent studies in humans showing that the human motor system is provided with a multisensory interface that allows for continuous monitoring of the space near the body (i.e., peripersonal space). We further demonstrate that multisensory processing can be modulated on-line as a consequence of interacting with objects. This indicates that, far from being passive, the monitoring of peripersonal space is an active process subserving actions between our body and objects located in the space around us.

Pronoun processing in Broca's aphasia: discourse-syntax effects in ambiguous anaphora resolution

Background: The interpretation of ambiguous subject pronouns in a null subject language, like Greek, requires that one possesses grammatical knowledge of the two subject pronominal forms, i.e., null and overt, and that discourse constraints regulating the distribution of the two pronouns in context are respected. Aims: We investigated whether the topic-shift feature encoded in overt subject pronouns would exert similar interpretive effects in a group of seven participants with Broca’s aphasia and a group of language-unimpaired adults during online processing of null and overt subject pronouns in referentially ambiguous contexts. Method & Procedures: An offline picture–sentence matching task was initially administered to investigate whether the participants with Broca’s aphasia had access to the gender and number features of clitic pronouns. An online self-paced listening picture-verification task was subsequently administered to examine how the aphasic individuals resolve pronoun ambiguities in contexts with either null or overt subject pronouns and how their performance compares to that of language-unimpaired adults. Outcomes & Results: Results demonstrate that the Broca group, along with controls, had intact access to the morphosyntactic features of clitic pronouns. However, the aphasic individuals showed decreased preference for non-salient antecedents in object position during the online resolution of ambiguous overt subject pronouns and preferred to pick the subject antecedent instead. Conclusions: Broca’s aphasic participants’ parsing decisions in the online task reflect their difficulty with establishing topic-shifted interpretations of the ambiguous overt subject pronouns. The presence of a local topic-shift effect in the immediate temporal vicinity of the overt pronoun suggests that sensitivity to the marked informational status of overt pronouns is preserved in the aphasic individuals, yet, it is blocked under conditions of global sentential processing.

The impact of number mismatch and passives on the real-time processing of relative clauses

Language processing plays a crucial role in language development, providing the ability to assign structural representations to input strings (e.g., Fodor, 1998). In this paper we aim at contributing to the study of children's processing routines, examining the operations underlying the auditory processing of relative clauses in children compared to adults. English-speaking children (6–8;11) and adults participated in the study, which employed a self-paced listening task with a final comprehension question. The aim was to determine (i) the role of number agreement in object relative clauses in which the subject and object NPs differ in terms of number properties, and (ii) the role of verb morphology (active vs. passive) in subject relative clauses. Even though children's off-line accuracy was not always comparable to that of adults, analyses of reaction times results support the view that children have the same structural processing reflexes observed in adults.

Children’s and Adults’ Processing of Anomaly and Implausibility during Reading: Evidence from Eye Movements

The eye movements of 24 children and 24 adults were monitored to compare how they read sentences containing plausible, implausible, and anomalous thematic relations. In the implausible condition the incongruity occurred due to the incompatibility of two objects involved in the event denoted by the main verb. In the anomalous condition the direct object of the verb was not a possible verb argument. Adults exhibited immediate disruption with the anomalous sentences as compared to the implausible sentences as indexed by longer gaze durations on the target word. Children exhibited the same pattern of effects as adults as far as the anomalous sentences were concerned, but exhibited delayed effects of implausibility. These data indicate that while children and adults are alike in their basic thematic assignment processes during reading, children may be delayed in the efficiency with which they are able to integrate pragmatic and real world knowledge into their discourse representation.

Incremental processing in head-final child language: online comprehension of relative clauses in Turkish-speaking children and adults

The present study investigates the parsing of pre-nominal relative clauses (RCs) in children for the first time with a realtime methodology that reveals moment-to-moment processing patterns as the sentence unfolds. A self-paced listening experiment with Turkish-speaking children (aged 5–8) and adults showed that both groups display a sign of processing cost both in subject and object RCs at different points through the flow of the utterance when integrating the cues that are uninformative (i.e., ambiguous in function) and that are structurally and probabilistically unexpected. Both groups show a processing facilitation as soon as the morphosyntactic dependencies are completed and parse the unbounded dependencies rapidly using the morphosyntactic cues rather than waiting for the clause-final filler. These findings show that five-year-old children show similar patterns to adults in processing the morphosyntactic cues incrementally and in forming expectations about the rest of the utterance on the basis of the probabilistic model of their language.

Pose Refinement of Active Models using Forces in 3D

A new algorithm is described for refining the pose of a model of a rigid object, to conform more accurately to the image structure. Elemental 3D forces are considered to act on the model. These are derived from directional derivatives of the image local to the projected model features. The convergence properties of the algorithm is investigated and compared to a previous technique. Its use in a video sequence of a cluttered outdoor traffic scene is also illustrated and assessed.

Pose and Structure Recovery using Active Models

A new formulation of a pose refinement technique using ``active'' models is described. An error term derived from the detection of image derivatives close to an initial object hypothesis is linearised and solved by least squares. The method is particularly well suited to problems involving external geometrical constraints (such as the ground-plane constraint). We show that the method is able to recover both the pose of a rigid model, and the structure of a deformable model. We report an initial assessment of the performance and cost of pose and structure recovery using the active model in comparison with our previously reported ``passive'' model-based techniques in the context of traffic surveillance. The new method is more stable, and requires fewer iterations, especially when the number of free parameters increases, but shows somewhat poorer convergence.

A Newton method for pose refinement of 3D models

Different optimization methods can be employed to optimize a numerical estimate for the match between an instantiated object model and an image. In order to take advantage of gradient-based optimization methods, perspective inversion must be used in this context. We show that convergence can be very fast by extrapolating to maximum goodness-of-fit with Newton's method. This approach is related to methods which either maximize a similar goodness-of-fit measure without use of gradient information, or else minimize distances between projected model lines and image features. Newton's method combines the accuracy of the former approach with the speed of convergence of the latter.

Use of fused airborne scanning laser altimetry and digital map data for urban flood modelling

Flood modelling of urban areas is still at an early stage, partly because until recently topographic data of sufficiently high resolution and accuracy have been lacking in urban areas. However, Digital Surface Models (DSMs) generated from airborne scanning laser altimetry (LiDAR) having sub-metre spatial resolution have now become available, and these are able to represent the complexities of urban topography. The paper describes the development of a LiDAR post-processor for urban flood modelling based on the fusion of LiDAR and digital map data. The map data are used in conjunction with LiDAR data to identify different object types in urban areas, though pattern recognition techniques are also employed. Post-processing produces a Digital Terrain Model (DTM) for use as model bathymetry, and also a friction parameter map for use in estimating spatially-distributed friction coefficients. In vegetated areas, friction is estimated from LiDAR-derived vegetation height, and (unlike most vegetation removal software) the method copes with short vegetation less than ~1m high, which may occupy a substantial fraction of even an urban floodplain. The DTM and friction parameter map may also be used to help to generate an unstructured mesh of a vegetated urban floodplain for use by a 2D finite element model. The mesh is decomposed to reflect floodplain features having different frictional properties to their surroundings, including urban features such as buildings and roads as well as taller vegetation features such as trees and hedges. This allows a more accurate estimation of local friction. The method produces a substantial node density due to the small dimensions of many urban features.