Neural mechanisms of visual object priming: evidence for perceptual and semantic distinctions in fusiform cortex

Previous functional imaging studies have shown that facilitated processing of a visual object on repeated, relative to initial, presentation (i.e., repetition priming) is associated with reductions in neural activity in multiple regions, including fusiforin/lateral occipital cortex. Moreover, activity reductions have been found, at diminished levels, when a different exemplar of an object is presented on repetition. In one previous study, the magnitude of diminished priming across exemplars was greater in the right relative to the left fusiform, suggesting greater exemplar specificity in the right. Another previous study, however, observed fusiform lateralization modulated by object viewpoint, but not object exemplar. The present fMRI study sought to determine whether the result of differential fusiform responses for perceptually different exemplars could be replicated. Furthermore, the role of the left fusiform cortex in object recognition was investigated via the inclusion of a lexical/semantic manipulation. Right fusiform cortex showed a significantly greater effect of exemplar change than left fusiform, replicating the previous result of exemplar-specific fusiform lateralization. Right fusiform and lateral occipital cortex were not differentially engaged by the lexical/semantic manipulation, suggesting that their role in visual object recognition is predominantly in the. C visual discrimination of specific objects. Activation in left fusiform cortex, but not left lateral occipital cortex, was modulated by both exemplar change and lexical/semantic manipulation, with further analysis suggesting a posterior-to-anterior progression between regions involved in processing visuoperceptual and lexical/semantic information about objects. The results are consistent with the view that the right fusiform plays a greater role in processing specific visual form information about objects, whereas the left fusiform is also involved in lexical/semantic processing. (C) 2003 Elsevier Science (USA). All rights reserved.

Considering the Feasibility of Semantic Model Design in the Built-Environment

Building Information Modeling (BIM) is the process of structuring, capturing, creating, and managing a digital representation of physical and/or functional characteristics of a built space [1]. Current BIM has limited ability to represent dynamic semantics, social information, often failing to consider building activity, behavior and context; thus limiting integration with intelligent, built-environment management systems. Research, such as the development of Semantic Exchange Modules, and/or the linking of IFC with semantic web structures, demonstrates the need for building models to better support complex semantic functionality. To implement model semantics effectively, however, it is critical that model designers consider semantic information constructs. This paper discusses semantic models with relation to determining the most suitable information structure. We demonstrate how semantic rigidity can lead to significant long-term problems that can contribute to model failure. A sufficiently detailed feasibility study is advised to maximize the value from the semantic model. In addition we propose a set of questions, to be used during a model’s feasibility study, and guidelines to help assess the most suitable method for managing semantics in a built environment.

Multiresolution semantic activity characterisation and abnormality discovery in videos

This paper addresses the issue of activity understanding from video and its semantics-rich description. A novel approach is presented where activities are characterised and analysed at different resolutions. Semantic information is delivered according to the resolution at which the activity is observed. Furthermore, the multiresolution activity characterisation is exploited to detect abnormal activity. To achieve these system capabilities, the focus is given on context modelling by employing a soft computing-based algorithm which automatically enables the determination of the main activity zones of the observed scene by taking as input the trajectories of detected mobiles. Such areas are learnt at different resolutions (or granularities). In a second stage, learned zones are employed to extract people activities by relating mobile trajectories to the learned zones. In this way, the activity of a person can be summarised as the series of zones that the person has visited. Employing the inherent soft relation properties, the reported activities can be labelled with meaningful semantics. Depending on the granularity at which activity zones and mobile trajectories are considered, the semantic meaning of the activity shifts from broad interpretation to detailed description.Activity information at different resolutions is also employed to perform abnormal activity detection.

Multicamera trajectory analysis for semantic behaviour characterisation

In this paper we propose an innovative approach for behaviour recognition, from a multicamera environment, based on translating video activity into semantics. First, we fuse tracks from individual cameras through clustering employing soft computing techniques. Then, we introduce a higher-level module able to translate fused tracks into semantic information. With our proposed approach, we address the challenge set in PETS 2014 on recognising behaviours of interest around a parked vehicle, namely the abnormal behaviour of someone walking around the vehicle.

The neural response to emotional prosody, as revealed by functional magnetic resonance imaging

Prosody is an important feature of language, comprising intonation, loudness, and tempo. Emotional prosodic processing forms an integral part of our social interactions. The main aim of this study was to use bold contrast fMRI to clarify the normal functional neuroanatomy of emotional prosody, in passive and active contexts. Subjects performed six separate scanning studies, within which two different conditions were contrasted: (1) "pure" emotional prosody versus rest; (2) congruent emotional prosody versus 'neutral' sentences; (3) congruent emotional prosody versus rest; (4) incongruent emotional prosody versus rest; (5) congruent versus incongruent emotional prosody; and (6) an active experiment in which subjects were instructed to either attend to the emotion conveyed by semantic content or that conveyed by tone of voice. Data resulting from these contrasts were analysed using SPM99. Passive listening to emotional prosody consistently activated the lateral temporal lobe (superior and/or middle temporal gyri). This temporal lobe response was relatively right-lateralised with or without semantic information. Both the separate and direct comparisons of congruent and incongruent emotional prosody revealed that subjects used fewer brain regions to process incongruent emotional prosody than congruent. The neural response to attention to semantics, was left lateralised, and recruited an extensive network not activated by attention to emotional prosody. Attention to emotional prosody modulated the response to speech, and induced right-lateralised activity, including the middle temporal gyrus. In confirming the results of lesion and neuropsychological studies, the current study emphasises the importance of the right hemisphere in the processing of emotional prosody, specifically the lateral temporal lobes. (C) 2003 Elsevier Science Ltd. All rights reserved.

Motor brain regions are involved in the encoding of delayed intentions: a fMRI study

In studies of prospective memory, recall of the content of delayed intentions is normally excellent, probably because they contain actions that have to be enacted at a later time. Action words encoded for later enactment are more accessible from memory than those encoded for later verbal report [Freeman, J.E., and Ellis, J.A. 2003a. The representation of delayed intentions: A prospective subject-performed task? Journal of Experimental Psychology: Learning, Memory, and Cognition, 29, 976-992.]. As this higher assessibility is lost when the intended actions have to be enacted during encoding, or when a motor interference task is introduced concurrent to intention encoding, Freeman and Ellis suggested that the advantage of to-be-enacted actions is due to additional preparatory motor operations during encoding. Accordingly, in a fMRI study with 10 healthy young participants, we investigated whether motor brain regions are differentially activated during verbal encoding of actions for later enactment with the right hand in contrast to verbal encoding of actions for later verbal report. We included an additional condition of verbal encoding of abstract verbs for later verbal report to investigate whether the semantic motor information inherent in action verbs in contrast to abstract verbs activates motor brain regions different from those involved in the verbal encoding of actions for later enactment. Differential activation for the verbal encoding of to-be-enacted actions in contrast to to-be-reported actions was found in brain regions known to be involved in covert motor preparation for hand movements, i.e. the postcentral gyrus, the precuneus, the dorsal and ventral premotor cortex, the posterior middle temporal gyrus and the inferior parietal lobule. There was no overlap between these brain regions and those differentially activated during the verbal encoding of actions in contrast to abstract verbs for later verbal report. Consequently, the results of this fMRI study suggest the presence of preparatory motor operations during the encoding of delayed intentions requiring a future motor response, which cannot be attributed to semantic information inherent to action verbs. (c) 2006 Elsevier B.V. All rights reserved.

Verb retrieval in fluent aphasia: a clinical study

Background: Problems with lexical retrieval are common across all types of aphasia but certain word classes are thought to be more vulnerable in some aphasia types. Traditionally, verb retrieval problems have been considered characteristic of non-fluent aphasias but there is growing evidence that verb retrieval problems are also found in fluent aphasia. As verbs are retrieved from the mental lexicon with syntactic as well as phonological and semantic information, it is speculated that an improvement in verb retrieval should enhance communicative abilities in this population as in others. We report on an investigation into the effectiveness of verb treatment for three individuals with fluent aphasia. Methods & Procedures: Multiple pre-treatment baselines were established over 3 months in order to monitor language change before treatment. The three participants then received twice-weekly verb treatment over approximately 4 months. All pre-treatment assessments were administered immediately after treatment and 3 months post-treatment. Outcome & Results: Scores fluctuated in the pre-treatment period. Following treatment, there was a significant improvement in verb retrieval for two of the three participants on the treated items. The increase in scores for the third participant was statistically nonsignificant but post-treatment scores moved from below the normal range to within the normal range. All participants were significantly quicker in the verb retrieval task following treatment. There was an increase in well-formed sentences in the sentence construction test and in some samples of connected speech. Conclusions: Repeated systematic treatment can produce a significant improvement in verb retrieval of practised items and generalise to unpractised items for some participants. An increase in well-formed sentences is seen for some speakers. The theoretical and clinical implications of the results are discussed.

Visual motion interferes with lexical decision on motion words

Embodied theories of cognition propose that neural substrates used in experiencing the referent of a word, for example perceiving upward motion, should be engaged in weaker form when that word, for example ‘rise’, is comprehended. Motivated by the finding that the perception of irrelevant background motion at near-threshold, but not supra-threshold, levels interferes with task execution, we assessed whether interference from near-threshold background motion was modulated by its congruence with the meaning of words (semantic content) when participants completed a lexical decision task (deciding if a string of letters is a real word or not). Reaction times for motion words, such as ‘rise’ or ‘fall’, were slower when the direction of visual motion and the ‘motion’ of the word were incongruent — but only when the visual motion was at nearthreshold levels. When motion was supra-threshold, the distribution of error rates, not reaction times, implicated low-level motion processing in the semantic processing of motion words. As the perception of near-threshold signals is not likely to be influenced by strategies, our results support a close contact between semantic information and perceptual systems.

Processing empty categories in a second language: When naturalistic exposure fills the (intermediate) gap

An ongoing debate on second language (L2) processing revolves around whether or not L2 learners process syntactic information similarly to monolinguals (L1), and what factors lead to a native-like processing. According to the Shallow Structure Hypothesis (Clahsen & Felser, 2006a), L2 learners’ processing does not include abstract syntactic features, such as intermediate gaps of wh-movement, but relies more on lexical/semantic information. Other researchers have suggested that naturalistic L2 exposure can lead to native-like processing (Dussias, 2003). This study investigates the effect of naturalistic exposure in processing wh-dependencies. Twenty-six advanced Greek learners of L2 English with an average nine years of naturalistic exposure, 30 with classroom exposure, and 30 native speakers of English completed a self-paced reading task with sentences involving intermediate gaps. L2 learners with naturalistic exposure showed evidence of native-like processing of the intermediate gaps, suggesting that linguistic immersion can lead to native-like abstract syntactic processing in the L2.

The time-course of morphological constraints: Evidence from eye-movements during reading

Lexical compounds in English are constrained in that the non-head noun can be an irregular but not a regular plural (e.g. mice eater vs. *rats eater), a contrast that has been argued to derive from a morphological constraint on modifiers inside compounds. In addition, bare nouns are preferred over plural forms inside compounds (e.g. mouse eater vs. mice eater), a contrast that has been ascribed to the semantics of compounds. Measuring eyemovements during reading, this study examined how morphological and semantic information become available over time during the processing of a compound. We found that the morphological constraint affected both early and late eye-movement measures, whereas the semantic constraint for singular non-heads only affected late measures of processing. These results indicate that morphological information becomes available earlier than semantic information during the processing of compounds.

When semantics aids phonology: a processing advantage for iconic word forms in aphasia

Iconicity is the non-arbitrary relation between properties of a phonological form and semantic content (e.g. “moo”, “splash”). It is a common feature of both spoken and signed languages, and recent evidence shows that iconic forms confer an advantage during word learning. We explored whether iconic forms conferred a processing advantage for 13 individuals with aphasia following left-hemisphere stroke. Iconic and control words were compared in four different tasks: repetition, reading aloud, auditory lexical decision and visual lexical decision. An advantage for iconic words was seen for some individuals in all tasks, with consistent group effects emerging in reading aloud and auditory lexical decision. Both these tasks rely on mapping between semantics and phonology. We conclude that iconicity aids spoken word processing for individuals with aphasia. This advantage may be due to a stronger connection between semantic information and phonological forms.

Selecting object pairs for action: is the active object always first?

Perception is linked to action via two routes: a direct route based on affordance information in the environment and an indirect route based on semantic knowledge about objects. The present study explored the factors modulating the recruitment of the two routes, in particular which factors affecting the selection of paired objects. In Experiment 1, we presented real objects among semantically related or unrelated distracters. Participants had to select two objects that can interact. The presence of distracters affected selection times, but not the semantic relations of the objects with the distracters. Furthermore, participants first selected the active object (e.g. teaspoon) with their right hand, followed by the passive object (e.g. mug), often with their left hand. In Experiment 2, we presented pictures of the same objects with no hand grip, congruent or incongruent hand grip. Participants had to decide whether the two objects can interact. Action decisions were faster when the presentation of the active object preceded the presentation of the passive object, and when the grip was congruent. Interestingly, participants were slower when the objects were semantically but not functionally related; this effect increased with congruently gripped objects. Our data showed that action decisions in the presence of strong affordance cues (real objects, pictures of congruently gripped objects) relied on sensory-motor representation, supporting the direct route from perception-to-action that bypasses semantic knowledge. However, in the case of weak affordance cues (pictures), semantic information interfered with action decisions, indicating that semantic knowledge impacts action decisions. The data support the dual-route account from perception-to-action.

Verbs in space: axis and direction of motion norms for 299 English verbs

A strong body of work has explored the interaction between visual perception and language comprehension; for example, recent studies exploring predictions from embodied cognition have focused particularly on the common representation of sensory—motor and semantic information. Motivated by this background, we provide a set of norms for the axis and direction of motion implied in 299 English verbs, collected from approximately 100 native speakers of British English. Until now, there have been no freely available norms of this kind for a large set of verbs that can be used in any area of language research investigating the semantic representation of motion. We have used these norms to investigate the interaction between language comprehension and low-level visual processes involved in motion perception, validating the norming procedure’s ability to capture the motion content of individual verbs. Supplemental materials for this study may be downloaded from brm.psychonomic-journals.org/content/supplemental.

Semantic-driven context-aware visual information indexing and retrieval: applied in the film post-production domain

A large volume of visual content is inaccessible until effective and efficient indexing and retrieval of such data is achieved. In this paper, we introduce the DREAM system, which is a knowledge-assisted semantic-driven context-aware visual information retrieval system applied in the film post production domain. We mainly focus on the automatic labelling and topic map related aspects of the framework. The use of the context- related collateral knowledge, represented by a novel probabilistic based visual keyword co-occurrence matrix, had been proven effective via the experiments conducted during system evaluation. The automatically generated semantic labels were fed into the Topic Map Engine which can automatically construct ontological networks using Topic Maps technology, which dramatically enhances the indexing and retrieval performance of the system towards an even higher semantic level.

Automatically constructing the semantic web

The storage and processing capacity realised by computing has lead to an explosion of data retention. We now reach the point of information overload and must begin to use computers to process more complex information. In particular, the proposition of the Semantic Web has given structure to this problem, but has yet realised practically. The largest of its problems is that of ontology construction; without a suitable automatic method most will have to be encoded by hand. In this paper we discus the current methods for semi and fully automatic construction and their current shortcomings. In particular we pay attention the application of ontologies to products and the particle application of the ontologies.