Activation and persistence of implicit causality information in spoken language comprehension

A visual world eye-tracking study investigated the activation and persistence of implicit causality information in spoken language comprehension. We showed that people infer the implicit causality of verbs as soon as they encounter such verbs in discourse, as is predicted by proponents of the immediate focusing account (Greene & McKoon, 1995; Koornneef & Van Berkum, 2006; Van Berkum, Koornneef, Otten, & Nieuwland, 2007). Interestingly, we observed activation of implicit causality information even before people encountered the causal conjunction. However, while implicit causality information was persistent as the discourse unfolded, it did not have a privileged role as a focusing cue immediately at the ambiguous pronoun when people were resolving its antecedent. Instead, our study indicated that implicit causality does not affect all referents to the same extent, rather it interacts with other cues in the discourse, especially when one of the referents is already prominently in focus.

Three-year-olds are sensitive to semantic prominence during online language comprehension

Recent evidence from adult pronoun comprehension suggests that semantic factors such as verb transitivity affect referent salience and thereby anap- hora resolution. We tested whether the same semantic factors influence pronoun comprehension in young children. In a visual world study, 3-year- olds heard stories that began with a sentence containing either a high or a low transitivity verb. Looking behaviour to pictures depicting the subject and object of this sentence was recorded as children listened to a subsequent sentence containing a pronoun. Children showed a stronger preference to look to the subject as opposed to the object antecedent in the low transitivity condition. In addition there were general preferences (1) to look to the subject in both conditions and (2) to look more at both potential antecedents in the high transitivity condition. This suggests that children, like adults, are affected by semantic factors, specifically semantic prominence, when interpreting anaphoric pronouns.

The Role of Epistemic Monitoring in Language Comprehension

In der psycholinguistischen Forschung ist die Annahme weitverbreitet, dass die Bewertung von Informationen hinsichtlich ihres Wahrheitsgehaltes oder ihrer Plausibilität (epistemische Validierung; Richter, Schroeder & Wöhrmann, 2009) ein strategischer, optionaler und dem Verstehen nachgeschalteter Prozess ist (z.B. Gilbert, 1991; Gilbert, Krull & Malone, 1990; Gilbert, Tafarodi & Malone, 1993; Herbert & Kübler, 2011). Eine zunehmende Anzahl an Studien stellt dieses Zwei-Stufen-Modell von Verstehen und Validieren jedoch direkt oder indirekt in Frage. Insbesondere Befunde zu Stroop-artigen Stimulus-Antwort-Kompatibilitätseffekten, die auftreten, wenn positive und negative Antworten orthogonal zum aufgaben-irrelevanten Wahrheitsgehalt von Sätzen abgegeben werden müssen (z.B. eine positive Antwort nach dem Lesen eines falschen Satzes oder eine negative Antwort nach dem Lesen eines wahren Satzes; epistemischer Stroop-Effekt, Richter et al., 2009), sprechen dafür, dass Leser/innen schon beim Verstehen eine nicht-strategische Überprüfung der Validität von Informationen vornehmen. Ausgehend von diesen Befunden war das Ziel dieser Dissertation eine weiterführende Überprüfung der Annahme, dass Verstehen einen nicht-strategischen, routinisierten, wissensbasierten Validierungsprozesses (epistemisches Monitoring; Richter et al., 2009) beinhaltet. Zu diesem Zweck wurden drei empirische Studien mit unterschiedlichen Schwerpunkten durchgeführt. Studie 1 diente der Untersuchung der Fragestellung, ob sich Belege für epistemisches Monitoring auch bei Informationen finden lassen, die nicht eindeutig wahr oder falsch, sondern lediglich mehr oder weniger plausibel sind. Mithilfe des epistemischen Stroop-Paradigmas von Richter et al. (2009) konnte ein Kompatibilitätseffekt von aufgaben-irrelevanter Plausibilität auf die Latenzen positiver und negativer Antworten in zwei unterschiedlichen experimentellen Aufgaben nachgewiesen werden, welcher dafür spricht, dass epistemisches Monitoring auch graduelle Unterschiede in der Übereinstimmung von Informationen mit dem Weltwissen berücksichtigt. Darüber hinaus belegen die Ergebnisse, dass der epistemische Stroop-Effekt tatsächlich auf Plausibilität und nicht etwa auf der unterschiedlichen Vorhersagbarkeit von plausiblen und unplausiblen Informationen beruht. Das Ziel von Studie 2 war die Prüfung der Hypothese, dass epistemisches Monitoring keinen evaluativen Mindset erfordert. Im Gegensatz zu den Befunden anderer Autoren (Wiswede, Koranyi, Müller, Langner, & Rothermund, 2013) zeigte sich in dieser Studie ein Kompatibilitätseffekt des aufgaben-irrelevanten Wahrheitsgehaltes auf die Antwortlatenzen in einer vollständig nicht-evaluativen Aufgabe. Die Ergebnisse legen nahe, dass epistemisches Monitoring nicht von einem evaluativen Mindset, möglicherweise aber von der Tiefe der Verarbeitung abhängig ist. Studie 3 beleuchtete das Verhältnis von Verstehen und Validieren anhand einer Untersuchung der Online-Effekte von Plausibilität und Vorhersagbarkeit auf Augenbewegungen beim Lesen kurzer Texte. Zusätzlich wurde die potentielle Modulierung dieser Effeke durch epistemische Marker, die die Sicherheit von Informationen anzeigen (z.B. sicherlich oder vielleicht), untersucht. Entsprechend der Annahme eines schnellen und nicht-strategischen epistemischen Monitoring-Prozesses zeigten sich interaktive Effekte von Plausibilität und dem Vorhandensein epistemischer Marker auf Indikatoren früher Verstehensprozesse. Dies spricht dafür, dass die kommunizierte Sicherheit von Informationen durch den Monitoring-Prozess berücksichtigt wird. Insgesamt sprechen die Befunde gegen eine Konzeptualisierung von Verstehen und Validieren als nicht-überlappenden Stufen der Informationsverarbeitung. Vielmehr scheint eine Bewertung des Wahrheitsgehalts oder der Plausibilität basierend auf dem Weltwissen – zumindest in gewissem Ausmaß – eine obligatorische und nicht-strategische Komponente des Sprachverstehens zu sein. Die Bedeutung der Befunde für aktuelle Modelle des Sprachverstehens und Empfehlungen für die weiterführende Forschung zum Vehältnis von Verstehen und Validieren werden aufgezeigt.

Complexity of Human Language Comprehension

The goal of this article is to reveal the computational structure of modern principle-and-parameter (Chomskian) linguistic theories: what computational problems do these informal theories pose, and what is the underlying structure of those computations? To do this, I analyze the computational complexity of human language comprehension: what linguistic representation is assigned to a given sound? This problem is factored into smaller, interrelated (but independently statable) problems. For example, in order to understand a given sound, the listener must assign a phonetic form to the sound; determine the morphemes that compose the words in the sound; and calculate the linguistic antecedent of every pronoun in the utterance. I prove that these and other subproblems are all NP-hard, and that language comprehension is itself PSPACE-hard.

Theory of mind and language comprehension in schizophrenia

Resumen tomado de la publicación

Language and Embodiment: sensory-motor and linguistic-social experience. Evidence on sentence comprehension

In this work I address the study of language comprehension in an “embodied” framework. Firstly I show behavioral evidence supporting the idea that language modulates the motor system in a specific way, both at a proximal level (sensibility to the effectors) and at the distal level (sensibility to the goal of the action in which the single motor acts are inserted). I will present two studies in which the method is basically the same: we manipulated the linguistic stimuli (the kind of sentence: hand action vs. foot action vs. mouth action) and the effector by which participants had to respond (hand vs. foot vs. mouth; dominant hand vs. non-dominant hand). Response times analyses showed a specific modulation depending on the kind of sentence: participants were facilitated in the task execution (sentence sensibility judgment) when the effector they had to use to respond was the same to which the sentences referred. Namely, during language comprehension a pre-activation of the motor system seems to take place. This activation is analogous (even if less intense) to the one detectable when we practically execute the action described by the sentence. Beyond this effector specific modulation, we also found an effect of the goal suggested by the sentence. That is, the hand effector was pre-activated not only by hand-action-related sentences, but also by sentences describing mouth actions, consistently with the fact that to execute an action on an object with the mouth we firstly have to bring it to the mouth with the hand. After reviewing the evidence on simulation specificity directly referring to the body (for instance, the kind of the effector activated by the language), I focus on the specific properties of the object to which the words refer, particularly on the weight. In this case the hypothesis to test was if both lifting movement perception and lifting movement execution are modulated by language comprehension. We used behavioral and kinematics methods, and we manipulated the linguistic stimuli (the kind of sentence: the lifting of heavy objects vs. the lifting of light objects). To study the movement perception we measured the correlations between the weight of the objects lifted by an actor (heavy objects vs. light objects) and the esteems provided by the participants. To study the movement execution we measured kinematics parameters variance (velocity, acceleration, time to the first peak of velocity) during the actual lifting of objects (heavy objects vs. light objects). Both kinds of measures revealed that language had a specific effect on the motor system, both at a perceptive and at a motoric level. Finally, I address the issue of the abstract words. Different studies in the “embodied” framework tried to explain the meaning of abstract words The limit of these works is that they account only for subsets of phenomena, so results are difficult to generalize. We tried to circumvent this problem by contrasting transitive verbs (abstract and concrete) and nouns (abstract and concrete) in different combinations. The behavioral study was conducted both with German and Italian participants, as the two languages are syntactically different. We found that response times were faster for both the compatible pairs (concrete verb + concrete noun; abstract verb + abstract noun) than for the mixed ones. Interestingly, for the mixed combinations analyses showed a modulation due to the specific language (German vs. Italian): when the concrete word precedes the abstract one responses were faster, regardless of the word grammatical class. Results are discussed in the framework of current views on abstract words. They highlight the important role of developmental and social aspects of language use, and confirm theories assigning a crucial role to both sensorimotor and linguistic experience for abstract words.

Sentence comprehension before and after 1970: Topics, debates and techniques

[EN] Language Down the Garden Path traces the lines of research that grew out of Bever's classic paper. Leading scientists review over 40 years of debates on the factors at play in language comprehension, production, and acquisition (the role of prediction, grammar, working memory, prosody, abstractness, syntax and semantics mapping); the current status of universals and narrow syntax; and virtually every topic relevant in psycholinguistics since 1970. Written in an accessible and engaging style, the book will appeal to all those interested in understanding the questions that shaped, and are still shaping, this field and the ways in which linguists, cognitive scientists, psychologists, and neuroscientists are seeking to answer them.

H.M.'s personal crossword puzzles: understanding memory and language.

The amnesic patient H.M. has been solving crossword puzzles nearly all his life. Here, we analysed the linguistic content of 277 of H.M.'s crossword-puzzle solutions. H.M. did not have any unusual difficulties with the orthographic and grammatical components inherent to the puzzles. He exhibited few spelling errors, responded with appropriate parts of speech, and provided answers that were, at times, more convincing to observers than those supplied by the answer keys. These results suggest that H.M.'s lexical word-retrieval skills remain fluid despite his profound anterograde amnesia. Once acquired, the maintenance of written language comprehension and production does not seem to require intact medial temporal lobe structures.

An embodied approach to language: the role of bodily feedback in discourse processing

Theories of embodied cognition argue that language processing arises not from amodal symbols that redescribe sensorimotor and affective experiences, but from partial simulations (reenactments) of modality-specific states. Recent findings on processing of words and sentences support such a stance emphasizing that the role of the body in the domain of language comprehension should not be overlooked or dismissed. The present research was conducted to extend prior work in two important ways. First, the role of simulation was tested with connected discourse rather than words or sentences presented in isolation. Second, both “online” and “offline” measures of discourse comprehension were taken. In Experiments 1 and 2 participants’ facial postures were manipulated to show that preparing the body for processing of emotion-congruent information improves discourse comprehension. In Experiment 3 the direction of body posture was manipulated to show that implicit properties of simulations, such as spatial dimension or location, are at least somewhat involved in processing of large language segments such as discourse. Finally, in Experiments 4 and 5 participants’ body movement and body posture were manipulated to show that even understanding of language describing metaphorical actions physically impossible to perform involves constructing a sensorimotor simulation of the described event. The major result was that compatibility between embodiment and language strongly modulated performance effectiveness in experiments on simulation of emotion and metaphorical action. The effect of simulation on comprehension of discourse implying spatial dimension was fragile. These findings support an embodied simulation account of cognition suggesting that sensorimotor and affective states are at least partially implicated in “online” and “offline” discourse comprehension.

Computational Structure of Human Language

The central thesis of this report is that human language is NP-complete. That is, the process of comprehending and producing utterances is bounded above by the class NP, and below by NP-hardness. This constructive complexity thesis has two empirical consequences. The first is to predict that a linguistic theory outside NP is unnaturally powerful. The second is to predict that a linguistic theory easier than NP-hard is descriptively inadequate. To prove the lower bound, I show that the following three subproblems of language comprehension are all NP-hard: decide whether a given sound is possible sound of a given language; disambiguate a sequence of words; and compute the antecedents of pronouns. The proofs are based directly on the empirical facts of the language user's knowledge, under an appropriate idealization. Therefore, they are invariant across linguistic theories. (For this reason, no knowledge of linguistic theory is needed to understand the proofs, only knowledge of English.) To illustrate the usefulness of the upper bound, I show that two widely-accepted analyses of the language user's knowledge (of syntactic ellipsis and phonological dependencies) lead to complexity outside of NP (PSPACE-hard and Undecidable, respectively). Next, guided by the complexity proofs, I construct alternate linguisitic analyses that are strictly superior on descriptive grounds, as well as being less complex computationally (in NP). The report also presents a new framework for linguistic theorizing, that resolves important puzzles in generative linguistics, and guides the mathematical investigation of human language.

Revealing and quantifying the impaired phonological analysis underpinning impaired comprehension in Wernicke’s aphasia

Wernicke’s aphasia is a condition which results in severely disrupted language comprehension following a lesion to the left temporo-parietal region. A phonological analysis deficit has traditionally been held to be at the root of the comprehension impairment in WA, a view consistent with current functional neuroimaging which finds areas in the superior temporal cortex responsive to phonological stimuli. However behavioural evidence to support the link between a phonological analysis deficit and auditory comprehension has not been yet shown. This study extends seminal work by Blumstein et al. (1977) to investigate the relationship between acoustic-phonological perception, measured through phonological discrimination, and auditory comprehension in a case series of Wernicke’s aphasia participants. A novel adaptive phonological discrimination task was used to obtain reliable thresholds of the phonological perceptual distance required between nonwords before they could be discriminated. Wernicke’s aphasia participants showed significantly elevated thresholds compared to age and hearing matched control participants. Acoustic-phonological thresholds correlated strongly with auditory comprehension abilities in Wernicke’s aphasia. In contrast, nonverbal semantic skills showed no relationship with auditory comprehension. The results are evaluated in the context of recent neurobiological models of language and suggest that impaired acoustic-phonological perception underlies the comprehension impairment in Wernicke’s aphasia and favour models of language which propose a leftward asymmetry in phonological analysis.

The anterior temporal lobes support residual comprehension in Wernicke’s aphasia

Wernicke’s aphasia occurs following a stroke to classical language comprehension regions in the left temporoparietal cortex. Consequently, auditory-verbal comprehension is significantly impaired in Wernicke’s aphasia but the capacity to comprehend visually presented materials (written words and pictures) is partially spared. This study used fMRI to investigate the neural basis of written word and picture semantic processing in Wernicke’s aphasia, with the wider aim of examining how the semantic system is altered following damage to the classical comprehension regions. Twelve participants with Wernicke’s aphasia and twelve control participants performed semantic animate-inanimate judgements and a visual height judgement baseline task. Whole brain and ROI analysis in Wernicke’s aphasia and control participants found that semantic judgements were underpinned by activation in the ventral and anterior temporal lobes bilaterally. The Wernicke’s aphasia group displayed an “over-activation” in comparison to control participants, indicating that anterior temporal lobe regions become increasingly influential following reduction in posterior semantic resources. Semantic processing of written words in Wernicke’s aphasia was additionally supported by recruitment of the right anterior superior temporal lobe, a region previously associated with recovery from auditory-verbal comprehension impairments. Overall, the results concord with models which indicate that the anterior temporal lobes are crucial for multimodal semantic processing and that these regions may be accessed without support from classic posterior comprehension regions.

Mismatch Negativity (MMN) reveals inefficient auditory ventral stream function in chronic auditory comprehension impairments

Background: Auditory discrimination is significantly impaired in Wernicke’s aphasia (WA) and thought to be causatively related to the language comprehension impairment which characterises the condition. This study used mismatch negativity (MMN) to investigate the neural responses corresponding to successful and impaired auditory discrimination in WA. Methods: Behavioural auditory discrimination thresholds of CVC syllables and pure tones were measured in WA (n=7) and control (n=7) participants. Threshold results were used to develop multiple-deviant mismatch negativity (MMN) oddball paradigms containing deviants which were either perceptibly or non-perceptibly different from the standard stimuli. MMN analysis investigated differences associated with group, condition and perceptibility as well as the relationship between MMN responses and comprehension (within which behavioural auditory discrimination profiles were examined). Results: MMN waveforms were observable to both perceptible and non-perceptible auditory changes. Perceptibility was only distinguished by MMN amplitude in the PT condition. The WA group could be distinguished from controls by an increase in MMN response latency to CVC stimuli change. Correlation analyses displayed relationship between behavioural CVC discrimination and MMN amplitude in the control group, where greater amplitude corresponded to better discrimination. The WA group displayed the inverse effect; both discrimination accuracy and auditory comprehension scores were reduced with increased MMN amplitude. In the WA group, a further correlation was observed between the lateralisation of MMN response and CVC discrimination accuracy; the greater the bilateral involvement the better the discrimination accuracy. Conclusions: The results from this study provide further evidence for the nature of auditory comprehension impairment in WA and indicate that the auditory discrimination deficit is grounded in a reduced ability to engage in efficient hierarchical processing and the construction of invariant auditory objects. Correlation results suggest that people with chronic WA may rely on an inefficient, noisy right hemisphere auditory stream when attempting to process speech stimuli.

The language of action. How language translates the dynamics of our actions.

The general aim of the thesis was to investigate how and to what extent the characteristics of action organization are reflected in language, and how they influence language processing and understanding. Even though a huge amount of research has been devoted to the study of the motor effects of language, this issue is very debated in literature. Namely, the majority of the studies have focused on low-level motor effects such as effector-relatedness of action, whereas only a few studies have started to systematically investigate how specific aspects of action organization are encoded and reflected in language. After a review of previous studies on the relationship between language comprehension and action (chapter 1) and a critical discussion of some of them (chapter 2), the thesis is composed by three experimental chapters, each devoted to a specific aspect of action organization. Chapter 3 presents a study designed with the aim to disentangle the effective time course of the involvement of the motor system during language processing. Three kinematics experiments were designed in order to determine whether and, at which stage of motor planning and execution effector-related action verbs influence actions executed with either the same or a different effector. Results demonstrate that the goal of an action can be linguistically re-activated, producing a modulation of the motor response. In chapter 4, a second study investigates the interplay between the role of motor perspective (agent) and the organization of action in motor chains. More specifically, this kinematics study aims at deepening how goal can be translated in language, using as stimuli simple sentences composed by a pronoun (I, You, He/She) and a verb. Results showed that the perspective activated by the pronoun You reflects the motor pattern of the “agent” combined with the chain structure of the verb. These data confirm an early involvement of the motor system in language processing, suggesting that it is specifically modulated by the activation of the agent’s perspective. In chapter 5, the issue of perspective is specifically investigated, focusing on its role in language comprehension. In particular, this study aimed at determining how a specific perspective (induced for example by a personal pronoun) modulates motor behaviour during and after language processing. A classical compatibility effect (the Action-sentence compatibility effect) has been used to this aim. In three behavioural experiments the authors investigated how the ACE is modulated by taking first or third person perspective. Results from these experiments showed that the ACE effect occurs only when a first-person perspective is activated by the sentences used as stimuli. Overall, the data from this thesis contributed to disentangle several aspects of how action organization is translated in language, and then reactivated during language processing. This constitutes a new contribution to the field, adding lacking information on how specific aspects such as goal and perspective are linguistically described. In addition, these studies offer a new point of view to understand the functional implications of the involvement of the motor system during language comprehension, specifically from the point of view of our social interactions.

Cognition in context: Evidence on affordances and verbal language

This thesis investigated affordances and verbal language to demonstrate the flexibility of embodied simulation processes. Starting from the assumption that both object/action understanding and language comprehension are tied to the context in which they take place, six studies clarified the factors that modulate simulation. The studies in chapter 4 and 5 investigated affordance activation in complex scenes, revealing the strong influence of the visual context, which included either objects and actions, on compatibility effects. The study in chapter 6 compared the simulation triggered by visual objects and objects names, showing differences depending on the kind of materials processed. The study in chapter 7 tested the predictions of the WAT theory, confirming that the different contexts in which words are acquired lead to the difference typically observed in the literature between concrete and abstract words. The study in chapter 8 on the grounding of abstract concepts tested the mapping of temporal contents on the spatial frame of reference of the mental timeline, showing that metaphoric congruency effects are not automatic, but flexibly mediated by the context determined by the goals of different tasks. The study in chapter 9 investigated the role of iconicity in verbal language, showing sound-to-shape correspondences when every-day object figures, result that validated the reality of sound-symbolism in ecological contexts. On the whole, this evidence favors embodied views of cognition, and supports the hypothesis of a high flexibility of simulation processes. The reported conceptual effects confirm that the context plays a crucial role in affordances emergence, metaphoric mappings activation and language grounding. In conclusion, this thesis highlights that in an embodied perspective cognition is necessarily situated and anchored to a specific context, as it is sustained by the existence of a specific body immersed in a specific environment.