Language and Embodiment: sensory-motor and linguistic-social experience. Evidence on sentence comprehension

In this work I address the study of language comprehension in an “embodied” framework. Firstly I show behavioral evidence supporting the idea that language modulates the motor system in a specific way, both at a proximal level (sensibility to the effectors) and at the distal level (sensibility to the goal of the action in which the single motor acts are inserted). I will present two studies in which the method is basically the same: we manipulated the linguistic stimuli (the kind of sentence: hand action vs. foot action vs. mouth action) and the effector by which participants had to respond (hand vs. foot vs. mouth; dominant hand vs. non-dominant hand). Response times analyses showed a specific modulation depending on the kind of sentence: participants were facilitated in the task execution (sentence sensibility judgment) when the effector they had to use to respond was the same to which the sentences referred. Namely, during language comprehension a pre-activation of the motor system seems to take place. This activation is analogous (even if less intense) to the one detectable when we practically execute the action described by the sentence. Beyond this effector specific modulation, we also found an effect of the goal suggested by the sentence. That is, the hand effector was pre-activated not only by hand-action-related sentences, but also by sentences describing mouth actions, consistently with the fact that to execute an action on an object with the mouth we firstly have to bring it to the mouth with the hand. After reviewing the evidence on simulation specificity directly referring to the body (for instance, the kind of the effector activated by the language), I focus on the specific properties of the object to which the words refer, particularly on the weight. In this case the hypothesis to test was if both lifting movement perception and lifting movement execution are modulated by language comprehension. We used behavioral and kinematics methods, and we manipulated the linguistic stimuli (the kind of sentence: the lifting of heavy objects vs. the lifting of light objects). To study the movement perception we measured the correlations between the weight of the objects lifted by an actor (heavy objects vs. light objects) and the esteems provided by the participants. To study the movement execution we measured kinematics parameters variance (velocity, acceleration, time to the first peak of velocity) during the actual lifting of objects (heavy objects vs. light objects). Both kinds of measures revealed that language had a specific effect on the motor system, both at a perceptive and at a motoric level. Finally, I address the issue of the abstract words. Different studies in the “embodied” framework tried to explain the meaning of abstract words The limit of these works is that they account only for subsets of phenomena, so results are difficult to generalize. We tried to circumvent this problem by contrasting transitive verbs (abstract and concrete) and nouns (abstract and concrete) in different combinations. The behavioral study was conducted both with German and Italian participants, as the two languages are syntactically different. We found that response times were faster for both the compatible pairs (concrete verb + concrete noun; abstract verb + abstract noun) than for the mixed ones. Interestingly, for the mixed combinations analyses showed a modulation due to the specific language (German vs. Italian): when the concrete word precedes the abstract one responses were faster, regardless of the word grammatical class. Results are discussed in the framework of current views on abstract words. They highlight the important role of developmental and social aspects of language use, and confirm theories assigning a crucial role to both sensorimotor and linguistic experience for abstract words.

The language of action. How language translates the dynamics of our actions.

The general aim of the thesis was to investigate how and to what extent the characteristics of action organization are reflected in language, and how they influence language processing and understanding. Even though a huge amount of research has been devoted to the study of the motor effects of language, this issue is very debated in literature. Namely, the majority of the studies have focused on low-level motor effects such as effector-relatedness of action, whereas only a few studies have started to systematically investigate how specific aspects of action organization are encoded and reflected in language. After a review of previous studies on the relationship between language comprehension and action (chapter 1) and a critical discussion of some of them (chapter 2), the thesis is composed by three experimental chapters, each devoted to a specific aspect of action organization. Chapter 3 presents a study designed with the aim to disentangle the effective time course of the involvement of the motor system during language processing. Three kinematics experiments were designed in order to determine whether and, at which stage of motor planning and execution effector-related action verbs influence actions executed with either the same or a different effector. Results demonstrate that the goal of an action can be linguistically re-activated, producing a modulation of the motor response. In chapter 4, a second study investigates the interplay between the role of motor perspective (agent) and the organization of action in motor chains. More specifically, this kinematics study aims at deepening how goal can be translated in language, using as stimuli simple sentences composed by a pronoun (I, You, He/She) and a verb. Results showed that the perspective activated by the pronoun You reflects the motor pattern of the “agent” combined with the chain structure of the verb. These data confirm an early involvement of the motor system in language processing, suggesting that it is specifically modulated by the activation of the agent’s perspective. In chapter 5, the issue of perspective is specifically investigated, focusing on its role in language comprehension. In particular, this study aimed at determining how a specific perspective (induced for example by a personal pronoun) modulates motor behaviour during and after language processing. A classical compatibility effect (the Action-sentence compatibility effect) has been used to this aim. In three behavioural experiments the authors investigated how the ACE is modulated by taking first or third person perspective. Results from these experiments showed that the ACE effect occurs only when a first-person perspective is activated by the sentences used as stimuli. Overall, the data from this thesis contributed to disentangle several aspects of how action organization is translated in language, and then reactivated during language processing. This constitutes a new contribution to the field, adding lacking information on how specific aspects such as goal and perspective are linguistically described. In addition, these studies offer a new point of view to understand the functional implications of the involvement of the motor system during language comprehension, specifically from the point of view of our social interactions.

Cognition in context: Evidence on affordances and verbal language

This thesis investigated affordances and verbal language to demonstrate the flexibility of embodied simulation processes. Starting from the assumption that both object/action understanding and language comprehension are tied to the context in which they take place, six studies clarified the factors that modulate simulation. The studies in chapter 4 and 5 investigated affordance activation in complex scenes, revealing the strong influence of the visual context, which included either objects and actions, on compatibility effects. The study in chapter 6 compared the simulation triggered by visual objects and objects names, showing differences depending on the kind of materials processed. The study in chapter 7 tested the predictions of the WAT theory, confirming that the different contexts in which words are acquired lead to the difference typically observed in the literature between concrete and abstract words. The study in chapter 8 on the grounding of abstract concepts tested the mapping of temporal contents on the spatial frame of reference of the mental timeline, showing that metaphoric congruency effects are not automatic, but flexibly mediated by the context determined by the goals of different tasks. The study in chapter 9 investigated the role of iconicity in verbal language, showing sound-to-shape correspondences when every-day object figures, result that validated the reality of sound-symbolism in ecological contexts. On the whole, this evidence favors embodied views of cognition, and supports the hypothesis of a high flexibility of simulation processes. The reported conceptual effects confirm that the context plays a crucial role in affordances emergence, metaphoric mappings activation and language grounding. In conclusion, this thesis highlights that in an embodied perspective cognition is necessarily situated and anchored to a specific context, as it is sustained by the existence of a specific body immersed in a specific environment.

The language of others mirrored in thy face: The role of political affiliation in automatic facial effects of language

People tend to automatically mimic facial expressions of others. If clear evidence exists on the effect of non-verbal behavior (emotion faces) on automatic facial mimicry, little is known about the role of verbal behavior (emotion language) in triggering such effects. Whereas it is well-established that political affiliation modulates facial mimicry, no evidence exists on whether this modulation passes also through verbal means. This research addressed the role of verbal behavior in triggering automatic facial effects depending on whether verbal stimuli are attributed to leaders of different political parties. Study 1 investigated the role of interpersonal verbs, referring to positive and negative emotion expressions and encoding them at different levels of abstraction, in triggering corresponding facial muscle activation in a reader. Study 2 examined the role of verbs expressing positive and negative emotional behaviors of political leaders in modulating automatic facial effects depending on the matched or mismatched political affiliation of participants and politicians of left-and right-wing. Study 3 examined whether verbs expressing happiness displays of ingroup politicians induce a more sincere smile (Duchenne) pattern among readers of same political affiliation relative to happiness expressions of outgroup politicians. Results showed that verbs encoding facial actions at different levels of abstraction elicited differential facial muscle activity (Study 1). Furthermore, political affiliation significantly modulated facial activation triggered by emotion verbs as participants showed more congruent and enhanced facial activity towards ingroup politicians’ smiles and frowns compared to those of outgroup politicians (Study 2). Participants facially responded with a more sincere smile pattern towards verbs expressing smiles of ingroup compared to outgroup politicians (Study 3). Altogether, results showed that the role of political affiliation in modulating automatic facial effects passes also through verbal channels and is revealed at a fine-grained level by inducing quantitative and qualitative differences in automatic facial reactions of readers.