Locating the semiotic power of multimodality

This article reports research that attempts to characterize what is powerful about digital multimodal texts. Building from recent theoretical work on understanding the workings and implications of multimodal communication, the authors call for a continuing empirical investigation into the roles that digital multimodal texts play in real-world contexts, and they offer one example of how such investigations might be approached. Drawing on data from the practice of multimedia digital storytelling, specifically a piece titled “Lyfe-N-Rhyme,” created by Oakland, California, artist Randy Young (accessible at http://www.oaklanddusty.org/videos.php), the authors detail the method and results of a fine-grained multimodal analysis, revealing semiotic relationships between and among different, copresent modes. It is in these relationships, the authors argue, that the expressive power of multimodality resides.

Semiotic, rhetoric and democracy

This paper unites Deely’s call for a better understanding of semiotics with Jaeger’s insight into the sophists and the cultural history of the Ancient Greeks. The two bodies of knowledge are brought together to try to better understand the importance of rhetorical processes to political forms such as democracy. Jaeger explains how cultural expression, particularly poetry, changed through the archaic and classical eras to deliver, or at least to be commensurate with contemporary politics and ideologies. He explains how Plato (429-347 BCE) struggled against certain poetry and prose manifestations in his ambition to create a ‘perfect man’ – a humanity which would think in a way which would enable the ideal Republic to flourish. Deely’s approach based on Poinsot and Peirce presents a theoretical framework by means of which we can think of the struggle to influence individual and communal conceptualisation as a struggle within semiotics. This is a struggle over the ways reality is signified by signs. Signs are physical and mental indications which, in the semiotic tradition, are taken to produce human subjectivity – human ‘being’. Deely’s extensive body of work is about how these signs are the building blocks of realist constructions of understanding. This paper is concerned with the deliberate use of oral and written signs in rhetorical activity which has been deliberately crafted to change subjectivity. We discuss: (1) what thought and culture is in terms of semiotics and (2) Jaeger’s depiction of Ancient Greece as an illustration of the conjunction between culture and subjectivity. These two fields are brought together in order to make the argument that rhetoric can be theorised as the deliberate harnessing of semiotic affects. The implication is that the same semiotic, subjectivity-changing potency holds for 21st century rhetoric. However fourth century BCE Athens is the best setting for a preliminary discussion of rhetoric as deliberate semiotic practice because this was when rhetoric was most clearly understood for what it is. By contrast a discussion concentrating on modern rhetoric: public relations; advertising; lobbying; and public affairs would open wider controversies requiring considerably more complex explanation.

Scientific concepts and public policies: Semiotic-cultural obstacles concerning intergroup and intercultural relationships

I attempt to articulate Jahoda's (2012) critical reflections regarding definitions of culture in recent cross-cultural studies and Moghaddam's (2012) claims of an omnicultural imperative to guide the elaboration of public policies for managing relationships among human groups from different cultural origins. For this, I will approach some aspects of the socio-historical and ontogenetic roots of the notion of culture. The notion of culture and the consequent public policies involving intercultural managing are being transformed as our global society develops. It has been proposed that some ways of dealing with the culture of the other are crucial to achieve awareness in respect of one's own cultural positioning when making science and attempting social interventions. Finally, the experience of Brazilian psychologists working on challenges faced by Amerindians dealing with the national society they live in will be presented as a pioneering work aiming to interfere in the development of public policies ethically concerned with the assurance of cultural integrity of currently marginalized social groups.

Semiotic Analysis in the Study of Social Work

Research and professional practices have the joint aim of re-structuring the preconceived notions of reality. They both want to gain the understanding about social reality. Social workers use their professional competence in order to grasp the reality of their clients, while researchers’ pursuit is to open the secrecies of the research material. Development and research are now so intertwined and inherent in almost all professional practices that making distinctions between practising, developing and researching has become difficult and in many aspects irrelevant. Moving towards research-based practices is possible and it is easily applied within the framework of the qualitative research approach (Dominelli 2005, 235; Humphries 2005, 280). Social work can be understood as acts and speech acts crisscrossing between social workers and clients. When trying to catch the verbal and non-verbal hints of each others’ behaviour, the actors have to do a lot of interpretations in a more or less uncertain mental landscape. Our point of departure is the idea that the study of social work practices requires tools which effectively reveal the internal complexity of social work (see, for example, Adams & Dominelli & Payne 2005, 294 – 295). The boom of qualitative research methodologies in recent decades is associated with much profound the rupture in humanities, which is called the linguistic turn (Rorty 1967). The idea that language is not transparently mediating our perceptions and thoughts about reality, but on the contrary it constitutes it was new and even confusing to many social scientists. Nowadays we have got used to read research reports which have applied different branches of discursive analyses or narratologic or semiotic approaches. Although differences are sophisticated between those orientations they share the idea of the predominance of language. Despite the lively research work of today’s social work and the research-minded atmosphere of social work practice, semiotics has rarely applied in social work research. However, social work as a communicative practice concerns symbols, metaphors and all kinds of the representative structures of language. Those items are at the core of semiotics, the science of signs, and the science which examines people using signs in their mutual interaction and their endeavours to make the sense of the world they live in, their semiosis. When thinking of the practice of social work and doing the research of it, a number of interpretational levels ought to be passed before reaching the research phase in social work. First of all, social workers have to interpret their clients’ situations, which will be recorded in the files. In some very rare cases those past situations will be reflected in discussions or perhaps interviews or put under the scrutiny of some researcher in the future. Each and every new observation adds its own flavour to the mixture of meanings. Social workers have combined their observations with previous experience and professional knowledge, furthermore, the situation on hand also influences the reactions. In addition, the interpretations made by social workers over the course of their daily working routines are never limited to being part of the personal process of the social worker, but are also always inherently cultural. The work aiming at social change is defined by the presence of an initial situation, a specific goal, and the means and ways of achieving it, which are – or which should be – agreed upon by the social worker and the client in situation which is unique and at the same time socially-driven. Because of the inherent plot-based nature of social work, the practices related to it can be analysed as stories (see Dominelli 2005, 234), given, of course, that they are signifying and told by someone. The research of the practices is concentrating on impressions, perceptions, judgements, accounts, documents etc. All these multifarious elements can be scrutinized as textual corpora, but not whatever textual material. In semiotic analysis, the material studied is characterised as verbal or textual and loaded with meanings. We present a contribution of research methodology, semiotic analysis, which has to our mind at least implicitly references to the social work practices. Our examples of semiotic interpretation have been picked up from our dissertations (Laine 2005; Saurama 2002). The data are official documents from the archives of a child welfare agency and transcriptions of the interviews of shelter employees. These data can be defined as stories told by the social workers of what they have seen and felt. The official documents present only fragmentations and they are often written in passive form. (Saurama 2002, 70.) The interviews carried out in the shelters can be described as stories where the narrators are more familiar and known. The material is characterised by the interaction between the interviewer and interviewee. The levels of the story and the telling of the story become apparent when interviews or documents are examined with the use of semiotic tools. The roots of semiotic interpretation can be found in three different branches; the American pragmatism, Saussurean linguistics in Paris and the so called formalism in Moscow and Tartu; however in this paper we are engaged with the so called Parisian School of semiology which prominent figure was A. J. Greimas. The Finnish sociologists Pekka Sulkunen and Jukka Törrönen (1997a; 1997b) have further developed the ideas of Greimas in their studies on socio-semiotics, and we lean on their ideas. In semiotics social reality is conceived as a relationship between subjects, observations, and interpretations and it is seen mediated by natural language which is the most common sign system among human beings (Mounin 1985; de Saussure 2006; Sebeok 1986). Signification is an act of associating an abstract context (signified) to some physical instrument (signifier). These two elements together form the basic concept, the “sign”, which never constitutes any kind of meaning alone. The meaning will be comprised in a distinction process where signs are being related to other signs. In this chain of signs, the meaning becomes diverged from reality. (Greimas 1980, 28; Potter 1996, 70; de Saussure 2006, 46-48.) One interpretative tool is to think of speech as a surface under which deep structures – i.e. values and norms – exist (Greimas & Courtes 1982; Greimas 1987). To our mind semiotics is very much about playing with two different levels of text: the syntagmatic surface which is more or less faithful to the grammar, and the paradigmatic, semantic structure of values and norms hidden in the deeper meanings of interpretations. Semiotic analysis deals precisely with the level of meaning which exists under the surface, but the only way to reach those meanings is through the textual level, the written or spoken text. That is why the tools are needed. In our studies, we have used the semiotic square and the actant analysis. The former is based on the distinctions and the categorisations of meanings, and the latter on opening the plotting of narratives in order to reach the value structures.

A distributed, semiotic-inductive and human-Oriented approach to web-scale knowledge retrieval

Web-scale knowledge retrieval can be enabled by distributed information retrieval, clustering Web clients to a large-scale computing infrastructure for knowledge discovery from Web documents. Based on this infrastructure, we propose to apply semiotic (i.e., sub-syntactical) and inductive (i.e., probabilistic) methods for inferring concept associations in human knowledge. These associations can be combined to form a fuzzy (i.e.,gradual) semantic net representing a map of the knowledge in the Web. Thus, we propose to provide interactive visualizations of these cognitive concept maps to end users, who can browse and search the Web in a human-oriented, visual, and associative interface.

Multimodal viral communication: a social semiotic analysis of the popular Spanish "Ola ke ase" meme

Multimodality – the interdependence of semiotic resources in text – is an existential element of today’s media. The term multimodality attends systematically to the social interpretation of a wide range of communicational forms used in meaning making. A primary focus of social- semiotic multimodal analysis is on mapping how modal resources are used by people in a given social context. In November 2012 the “Ola ke ase” catchphrase, which is a play on “Hola ¿qué hace?”, appeared for the first time in Spain and immediately has been adopted as a Twitter hashtag and an image macro series. Its viral spread on social networks has been tremendous, being a trending topic in various Spanish-speaking countries. The objective of analysis is how language and image work together in the “Ola ke ase” meme. The interplay between text and image in one of the original memes and some of its variations is quantitatively analysed applying a social-semiotic approach. Results demonstrate how the “Ola ke ase” meme functions through its multimodal character and the non-standard orthography. The spread of uncountable variations of the meme shows the social process that goes on in the meaning making of the semiotic elements.

Semiotic Vision of Ideologies

A semiotic theory of systems derived from language would have the purpose of classifying all the systems of linguistic expression: philosophy, ideology, myth, poetry, art, as much as the dream, lapsus, and free association in a pluridimensional matrix that will interact with many diversified fields. In each one of these discourses it is necessary to consider a plurality of questions, the essence of which will only be comprehensible by the totality; it will be necessary to ask, in the first place, what will be the purpose of this language, what function does it fulfill and for which reason has it been constructed. The concept of World vision (WV) is introduced and its relation with Generalized Collective Conscience (GCC) and Particularized Collective Conscience. Culture implies a particular WV. Culture creates GCC. The semantic field is a structure that formalizes the units of a certain culture constituting a portion of the vision of the Reality that owns this culture. An ecological case is explained.