Testo e memoria tra semiotica e scienze cognitive: una situazione sperimentale intorno al ricordo di un film

The research project presented in this dissertation is about text and memory. The title of the work is "Text and memory between Semiotics and Cognitive Science: an experimental setting about remembering a movie". The object of the research is the relationship between texts or "textuality" - using a more general semiotic term - and memory. The goal is to analyze the link between those semiotic artifacts that a culture defines as autonomous meaningful objects - namely texts - and the cognitive performance of memory that allows to remember them. An active dialogue between Semiotics and Cognitive Science is the theoretical paradigm in which this research is set, the major intend is to establish a productive alignment between the "theory of text" developed in Semiotics and the "theory of memory" outlined in Cognitive Science. In particular the research is an attempt to study how human subjects remember and/or misremember a film, as a specific case study; in semiotics, films are “cinematographic texts”. The research is based on the production of a corpus of data gained through the qualitative method of interviewing. After an initial screening of a fulllength feature film each participant of the experiment has been interviewed twice, according to a pre-established set of questions. The first interview immediately after the screening: the subsequent, follow-up interview three months from screening. The purpose of this design is to elicit two types of recall from the participants. In order to conduce a comparative inquiry, three films have been used in the experimental setting. Each film has been watched by thirteen subjects, that have been interviewed twice. The corpus of data is then made by seventy-eight interviews. The present dissertation displays the results of the investigation of these interviews. It is divided into six main parts. Chapter one presents a theoretical framework about the two main issues: memory and text. The issue of the memory is introduced through many recherches drown up in the field of Cognitive Science and Neuroscience. It is developed, at the same time, a possible relationship with a semiotic approach. The theoretical debate about textuality, characterizing the field of Semiotics, is examined in the same chapter. Chapter two deals with methodology, showing the process of definition of the whole method used for production of the corpus of data. The interview is explored in detail: how it is born, what are the expected results, what are the main underlying hypothesis. In Chapter three the investigation of the answers given by the spectators starts. It is examined the phenomenon of the outstanding details of the process of remembering, trying to define them in a semiotic way. Moreover there is an investigation of the most remembered scenes in the movie. Chapter four considers how the spectators deal with the whole narrative. At the same time it is examined what they think about the global meaning of the film. Chapter five is about affects. It tries to define the role of emotions in the process of comprehension and remembering. Chapter six presents a study of how the spectators account for a single scene of the movie. The complete work offers a broad perspective about the semiotic issue of textuality, using both a semiotic competence and a cognitive one. At the same time it presents a new outlook on the issue of memory, opening several direction of research.

Macchine esperienziali: Narrativa, cognizione e atto di lettura

La mia tesi si riallaccia al dibattito teorico-letterario contemporaneo sulla possibilità di un approccio cognitivo alla narrativa e alla letteratura in particolare. Essa si propone di esplorare il rapporto tra narrazione ed esperienza, ridefinendo il concetto di “esperienzialità” della narrativa introdotto da Monika Fludernik nel suo Towards a “Natural” Narratology (1996). A differenza di Fludernik, che ha identificato l’esperienzialità con la rappresentazione dell’esperienza dei personaggi, la mia trattazione assegna un ruolo di primo piano al lettore, cercando di rispondere alla domanda: perché leggere una storia è – o si costituisce come – un’esperienza? L’intuizione dietro tutto ciò è che le teorizzazioni dell’esperienza e della coscienza nella filosofia della mente degli ultimi venti anni possano gettare luce sull’interazione tra lettori e testi narrativi. Il mio punto di riferimento principale è la scienza cognitiva “di seconda generazione”, secondo cui l’esperienza è un relazionarsi attivo e corporeo al mondo. La prima parte del mio studio è dedicata all’intreccio tra la narrativa e quello che chiamo lo “sfondo esperienziale” di ogni lettore, un repertorio di esperienze già note ai lettori attraverso ripetute interazioni con il mondo fisico e socio-culturale. Mi soffermo inoltre sul modo in cui relazionarsi a un testo narrativo può causare cambiamenti e slittamenti in questo sfondo esperienziale, incidendo sulla visione del mondo del lettore. Mi rivolgo poi al coinvolgimento corporeo del lettore, mostrando che la narrativa può attingere allo sfondo esperienziale dei suoi fruitori anche sul piano dell’esperienza di base: le simulazioni corporee della percezione contribuiscono alla nostra comprensione delle storie, incidendo sia sulla ricostruzione dello spazio dell’ambientazione sia sulla relazione intersoggettiva tra lettori e personaggi. Infine, mi occupo del rapporto tra l’esperienza della lettura e la pratica critico-letteraria dell’interpretazione, sostenendo che – lungi dal costituire due modalità opposte di fruizione dei testi – esse sono intimamente connesse.

Ai confini del senso. La schizofrenia tra semiotica, psicopatologia e scienze cognitive

La schizofrenia rappresenta uno dei più grandi enigmi per l’impresa conoscitiva umana: non si conosce la sua eziologia, né le sue basi biologiche e cerebrali. Non è neanche chiaro cosa accada nell’esperienza di chi ne soffre, che sembra vivere in un mondo altro. La scarsa conoscenza dell’esperienza schizofrenica e la distanza tra questa e il senso comune hanno portato molti studiosi a inquadrare questo disturbo come illogico, irrazionale, insensato. Il presente lavoro tenta di confutare tale impostazione, mostrando come il mondo di senso dello schizofrenico si altera, non si disgrega; si trasforma, non si annulla. Il campo di studi all’interno del quale si colloca la ricerca è la semiotica, disciplina che studia i sistemi e i processi di significazione e i modi attraverso cui l’essere umano dà senso al mondo. L’intera indagine è inserita in un quadro interdisciplinare in costante dialogo con la psicopatologia fenomenologica e le scienze cognitive contemporanee, e si sviluppa a partire da numerosi testi autobiografici di pazienti schizofrenici, report psichiatrici, articoli di giornale, film e romanzi sul tema. L’ipotesi su cui si muove il lavoro è che sia possibile comprendere la schizofrenia come un problema costitutivamente semiotico, il cui nucleo è da rintracciarsi in una radicale metamorfosi delle modalità di produrre e interpretare il significato. La scommessa sottesa è che la semiotica possa contribuire in modo sostanziale alla comprensione delle modalità attraverso cui la nostra cultura concettualizza la schizofrenia e dei modi in cui gli schizofrenici danno senso al mondo. Il lavoro indaga, quindi, i legami tra schizofrenia e cultura, la storia del concetto nosografico, e le alterazioni dei processi di significazione nei casi di eloquio disorganizzato, nei racconti autobiografici e nei deliri, cercando anche di fornire strumenti utili alla pratica clinica.

Mind the gaps: cognitive-inspired AI for high-level visual sensemaking. Towards abstract concept image classification

The abundance of visual data and the push for robust AI are driving the need for automated visual sensemaking. Computer Vision (CV) faces growing demand for models that can discern not only what images "represent," but also what they "evoke." This is a demand for tools mimicking human perception at a high semantic level, categorizing images based on concepts like freedom, danger, or safety. However, automating this process is challenging due to entropy, scarcity, subjectivity, and ethical considerations. These challenges not only impact performance but also underscore the critical need for interoperability. This dissertation focuses on abstract concept-based (AC) image classification, guided by three technical principles: situated grounding, performance enhancement, and interpretability. We introduce ART-stract, a novel dataset of cultural images annotated with ACs, serving as the foundation for a series of experiments across four key domains: assessing the effectiveness of the end-to-end DL paradigm, exploring cognitive-inspired semantic intermediaries, incorporating cultural and commonsense aspects, and neuro-symbolic integration of sensory-perceptual data with cognitive-based knowledge. Our results demonstrate that integrating CV approaches with semantic technologies yields methods that surpass the current state of the art in AC image classification, outperforming the end-to-end deep vision paradigm. The results emphasize the role semantic technologies can play in developing both effective and interpretable systems, through the capturing, situating, and reasoning over knowledge related to visual data. Furthermore, this dissertation explores the complex interplay between technical and socio-technical factors. By merging technical expertise with an understanding of human and societal aspects, we advocate for responsible labeling and training practices in visual media. These insights and techniques not only advance efforts in CV and explainable artificial intelligence but also propel us toward an era of AI development that harmonizes technical prowess with deep awareness of its human and societal implications.

Bridging the gap:exploring the cognitive impact of InterpretBank on Chinese interpreting trainees

This exploratory research project developed a cognitive situated approach to studying aspects of simultaneous interpreting with quantitative, confirmatory methods. To do so, it explored how to determine the potential benefits of using a computer-assisted interpreting tool, InterpretBank, among 22 Chinese interpreting trainees with Chinese L1 and English L2. The informants were mostly 2nd-year female students with an average age of 24.7 enrolled in Chinese MA interpreting programs. The study adopted a pretest and posttest design with three cycles. The independent variable was using Excel or InterpretBank. After Cycle I (pre-test), the sample split into control (Excel) and experimental (InterpretBank) groups. Tool choice was compulsory in Cycle II but not Cycle III. The source materials for each cycle were pairs of matching transcripts from popular science podcasts. Informants compiled glossaries out of one transcript, while the other one was edited for simultaneous interpreting, with 39 terms as potential problem triggers. Quantitative profiling results showed that InterpretBank informants spent less time on glossary compilation, generated more terms faster than Excel informants, but their glossaries were less diverse (personal) and longer. The booth tasks yielded no significant differences in fluency indicators except for more bumps (200-600ms silent time gaps) for InterpretBank in Cycle II. InterpretBank informants had more correct renditions in Cycles II and III but there was no statistically significant difference among accuracy indicators per cycle. Holistic quality assessments by PhD raters showed InterpretBank consistently outperforming Excel, suggesting a positive InterpretBank impact on SI quality. However, some InterpretBank implementations raised cognitive ergonomic concerns for Chinese, potentially undermining its utility. Overall, results were mixed regarding InterpretBank benefits for Chinese trainees, but the project was successful in developing cognitive situated interpreting study methods, constructs and indicators.