Language, action, and sensoriality: a multimodal analysis of interactions in inclusive sport climbing with visually impaired athletes

In sport climbing, athletes with vision impairments are constantly accompanied by their guides – usually trainers – both during the preparatory inspection of the routes and whilst climbing. Trainers are, so to speak, the climbers’ eyes, in the sense that they systematically put their vision in the service of the climbers’ mobility and sporting performance. The synergy between trainers and athletes is based on peculiar, strictly multimodal interactive practices that are focused on the body and on its constantly evolving sensory engagement with the materiality of routes. In this context, sensory perception and embodied actions required to plan and execute the climb are configured as genuinely interactive accomplishments. Drawing on the theoretical framework of Embodied and Situated Cognition and on the methodology of Conversation Analysis, this thesis engages in the multimodal analysis of trainer-athlete interactions in paraclimbing. The analysis is based on a corpus of video recorded climbing sessions. The major findings of the study can be summarized as follows. 1) Intercorporeality is key to interactions between trainers and athletes with visual impairments. The participants orient to perceiving the climbing space and acting in it as a ‘We’. 2) The grammar, lexicon, prosody, and timing of the trainers’ instructions are finely tuned to the ongoing corporeal experience of the climbers. 3) Climbers with visual impairments build their actions by using sensory resources that are provided by their trainers. This result is of particular importance as it shows that resources and constraints for action are in a fundamental way constituted in interaction with Others and with specific socio-material ecologies, rather than being defined a priori by the organs and functions of individuals’ body and mind. Individual capabilities are thus enhanced and extended in interaction, which encourages a more ecological view of (dis)ability.

Towards Unstructured Knowledge Integration in Natural Language Processing

In the last decades, Artificial Intelligence has witnessed multiple breakthroughs in deep learning. In particular, purely data-driven approaches have opened to a wide variety of successful applications due to the large availability of data. Nonetheless, the integration of prior knowledge is still required to compensate for specific issues like lack of generalization from limited data, fairness, robustness, and biases. In this thesis, we analyze the methodology of integrating knowledge into deep learning models in the field of Natural Language Processing (NLP). We start by remarking on the importance of knowledge integration. We highlight the possible shortcomings of these approaches and investigate the implications of integrating unstructured textual knowledge. We introduce Unstructured Knowledge Integration (UKI) as the process of integrating unstructured knowledge into machine learning models. We discuss UKI in the field of NLP, where knowledge is represented in a natural language format. We identify UKI as a complex process comprised of multiple sub-processes, different knowledge types, and knowledge integration properties to guarantee. We remark on the challenges of integrating unstructured textual knowledge and bridge connections with well-known research areas in NLP. We provide a unified vision of structured knowledge extraction (KE) and UKI by identifying KE as a sub-process of UKI. We investigate some challenging scenarios where structured knowledge is not a feasible prior assumption and formulate each task from the point of view of UKI. We adopt simple yet effective neural architectures and discuss the challenges of such an approach. Finally, we identify KE as a form of symbolic representation. From this perspective, we remark on the need of defining sophisticated UKI processes to verify the validity of knowledge integration. To this end, we foresee frameworks capable of combining symbolic and sub-symbolic representations for learning as a solution.