Some Reflections on the Emergence of Multimodal Assemblages of Meaning

In this commentary I reflect upon the conceptualisation of human meaning-making, utilised in the two target articles, that relies heavily on speech as the main mode of semiosis and considers time only in its chronological form. Instead I argue that human existence is embodied and lived through multiple modalities, and involves not only sequential experience of time, but also experience of emergence. In order to move towards a conception of meaning-making that takes this into account, I introduce the social-semiotic theory of multimodality (Kress 2010) and discuss notions of ‘real duration’ (Bergson 1907/1998) and ‘lived time’ (Martin-Vallas 2009). I argue that dialogical (idiographic) researchers need to develop analytic and methodological tools that allow exploring the emergence of multimodal assemblages of meaning in addition to trying to avoid the monologisation of complex dynamic dialogical phenomena.

Decolonizing digital heritage practices for Indigenous literacy: A multimodal analysis of iPad Tellagami videos

Purpose Drawing on multimodal texts produced by an Indigenous school community in Australia, we apply critical race theory and multimodal analysis to decolonize digital heritage practices for Indigenous students. This study focuses on the particular ways in which students’ counter-­‐narratives about race were embedded in multimodal and digital design in the development of a digital cultural heritage (Giaccardi, 2012). Pedagogies that explore counter-­‐narratives of cultural heritage in the official curriculum can encourage students to reframe their own racial identity, while challenging dominant white, historical narratives of colonial conquest, race, and power (Gutierrez, 2008). The children’s digital “Gami” videos, created with the iPad application, Tellagami, enabled the students to imagine hybrid, digital social identities and perspectives of Australian history that were tied to their Indigenous cultural heritage (Kamberelis, 2001).

The YOU CALL-WE CALL randomized clinical trial: Impact of a multimodal support intervention after a mild stroke

Background Comparison of a multimodal intervention WE CALL (study initiated phone support/information provision) versus a passive intervention YOU CALL (participant can contact a resource person) in individuals with first mild stroke. Methods and Results This study is a single-blinded randomized clinical trial. Primary outcome includes unplanned use of health services (participant diaries) for adverse events and quality of life (Euroquol-5D, Quality of Life Index). Secondary outcomes include planned use of health services (diaries), mood (Beck Depression Inventory II), and participation (Assessment of Life Habits [LIFE-H]). Blind assessments were done at baseline, 6, and 12 months. A mixed model approach for statistical analysis on an intention-to-treat basis was used where the group factor was intervention type and occasion factor time, with a significance level of 0.01. We enrolled 186 patients (WE=92; YOU=94) with a mean age of 62.5±12.5 years, and 42.5% were women. No significant differences were seen between groups at 6 months for any outcomes with both groups improving from baseline on all measures (effect sizes ranged from 0.25 to 0.7). The only significant change for both groups from 6 months to 1 year (n=139) was in the social domains of the LIFE-H (increment in score, 0.4/9±1.3 [95% confidence interval, 0.1–0.7]; effect size, 0.3). Qualitatively, the WE CALL intervention was perceived as reassuring, increased insight, and problem solving while decreasing anxiety. Only 6 of 94 (6.4%) YOU CALL participants availed themselves of the intervention. Conclusions Although the 2 groups improved equally over time, WE CALL intervention was perceived as helpful, whereas YOU CALL intervention was not used.

Study protocol of the YOU CALL - WE CALL TRIAL: Impact of a multimodal support intervention after a "mild" stroke

Background More than 60% of new strokes each year are "mild" in severity and this proportion is expected to rise in the years to come. Within our current health care system those with "mild" stroke are typically discharged home within days, without further referral to health or rehabilitation services other than advice to see their family physician. Those with mild stroke often have limited access to support from health professionals with stroke-specific knowledge who would typically provide critical information on topics such as secondary stroke prevention, community reintegration, medication counselling and problem solving with regard to specific concerns that arise. Isolation and lack of knowledge may lead to a worsening of health problems including stroke recurrence and unnecessary and costly health care utilization. The purpose of this study is to assess the effectiveness, for individuals who experience a first "mild" stroke, of a sustainable, low cost, multimodal support intervention (comprising information, education and telephone support) - "WE CALL" compared to a passive intervention (providing the name and phone number of a resource person available if they feel the need to) - "YOU CALL", on two primary outcomes: unplanned-use of health services for negative events and quality of life. Method/Design We will recruit 384 adults who meet inclusion criteria for a first mild stroke across six Canadian sites. Baseline measures will be taken within the first month after stroke onset. Participants will be stratified according to comorbidity level and randomised to one of two groups: YOU CALL or WE CALL. Both interventions will be offered over a six months period. Primary outcomes include unplanned use of heath services for negative event (frequency calendar) and quality of life (EQ-5D and Quality of Life Index). Secondary outcomes include participation level (LIFE-H), depression (Beck Depression Inventory II) and use of health services for health promotion or prevention (frequency calendar). Blind assessors will gather data at mid-intervention, end of intervention and one year follow up. Discussion If effective, this multimodal intervention could be delivered in both urban and rural environments. For example, existing infrastructure such as regional stroke centers and existing secondary stroke prevention clinics, make this intervention, if effective, deliverable and sustainable.

Changes in disability, physical/mental health states and quality of life during an 8-Week multimodal physiotherapy programme in patients with chronic non-specific neck pain: A prospective cohort study

Aim The aim of this study was to analyse the effect of an 8-week multimodal physiotherapy programme (MPP), integrating physical land-based therapeutic exercise (TE), adapted swimming and health education, as a treatment for patients with chronic non-specific neck pain (CNSNP), on disability, general health/mental states and quality of life. Methods 175 CNSNP patients from a community-based centre were recruited to participate in this prospective study. Intervention: 60-minute session (30 minutes of land-based exercise dedicated to improving mobility, motor control, resistance and strengthening of the neck muscles, and 30 minutes of adapted swimming with aerobic exercise keeping a neutral neck position using a snorkel). Health education was provided using a decalogue on CNSNP and constant repetition of brief advice by the physiotherapist during the supervision of the exercises in each session. Study outcomes: primary: disability (Neck Disability Index); secondary: physical and mental health states and quality of life of patients (SF-12 and EuroQoL-5D respectively). Differences between baseline data and that at the 8-week follow-up were calculated for all outcome variables. Results Disability showed a significant improvement of 24.6% from a mean (SD) of 28.2 (13.08) at baseline to 16.88 (11.62) at the end of the 8-week intervention. All secondary outcome variables were observed to show significant, clinically relevant improvements with increase ranges between 13.0% and 16.3% from a mean of 0.70 (0.2) at baseline to 0.83 (0.2), for EuroQoL-5D, and from a mean of 40.6 (12.7) at baseline to 56.9 (9.5), for mental health state, at the end of the 8-week intervention. Conclusion After 8 weeks of a MPP that integrated land-based physical TE, health education and adapted swimming, clinically-relevant and statistically-significant improvements were observed for disability, physical and mental health states and quality of life in patients who suffer CNSNP. The clinical efficacy requires verification using a randomised controlled study design.

Indigenous ways with literacies: Transgenerational, multimodal, placed, and collective

This research describes some of the salient features of Indigenous ways of working with multimodal literacies in digital contexts of use that emerged within an Indigenous school community with the oversight of Aboriginal Elders. This is significant because the use of multimodal literacy practices among a growing number of Indigenous school community groups has not been an emphasis of multimodal literacy research to date. Furthermore, authentic examples of Indigenous multimodal texts are often difficult to locate within Euro-centric educational systems of postcolonial countries. The research was conducted over a full year with students from middle and upper primary (aged 8.5–12.5 years) in an Indigenous Independent school in South-East Queensland, Australia. The project applied participatory research methods in which the research agenda was negotiated with the cultural community. Indigenous ways of multimodal literacy practices emerged as transgenerational, multimodal, placed, and collective. The findings have implications for teachers and researchers to re-envisage Indigenous ways of multimodal literacy practices in the digital age.

Multimodal imaging of the collagen and elastic fibre networks in the bovine intervertebral disc

INTRODUCTION. The intervertebral disc is the largest avascular structure in the human body, withstanding transient loads of up to nine times body weight during rigorous physical activity. The key structural elements of the disc are a gel-like nucleus pulposus surrounded by concentric lamellar rings containing criss-crossed collagen fibres. The disc also contains an elastic fiber network which has been suggested to play a structural role, but to date the relationship between the collagen and elastic fiber networks is unclear. CONCLUSION. The multimodal transmitted and reflected polarized light microscopy technique developed here allows clear differentiation between the collagen and elastic fiber networks of the intervertebral disc. The ability to image unstained specimens avoids concerns with uneven stain penetration or specificity of staining. In bovine tail discs, the elastic fiber network is intimately associated with the collagen network.

Synthesis of a multimodal molecular imaging probe based on a hyperbranched polymer architecture

Molecular imaging is utilised in modern medicine to aid in the diagnosis and treatment of disease by allowing its spatiotemporal state to be examined in vivo. This study focuses on the development of novel multimodal molecular imaging agents based on hyperbranched polymers that combine the complementary capabilities of optical fluorescence imaging and positron emission tomography-computed tomography (PET/CT) into one construct. RAFT-mediated polymerisation was used to prepare two hydrophilic hyperbranched polymers that were differentiated by their size and level of branching. The multiple functional end-groups facilitated covalent attachment of both near infrared fluorescent dyes for optical imaging, as well as a copper chelator allowing binding of 64Cu as a PET radio nuclei. In vivo multimodal imaging of mice using PET/CT and planar optical imaging was first used to assess the biodistribution of the polymeric materials and it was shown that the larger and more branched polymer had a significantly longer circulation time. The larger constructs were also shown to exhibit enhanced accumulation in solid tumours in a murine B16 melanoma model. Importantly, it was demonstrated that the PET modality gave rise to high sensitivity immediately after injection of the agent, while the optical modality facilitated extended longitudinal studies, thus highlighting how the complementary capabilities of the molecular imaging agents can be useful for studying various diseases, including cancer.

Multimodal polymer nanoparticles with combined 19F magnetic resonance and optical detection for tunable, targeted, multimodal imagingin vivo

Understanding the complex nature of diseased tissue in vivo requires development of more advanced nanomedicines, where synthesis of multifunctional polymers combines imaging multimodality with a biocompatible, tunable, and functional nanomaterial carrier. Here we describe the development of polymeric nanoparticles for multimodal imaging of disease states in vivo. The nanoparticle design utilizes the abundant functionality and tunable physicochemical properties of synthetically robust polymeric systems to facilitate targeted imaging of tumors in mice. For the first time, high-resolution 19F/1H magnetic resonance imaging is combined with sensitive and versatile fluorescence imaging in a polymeric material for in vivo detection of tumors. We highlight how control over the chemistry during synthesis allows manipulation of nanoparticle size and function and can lead to very high targeting efficiency to B16 melanoma cells, both in vitro and in vivo. Importantly, the combination of imaging modalities within a polymeric nanoparticle provides information on the tumor mass across various size scales in vivo, from millimeters down to tens of micrometers.

Critically deconstructing race through a multimodal analysis of iPad Tellagami videos

Drawing on multimodal texts produced by an Indigenous school community in Australia, I apply critical race theory and multimodal analysis (Jewitt, 2011) to decolonise digital heritage practices for Indigenous students. This study focuses on the particular ways in which students’ counter-narratives about race were embedded in multimodal and digital design in the development of a digital cultural heritage (Giaccardi, 2012). Data analysis involved applying multimodal analysis to the students’ Gamis, following social semiotic categories and principles theorised by Kress and Bezemer (2008), and Jewitt (2006, 2011). This includes attending to the following semiotic elements: visual design, movement and gesture, gaze, and recorded speech, and their interrelationships. The analysis also draws on critical race theory to interpret the students’ representations of race. In particular, the multimodal texts were analysed as a site for students’ views of Indigenous oppression in relation to the colonial powers and ownership of the land in Australian history (Ladson-Billings, 2009). Pedagogies that explore counter-narratives of cultural heritage in the official curriculum can encourage students to reframe their own racial identity, while challenging dominant white, historical narratives of colonial conquest, race, and power (Gutierrez, 2008). The children’s multimodal “Gami” videos, created with the iPad application, Tellagami, enabled the students to imagine hybrid, digital social identities and perspectives of Australian history that were tied to their Indigenous cultural heritage (Kamberelis, 2001).

Valuing Indigenous heritage through multimodal literacy

The study on which this presentation is based focuses on the particular ways in which students’ counter-narratives about race were embedded in multimodal and digital design in the development of a digital cultural heritage. The multimodal texts were analysed as a site for students’ views of Indigenous oppression in relation to the colonial powers and ownership of the land in Australian history. In this presentation, Kathy will demonstrate how pedagogies that explore counter-narratives of cultural heritage in the official curriculum can encourage students to reframe their own racial identity, while challenging dominant white, historical narratives of colonial conquest, race, and power. In the second part of this session, Indigenous Principal, John Davis and teachers from HymbaYumba Community Hub will provide a school-based, Indigenous panel to inspire educators with authentic ways to embed Indigenous knowledge in the curriculum.

2D-3D Face Recognition Based on PCA and Feature Modelling

Hybrid face recognition, using image (2D) and structural (3D) information, has explored the fusion of Nearest Neighbour classifiers. This paper examines the effectiveness of feature modelling for each individual modality, 2D and 3D. Furthermore, it is demonstrated that the fusion of feature modelling techniques for the 2D and 3D modalities yields performance improvements over the individual classifiers. By fusing the feature modelling classifiers for each modality with equal weights the average Equal Error Rate improves from 12.60% for the 2D classifier and 12.10% for the 3D classifier to 7.38% for the Hybrid 2D+3D clasiffier.

here/there/then/now

here/there/then/now was a practice-led research project that brought together 10 independent artists in dance, music, theatre and visual/media arts to create a site-specific program within the walls of the Brisbane Powerhouse. The purpose was to explore how to best conceive flexible performance platforms, theatricalise site-specific work and engage new audiences through forms of promenade experience that could provide open choices on how and where to view it. The sold out season of 6 performances, which took place 14-19 May 2002, presented three discrete performance installations set in intimate parts of the building, each with their own aesthetic and communicative intention, culminating in a fourth in-theatre installation, where memories of the first three coalesced and were reinterrogated. Each site thereby investigated meaning-making via the moving body and its critical relationship with space and objects, in a dramatic re-contextualisation of traditional solo dance forms, now re-articulated through interdisciplinary practices. The benefit of this approach was the creation of a layered and multimodal experience that could be both shared and subsequently critiqued by performers and audience alike.