Investigating human audio-visual object perception with a combination of hypothesis-generating and hypothesis-testing fMRI analysis tools

Primate multisensory object perception involves distributed brain regions. To investigate the network character of these regions of the human brain, we applied data-driven group spatial independent component analysis (ICA) to a functional magnetic resonance imaging (fMRI) data set acquired during a passive audio-visual (AV) experiment with common object stimuli. We labeled three group-level independent component (IC) maps as auditory (A), visual (V), and AV, based on their spatial layouts and activation time courses. The overlap between these IC maps served as definition of a distributed network of multisensory candidate regions including superior temporal, ventral occipito-temporal, posterior parietal and prefrontal regions. During an independent second fMRI experiment, we explicitly tested their involvement in AV integration. Activations in nine out of these twelve regions met the max-criterion (A < AV > V) for multisensory integration. Comparison of this approach with a general linear model-based region-of-interest definition revealed its complementary value for multisensory neuroimaging. In conclusion, we estimated functional networks of uni- and multisensory functional connectivity from one dataset and validated their functional roles in an independent dataset. These findings demonstrate the particular value of ICA for multisensory neuroimaging research and using independent datasets to test hypotheses generated from a data-driven analysis.

Internet video telephony allows speech reading by deaf individuals and improves speech perception by cochlear implant users

OBJECTIVE To analyze speech reading through Internet video calls by profoundly hearing-impaired individuals and cochlear implant (CI) users. METHODS Speech reading skills of 14 deaf adults and 21 CI users were assessed using the Hochmair Schulz Moser (HSM) sentence test. We presented video simulations using different video resolutions (1280 × 720, 640 × 480, 320 × 240, 160 × 120 px), frame rates (30, 20, 10, 7, 5 frames per second (fps)), speech velocities (three different speakers), webcameras (Logitech Pro9000, C600 and C500) and image/sound delays (0-500 ms). All video simulations were presented with and without sound and in two screen sizes. Additionally, scores for live Skype™ video connection and live face-to-face communication were assessed. RESULTS Higher frame rate (>7 fps), higher camera resolution (>640 × 480 px) and shorter picture/sound delay (<100 ms) were associated with increased speech perception scores. Scores were strongly dependent on the speaker but were not influenced by physical properties of the camera optics or the full screen mode. There is a significant median gain of +8.5%pts (p = 0.009) in speech perception for all 21 CI-users if visual cues are additionally shown. CI users with poor open set speech perception scores (n = 11) showed the greatest benefit under combined audio-visual presentation (median speech perception +11.8%pts, p = 0.032). CONCLUSION Webcameras have the potential to improve telecommunication of hearing-impaired individuals.

Perception of co-speech gestures in aphasic patients: A visual exploration study during the observation of dyadic conversations.

BACKGROUND Co-speech gestures are part of nonverbal communication during conversations. They either support the verbal message or provide the interlocutor with additional information. Furthermore, they prompt as nonverbal cues the cooperative process of turn taking. In the present study, we investigated the influence of co-speech gestures on the perception of dyadic dialogue in aphasic patients. In particular, we analysed the impact of co-speech gestures on gaze direction (towards speaker or listener) and fixation of body parts. We hypothesized that aphasic patients, who are restricted in verbal comprehension, adapt their visual exploration strategies. METHODS Sixteen aphasic patients and 23 healthy control subjects participated in the study. Visual exploration behaviour was measured by means of a contact-free infrared eye-tracker while subjects were watching videos depicting spontaneous dialogues between two individuals. Cumulative fixation duration and mean fixation duration were calculated for the factors co-speech gesture (present and absent), gaze direction (to the speaker or to the listener), and region of interest (ROI), including hands, face, and body. RESULTS Both aphasic patients and healthy controls mainly fixated the speaker's face. We found a significant co-speech gesture × ROI interaction, indicating that the presence of a co-speech gesture encouraged subjects to look at the speaker. Further, there was a significant gaze direction × ROI × group interaction revealing that aphasic patients showed reduced cumulative fixation duration on the speaker's face compared to healthy controls. CONCLUSION Co-speech gestures guide the observer's attention towards the speaker, the source of semantic input. It is discussed whether an underlying semantic processing deficit or a deficit to integrate audio-visual information may cause aphasic patients to explore less the speaker's face.

Perception of co-speech gestures in aphasic patients: A visual exploration study during the observation of dyadic conversations

Background: Co-speech gestures are part of nonverbal communication during conversations. They either support the verbal message or provide the interlocutor with additional information. Furthermore, they prompt as nonverbal cues the cooperative process of turn taking. In the present study, we investigated the influence of co-speech gestures on the perception of dyadic dialogue in aphasic patients. In particular, we analysed the impact of co-speech gestures on gaze direction (towards speaker or listener) and fixation of body parts. We hypothesized that aphasic patients, who are restricted in verbal comprehension, adapt their visual exploration strategies. Methods: Sixteen aphasic patients and 23 healthy control subjects participated in the study. Visual exploration behaviour was measured by means of a contact-free infrared eye-tracker while subjects were watching videos depicting spontaneous dialogues between two individuals. Cumulative fixation duration and mean fixation duration were calculated for the factors co-speech gesture (present and absent), gaze direction (to the speaker or to the listener), and region of interest (ROI), including hands, face, and body. Results: Both aphasic patients and healthy controls mainly fixated the speaker’s face. We found a significant co-speech gesture x ROI interaction, indicating that the presence of a co-speech gesture encouraged subjects to look at the speaker. Further, there was a significant gaze direction x ROI x group interaction revealing that aphasic patients showed reduced cumulative fixation duration on the speaker’s face compared to healthy controls. Conclusion: Co-speech gestures guide the observer’s attention towards the speaker, the source of semantic input. It is discussed whether an underlying semantic processing deficit or a deficit to integrate audio-visual information may cause aphasic patients to explore less the speaker’s face. Keywords: Gestures, visual exploration, dialogue, aphasia, apraxia, eye movements

Comparing three CPR feedback devices and standard BLS in a single rescuer scenario: a randomised simulation study

BACKGROUND Efficiently performed basic life support (BLS) after cardiac arrest is proven to be effective. However, cardiopulmonary resuscitation (CPR) is strenuous and rescuers' performance declines rapidly over time. Audio-visual feedback devices reporting CPR quality may prevent this decline. We aimed to investigate the effect of various CPR feedback devices on CPR quality. METHODS In this open, prospective, randomised, controlled trial we compared three CPR feedback devices (PocketCPR, CPRmeter, iPhone app PocketCPR) with standard BLS without feedback in a simulated scenario. 240 trained medical students performed single rescuer BLS on a manikin for 8min. Effective compression (compressions with correct depth, pressure point and sufficient decompression) as well as compression rate, flow time fraction and ventilation parameters were compared between the four groups. RESULTS Study participants using the PocketCPR performed 17±19% effective compressions compared to 32±28% with CPRmeter, 25±27% with the iPhone app PocketCPR, and 35±30% applying standard BLS (PocketCPR vs. CPRmeter p=0.007, PocketCPR vs. standard BLS p=0.001, others: ns). PocketCPR and CPRmeter prevented a decline in effective compression over time, but overall performance in the PocketCPR group was considerably inferior to standard BLS. Compression depth and rate were within the range recommended in the guidelines in all groups. CONCLUSION While we found differences between the investigated CPR feedback devices, overall BLS quality was suboptimal in all groups. Surprisingly, effective compression was not improved by any CPR feedback device compared to standard BLS. All feedback devices caused substantial delay in starting CPR, which may worsen outcome.

Finding a voice - Nollywood inspired filmmaking practices in Switzerland

Under the name Nollywood a unique video film industry has developed in Nigeria in the last few decades, which now forms one of the world’s biggest entertainment industries. With its focus on stories reflecting „the values, desires and fears” (Haynes 2007: 133) of African viewers and its particular way of production, Nollywood brings „lived practices and its representation together in ways that make the films deeply accessible and entirely familiar to their audience“ (Marston et al. 2007: 57). In doing so, Nollywood shows its spectators new postcolonial forms of performative self‐expression and becomes a point of reference for a wide range of people. However, Nollywood not only excites a large number of viewers inside and outside Nigeria, it also inspires some of them to become active themselves and make their own films. This effect of Nigerian filmmaking can be found in many parts of sub‐Saharan Africa as well as in African diasporas all over the world – including Switzerland (Mooser 2011: 63‐66). As a source of inspiration, Nollywood and its unconventional ways of filmmaking offer African migrants a benchmark that meets their wish to express themselves as minority group in a foreign country. As Appadurai (1996: 53), Ginsburg (2003: 78) and Marks (2000: 21) assume, filmmakers with a migratory background have a specific need to express themselves through media. As minority group members in their country of residence they not only wish to reflect upon their situation within the diaspora and illustrate their everyday struggles as foreigners, but to also express their own views and ideas in order to challenge dominant public opinion (Ginsburg 2003: 78). They attempt to “talk back to the structures of power” (2003: 78) they live in. In this process, their audio-visual works become a means of response and “an answering echo to a previous presentation or representation” (Mitchell 1994: 421). The American art historian Mitchell, therefore, suggests interpreting representation as “the relay mechanism in exchange of power, value, and publicity” (1994: 420). This desire of interacting with the local public has also been expressed during a film project of African, mainly Nigerian, first-generation migrants in Switzerland I am currently partnering in. Several cast and crew members have expressed feelings of being under-represented, even misrepresented, in the dominant Swiss media discourse. In order to create a form of exchange and give themselves a voice, they consequently produce a Nollywood inspired film and wish to present it to the society they live in. My partnership in this on‐going film production (which forms the foundation of my PhD field study) allows me to observe and experience this process. By employing qualitative media anthropological methods and in particular Performance Ethnography, I seek to find out more about the ways African migrants represent themselves as a community through audio‐visual media and the effect the transnational use of Nollywood has on their form of self‐representations as well as the ways they express themselves.

Job-shadowing: Swiss health librarians observing experienced search specialists and information skills trainers in London

Abstract As librarians of the Social & Preventive Medicine Library in Bern, we help researchers perform systematic literature searches and teach students to use medical databases. We developed our skills mainly “on the job”, and we wondered how other health librarians in Europe were trained to become experts in searching. We had a great opportunity to “job shadow” specialists in this area of library service during a 5-day-internship at the Royal Free Hospital Medical Library in London, Great Britain.

Short structured feedback training is equivalent to a mechanical feedback device in two-rescuer BLS: a randomised simulation study.

BACKGROUND Resuscitation guidelines encourage the use of cardiopulmonary resuscitation (CPR) feedback devices implying better outcomes after sudden cardiac arrest. Whether effective continuous feedback could also be given verbally by a second rescuer ("human feedback") has not been investigated yet. We, therefore, compared the effect of human feedback to a CPR feedback device. METHODS In an open, prospective, randomised, controlled trial, we compared CPR performance of three groups of medical students in a two-rescuer scenario. Group "sCPR" was taught standard BLS without continuous feedback, serving as control. Group "mfCPR" was taught BLS with mechanical audio-visual feedback (HeartStart MRx with Q-CPR-Technology™). Group "hfCPR" was taught standard BLS with human feedback. Afterwards, 326 medical students performed two-rescuer BLS on a manikin for 8 min. CPR quality parameters, such as "effective compression ratio" (ECR: compressions with correct hand position, depth and complete decompression multiplied by flow-time fraction), and other compression, ventilation and time-related parameters were assessed for all groups. RESULTS ECR was comparable between the hfCPR and the mfCPR group (0.33 vs. 0.35, p = 0.435). The hfCPR group needed less time until starting chest compressions (2 vs. 8 s, p < 0.001) and showed fewer incorrect decompressions (26 vs. 33 %, p = 0.044). On the other hand, absolute hands-off time was higher in the hfCPR group (67 vs. 60 s, p = 0.021). CONCLUSIONS The quality of CPR with human feedback or by using a mechanical audio-visual feedback device was similar. Further studies should investigate whether extended human feedback training could further increase CPR quality at comparable costs for training.

Misleading Population Estimates: Biases and Consistency of Visual Surveys and Matrix Modelling in the Endangered Bearded Vulture

Conservation strategies for long-lived vertebrates require accurate estimates of parameters relative to the populations' size, numbers of non-breeding individuals (the “cryptic” fraction of the population) and the age structure. Frequently, visual survey techniques are used to make these estimates but the accuracy of these approaches is questionable, mainly because of the existence of numerous potential biases. Here we compare data on population trends and age structure in a bearded vulture (Gypaetus barbatus) population from visual surveys performed at supplementary feeding stations with data derived from population matrix-modelling approximations. Our results suggest that visual surveys overestimate the number of immature (<2 years old) birds, whereas subadults (3–5 y.o.) and adults (>6 y.o.) were underestimated in comparison with the predictions of a population model using a stable-age distribution. In addition, we found that visual surveys did not provide conclusive information on true variations in the size of the focal population. Our results suggest that although long-term studies (i.e. population matrix modelling based on capture-recapture procedures) are a more time-consuming method, they provide more reliable and robust estimates of population parameters needed in designing and applying conservation strategies. The findings shown here are likely transferable to the management and conservation of other long-lived vertebrate populations that share similar life-history traits and ecological requirements.

A semi-structured interview to assess visual hallucinations in older people

OBJECTIVE: Visual hallucinations are under-reported by patients and are often undiscovered by health professionals. There is no gold standard available to assess hallucinations. Our objective was to develop a reliable, valid, semi-structured interview for identifying and assessing visual hallucinations in older people with eye disease and cognitive impairment. METHODS: We piloted the North-East Visual Hallucinations Interview (NEVHI) in 80 older people with visual and/or cognitive impairment (patient group) and 34 older people without known risks of hallucinations (control group). The informants of 11 patients were interviewed separately. We established face validity, content validity, criterion validity, inter-rater agreement and the internal consistency of the NEVHI, and assessed the factor structure for questions evaluating emotions, cognitions, and behaviours associated with hallucinations. RESULTS: Recurrent visual hallucinations were common in the patient group (68.8%) and absent in controls (0%). The criterion, face and content validities were good and the internal consistency of screening questions for hallucinations was high (Cronbach alpha: 0.71). The inter-rater agreements for simple and complex hallucinations were good (Kappa 0.72 and 0.83, respectively). Four factors associated with experiencing hallucinations (perceived control, pleasantness, distress and awareness) were identified and explained a total variance of 73%. Informants gave more 'don't know answers' than patients throughout the interview (p = 0.008), especially to questions evaluating cognitions and emotions associated with hallucinations (p = 0.02). CONCLUSIONS: NEVHI is a comprehensive assessment tool, helpful to identify the presence of visual hallucinations and to quantify cognitions, emotions and behaviours associated with hallucinations.

The James Lind Library: explaining and illustrating the evolution of fair tests of medical treatments

The James Lind Library (www.jameslindlibrary.org) has been established to improve public and professional general knowledge about fair tests of treatments in healthcare and their history. Its foundation was laid ten years ago at the Royal College of Physicians of Edinburgh, and its administrative centre is in the College's Sibbald Library, one of the most important collections of historic medical manuscripts, papers and books in the world. The James Lind Library is a website that introduces visitors to the principles of fair tests of treatments, with a series of short, illustrated essays, which are currently available in English, Arabic, Chinese, French, Portuguese, Russian and Spanish. A 100-page book-- Testing Treatments--is now available free through the website, both in English and in Arabic and Spanish translations. To illustrate the evolution of ideas related to fair tests of treatments from 2000 BC to the present, the James Lind Library contains key passages and images from manuscripts, books and journal articles, many of them accompanied by commentaries, biographies, portraits and other relevant documents and images, including audio and video files. New material is being added to the website continuously, as relevant new records are identified and as methods for testing treatments evolve. A multinational, multilingual editorial team oversees the development of the website, which currently receives tens of thousands of visitors every month.

The Eyes Have It: Regulatory and Structural Changes Both Underlie Cichlid Visual Pigment Diversity

A major goal of evolutionary biology is to unravel the molecular genetic mechanisms that underlie functional diversification and adaptation. We investigated how changes in gene regulation and coding sequence contribute to sensory diversification in two replicate radiations of cichlid fishes. In the clear waters of Lake Malawi, differential opsin expression generates diverse visual systems, with sensitivities extending from the ultraviolet to the red regions of the spectrum. These sensitivities fall into three distinct clusters and are correlated with foraging habits. In the turbid waters of Lake Victoria, visual sensitivity is constrained to longer wavelengths, and opsin expression is correlated with ambient light. In addition to regulatory changes, we found that the opsins coding for the shortest-and longest-wavelength visual pigments have elevated numbers of potentially functional substitutions. Thus, we present a model of sensory evolution in which both molecular genetic mechanisms work in concert. Changes in gene expression generate large shifts in visual pigment sensitivity across the collective opsin spectral range, but changes in coding sequence appear to fine-tune visual pigment sensitivity at the short-and long-wavelength ends of this range, where differential opsin expression can no longer extend visual pigment sensitivity.

The Frontal Eye Field Is Involved in Visual Vector Inversion in Humans – A Theta Burst Stimulation Study

In the antisaccade task, subjects are requested to suppress a reflexive saccade towards a visual target and to perform a saccade towards the opposite side. In addition, in order to reproduce an accurate saccadic amplitude, the visual saccade vector (i.e., the distance between a central fixation point and the peripheral target) must be exactly inverted from one visual hemifield to the other. Results from recent studies using a correlational approach (i.e., fMRI, MEG) suggest that not only the posterior parietal cortex (PPC) but also the frontal eye field (FEF) might play an important role in such a visual vector inversion process. In order to assess whether the FEF contributes to visual vector inversion, we applied an interference approach with continuous theta burst stimulation (cTBS) during a memory-guided antisaccade task. In 10 healthy subjects, one train of cTBS was applied over the right FEF prior to a memory-guided antisaccade task. In comparison to the performance without stimulation or with sham stimulation, cTBS over the right FEF induced a hypometric gain for rightward but not leftward antisaccades. These results obtained with an interference approach confirm that the FEF is also involved in the process of visual vector inversion.