Development of Multi-modal and Super-resolved Retinal Imaging Systems

Advancements in retinal imaging technologies have drastically improved the quality of eye care in the past couple decades. Scanning laser ophthalmoscopy (SLO) and optical coherence tomography (OCT) are two examples of critical imaging modalities for the diagnosis of retinal pathologies. However current-generation SLO and OCT systems have limitations in diagnostic capability due to the following factors: the use of bulky tabletop systems, monochromatic imaging, and resolution degradation due to ocular aberrations and diffraction.

Bulky tabletop SLO and OCT systems are incapable of imaging patients that are supine, under anesthesia, or otherwise unable to maintain the required posture and fixation. Monochromatic SLO and OCT imaging prevents the identification of various color-specific diagnostic markers visible with color fundus photography like those of neovascular age-related macular degeneration. Resolution degradation due to ocular aberrations and diffraction has prevented the imaging of photoreceptors close to the fovea without the use of adaptive optics (AO), which require bulky and expensive components that limit the potential for widespread clinical use.

In this dissertation, techniques for extending the diagnostic capability of SLO and OCT systems are developed. These techniques include design strategies for miniaturizing and combining SLO and OCT to permit multi-modal, lightweight handheld probes to extend high quality retinal imaging to pediatric eye care. In addition, a method for extending true color retinal imaging to SLO to enable high-contrast, depth-resolved, high-fidelity color fundus imaging is demonstrated using a supercontinuum light source. Finally, the development and combination of SLO with a super-resolution confocal microscopy technique known as optical photon reassignment (OPRA) is demonstrated to enable high-resolution imaging of retinal photoreceptors without the use of adaptive optics.

Multi-modal ultrasound imaging for breast cancer detection

This work describes preliminary results of a two-modality imaging system aimed at the early detection of breast cancer. The first technique is based on compounding conventional echographic images taken at regular angular intervals around the imaged breast. The other modality obtains tomographic images of propagation velocity using the same circular geometry. For this study, a low-cost prototype has been built. It is based on a pair of opposed 128-element, 3.2 MHz array transducers that are mechanically moved around tissue mimicking phantoms. Compounded images around 360 degrees provide improved resolution, clutter reduction, artifact suppression and reinforce the visualization of internal structures. However, refraction at the skin interface must be corrected for an accurate image compounding process. This is achieved by estimation of the interface geometry followed by computing the internal ray paths. On the other hand, sound velocity tomographic images from time of flight projections have been also obtained. Two reconstruction methods, Filtered Back Projection (FBP) and 2D Ordered Subset Expectation Maximization (2D OSEM), were used as a first attempt towards tomographic reconstruction. These methods yield useable images in short computational times that can be considered as initial estimates in subsequent more complex methods of ultrasound image reconstruction. These images may be effective to differentiate malignant and benign masses and are very promising for breast cancer screening. (C) 2015 The Authors. Published by Elsevier B.V.

User experience with immersive and interactive media = Experiencia de usuario en entornos 3D inmersivos e interactivos

Estudios recientes promueven la integración de estímulos multisensoriales en activos multimedia con el fin de mejorar la experiencia de usuario mediante la estimulación de nuevos sentidos, más allá de la tradicional experiencia audiovisual. Del mismo modo, varios trabajos proponen la introducción de componentes de interacción capaces de complementar con nuevas características, funcionalidades y/o información la experiencia multimedia. Efectos sensoriales basados en el uso de nuevas técnicas de audio, olores, viento, vibraciones y control de la iluminación, han demostrado tener un impacto favorable en la sensación de Presencia, en el disfrute de la experiencia multimedia y en la calidad, relevancia y realismo de la misma percibidos por el usuario. Asimismo, los servicios basados en dos pantallas y la manipulación directa de (elementos en) la escena de video tienen el potencial de mejorar la comprensión, la concentración y la implicación proactiva del usuario en la experiencia multimedia. El deporte se encuentra entre los géneros con mayor potencial para integrar y explotar éstas soluciones tecnológicas. Trabajos previos han demostrado asimismo la viabilidad técnica de integrar éstas tecnologías con los estándares actualmente adoptados a lo largo de toda la cadena de transmisión de televisión. De este modo, los sistemas multimedia enriquecidos con efectos sensoriales, los servicios interactivos multiplataforma y un mayor control del usuario sobre la escena de vídeo emergen como nuevas formas de llevar la multimedia immersiva e interactiva al mercado de consumo de forma no disruptiva. Sin embargo, existen numerosas interrogantes relativas a los efectos sensoriales y/o soluciones interactivas más adecuadas para complementar un contenido audiovisual determinado o a la mejor manera de de integrar y combinar dichos componentes para mejorar la experiencia de usuario de un segmento de audiencia objetivo. Además, la evidencia científica sobre el impacto de factores humanos en la experiencia de usuario con estas nuevas formas de immersión e interacción en el contexto multimedia es aún insuficiente y en ocasiones, contradictoria. Así, el papel de éstos factores en el potencial de adopción de éstas tecnologías ha sido amplia-mente ignorado. La presente tesis analiza el impacto del audio binaural, efectos sensoriales (de iluminación y olfativos), interacción con objetos 3D integrados en la escena de vídeo e interacción con contenido adicional utilizando una segunda pantalla en la experiencia de usuario con contenidos de deporte. La posible influencia de dichos componentes en las variables dependientes se explora tanto a nivel global (efecto promedio) como en función de las características de los usuarios (efectos heterogéneos). Para ello, se ha llevado a cabo un experimento con usuarios orientado a explorar la influencia de éstos componentes immersivos e interactivos en dos grandes dimensiones de la experiencia multimedia: calidad y Presencia. La calidad de la experiencia multimedia se analiza en términos de las posibles variaciones asociadas a la calidad global y a la calidad del contenido, la imagen, el audio, los efectos sensoriales, la interacción con objetos 3D y la interacción con la segunda pantalla. El posible impacto en la Presencia considera dos de las dimensiones definidas por el cuestionario ITC-SOPI: Presencia Espacial (Spatial Presence) e Implicación (Engagement). Por último, los individuos son caracterizados teniendo en cuenta los siguientes atributos afectivos, cognitivos y conductuales: preferencias y hábitos en relación con el contenido, grado de conocimiento de las tecnologías integradas en el sistema, tendencia a involucrarse emocionalmente, tendencia a concentrarse en una actividad bloqueando estímulos externos y los cinco grandes rasgos de la personalidad: extroversión, amabilidad, responsabilidad, inestabilidad emocional y apertura a nuevas experiencias. A nivel global, nuestro estudio revela que los participantes prefieren el audio binaural frente al sistema estéreo y que los efectos sensoriales generan un aumento significativo del nivel de Presencia Espacial percibido por los usuarios. Además, las manipulaciones experimentales realizadas permitieron identificar una gran variedad de efectos heterogéneos. Un resultado interesante es que dichos efectos no se encuentran distribuidos de forma equitativa entre las medidas de calidad y Presencia. Nuestros datos revelan un impacto generalizado del audio binaural en la mayoría de las medidas de calidad y Presencia analizadas. En cambio, la influencia de los efectos sensoriales y de la interacción con la segunda pantalla se concentran en las medidas de Presencia y calidad, respectivamente. La magnitud de los efectos heterogéneos identificados está modulada por las siguientes características personales: preferencias en relación con el contenido, frecuencia con la que el usuario suele ver contenido similar, conocimiento de las tecnologías integradas en el demostrador, sexo, tendencia a involucrarse emocionalmente, tendencia a a concentrarse en una actividad bloqueando estímulos externos y niveles de amabilidad, responsabilidad y apertura a nuevas experiencias. Las características personales consideradas en nuestro experimento explicaron la mayor parte de la variación en las variables dependientes, confirmando así el importante (y frecuentemente ignorado) papel de las diferencias individuales en la experiencia multimedia. Entre las características de los usuarios con un impacto más generalizado se encuentran las preferencias en relación con el contenido, el grado de conocimiento de las tecnologías integradas en el sistema y la tendencia a involucrarse emocionalmente. En particular, los primeros dos factores parecen generar un conflicto de atención hacia el contenido versus las características/elementos técnicos del sistema, respectivamente. Asimismo, la experiencia multimedia de los fans del fútbol parece estar modulada por procesos emociona-les, mientras que para los no-fans predominan los procesos cognitivos, en particular aquellos directamente relacionados con la percepción de calidad. Abstract Recent studies encourage the integration of multi-sensorial stimuli into multimedia assets to enhance the user experience by stimulating other senses beyond sight and hearing. Similarly, the introduction of multi-modal interaction components complementing with new features, functionalities and/or information the multimedia experience is promoted. Sensory effects as odor, wind, vibration and light effects, as well as an enhanced audio quality, have been found to favour media enjoyment and to have a positive influence on the sense of Presence and on the perceived quality, relevance and reality of a multimedia experience. Two-screen services and a direct manipulation of (elements in) the video scene have the potential to enhance user comprehension, engagement and proactive involvement of/in the media experience. Sports is among the genres that could benefit the most from these solutions. Previous works have demonstrated the technical feasibility of implementing and deploying end-to-end solutions integrating these technologies into legacy systems. Thus, sensorially-enhanced media, two-screen services and an increased user control over the displayed scene emerge as means to deliver a new form of immersive and interactive media experiences to the mass market in a non-disruptive manner. However, many questions remain concerning issues as the specific interactive solutions or sensory effects that can better complement a given audiovisual content or the best way in which to integrate and combine them to enhance the user experience of a target audience segment. Furthermore, scientific evidence on the impact of human factors on the user experience with these new forms of immersive and interactive media is still insufficient and sometimes, contradictory. Thus, the role of these factors on the potential adoption of these technologies has been widely ignored. This thesis analyzes the impact of binaural audio, sensory (light and olfactory) effects, interaction with 3D objects integrated into the video scene and interaction with additional content using a second screen on the sports media experience. The potential influence of these components on the dependent variables is explored both at the overall level (average effect) and as a function of users’ characteristics (heterogeneous effects). To these aims, we conducted an experimental study exploring the influence of these immersive and interactive elements on the quality and Presence dimensions of the media experience. Along the quality dimension, we look for possible variations on the quality scores as-signed to the overall media experience and to the media components content, image, audio, sensory effects, interaction with 3D objects and interaction using the tablet device. The potential impact on Presence is analyzed by looking at two of the four dimensions defined by the ITC-SOPI questionnaire, namely Spatial Presence and Engagement. The users’ characteristics considered encompass the following personal affective, cognitive and behavioral attributes: preferences and habits in relation to the content, knowledge of the involved technologies, tendency to get emotionally involved and tendency to get absorbed in an activity and block out external distractors and the big five personality traits extraversion, agreeableness, conscientiousness, neuroticism and openness to experience. At the overall level, we found that participants preferred binaural audio than standard stereo audio and that sensory effects increase significantly the level of Spatial Presence. Several heterogeneous effects were also revealed as a result of our experimental manipulations. Interestingly, these effects were not equally distributed across the quality and Presence measures analyzed. Whereas binaural audio was foud to have an influence on the majority of the quality and Presence measures considered, the effects of sensory effects and of interaction with additional content through the tablet device concentrate mainly on the dimensions of Presence and on quality measures, respectively. The magnitude of these effects was modulated by individual’s characteristics, such as: preferences in relation to the content, frequency of viewing similar content, knowledge of involved technologies, gender, tendency to get emotionally involved, tendency to absorption and levels of agreeableness, conscientiousness and openness to experience. The personal characteristics collected in our experiment explained most of the variation in the dependent variables, confirming the frequently neglected role of individual differences on the media experience. Preferences in relation to the content, knowledge of involved technologies and tendency to get emotionally involved were among the user variables with the most generalized influence. In particular, the former two features seem to present a conflict in the allocation of attentional resources towards the media content versus the technical features of the system, respectively. Additionally, football fans’ experience seems to be modulated by emotional processes whereas for not fans, cognitive processes (and in particular those related to quality judgment) prevail.

Interaction design for teaching visually impaired students

This paper reports on the findings for a study on improving interaction design for teaching visually impaired students. The crux of the problem is the ability to draw and understand diagrams. The cognitive issues are often underestimated with insufficient attention being given to the use of metaphors, etc. and "one size fits all solutions" are often the norm. The findings of the original seed funded project have led to design criteria and to an application for a large scale project, to produce generic tools and to enable multi-modal teaching and learning, with connotations for the mentally as well as physically impaired.

The functional subdivision of the visual brain : Is there a real illusion effect on action? A multi-lab replication study

Acknowledgements We thank Brian Roberts and Mike Harris for responding to our questions regarding their paper; Zoltan Dienes for advice on Bayes factors; Denise Fischer, Melanie Römer, Ioana Stanciu, Aleksandra Romanczuk, Stefano Uccelli, Nuria Martos Sánchez, and Rosa María Beño Ruiz de la Sierra for help collecting data; Eva Viviani for managing data collection in Parma. We thank Maurizio Gentilucci for letting us use his lab, and the Centro Intradipartimentale Mente e Cervello (CIMeC), University of Trento, and especially Francesco Pavani for lending us his motion tracking equipment. We thank Rachel Foster for proofreading. KKK was supported by a Ph.D. scholarship as part of a grant to VHF within the International Graduate Research Training Group on Cross-Modal Interaction in Natural and Artificial Cognitive Systems (CINACS; DFG IKG-1247) and TS by a grant (DFG – SCHE 735/3-1); both from the German Research Council.

UGI: a multi-dimensional ultrasonic-based interaction approach

We are currently witnessing an era where interaction with computers is no longer limited to conventional methods (i.e. keyboard and mouse). Human Computer Interaction (HCI) as a progressive field of research, has opened up alternatives to the traditional interaction techniques. Embedded Infrared (IR) sensors, Accelerometers and RGBD cameras have become common inputs for devices to recognize gestures and body movements. These sensors are vision based and as a result the devices that incorporate them will be reliant on presence of light. Ultrasonic sensors on the other hand do not suffer this limitation as they utilize properties of sound waves. These sensors however, have been mainly used for distance detection and not with HCI devices. This paper presents our approach in developing a multi-dimensional interaction input method and tool Ultrasonic Gesture-based Interaction (UGI) that utilizes ultrasonic sensors. We demonstrate how these sensors can detect object movements and recognize gestures. We present our approach in building the device and demonstrate sample interactions with it. We have also conducted a user study to evaluate our tool and its distance and micro gesture detection accuracy. This paper reports these results and outlines our future work in the area.

Visual voice activity detection using frontal versus profile views

Visual activity detection of lip movements can be used to overcome the poor performance of voice activity detection based solely in the audio domain, particularly in noisy acoustic conditions. However, most of the research conducted in visual voice activity detection (VVAD) has neglected addressing variabilities in the visual domain such as viewpoint variation. In this paper we investigate the effectiveness of the visual information from the speaker’s frontal and profile views (i.e left and right side views) for the task of VVAD. As far as we are aware, our work constitutes the first real attempt to study this problem. We describe our visual front end approach and the Gaussian mixture model (GMM) based VVAD framework, and report the experimental results using the freely available CUAVE database. The experimental results show that VVAD is indeed possible from profile views and we give a quantitative comparison of VVAD based on frontal and profile views The results presented are useful in the development of multi-modal Human Machine Interaction (HMI) using a single camera, where the speaker’s face may not always be frontal.

“A problem of versions” : laying down the law in the school playground

Purpose This chapter investigates an episode where a supervising teacher on playground duty asks two boys to each give an account of their actions over an incident that had just occurred on some climbing equipment in the playground. Methodology This paper employs an ethnomethodological approach using conversation analysis. The data are taken from a corpus of video recorded interactions of children, aged 7-9 years, and the teacher, in school playgrounds during the lunch recess. Findings The findings show the ways that children work up accounts of their playground practices when asked by the teacher. The teacher initially provided interactional space for each child to give their version of the events. Ultimately, the teacher’s version of how to act in the playground became the sanctioned one. The children and the teacher formulated particular social orders of behavior in the playground through multi-modal devices, direct reported speech and scripts. Such public displays of talk work as socialization practices that frame teacher-sanctioned morally appropriate actions in the playground. Value of paper This chapter shows the pervasiveness of the teacher’s social order, as she presented an institutional social order of how to interact in the playground, showing clearly the disjunction of adult-child orders between the teacher and children.

Street reads - Creating mobile walking events

By 2012 mobile devices had become the main interface for people to access information about anything from their current GPS position to the latest book reviews. What was less accessible were tools and techniques for writers to leverage this new technology to construct and distribute located stories. This project began with a series of master classes for local Brisbane writers to demonstrate processes and techniques for imagining, constructing and distributing stories. Most significantly, this project equipped writers with how to identify and adopt various mobile services and applications to research, produce and deliver packaged multi-modal content for readers to access and experience stories in the very locations from which they were inspired. Four stories by four writers were selected to be developed and published as location-based events in four different neighbourhoods across Brisbane. These writers were mentored throughout the writing process and a model was developed for them to simply upload several multi-modal chapters for access on location by readers using QR codes. These activities culminated in a major 25 day event presented by Brisbane City Council and supported by Brisbane Writers Festival and Queensland Writers Centre. The 'Street Reads' event presented the four stories on location in Cannon Hill, Darra, Toowong and West End. The significance of the Street Reads project went beyond extending the capacity for writers to access mobile technologies as a new platform for distributing stories. This event also motivated readers to travel to neighbourhoods to experience them in ways that had not previously imagined possible. These located stories were fictionalisations of actual events and characters that have current and historic importance to these places. These histories are hidden from view and yet can provide locals and visitors with a new found appreciation for the past and set an example for how neighbourhoods can become active stages for the sharing of stories inspiring a deeper connection with each other and an agency for participating in the development of the identity of the local places they inhabit together. Due to the success of the project and by employing more advanced tools now available, Street reads has been further developed by Brisbane City Council and is now available as a the Story City App available for download at itunes.

IdeaWall: Mixed mode distributed collaboration in enterprise environments

In early stages of design and modeling, computers and computer applications are often considered an obstacle, rather than a facilitator of the process. Most notably, brainstorms, process modeling with business experts, or development planning, are often performed by a team in front of a whiteboard. While "whiteboarding" is recognized as an effective tool, low-tech solutions that allow remote participants to contribute are still not generally available. This is a striking observation, considering that vast majority of teams in large organizations are distributed teams. And this has also been one of the key triggers behind the project described in this article, where a team of corporate researchers decided to identify state of the art technologies that could facilitate the scenario mentioned above. This paper is an account of a research project in the area of enterprise collaboration, with a strong focus on the aspects of human computer interaction in mixed mode environments, especially in areas of collaboration where computers still play a secondary role. It is describing a currently running corporate research project. © 2012 Springer-Verlag.

EU Multi-level Governance in the Making - The Community Initiative LEADER+ in Finland and Germany

In order to fully understand the process of European integration it is of paramount importance to consider developments at the sub-national and local level. EU integration scholars shifted their attention to the local level only at the beginning of the 1990s with the concept of multi-level governance (MLG). While MLG is the first concept to scrutinise the position of local levels of public administration and other actors within the EU polity, I perceive it as too optimistic in the degree of influence it ascribes to local levels. Thus, learning from and combining MLG with other concepts, such as structural constructivism, helps to reveal some of the hidden aspects of EU integration and paint a more realistic picture of multi-level interaction. This thesis also answers the call for more case studies in order to conceptualise MLG further. After a critical study of theories and concepts of European integration, above all, MLG, I will analyse sub-national and local government in Finland and Germany. I show how the sub-national level and local governments are embedded in the EU s multi-level structure of governance and how, through EU integration, those levels have been empowered but also how their scope of action has partially decreased. After theoretical and institutional contextualisation, I present the results of my empirical study of the EU s Community Initiative LEADER+. LEADER stands for Liaison Entre Actions de Développement de l'Économie Rurale , and aims at improving the economic conditions in Europe s rural areas. I was interested in how different actors construct and shape EU financed rural development, especially in how local actors organised in so-called local action groups (LAGs) cooperate with other administrative units within the LEADER+ administrative chain. I also examined intra-institutional relations within those groups, in order to find out who are the most influential and powerful actors within them. Empirical data on the Finnish and German LAGs was first gathered through a survey, which was then supplemented and completed by interviewing LAG members, LAG-managers, several civil servants from Finnish and German decision-making and managing authorities and a civil servant from the EU Commission. My main argument is that in both Germany and Finland, the Community Initiative LEADER+ offered a space for multi-level interaction and local-level involvement, a space that on the one hand consists of highly motivated people actively contributing to the improvement of the quality of life and economy in Europe s countryside but which is dependent and also restricted by national administrative practices, implementation approaches and cultures on the other. In Finland, the principle of tri-partition (kolmikantaperiaatte) in organising the executive committees of LAGs is very noticeable. In comparison to Germany, for instance, the representation of public administration in those committees is much more limited due to this principle. Furthermore, the mobilisation of local residents and the bringing together of actors from the local area with different social and institutional backgrounds to become an active part of LEADER+ was more successful in Finland than in Germany. Tri-partition as applied in Finland should serve as a model for similar policies in other EU member states. EU integration changed the formal and informal inter-institutional relations linking the different levels of government. The third sector including non-governmental institutions and interest groups gained access to policy-making processes and increasingly interact with government institutions at all levels of public administration. These developments do not necessarily result in the empowering of the local level.

An Event Driven Fusion Approach for Enjoyment Recognition in Real-time

Social signals and interpretation of carried information is of high importance in Human Computer Interaction. Often used for affect recognition, the cues within these signals are displayed in various modalities. Fusion of multi-modal signals is a natural and interesting way to improve automatic classification of emotions transported in social signals. Throughout most present studies, uni-modal affect recognition as well as multi-modal fusion, decisions are forced for fixed annotation segments across all modalities. In this paper, we investigate the less prevalent approach of event driven fusion, which indirectly accumulates asynchronous events in all modalities for final predictions. We present a fusion approach, handling short-timed events in a vector space, which is of special interest for real-time applications. We compare results of segmentation based uni-modal classification and fusion schemes to the event driven fusion approach. The evaluation is carried out via detection of enjoyment-episodes within the audiovisual Belfast Story-Telling Corpus.

Using fuction-based CBT with parent involvement to treat OCD in two school-age children with high-functioning autism

Obsessive Compulsive Disorder (OCD) involves excessive worry coupled with engaging in rituals that are believed to help alleviate the worry. Pervasive Developmental Disorders (PODs) are characterized by impairments in social interaction, communication, and the presence of repetitive and/or restrictive behaviours (American Psychiatric Association, 2000). Research suggests that as many as 81% of children with a POD also meet criteria for a diagnosis ofOCD. Currently, only a handful of studies have investigated the use of Cognitive Behavioural Therapy (CBT) in treating OCD in children with autism (Reaven & Hepburn, 2003 ; Sze & Wood, 2007; Lehmkuhl, Storch, Bodtish & Geflken, 2008). In these case studies. the use of a multi-modal CBT treatment package was successful in alleviating OCD behaviours. The current study used function-based CBT with parent involvement and behavioural supplements to treat 2 children with POD and OCD. Using a multiple baseline design across behaviours and participants, parents reported that their child 's anxiety was alleviated and these gains were maintained at 6-month follow-up. According to results of the Children 's Yale-Brown Obsessive Compulsive Scale (Goodman, Price, Rasmussen, Riddle, & Rapoport, 1986) from preto post-test, OCD behaviours of the children decreased II"om the severe to the mild range. In addition, the parents rated the family's level of interference related to their child 's OCD as substantially lower. Last, the CBT treatment received high ratings of consumer satisfaction.

Advances in Discourse Studies

Methods of approaching the study of discourse have developed rapidly in the last ten years, influenced by a growing interdisciplinary spirit among linguistics and anthropology, sociology, cognitive and cultural psychology and cultural studies, as well as among established sub-fields within linguistics itself. Among the more recent developments are an increasing ‘critical’ turn in discourse analysis, a growing interest in historical, ethnographic and corpus-based approaches to discourse, more concern with the social contexts in which discourse occurs, the social actions that it is used to take and the identities that are constructed through it, as well as a revaluation of what counts as ‘discourse’ to include multi-modal texts and interaction. Advances in Discourse Studies brings together contributions from leading scholars in the field, investigating the historical and theoretical relationships between new advances in discourse studies and pointing towards new directions for the future of the discipline. Featuring discussion questions, classroom projects and recommended readings at the end of each section, as well as case studies illustrating each approach discussed, this is an invaluable resource for students of interdisciplinary discourse analysis.