A multimodal and adaptive real-time human -computer interface for peoples with severe motor disability

This dissertation introduces the design of a multimodal, adaptive real-time assistive system as an alternate human computer interface that can be used by individuals with severe motor disabilities. The proposed design is based on the integration of a remote eye-gaze tracking system, voice recognition software, and a virtual keyboard. The methodology relies on a user profile that customizes eye gaze tracking using neural networks. The user profiling feature facilitates the notion of universal access to computing resources for a wide range of applications such as web browsing, email, word processing and editing. ^ The study is significant in terms of the integration of key algorithms to yield an adaptable and multimodal interface. The contributions of this dissertation stem from the following accomplishments: (a) establishment of the data transport mechanism between the eye-gaze system and the host computer yielding to a significantly low failure rate of 0.9%; (b) accurate translation of eye data into cursor movement through congregate steps which conclude with calibrated cursor coordinates using an improved conversion function; resulting in an average reduction of 70% of the disparity between the point of gaze and the actual position of the mouse cursor, compared with initial findings; (c) use of both a moving average and a trained neural network in order to minimize the jitter of the mouse cursor, which yield an average jittering reduction of 35%; (d) introduction of a new mathematical methodology to measure the degree of jittering of the mouse trajectory; (e) embedding an onscreen keyboard to facilitate text entry, and a graphical interface that is used to generate user profiles for system adaptability. ^ The adaptability nature of the interface is achieved through the establishment of user profiles, which may contain the jittering and voice characteristics of a particular user as well as a customized list of the most commonly used words ordered according to the user's preferences: in alphabetical or statistical order. This allows the system to successfully provide the capability of interacting with a computer. Every time any of the sub-system is retrained, the accuracy of the interface response improves even more. ^

(Re)criando (mini)contos de terror: uma experiência de ensino de leitura e escrita de gênero multimodal

Given the multiplicity of languages and media present in contemporary texts, the work with digital genres characterizes itself as essential for teaching, reading and writing. Virtual media is already present in many dayly activities that require the use of language. This shows that the globalized world brings new demands of literacy and various reading practices. Given this perspective, we propose to work the multiliteracies present in the new texts from the enunciative discourse Bakhtinian assumptions. For this, we chose to be as the object of research / intervention, the horror flash fiction multimodal discursive genre, because it is a multissemiótico digital statement of virtual circulation. In this context, this study aimed to understand how the teaching of this kind can contribute to the development of knowledge related to reading and text production, required by multiliteracies, by performing a Didactic Sequence in the classroom, specifically for two classes of elementary school, 7th and 8th grades of public school. The research was based on Bakhtin's theory and the Circle (2009, 2011) on gender perspective in a dialogic and the proposed Dolz and Schneuwly (2004) for the text of teaching through sequences Teaching. We also use the precepts of multiliteracies focused on Rojo (2012, 2013). The methodology used was based on a qualitative approach. We consider the analysis of minicontos produced by students, it's own hibridism of multiliteracies, the discursive characteristics such as composition, style and subject content, in addition to relations dialogicity present in these statements. At the end of the study, we realized that our intervention contributed to the expansion of knowledge of the subjects involved related to reading and multimodal genre production.

Beat that word : How listeners integrate beat gesture and focus in multimodal speech discourse

Peer reviewed

Beat that Word : How Listeners Integrate Beat Gesture and Focus in Multimodal Speech Discourse

Peer reviewed

Aerodynamic Testing of Variable Message Signs

The increasing nationwide interest in intelligent transportation systems (ITS) and the need for more efficient transportation have led to the expanding use of variable message sign (VMS) technology. VMS panels are substantially heavier than flat panel aluminum signs and have a larger depth (dimension parallel to the direction of traffic). The additional weight and depth can have a significant effect on the aerodynamic forces and inertial loads transmitted to the support structure. The wind induced drag forces and the response of VMS structures is not well understood. Minimum design requirements for VMS structures are contained in the American Association of State Highway Transportation Officials Standard Specification for Structural Support for Highway Signs, Luminaires, and Traffic Signals (AASHTO Specification). However the Specification does not take into account the prismatic geometry of VMS and the complex interaction of the applied aerodynamic forces to the support structure. In view of the lack of code guidance and the limited number research performed so far, targeted experimentation and large scale testing was conducted at the Florida International University (FIU) Wall of Wind (WOW) to provide reliable drag coefficients and investigate the aerodynamic instability of VMS. A comprehensive range of VMS geometries was tested in turbulence representative of the high frequency end of the spectrum in a simulated suburban atmospheric boundary layer. The mean normal, lateral and vertical lift force coefficients, in addition to the twisting moment coefficient and eccentricity ratio, were determined using the measured data for each model. Wind tunnel testing confirmed that drag on a prismatic VMS is smaller than the 1.7 suggested value in the current AASHTO Specification (2013). An alternative to the AASHTO Specification code value is presented in the form of a design matrix. Testing and analysis also indicated that vortex shedding oscillations and galloping instability could be significant for VMS signs with a large depth ratio attached to a structure with a low natural frequency. The effect of corner modification was investigated by testing models with chamfered and rounded corners. Results demonstrated an additional decrease in the drag coefficient but a possible Reynolds number dependency for the rounded corner configuration.

The Material Dynamics of a London-French Blog: A Multimodal Reading of Migrant Habitus

Through a fine-grained reading of a London-French blog, this article aims to shed light on the lived experience of the French community in London. The ethnosemiotic conceptual framework brings together ethnographic and semiotic schools of thought, focusing in particular on Pierre Bourdieu’s concept of habitus and Gunther Kress’s multimodal social semiotic analytical model. Habitus is broken down into its material manifestations of habitat, habit and habituation, all displayed in the blog and revealing of the blogger’s identity and positioning within the migration setting. As all modes are considered to be of equal semiotic potential, equivalent emphasis is placed on the multiple modes of meaning-making present in the blog, such as layout, colour, typography and language. By examining the dynamic relationships between blogger and audience, subjectivity and objectivity, on-line and on-land habitus, and intermodal dynamics themselves, through the prism of multimodality, hidden facets of the blogger’s cultural identity and sense of community belonging within the diasporic context begin to materialise.

Multimodal Education for Very Young Learners

Invited Plenary Speaker

Deep Head Pose: Gaze-Direction Estimation in Multimodal Video

In this paper we present a convolutional neuralnetwork (CNN)-based model for human head pose estimation inlow-resolution multi-modal RGB-D data. We pose the problemas one of classification of human gazing direction. We furtherfine-tune a regressor based on the learned deep classifier. Next wecombine the two models (classification and regression) to estimateapproximate regression confidence. We present state-of-the-artresults in datasets that span the range of high-resolution humanrobot interaction (close up faces plus depth information) data tochallenging low resolution outdoor surveillance data. We buildupon our robust head-pose estimation and further introduce anew visual attention model to recover interaction with theenvironment. Using this probabilistic model, we show thatmany higher level scene understanding like human-human/sceneinteraction detection can be achieved. Our solution runs inreal-time on commercial hardware

Multimodal attention system for an interactive robot

[EN]Social robots are receiving much interest in the robotics community. The most important goal for such robots lies in their interaction capabilities. An attention system is crucial, both as a filter to center the robot’s perceptual resources and as a mean of letting the observer know that the robot has intentionality. In this paper a simple but flexible and functional attentional model is described. The model, which has been implemented in an interactive robot currently under development, fuses both visual and auditive information extracted from the robot’s environment, and can incorporate knowledge-based influences on attention.

Mémoire de licence La réception du message donné dans Le Petit Prince : Une étude sur la réception et l’interprétation du message donné dans Le Petit Prince par Antoine de Saint-Exupéry chez l’adulte et l’enfant.

Le présent travail est une étude sur Le Petit Prince, d’Antoine de Saint-Exupéry. Le but principal du mémoire est d’analyser la réception du message chez le lecteur adulte et chez le lecteur enfant. Nous avons analysé comment l’enfant et l’adulte interprètent le message donné par l’oeuvre, parsemée d’images et d’allégories, des citations qui son devenues symboliques avec le temps. Ensuite, nous avons analysé les personnages dans le but d’interpréter leur sens symbolique. Pour réaliser cette recherche, nous avons regardé plusieurs interprétations allégoriques. Les résultats de notre étude nous ont permis d’affirmer que la réception du message n’est pas la même chez le lecteur adulte et le lecteur enfant. Les résultats de cette étude montrent enfin qu’il est trop difficile pour le lecteur enfant de comprendre le sens moral d’une image allégorique, tandis que le lecteur adulte est plus apte à interpréter l’image allégorique en s’appuyant sur son expérience de la lecture et de son expérience de la vie quotidienne. L’auteur utilise l’image symbolique dans la littérature pour montrer avec plus de clarté la condition humaine. En conclusion nous apprenons que Le Petit Prince nous fait réfléchir sur la vie de la façon plus profonde et philosophique.

Clinton Multimodal (U. S. 30 Mississippi River Bridge) Study, 2003

The purpose of this memorandum is to document the benefit-cost analysis of the river crossing concept alternatives described in the "Concept Alternatives Technical Memo." Benefit-cost studies are designed to measure, in dollars, the potential positive or negative impacts of large-scale construction projects. The concept alternatives analyzed include improvements to the Union Pacific Railroad (UPRR) River Crossing and the U.S. Highway 30 (U.S. 30) River Crossing.

Evaluation of Mobile Telephone Text Message Reminders for People with Antipsychotic Medication

Nonadherence to treatment is a worldwide problem among people with severe mental disorders. Patient treatment adherence may be supported with simple reminding methods e.g. text message reminders. However, there is limited evidence of its benefits. Intervention evaluation is essential in mHealth research. Therefore, this evaluative study was conducted. This study aimed to evaluate text message reminder use in encouraging patients’treatment adherence among people with antipsychotic medication. The data were collected between September 2011 and December 2013. First, a systematic literature review revealed that text message reminders were widely used in healthcare. However, its impacts were conflicting. Second, a sub-sample (n = 562) analysis showed that patients preferred humorous text message reminders and preferred to receive them in the morning, at the beginning of the week. Age, gender and marital status seemed to have different effects on the preferred amount and timing of the selected reminders. Third, a cross-sectional survey revealed that people with antipsychotic medication (n = 408) expressed overall satisfaction towards the reminder system. Finally, the evaluative design showed that patient recruitment for a randomized controlled trial concerning people with antipsychotic medication was challenging due to low rates of eligible participants. Follow-up drop-out rates varied depending on the data collection method. Participants’ demographic characteristics were associated with the risk of dropping out from the trial. This study suggests that text messages are a potential reminder system in healthcare services among people with antipsychotic medication. More research is needed to gain a comprehensive picture of the impacts and effectiveness of text message reminders.

You ain´t seen nothing yet. : Transparency’s (lack of) Effect on Source and Message Credibility

Transparency has been proposed to both change the way journalism is being produced and increase its credibility. However, little research has been conducted to assess the connection between transparency and credibility. This study utilizes an experimental setting (N=1320) to measure what impact transparency have on source and message credibility from the user perspective. The results reveals an almost absence of any transparency effect on both source and message credibility although some small significant effects could be observed primarily regarding internal hyperlinks, comments and contextual information. Although further research is desperately needed in this area the study suggest that transparency does not affect the credibility of journalism in the eyes of the contemporary audience and thus have limited appeal as a new norm in journalism.

The Voice of History and the Message of the National Curriculum: Recontextualising History to a Pedagogic Discourse for Upper Secondary VET

In the reform by the liberal-conservative government of Swedish upper secondary education in 2011, history was recognized as an important part of citizenship education and was introduced into the curriculum for vocational education and training (VET) tracks. Through the concepts of classification and framing, this article explores the process of constructing the history syllabus for VET. The data consist of archived material from the working group responsible for the history curriculum under the Swedish National Agency for Education. The analysis shows that there are competing discourses concerning the relative emphasis on competencies and skills and concerning the emphasis on contemporary and modern history. Although historians, history teachers and other agents are invited to respond to the content of the curriculum, the respondents have no influence on the knowledge structure of the curriculum, which is controlled by agents of the dominant educational ideology. From a critical perspective, this article suggests that the curriculum reflects the instrumental and neoconservative message of the reform through strong classification and framing and through the emphasis on general abilities and a contemporary history that has a more direct explanatory value to contemporary society.