Untersuchungen zur Visuomotorik des Menschen

Motorische Bewegungen werden über die visuelle Rückmeldung auf ihre Genauigkeit kontrolliert und ggf. korrigiert. Über einen technischen Eingriff, wie beispielsweise einer Prismenbrille, kann man eine Differenz zwischen optisch wahrgenommener und haptisch erlebter Umwelt erzeugen, um die Fähigkeiten des visuomotorischen Systems zu testen. In dieser Arbeit wurde eine computergestützte Methode entwickelt, eine solche visuomotorische Differenz zu simulieren. Die Versuchspersonen führen eine ballistische Bewegung mit Arm und Hand aus in der Absicht, ein vorgegebenes Ziel zu treffen. Die Trefferpunkte werden durch einen Computer mit Hilfe eines Digitalisierungstablettes aufgenommen. Die visuelle Umwelt, welche den Versuchspersonen präsentiert wird, ist auf einem Monitor dargestellt. Das Monitorabbild – ein Kreuz auf weißem Hintergrund – betrachten die Testpersonen über einen Spiegel. Dieser ist in einem entsprechenden Winkel zwischen Monitor und Digitalisierungstablett angebracht, so dass das Zielbild auf dem Digitalisierungstablett projiziert wird. Die Testpersonen nehmen das Zielkreuz auf dem Digitalisierungstablett liegend wahr. Führt die Versuchsperson eine Zielbewegung aus, können die aufgenommenen Koordinaten als Punkte auf dem Monitor dargestellt werden und die Testperson erhält über diese Punktanzeige ein visuelles Feedback ihrer Bewegung. Der Arbeitsbereich des Digitalisierungstabletts kann über den Computer eingerichtet und so motorische Verschiebungen simuliert werden. Die verschiedenartigen Möglichkeiten dieses Aufbaus wurden zum Teil in Vorversuchen getestet um Fragestellungen, Methodik und technische Einrichtungen aufeinander abzustimmen. Den Hauptversuchen galt besonderes Interesse an der zeitlichen Verzögerung des visuellen Feedbacks sowie dem intermanuellen Transfer. Hierbei ergaben sich folgende Ergebnisse: ● Die Versuchspersonen adaptieren an eine räumlich verschobene Umwelt. Der Adaptationsverlauf lässt sich mit einer Exponentialfunktion mathematisch berechnen und darstellen. ● Dieser Verlauf ist unabhängig von der Art des visuellen Feedbacks. Die Beobachtung der Handbewegung während der Adaptation zeigt die gleiche Zielabfolge wie eine einfache Punktprojektion, die den Trefferort der Bewegung darstellt. ● Der exponentielle Verlauf der Adaptationsbewegung ist unabhängig von den getesteten zeitlichen Verzögerungen des visuellen Feedbacks. ● Die Ergebnisse des Folgeeffektes zeigen, dass bei zunehmender zeitlicher Verzögerung des visuellen Feedbacks während der Adaptationsphase, die Größe des Folgeeffektwertes geringer wird, d.h. die anhaltende Anpassungsleistung an eine visuomotorische Differenz sinkt. ● Die Folgeeffekte weisen individuelle Eigenheiten auf. Die Testpersonen adaptieren verschieden stark an eine simulierte Verschiebung. Ein Vergleich mit den visuomotorischen Herausforderungen im Vorleben der Versuchspersonen ließ vermuten, dass das visuomotorische System des Menschen trainierbar ist und sich - je nach Trainingszustand – unterschiedlich an wahrgenommene Differenzen anpasst. ● Der intermanuelle Transfer konnte unter verschiedenen Bedingungen nachgewiesen werden. ● Ein deutlich stärkerer Folgeeffekt kann beobachtet werden, wenn die wahrgenommene visuomotorische Differenz zwischen Ziel und Trefferpunkt in eine Gehirnhälfte projiziert wird und der Folgeeffekt mit der Hand erfolgt, welche von dieser Hirnhemisphäre gesteuert wird. Der intermanuelle Transfer wird demnach begünstigt, wenn die visuelle Projektion der Fehlerbeobachtung in die Gehirnhälfte erfolgt, die während der Adaptationsphase motorisch passiv ist.

An interactive surgical planning tool for acetabular fractures: initial results

Background Acetabular fractures still are among the most challenging fractures to treat because of complex anatomy, involved surgical access to fracture sites and the relatively low incidence of these lesions. Proper evaluation and surgical planning is necessary to achieve anatomic reduction of the articular surface and stable fixation of the pelvic ring. The goal of this study was to test the feasibility of preoperative surgical planning in acetabular fractures using a new prototype planning tool based on an interactive virtual reality-style environment. Methods 7 patients (5 male and 2 female; median age 53 y (25 to 92 y)) with an acetabular fracture were prospectively included. Exclusion criterions were simple wall fractures, cases with anticipated surgical dislocation of the femoral head for joint debridement and accurate fracture reduction. According to the Letournel classification 4 cases had two column fractures, 2 cases had anterior column fractures and 1 case had a T-shaped fracture including a posterior wall fracture. The workflow included following steps: (1) Formation of a patient-specific bone model from preoperative computed tomography scans, (2) interactive virtual fracture reduction with visuo-haptic feedback, (3) virtual fracture fixation using common osteosynthesis implants and (4) measurement of implant position relative to landmarks. The surgeon manually contoured osteosynthesis plates preoperatively according to the virtually defined deformation. Screenshots including all measurements for the OR were available. The tool was validated comparing the preoperative planning and postoperative results by 3D-superimposition. Results Preoperative planning was feasible in all cases. In 6 of 7 cases superimposition of preoperative planning and postoperative follow-up CT showed a good to excellent correlation. In one case part of the procedure had to be changed due to impossibility of fracture reduction from an ilioinguinal approach. In 3 cases with osteopenic bone patient-specific prebent fixation plates were helpful in guiding fracture reduction. Additionally, anatomical landmark based measurements were helpful for intraoperative navigation. Conclusion The presented prototype planning tool for pelvic surgery was successfully integrated in a clinical workflow to improve patient-specific preoperative planning, giving visual and haptic information about the injury and allowing a patient-specific adaptation of osteosynthesis implants to the virtually reduced pelvis.

Retraining of interjoint arm coordination after stroke using robot-assisted time-independent functional training

We have developed a haptic-based approach for retraining of interjoint coordination following stroke called time-independent functional training (TIFT) and implemented this mode in the ARMin III robotic exoskeleton. The ARMin III robot was developed by Drs. Robert Riener and Tobias Nef at the Swiss Federal Institute of Technology Zurich (Eidgenossische Technische Hochschule Zurich, or ETH Zurich), in Zurich, Switzerland. In the TIFT mode, the robot maintains arm movements within the proper kinematic trajectory via haptic walls at each joint. These arm movements focus training of interjoint coordination with highly intuitive real-time feedback of performance; arm movements advance within the trajectory only if their movement coordination is correct. In initial testing, 37 nondisabled subjects received a single session of learning of a complex pattern. Subjects were randomized to TIFT or visual demonstration or moved along with the robot as it moved though the pattern (time-dependent [TD] training). We examined visual demonstration to separate the effects of action observation on motor learning from the effects of the two haptic guidance methods. During these training trials, TIFT subjects reduced error and interaction forces between the robot and arm, while TD subject performance did not change. All groups showed significant learning of the trajectory during unassisted recall trials, but we observed no difference in learning between groups, possibly because this learning task is dominated by vision. Further testing in stroke populations is warranted.

[Case report of osteo-odonto keratoprosthesis (Strampelli) and of Dacron keratoprosthesis (Pintucci)]

BACKGROUND: In severe forms of ocular surface disorders keratoprostheses provide the ultimate possibility to restore vision. They are made of an optical cylinder integrated with a supporting biocompatible or biological haptic. CASE REPORT: We report on two patients with different types of keratoprostheses. An 88-year-old woman with ocular pemphigoid received in 1970 a bilateral osteo-odonto-keratoprosthesis (Strampelli). A 59-year-old man with refractory corneal ulcer after corneal grafting received in 2003 a keratoprosthesis with supporting Dacron tissue (Pintucci). RESULTS: The course 35 years after implantation of the osteo-odonto-keratoprosthesis was uneventful. Histologically there were no signs of loosening, rejection or infection. The autologous dentin, which was used for the fixation, was still present. The eye with the Dacron fixated prosthesis (Pintucci) had to be enucleated due to a loosening with endophthalmitis one and a half year after implantation. CONCLUSIONS: Keratoprostheses with autologous fixation often show good long-term results. On the other hand, prostheses with synthetic material are more often complicated by dislocation and inflammation.

Driver Safety in Far-side and Far-oblique Crashes: A Study of Patrol Vehicles in the United States of America

The dissertation titled "Driver Safety in Far-side and Far-oblique Crashes" presents a novel approach to assessing vehicle cockpit safety by integrating Human Factors and Applied Mechanics. The methodology of this approach is aimed at improving safety in compact mobile workspaces such as patrol vehicle cockpits. A statistical analysis performed using Michigan state's traffic crash data to assess various contributing factors that affect the risk of severe driver injuries showed that the risk was greater for unrestrained drivers (OR=3.38, p<0.0001) and for incidents involving front and far-side crashes without seatbelts (OR=8.0 and 23.0 respectively, p<0.005). Statistics also showed that near-side and far-side crashes pose similar threat to driver injury severity. A Human Factor survey was conducted to assess various Human-Machine/Human-Computer Interaction aspects in patrol vehicle cockpits. Results showed that tasks requiring manual operation, especially the usage of laptop, would require more attention and potentially cause more distraction. A vehicle survey conducted to evaluate ergonomics-related issues revealed that some of the equipment was in airbag deployment zones. In addition, experiments were conducted to assess the effects on driver distraction caused by changing the position of in-car accessories. A driving simulator study was conducted to mimic HMI/HCI in a patrol vehicle cockpit (20 subjects, average driving experience = 5.35 years, s.d. = 1.8). It was found that the mounting locations of manual tasks did not result in a significant change in response times. Visual displays resulted in response times less than 1.5sec. It can also be concluded that the manual task was equally distracting regardless of mounting positions (average response time was 15 secs). Average speeds and lane deviations did not show any significant results. Data from 13 full-scale sled tests conducted to simulate far-side impacts at 70 PDOF and 40 PDOF was used to analyze head injuries and HIC/AIS values. It was found that accelerations generated by the vehicle deceleration alone were high enough to cause AIS 3 - AIS 6 injuries. Pretensioners could mitigated injuries only in 40 PDOF (oblique) impacts but are useless in 70 PDOF impacts. Seat belts were ineffective in protecting the driver's head from injuries. Head would come in contact with the laptop during a far-oblique (40 PDOF) crash and far-side door for an angle-type crash (70 PDOF). Finite Element analysis head-laptop impact interaction showed that the contact velocity was the most crucial factor in causing a severe (and potentially fatal) head injury. Results indicate that no equipment may be mounted in driver trajectory envelopes. A very narrow band of space is left in patrol vehicles for installation of manual-task equipment to be both safe and ergonomic. In case of a contact, the material stiffness and damping properties play a very significant role in determining the injury outcome. Future work may be done on improving the interiors' material properties to better absorb and dissipate kinetic energy of the head. The design of seat belts and pretensioners may also be seen as an essential aspect to be further improved.

Impaired verbal memory in Parkinson disease: relationship to prefrontal dysfunction and somatosensory discrimination

OBJECTIVE: To study the neurocognitive profile and its relationship to prefrontal dysfunction in non-demented Parkinson's disease (PD) with deficient haptic perception. METHODS: Twelve right-handed patients with PD and 12 healthy control subjects underwent thorough neuropsychological testing including Rey complex figure, Rey auditory verbal and figural learning test, figural and verbal fluency, and Stroop test. Test scores reflecting significant differences between patients and healthy subjects were correlated with the individual expression coefficients of one principal component, obtained in a principal component analysis of an oxygen-15-labeled water PET study exploring somatosensory discrimination that differentiated between the two groups and involved prefrontal cortices. RESULTS: We found significantly decreased total scores for the verbal learning trials and verbal delayed free recall in PD patients compared with normal volunteers. Further analysis of these parameters using Spearman's ranking correlation showed a significantly negative correlation of deficient verbal recall with expression coefficients of the principal component whose image showed a subcortical-cortical network, including right dorsolateral-prefrontal cortex, in PD patients. CONCLUSION: PD patients with disrupted right dorsolateral prefrontal cortex function and associated diminished somatosensory discrimination are impaired also in verbal memory functions. A negative correlation between delayed verbal free recall and PET activation in a network including the prefrontal cortices suggests that verbal cues and accordingly declarative memory processes may be operative in PD during activities that demand sustained attention such as somatosensory discrimination. Verbal cues may be compensatory in nature and help to non-specifically enhance focused attention in the presence of a functionally disrupted prefrontal cortex.

Haptische, visuelle und olfaktorische Sprachen: Westnilotische Wahrnehmungen

Cet article traite des expressions de la perception sensuelle de l’ouest nilotique. La première partie de l’article présente une terminologie ophrésiologique en louo et burun et démontre que cette catégorie lexicale détient aussi une catégorie grammaticale particulière. Phénomènes très rares dans les langues du monde, les termes ophrésiologiques sont seulement présentés sous forme introductive pour encourager davantage les recherches futures. La seconde partie de l’article porte sur les descriptions des couleurs utilisées pour les animaux domestiques. Quand les modalités de l’économie changent, les noms utilisés pour les couleurs des animaux peuvent aussi être employés pour d’autres concepts culturels. La troisième partie de l’article montre que les classificateurs nominaux en mabaan (burun) expriment des principes concernant le toucher en tant que structure cognitive. En conséquence, différents procédés de grammaticalisation sont assumés et corrélés avec des ponctuations dans l’histoire culturelle et mentale des ancêtres des locuteurs de l’ouest nilotique.

Real Walking through Virtual Environments by Redirection Techniques

We present redirection techniques that support exploration of large-scale virtual environments (VEs) by means of real walking. We quantify to what degree users can unknowingly be redirected in order to guide them through VEs in which virtual paths differ from the physical paths. We further introduce the concept of dynamic passive haptics by which any number of virtual objects can be mapped to real physical proxy props having similar haptic properties (i. e., size, shape, and surface structure), such that the user can sense these virtual objects by touching their real world counterparts. Dynamic passive haptics provides the user with the illusion of interacting with a desired virtual object by redirecting her to the corresponding proxy prop. We describe the concepts of generic redirected walking and dynamic passive haptics and present experiments in which we have evaluated these concepts. Furthermore, we discuss implications that have been derived from a user study, and we present approaches that derive physical paths which may vary from the virtual counterparts.

Stereocopic Photography Encounters the Staircase

The staircase is presented as the architectural component that most potently embodies thresholds, boundaries and passages due to its diagonal orientation and essence as an intermediary zone. Connections then are made between the kinesthetic requirements of traversing a staircase and viewing a stereoscopic photograph. From this foundation, the haptic essence of stereoscopic photography is proposed as uniquely qualified medium through which to view a staircase and therefore thresholds, boundaries, and passages within architecture. Analyses of stereoviews of staircases in the Palais de Justice in Brussels, the Library of Congress in Washington, and the Palais Garnier (Opéra) in Paris close the essay.

Bimanual anterior segment revision surgery for anterior capsule contraction syndrome associated with anterior flexion of intraocular lens haptics

PURPOSE To report the incidence of anterior capsule contraction syndrome (ACCS) and to present a novel minimally invasive bimanual technique for anterior segment revision surgery associated with ACCS with anterior flexion of the intraocular lens haptics. METHODS A consecutive cohort of 268 eyes of 161 patients undergoing phacoemulsification and implantation of the same type of hydrophilic acrylic aspheric intraocular lens cohort were analysed and a novel technique of minimally invasive bimanual technique for anterior segment revision surgery is described. RESULTS We identified four eyes (1.5%) of three patients with advanced ACCS. Successful restoration of a clear visual axis with minimal induction of astigmatism and rapid visual rehabilitation was achieved in all four cases. CONCLUSION This technique is a safe and minimally invasive alternative to laser or vitrector-cut capsulotomy to restore a clear visual axis. In cases of advanced ACCS, it offers the option for haptic reposition or amputation.

ARMin: a robot for patient-cooperative arm therapy

Task-oriented, repetitive and intensive arm training can enhance arm rehabilitation in patients with paralyzed upper extremities due to lesions of the central nervous system. There is evidence that the training duration is a key factor for the therapy progress. Robot-supported therapy can improve the rehabilitation allowing more intensive training. This paper presents the kinematics, the control and the therapy modes of the arm therapy robot ARMin. It is a haptic display with semi-exoskeleton kinematics with four active and two passive degrees of freedom. Equipped with position, force and torque sensors the device can deliver patient-cooperative arm therapy taking into account the activity of the patient and supporting him/her only as much as needed. The haptic display is combined with an audiovisual display that is used to present the movement and the movement task to the patient. It is assumed that the patient-cooperative therapy approach combined with a multimodal display can increase the patient's motivation and activity and, therefore, the therapeutic progress.

A novel paradigm for patient-cooperative control of upper-limb rehabilitation robots

Early intervention and intensive therapy improve the outcome of neuromuscular rehabilitation. There are indications that where a patient is motivated and premeditates their movement, the recovery is more effective. Therefore, a strategy for patient-cooperative control of rehabilitation devices for upper extremities is proposed and evaluated. The strategy is based on the minimal intervention principle allowing an efficient exploitation of task space redundancies and resulting in user-driven movement trajectories. The patient's effort is taken into consideration by enabling the machine to comply with forces exerted by the user. The interaction is enhanced through a multimodal display and a virtually generated environment that includes haptic, visual and sound modalities.

Improving Speech Interaction in Vehicles Using Context-Aware Information through A SCXML Framework

Speech Technologies can provide important benefits for the development of more usable and safe in-vehicle human-machine interactive systems (HMIs). However mainly due robustness issues, the use of spoken interaction can entail important distractions to the driver. In this challenging scenario, while speech technologies are evolving, further research is necessary to explore how they can be complemented with both other modalities (multimodality) and information from the increasing number of available sensors (context-awareness). The perceived quality of speech technologies can significantly be increased by implementing such policies, which simply try to make the best use of all the available resources; and the in vehicle scenario is an excellent test-bed for this kind of initiatives. In this contribution we propose an event-based HMI design framework which combines context modelling and multimodal interaction using a W3C XML language known as SCXML. SCXML provides a general process control mechanism that is being considered by W3C to improve both voice interaction (VoiceXML) and multimodal interaction (MMI). In our approach we try to anticipate and extend these initiatives presenting a flexible SCXML-based approach for the design of a wide range of multimodal context-aware HMI in-vehicle interfaces. The proposed framework for HMI design and specification has been implemented in an automotive OSGi service platform, and it is being used and tested in the Spanish research project MARTA for the development of several in-vehicle interactive applications.

New generation of human machine interfaces for controlling UAV through depth based gesture recognition

New forms of natural interactions between human operators and UAVs (Unmanned Aerial Vehicle) are demanded by the military industry to achieve a better balance of the UAV control and the burden of the human operator. In this work, a human machine interface (HMI) based on a novel gesture recognition system using depth imagery is proposed for the control of UAVs. Hand gesture recognition based on depth imagery is a promising approach for HMIs because it is more intuitive, natural, and non-intrusive than other alternatives using complex controllers. The proposed system is based on a Support Vector Machine (SVM) classifier that uses spatio-temporal depth descriptors as input features. The designed descriptor is based on a variation of the Local Binary Pattern (LBP) technique to efficiently work with depth video sequences. Other major consideration is the especial hand sign language used for the UAV control. A tradeoff between the use of natural hand signs and the minimization of the inter-sign interference has been established. Promising results have been achieved in a depth based database of hand gestures especially developed for the validation of the proposed system.

User Experience in Human-Technology Interaction. Communication, context and evaluation methodology

En esta Tesis se presentan dos líneas de investigación relacionadas y que contribuyen a las áreas de Interacción Hombre-Tecnología (o Máquina; siglas en inglés: HTI o HMI), lingüística computacional y evaluación de la experiencia del usuario. Las dos líneas en cuestión son el diseño y la evaluación centrada en el usuario de sistemas de Interacción Hombre-Máquina avanzados. En la primera parte de la Tesis (Capítulos 2 a 4) se abordan cuestiones fundamentales del diseño de sistemas HMI avanzados. El Capítulo 2 presenta una panorámica del estado del arte de la investigación en el ámbito de los sistemas conversacionales multimodales, con la que se enmarca el trabajo de investigación presentado en el resto de la Tesis. Los Capítulos 3 y 4 se centran en dos grandes aspectos del diseño de sistemas HMI: un gestor del diálogo generalizado para tratar la Interacción Hombre-Máquina multimodal y sensible al contexto, y el uso de agentes animados personificados (ECAs) para mejorar la robustez del diálogo, respectivamente. El Capítulo 3, sobre gestión del diálogo, aborda el tratamiento de la heterogeneidad de la información proveniente de las modalidades comunicativas y de los sensores externos. En este capítulo se propone, en un nivel de abstracción alto, una arquitectura para la gestión del diálogo con influjos heterogéneos de información, apoyándose en el uso de State Chart XML. En el Capítulo 4 se presenta una contribución a la representación interna de intenciones comunicativas, y su traducción a secuencias de gestos a ejecutar por parte de un ECA, diseñados específicamente para mejorar la robustez en situaciones de diálogo críticas que pueden surgir, por ejemplo, cuando se producen errores de entendimiento en la comunicación entre el usuario humano y la máquina. Se propone, en estas páginas, una extensión del Functional Mark-up Language definido en el marco conceptual SAIBA. Esta extensión permite representar actos comunicativos que realizan intenciones del emisor (la máquina) que no se pretende sean captadas conscientemente por el receptor (el usuario humano), pero con las que se pretende influirle a éste e influir el curso del diálogo. Esto se consigue mediante un objeto llamado Base de Intenciones Comunicativas (en inglés, Communication Intention Base, o CIB). La representación en el CIB de intenciones “no claradas” además de las explícitas permite la construcción de actos comunicativos que realizan simultáneamente varias intenciones comunicativas. En el Capítulo 4 también se describe un sistema experimental para el control remoto (simulado) de un asistente domótico, con autenticación de locutor para dar acceso, y con un ECA en el interfaz de cada una de estas tareas. Se incluye una descripción de las secuencias de comportamiento verbal y no verbal de los ECAs, que fueron diseñados específicamente para determinadas situaciones con objeto de mejorar la robustez del diálogo. Los Capítulos 5 a 7 conforman la parte de la Tesis dedicada a la evaluación. El Capítulo 5 repasa antecedentes relevantes en la literatura de tecnologías de la información en general, y de sistemas de interacción hablada en particular. Los principales antecedentes en el ámbito de la evaluación de la interacción sobre los cuales se ha desarrollado el trabajo presentado en esta Tesis son el Technology Acceptance Model (TAM), la herramienta Subjective Assessment of Speech System Interfaces (SASSI), y la Recomendación P.851 de la ITU-T. En el Capítulo 6 se describen un marco y una metodología de evaluación aplicados a la experiencia del usuario con sistemas HMI multimodales. Se desarrolló con este propósito un novedoso marco de evaluación subjetiva de la calidad de la experiencia del usuario y su relación con la aceptación por parte del mismo de la tecnología HMI (el nombre dado en inglés a este marco es Subjective Quality Evaluation Framework). En este marco se articula una estructura de clases de factores subjetivos relacionados con la satisfacción y aceptación por parte del usuario de la tecnología HMI propuesta. Esta estructura, tal y como se propone en la presente tesis, tiene dos dimensiones ortogonales. Primero se identifican tres grandes clases de parámetros relacionados con la aceptación por parte del usuario: “agradabilidad ” (likeability: aquellos que tienen que ver con la experiencia de uso, sin entrar en valoraciones de utilidad), rechazo (los cuales sólo pueden tener una valencia negativa) y percepción de utilidad. En segundo lugar, este conjunto clases se reproduce para distintos “niveles, o focos, percepción del usuario”. Éstos incluyen, como mínimo, un nivel de valoración global del sistema, niveles correspondientes a las tareas a realizar y objetivos a alcanzar, y un nivel de interfaz (en los casos propuestos en esta tesis, el interfaz es un sistema de diálogo con o sin un ECA). En el Capítulo 7 se presenta una evaluación empírica del sistema descrito en el Capítulo 4. El estudio se apoya en los mencionados antecedentes en la literatura, ampliados con parámetros para el estudio específico de los agentes animados (los ECAs), la auto-evaluación de las emociones de los usuarios, así como determinados factores de rechazo (concretamente, la preocupación por la privacidad y la seguridad). También se evalúa el marco de evaluación subjetiva de la calidad propuesto en el capítulo anterior. Los análisis de factores efectuados revelan una estructura de parámetros muy cercana conceptualmente a la división de clases en utilidad-agradabilidad-rechazo propuesta en dicho marco, resultado que da cierta validez empírica al marco. Análisis basados en regresiones lineales revelan estructuras de dependencias e interrelación entre los parámetros subjetivos y objetivos considerados. El efecto central de mediación, descrito en el Technology Acceptance Model, de la utilidad percibida sobre la relación de dependencia entre la intención de uso y la facilidad de uso percibida, se confirma en el estudio presentado en la presente Tesis. Además, se ha encontrado que esta estructura de relaciones se fortalece, en el estudio concreto presentado en estas páginas, si las variables consideradas se generalizan para cubrir más ampliamente las categorías de agradabilidad y utilidad contempladas en el marco de evaluación subjetiva de calidad. Se ha observado, asimismo, que los factores de rechazo aparecen como un componente propio en los análisis de factores, y además se distinguen por su comportamiento: moderan la relación entre la intención de uso (que es el principal indicador de la aceptación del usuario) y su predictor más fuerte, la utilidad percibida. Se presentan también resultados de menor importancia referentes a los efectos de los ECAs sobre los interfaces de los sistemas de diálogo y sobre los parámetros de percepción y las valoraciones de los usuarios que juegan un papel en conformar su aceptación de la tecnología. A pesar de que se observa un rendimiento de la interacción dialogada ligeramente mejor con ECAs, las opiniones subjetivas son muy similares entre los dos grupos experimentales (uno interactuando con un sistema de diálogo con ECA, y el otro sin ECA). Entre las pequeñas diferencias encontradas entre los dos grupos destacan las siguientes: en el grupo experimental sin ECA (es decir, con interfaz sólo de voz) se observó un efecto más directo de los problemas de diálogo (por ejemplo, errores de reconocimiento) sobre la percepción de robustez, mientras que el grupo con ECA tuvo una respuesta emocional más positiva cuando se producían problemas. Los ECAs parecen generar inicialmente expectativas más elevadas en cuanto a las capacidades del sistema, y los usuarios de este grupo se declaran más seguros de sí mismos en su interacción. Por último, se observan algunos indicios de efectos sociales de los ECAs: la “amigabilidad ” percibida los ECAs estaba correlada con un incremento la preocupación por la seguridad. Asimismo, los usuarios del sistema con ECAs tendían más a culparse a sí mismos, en lugar de culpar al sistema, de los problemas de diálogo que pudieran surgir, mientras que se observó una ligera tendencia opuesta en el caso de los usuarios del sistema con interacción sólo de voz. ABSTRACT This Thesis presents two related lines of research work contributing to the general fields of Human-Technology (or Machine) Interaction (HTI, or HMI), computational linguistics, and user experience evaluation. These two lines are the design and user-focused evaluation of advanced Human-Machine (or Technology) Interaction systems. The first part of the Thesis (Chapters 2 to 4) is centred on advanced HMI system design. Chapter 2 provides a background overview of the state of research in multimodal conversational systems. This sets the stage for the research work presented in the rest of the Thesis. Chapers 3 and 4 focus on two major aspects of HMI design in detail: a generalised dialogue manager for context-aware multimodal HMI, and embodied conversational agents (ECAs, or animated agents) to improve dialogue robustness, respectively. Chapter 3, on dialogue management, deals with how to handle information heterogeneity, both from the communication modalities or from external sensors. A highly abstracted architectural contribution based on State Chart XML is proposed. Chapter 4 presents a contribution for the internal representation of communication intentions and their translation into gestural sequences for an ECA, especially designed to improve robustness in critical dialogue situations such as when miscommunication occurs. We propose an extension of the functionality of Functional Mark-up Language, as envisaged in much of the work in the SAIBA framework. Our extension allows the representation of communication acts that carry intentions that are not for the interlocutor to know of, but which are made to influence him or her as well as the flow of the dialogue itself. This is achieved through a design element we have called the Communication Intention Base. Such r pr s ntation of “non- clar ” int ntions allows th construction of communication acts that carry several communication intentions simultaneously. Also in Chapter 4, an experimental system is described which allows (simulated) remote control to a home automation assistant, with biometric (speaker) authentication to grant access, featuring embodied conversation agents for each of the tasks. The discussion includes a description of the behavioural sequences for the ECAs, which were designed for specific dialogue situations with particular attention given to the objective of improving dialogue robustness. Chapters 5 to 7 form the evaluation part of the Thesis. Chapter 5 reviews evaluation approaches in the literature for information technologies, as well as in particular for speech-based interaction systems, that are useful precedents to the contributions of the present Thesis. The main evaluation precedents on which the work in this Thesis has built are the Technology Acceptance Model (TAM), the Subjective Assessment of Speech System Interfaces (SASSI) tool, and ITU-T Recommendation P.851. Chapter 6 presents the author’s work in establishing an valuation framework and methodology applied to the users’ experience with multimodal HMI systems. A novel user-acceptance Subjective Quality Evaluation Framework was developed by the author specifically for this purpose. A class structure arises from two orthogonal sets of dimensions. First we identify three broad classes of parameters related with user acceptance: likeability factors (those that have to do with the experience of using the system), rejection factors (which can only have a negative valence) and perception of usefulness. Secondly, the class structure is further broken down into several “user perception levels”; at the very least: an overall system-assessment level, task and goal-related levels, and an interface level (e.g., a dialogue system with or without an ECA). An empirical evaluation of the system described in Chapter 4 is presented in Chapter 7. The study was based on the abovementioned precedents in the literature, expanded with categories covering the inclusion of an ECA, the users’ s lf-assessed emotions, and particular rejection factors (privacy and security concerns). The Subjective Quality Evaluation Framework proposed in the previous chapter was also scrutinised. Factor analyses revealed an item structure very much related conceptually to the usefulness-likeability-rejection class division introduced above, thus giving it some empirical weight. Regression-based analysis revealed structures of dependencies, paths of interrelations, between the subjective and objective parameters considered. The central mediation effect, in the Technology Acceptance Model, of perceived usefulness on the dependency relationship of intention-to-use with perceived ease of use was confirmed in this study. Furthermore, the pattern of relationships was stronger for variables covering more broadly the likeability and usefulness categories in the Subjective Quality Evaluation Framework. Rejection factors were found to have a distinct presence as components in factor analyses, as well as distinct behaviour: they were found to moderate the relationship between intention-to-use (the main measure of user acceptance) and its strongest predictor, perceived usefulness. Insights of secondary importance are also given regarding the effect of ECAs on the interface of spoken dialogue systems and the dimensions of user perception and judgement attitude that may have a role in determining user acceptance of the technology. Despite observing slightly better performance values in the case of the system with the ECA, subjective opinions regarding both systems were, overall, very similar. Minor differences between two experimental groups (one interacting with an ECA, the other only through speech) include a more direct effect of dialogue problems (e.g., non-understandings) on perceived dialogue robustness for the voice-only interface test group, and a more positive emotional response for the ECA test group. Our findings further suggest that the ECA generates higher initial expectations, and users seem slightly more confident in their interaction with the ECA than do those without it. Finally, mild evidence of social effects of ECAs was also found: the perceived friendliness of the ECA increased security concerns, and ECA users may tend to blame themselves rather than the system when dialogue problems are encountered, while the opposite may be true for voice-only users.