Understanding "Just Enough" Computer Users: Motivation Style and Proficiency

Among daily computer users who are proficient, some are flexible at accomplishing unfamiliar tasks on their own and others have difficulty. Software designers and evaluators involved with Human Computer Interaction (HCI) should account for any group of proficient daily users that are shown to stumble over unfamiliar tasks. We define "Just Enough" (JE) users as proficient daily computer users with predominantly extrinsic motivation style who know just enough to get what they want or need from the computer. We hypothesize that JE users have difficulty with unfamiliar computer tasks and skill transfer, whereas intrinsically motivated daily users accomplish unfamiliar tasks readily. Intrinsic motivation can be characterized by interest, enjoyment, and choice and extrinsic motivation is externally regulated. In our study we identified users by motivation style and then did ethnographic observations. Our results confirm that JE users do have difficulty accomplishing unfamiliar tasks on their own but had fewer problems with near skill transfer. In contrast, intrinsically motivated users had no trouble with unfamiliar tasks nor with near skill transfer. This supports our assertion that JE users know enough to get routine tasks done and can transfer that knowledge, but become unproductive when faced with unfamiliar tasks. This study combines quantitative and qualitative methods. We identified 66 daily users by motivation style using an inventory adapted from Deci and Ryan (Ryan and Deci 2000) and from Guay, Vallerand, and Blanchard (Guay et al. 2000). We used qualitative ethnographic methods with a think aloud protocol to observe nine extrinsic users and seven intrinsic users. Observation sessions had three customized phases where the researcher directed the participant to: 1) confirm the participant's proficiency; 2) test the participant accomplishing unfamiliar tasks; and 3) test transfer of existing skills to unfamiliar software.

An Experience-Driven Pedagogy for the Instruction of Software Testing in Computer Science

In the realm of computer programming, the experience of writing a program is used to reinforce concepts and evaluate ability. This research uses three case studies to evaluate the introduction of testing through Kolb's Experiential Learning Model (ELM). We then analyze the impact of those testing experiences to determine methods for improving future courses. The first testing experience that students encounter are unit test reports in their early courses. This course demonstrates that automating and improving feedback can provide more ELM iterations. The JUnit Generation (JUG) tool also provided a positive experience for the instructor by reducing the overall workload. Later, undergraduate and graduate students have the opportunity to work together in a multi-role Human-Computer Interaction (HCI) course. The interactions use usability analysis techniques with graduate students as usability experts and undergraduate students as design engineers. Students get experience testing the user experience of their product prototypes using methods varying from heuristic analysis to user testing. From this course, we learned the importance of the instructors role in the ELM. As more roles were added to the HCI course, a desire arose to provide more complete, quality assured software. This inspired the addition of unit testing experiences to the course. However, we learned that significant preparations must be made to apply the ELM when students are resistant. The research presented through these courses was driven by the recognition of a need for testing in a Computer Science curriculum. Our understanding of the ELM suggests the need for student experience when being introduced to testing concepts. We learned that experiential learning, when appropriately implemented, can provide benefits to the Computer Science classroom. When examined together, these course-based research projects provided insight into building strong testing practices into a curriculum.

Multi-Contact Grasp Interaction for Virtual Environments

The grasping of virtual objects has been an active research field for several years. Solutions providing realistic grasping rely on special hardware or require time-consuming parameterizations. Therefore, we introduce a flexible grasping algorithm enabling grasping without computational complex physics. Objects can be grasped and manipulated with multiple fingers. In addition, multiple objects can be manipulated simultaneously with our approach. Through the usage of contact sensors the technique is easily configurable and versatile enough to be used in different scenarios.

Evaluation of Binocular Eye Trackers and Algorithms for 3D Gaze Interaction in Virtual Reality Environments

Tracking user���s visual attention is a fundamental aspect in novel human-computer interaction paradigms found in Virtual Reality. For example, multimodal interfaces or dialogue-based communications with virtual and real agents greatly benefit from the analysis of the user���s visual attention as a vital source for deictic references or turn-taking signals. Current approaches to determine visual attention rely primarily on monocular eye trackers. Hence they are restricted to the interpretation of two-dimensional fixations relative to a defined area of projection. The study presented in this article compares precision, accuracy and application performance of two binocular eye tracking devices. Two algorithms are compared which derive depth information as required for visual attention-based 3D interfaces. This information is further applied to an improved VR selection task in which a binocular eye tracker and an adaptive neural network algorithm is used during the disambiguation of partly occluded objects.

Efficient Bimanual Symmetric 3D Manipulation for Bare-Handed Interaction

Recently, stable markerless 6 DOF video based handtracking devices became available. These devices simultaneously track the positions and orientations of both user hands in different postures with at least 25 frames per second. Such hand-tracking allows for using the human hands as natural input devices. However, the absence of physical buttons for performing click actions and state changes poses severe challenges in designing an efficient and easy to use 3D interface on top of such a device. In particular, for coupling and decoupling a virtual object���s movements to the user���s hand (i.e. grabbing and releasing) a solution has to be found. In this paper, we introduce a novel technique for efficient two-handed grabbing and releasing objects and intuitively manipulating them in the virtual space. This technique is integrated in a novel 3D interface for virtual manipulations. A user experiment shows the superior applicability of this new technique. Last but not least, we describe how this technique can be exploited in practice to improve interaction by integrating it with RTT DeltaGen, a professional CAD/CAS visualization and editing tool.

NETWORK TOPOLOGY IN HUMAN PROTEIN INTERACTION DATA PREDICTS FUNCTIONAL ASSOCIATION

High-throughput assays, such as yeast two-hybrid system, have generated a huge amount of protein-protein interaction (PPI) data in the past decade. This tremendously increases the need for developing reliable methods to systematically and automatically suggest protein functions and relationships between them. With the available PPI data, it is now possible to study the functions and relationships in the context of a large-scale network. To data, several network-based schemes have been provided to effectively annotate protein functions on a large scale. However, due to those inherent noises in high-throughput data generation, new methods and algorithms should be developed to increase the reliability of functional annotations. Previous work in a yeast PPI network (Samanta and Liang, 2003) has shown that the local connection topology, particularly for two proteins sharing an unusually large number of neighbors, can predict functional associations between proteins, and hence suggest their functions. One advantage of the work is that their algorithm is not sensitive to noises (false positives) in high-throughput PPI data. In this study, we improved their prediction scheme by developing a new algorithm and new methods which we applied on a human PPI network to make a genome-wide functional inference. We used the new algorithm to measure and reduce the influence of hub proteins on detecting functionally associated proteins. We used the annotations of the Gene Ontology (GO) and the Kyoto Encyclopedia of Genes and Genomes (KEGG) as independent and unbiased benchmarks to evaluate our algorithms and methods within the human PPI network. We showed that, compared with the previous work from Samanta and Liang, our algorithm and methods developed in this study improved the overall quality of functional inferences for human proteins. By applying the algorithms to the human PPI network, we obtained 4,233 significant functional associations among 1,754 proteins. Further comparisons of their KEGG and GO annotations allowed us to assign 466 KEGG pathway annotations to 274 proteins and 123 GO annotations to 114 proteins with estimated false discovery rates of <21% for KEGG and <30% for GO. We clustered 1,729 proteins by their functional associations and made pathway analysis to identify several subclusters that are highly enriched in certain signaling pathways. Particularly, we performed a detailed analysis on a subcluster enriched in the transforming growth factor �� signaling pathway (P<10-50) which is important in cell proliferation and tumorigenesis. Analysis of another four subclusters also suggested potential new players in six signaling pathways worthy of further experimental investigations. Our study gives clear insight into the common neighbor-based prediction scheme and provides a reliable method for large-scale functional annotations in this post-genomic era.

User Experience in Human-Technology Interaction. Communication, context and evaluation methodology

En esta Tesis se presentan dos l��neas de investigaci��n relacionadas y que contribuyen a las ��reas de Interacci��n Hombre-Tecnolog��a (o M��quina; siglas en ingl��s: HTI o HMI), ling����stica computacional y evaluaci��n de la experiencia del usuario. Las dos l��neas en cuesti��n son el dise��o y la evaluaci��n centrada en el usuario de sistemas de Interacci��n Hombre-M��quina avanzados. En la primera parte de la Tesis (Cap��tulos 2 a 4) se abordan cuestiones fundamentales del dise��o de sistemas HMI avanzados. El Cap��tulo 2 presenta una panor��mica del estado del arte de la investigaci��n en el ��mbito de los sistemas conversacionales multimodales, con la que se enmarca el trabajo de investigaci��n presentado en el resto de la Tesis. Los Cap��tulos 3 y 4 se centran en dos grandes aspectos del dise��o de sistemas HMI: un gestor del di��logo generalizado para tratar la Interacci��n Hombre-M��quina multimodal y sensible al contexto, y el uso de agentes animados personificados (ECAs) para mejorar la robustez del di��logo, respectivamente. El Cap��tulo 3, sobre gesti��n del di��logo, aborda el tratamiento de la heterogeneidad de la informaci��n proveniente de las modalidades comunicativas y de los sensores externos. En este cap��tulo se propone, en un nivel de abstracci��n alto, una arquitectura para la gesti��n del di��logo con influjos heterog��neos de informaci��n, apoy��ndose en el uso de State Chart XML. En el Cap��tulo 4 se presenta una contribuci��n a la representaci��n interna de intenciones comunicativas, y su traducci��n a secuencias de gestos a ejecutar por parte de un ECA, dise��ados espec��ficamente para mejorar la robustez en situaciones de di��logo cr��ticas que pueden surgir, por ejemplo, cuando se producen errores de entendimiento en la comunicaci��n entre el usuario humano y la m��quina. Se propone, en estas p��ginas, una extensi��n del Functional Mark-up Language definido en el marco conceptual SAIBA. Esta extensi��n permite representar actos comunicativos que realizan intenciones del emisor (la m��quina) que no se pretende sean captadas conscientemente por el receptor (el usuario humano), pero con las que se pretende influirle a ��ste e influir el curso del di��logo. Esto se consigue mediante un objeto llamado Base de Intenciones Comunicativas (en ingl��s, Communication Intention Base, o CIB). La representaci��n en el CIB de intenciones ���no claradas��� adem��s de las expl��citas permite la construcci��n de actos comunicativos que realizan simult��neamente varias intenciones comunicativas. En el Cap��tulo 4 tambi��n se describe un sistema experimental para el control remoto (simulado) de un asistente dom��tico, con autenticaci��n de locutor para dar acceso, y con un ECA en el interfaz de cada una de estas tareas. Se incluye una descripci��n de las secuencias de comportamiento verbal y no verbal de los ECAs, que fueron dise��ados espec��ficamente para determinadas situaciones con objeto de mejorar la robustez del di��logo. Los Cap��tulos 5 a 7 conforman la parte de la Tesis dedicada a la evaluaci��n. El Cap��tulo 5 repasa antecedentes relevantes en la literatura de tecnolog��as de la informaci��n en general, y de sistemas de interacci��n hablada en particular. Los principales antecedentes en el ��mbito de la evaluaci��n de la interacci��n sobre los cuales se ha desarrollado el trabajo presentado en esta Tesis son el Technology Acceptance Model (TAM), la herramienta Subjective Assessment of Speech System Interfaces (SASSI), y la Recomendaci��n P.851 de la ITU-T. En el Cap��tulo 6 se describen un marco y una metodolog��a de evaluaci��n aplicados a la experiencia del usuario con sistemas HMI multimodales. Se desarroll�� con este prop��sito un novedoso marco de evaluaci��n subjetiva de la calidad de la experiencia del usuario y su relaci��n con la aceptaci��n por parte del mismo de la tecnolog��a HMI (el nombre dado en ingl��s a este marco es Subjective Quality Evaluation Framework). En este marco se articula una estructura de clases de factores subjetivos relacionados con la satisfacci��n y aceptaci��n por parte del usuario de la tecnolog��a HMI propuesta. Esta estructura, tal y como se propone en la presente tesis, tiene dos dimensiones ortogonales. Primero se identifican tres grandes clases de par��metros relacionados con la aceptaci��n por parte del usuario: ���agradabilidad ��� (likeability: aquellos que tienen que ver con la experiencia de uso, sin entrar en valoraciones de utilidad), rechazo (los cuales s��lo pueden tener una valencia negativa) y percepci��n de utilidad. En segundo lugar, este conjunto clases se reproduce para distintos ���niveles, o focos, percepci��n del usuario���. ��stos incluyen, como m��nimo, un nivel de valoraci��n global del sistema, niveles correspondientes a las tareas a realizar y objetivos a alcanzar, y un nivel de interfaz (en los casos propuestos en esta tesis, el interfaz es un sistema de di��logo con o sin un ECA). En el Cap��tulo 7 se presenta una evaluaci��n emp��rica del sistema descrito en el Cap��tulo 4. El estudio se apoya en los mencionados antecedentes en la literatura, ampliados con par��metros para el estudio espec��fico de los agentes animados (los ECAs), la auto-evaluaci��n de las emociones de los usuarios, as�� como determinados factores de rechazo (concretamente, la preocupaci��n por la privacidad y la seguridad). Tambi��n se eval��a el marco de evaluaci��n subjetiva de la calidad propuesto en el cap��tulo anterior. Los an��lisis de factores efectuados revelan una estructura de par��metros muy cercana conceptualmente a la divisi��n de clases en utilidad-agradabilidad-rechazo propuesta en dicho marco, resultado que da cierta validez emp��rica al marco. An��lisis basados en regresiones lineales revelan estructuras de dependencias e interrelaci��n entre los par��metros subjetivos y objetivos considerados. El efecto central de mediaci��n, descrito en el Technology Acceptance Model, de la utilidad percibida sobre la relaci��n de dependencia entre la intenci��n de uso y la facilidad de uso percibida, se confirma en el estudio presentado en la presente Tesis. Adem��s, se ha encontrado que esta estructura de relaciones se fortalece, en el estudio concreto presentado en estas p��ginas, si las variables consideradas se generalizan para cubrir m��s ampliamente las categor��as de agradabilidad y utilidad contempladas en el marco de evaluaci��n subjetiva de calidad. Se ha observado, asimismo, que los factores de rechazo aparecen como un componente propio en los an��lisis de factores, y adem��s se distinguen por su comportamiento: moderan la relaci��n entre la intenci��n de uso (que es el principal indicador de la aceptaci��n del usuario) y su predictor m��s fuerte, la utilidad percibida. Se presentan tambi��n resultados de menor importancia referentes a los efectos de los ECAs sobre los interfaces de los sistemas de di��logo y sobre los par��metros de percepci��n y las valoraciones de los usuarios que juegan un papel en conformar su aceptaci��n de la tecnolog��a. A pesar de que se observa un rendimiento de la interacci��n dialogada ligeramente mejor con ECAs, las opiniones subjetivas son muy similares entre los dos grupos experimentales (uno interactuando con un sistema de di��logo con ECA, y el otro sin ECA). Entre las peque��as diferencias encontradas entre los dos grupos destacan las siguientes: en el grupo experimental sin ECA (es decir, con interfaz s��lo de voz) se observ�� un efecto m��s directo de los problemas de di��logo (por ejemplo, errores de reconocimiento) sobre la percepci��n de robustez, mientras que el grupo con ECA tuvo una respuesta emocional m��s positiva cuando se produc��an problemas. Los ECAs parecen generar inicialmente expectativas m��s elevadas en cuanto a las capacidades del sistema, y los usuarios de este grupo se declaran m��s seguros de s�� mismos en su interacci��n. Por ��ltimo, se observan algunos indicios de efectos sociales de los ECAs: la ���amigabilidad ��� percibida los ECAs estaba correlada con un incremento la preocupaci��n por la seguridad. Asimismo, los usuarios del sistema con ECAs tend��an m��s a culparse a s�� mismos, en lugar de culpar al sistema, de los problemas de di��logo que pudieran surgir, mientras que se observ�� una ligera tendencia opuesta en el caso de los usuarios del sistema con interacci��n s��lo de voz. ABSTRACT This Thesis presents two related lines of research work contributing to the general fields of Human-Technology (or Machine) Interaction (HTI, or HMI), computational linguistics, and user experience evaluation. These two lines are the design and user-focused evaluation of advanced Human-Machine (or Technology) Interaction systems. The first part of the Thesis (Chapters 2 to 4) is centred on advanced HMI system design. Chapter 2 provides a background overview of the state of research in multimodal conversational systems. This sets the stage for the research work presented in the rest of the Thesis. Chapers 3 and 4 focus on two major aspects of HMI design in detail: a generalised dialogue manager for context-aware multimodal HMI, and embodied conversational agents (ECAs, or animated agents) to improve dialogue robustness, respectively. Chapter 3, on dialogue management, deals with how to handle information heterogeneity, both from the communication modalities or from external sensors. A highly abstracted architectural contribution based on State Chart XML is proposed. Chapter 4 presents a contribution for the internal representation of communication intentions and their translation into gestural sequences for an ECA, especially designed to improve robustness in critical dialogue situations such as when miscommunication occurs. We propose an extension of the functionality of Functional Mark-up Language, as envisaged in much of the work in the SAIBA framework. Our extension allows the representation of communication acts that carry intentions that are not for the interlocutor to know of, but which are made to influence him or her as well as the flow of the dialogue itself. This is achieved through a design element we have called the Communication Intention Base. Such r pr s ntation of ���non- clar ��� int ntions allows th construction of communication acts that carry several communication intentions simultaneously. Also in Chapter 4, an experimental system is described which allows (simulated) remote control to a home automation assistant, with biometric (speaker) authentication to grant access, featuring embodied conversation agents for each of the tasks. The discussion includes a description of the behavioural sequences for the ECAs, which were designed for specific dialogue situations with particular attention given to the objective of improving dialogue robustness. Chapters 5 to 7 form the evaluation part of the Thesis. Chapter 5 reviews evaluation approaches in the literature for information technologies, as well as in particular for speech-based interaction systems, that are useful precedents to the contributions of the present Thesis. The main evaluation precedents on which the work in this Thesis has built are the Technology Acceptance Model (TAM), the Subjective Assessment of Speech System Interfaces (SASSI) tool, and ITU-T Recommendation P.851. Chapter 6 presents the author���s work in establishing an valuation framework and methodology applied to the users��� experience with multimodal HMI systems. A novel user-acceptance Subjective Quality Evaluation Framework was developed by the author specifically for this purpose. A class structure arises from two orthogonal sets of dimensions. First we identify three broad classes of parameters related with user acceptance: likeability factors (those that have to do with the experience of using the system), rejection factors (which can only have a negative valence) and perception of usefulness. Secondly, the class structure is further broken down into several ���user perception levels���; at the very least: an overall system-assessment level, task and goal-related levels, and an interface level (e.g., a dialogue system with or without an ECA). An empirical evaluation of the system described in Chapter 4 is presented in Chapter 7. The study was based on the abovementioned precedents in the literature, expanded with categories covering the inclusion of an ECA, the users��� s lf-assessed emotions, and particular rejection factors (privacy and security concerns). The Subjective Quality Evaluation Framework proposed in the previous chapter was also scrutinised. Factor analyses revealed an item structure very much related conceptually to the usefulness-likeability-rejection class division introduced above, thus giving it some empirical weight. Regression-based analysis revealed structures of dependencies, paths of interrelations, between the subjective and objective parameters considered. The central mediation effect, in the Technology Acceptance Model, of perceived usefulness on the dependency relationship of intention-to-use with perceived ease of use was confirmed in this study. Furthermore, the pattern of relationships was stronger for variables covering more broadly the likeability and usefulness categories in the Subjective Quality Evaluation Framework. Rejection factors were found to have a distinct presence as components in factor analyses, as well as distinct behaviour: they were found to moderate the relationship between intention-to-use (the main measure of user acceptance) and its strongest predictor, perceived usefulness. Insights of secondary importance are also given regarding the effect of ECAs on the interface of spoken dialogue systems and the dimensions of user perception and judgement attitude that may have a role in determining user acceptance of the technology. Despite observing slightly better performance values in the case of the system with the ECA, subjective opinions regarding both systems were, overall, very similar. Minor differences between two experimental groups (one interacting with an ECA, the other only through speech) include a more direct effect of dialogue problems (e.g., non-understandings) on perceived dialogue robustness for the voice-only interface test group, and a more positive emotional response for the ECA test group. Our findings further suggest that the ECA generates higher initial expectations, and users seem slightly more confident in their interaction with the ECA than do those without it. Finally, mild evidence of social effects of ECAs was also found: the perceived friendliness of the ECA increased security concerns, and ECA users may tend to blame themselves rather than the system when dialogue problems are encountered, while the opposite may be true for voice-only users.

Human-robot interaction for telemanipulation in large workspaces

Independientemente de la existencia de t��cnicas altamente sofisticadas y capacidades de c��mputo cada vez m��s elevadas, los problemas asociados a los robots que interact��an con entornos no estructurados siguen siendo un desaf��o abierto en rob��tica. A pesar de los grandes avances de los sistemas rob��ticos aut��nomos, hay algunas situaciones en las que una persona en el bucle sigue siendo necesaria. Ejemplos de esto son, tareas en entornos de fusi��n nuclear, misiones espaciales, operaciones submarinas y cirug��a rob��tica. Esta necesidad se debe a que las tecnolog��as actuales no pueden realizar de forma fiable y aut��noma cualquier tipo de tarea. Esta tesis presenta m��todos para la teleoperaci��n de robots abarcando distintos niveles de abstracci��n que van desde el control supervisado, en el que un operador da instrucciones de alto nivel en la forma de acciones, hasta el control bilateral, donde los comandos toman la forma de se��ales de control de bajo nivel. En primer lugar, se presenta un enfoque para llevar a cabo la teleoperaci��n supervisada de robots humanoides. El objetivo es controlar robots terrestres capaces de ejecutar tareas complejas en entornos de b��squeda y rescate utilizando enlaces de comunicaci��n limitados. Esta propuesta incorpora comportamientos aut��nomos que el operador puede utilizar para realizar tareas de navegaci��n y manipulaci��n mientras se permite cubrir grandes ��reas de entornos remotos dise��ados para el acceso de personas. Los resultados experimentales demuestran la eficacia de los m��todos propuestos. En segundo lugar, se investiga el uso de dispositivos rentables para telemanipulaci��n guiada. Se presenta una aplicaci��n que involucra un robot humanoide bimanual y un traje de captura de movimiento basado en sensores inerciales. En esta aplicaci��n, se estudian las capacidades de adaptaci��n introducidas por el factor humano y c��mo estas pueden compensar la falta de sistemas rob��ticos de alta precisi��n. Este trabajo es el resultado de una colaboraci��n entre investigadores del Biorobotics Laboratory de la Universidad de Harvard y el Centro de Autom��tica y Rob��tica UPM-CSIC. En tercer lugar, se presenta un nuevo controlador h��ptico que combina velocidad y posici��n. Este controlador bilateral h��brido hace frente a los problemas relacionados con la teleoperaci��n de un robot esclavo con un gran espacio de trabajo usando un dispositivo h��ptico peque��o como maestro. Se pueden cubrir amplias ��reas de trabajo al cambiar autom��ticamente entre los modos de control de velocidad y posici��n. Este controlador h��ptico es ideal para sistemas maestro-esclavo con cinem��ticas diferentes, donde los comandos se transmiten en el espacio de la tarea del entorno remoto. El m��todo es validado para realizar telemanipulaci��n h��bil de objetos con un robot industrial. Por ��ltimo, se introducen dos contribuciones en el campo de la manipulaci��n rob��tica. Por un lado, se presenta un nuevo algoritmo de cinem��tica inversa, llamado m��todo iterativo de desacoplamiento cinem��tico. Este m��todo se ha desarrollado para resolver el problema cinem��tico inverso de un tipo de robot de seis grados de libertad donde una soluci��n cerrada no est�� disponible. La eficacia del m��todo se compara con m��todos num��ricos convencionales. Adem��s, se ha dise��ado una taxonom��a robusta de agarres que permite controlar diferentes manos rob��ticas utilizando una correspondencia, basada en gestos, entre los espacios de trabajo de la mano humana y de la mano rob��tica. El gesto de la mano humana se identifica mediante la lectura de los movimientos relativos del ��ndice, el pulgar y el dedo medio del usuario durante las primeras etapas del agarre. ABSTRACT Regardless of the availability of highly sophisticated techniques and ever increasing computing capabilities, the problems associated with robots interacting with unstructured environments remains an open challenge. Despite great advances in autonomous robotics, there are some situations where a humanin- the-loop is still required, such as, nuclear, space, subsea and robotic surgery operations. This is because the current technologies cannot reliably perform all kinds of task autonomously. This thesis presents methods for robot teleoperation strategies at different levels of abstraction ranging from supervisory control, where the operator gives high-level task actions, to bilateral teleoperation, where the commands take the form of low-level control inputs. These strategies contribute to improve the current human-robot interfaces specially in the case of slave robots deployed at large workspaces. First, an approach to perform supervisory teleoperation of humanoid robots is presented. The goal is to control ground robots capable of executing complex tasks in disaster relief environments under constrained communication links. This proposal incorporates autonomous behaviors that the operator can use to perform navigation and manipulation tasks which allow covering large human engineered areas of the remote environment. The experimental results demonstrate the efficiency of the proposed methods. Second, the use of cost-effective devices for guided telemanipulation is investigated. A case study involving a bimanual humanoid robot and an Inertial Measurement Unit (IMU) Motion Capture (MoCap) suit is introduced. Herein, it is corroborated how the adaptation capabilities offered by the human-in-the-loop factor can compensate for the lack of high-precision robotic systems. This work is the result of collaboration between researchers from the Harvard Biorobotics Laboratory and the Centre for Automation and Robotics UPM-CSIC. Thirdly, a new haptic rate-position controller is presented. This hybrid bilateral controller copes with the problems related to the teleoperation of a slave robot with large workspace using a small haptic device as master. Large workspaces can be covered by automatically switching between rate and position control modes. This haptic controller is ideal to couple kinematic dissimilar master-slave systems where the commands are transmitted in the task space of the remote environment. The method is validated to perform dexterous telemanipulation of objects with a robotic manipulator. Finally, two contributions for robotic manipulation are introduced. First, a new algorithm, the Iterative Kinematic Decoupling method, is presented. It is a numeric method developed to solve the Inverse Kinematics (IK) problem of a type of six-DoF robotic arms where a close-form solution is not available. The effectiveness of this IK method is compared against conventional numerical methods. Second, a robust grasp mapping has been conceived. It allows to control a wide range of different robotic hands using a gesture based correspondence between the human hand space and the robotic hand space. The human hand gesture is identified by reading the relative movements of the index, thumb and middle fingers of the user during the early stages of grasping.

Study and analysis of the Leap Motion sensor and the Software Development Kit (SDK) for the implementation of visual Human Computer Interfaces

En este proyecto, se presenta un informe t��cnico sobre la c��mara Leap Motion y el Software Development Kit correspondiente, el cual es un dispositivo con una c��mara de profundidad orientada a interfaces hombre-m��quina. Esto es realizado con el prop��sito de desarrollar una interfaz hombre-m��quina basada en un sistema de reconocimiento de gestos de manos. Despu��s de un exhaustivo estudio de la c��mara Leap Motion, se han realizado diversos programas de ejemplo con la intenci��n de verificar las capacidades descritas en el informe t��cnico, poniendo a prueba la Application Programming Interface y evaluando la precisi��n de las diferentes medidas obtenidas sobre los datos de la c��mara. Finalmente, se desarrolla un prototipo de un sistema de reconocimiento de gestos. Los datos sobre la posici��n y orientaci��n de la punta de los dedos obtenidos de la Leap Motion son usados para describir un gesto mediante un vector descriptor, el cual es enviado a una M��quina Vectores Soporte, utilizada como clasificador multi-clase.

VHDL-based system design of a cognitive sensorimotor loop (CSL) for haptic Human-Machine Interaction (HMI)

This document is a summary of the Bachelor thesis titled ���VHDL-Based System Design of a Cognitive Sensorimotor Loop (CSL) for Haptic Human-Machine Interaction (HMI)��� written by Pablo de Miguel Morales, Electronics Engineering student at the Universidad Polit��cnica de Madrid (UPM Madrid, Spain) during an Erasmus+ Exchange Program at the Beuth Hochschule f��r Technik (BHT Berlin, Germany). The tutor of this project is Dr. Prof. Hild. This project has been developed inside the Neurobotics Research Laboratory (NRL) in close collaboration with Benjamin Panreck, a member of the NRL, and another exchange student from the UPM Pablo Gabriel Lezcano. For a deeper comprehension of the content of the thesis, a deeper look in the document is needed as well as the viewing of the videos and the VHDL design. In the growing field of automation, a large amount of workforce is dedicated to improve, adapt and design motor controllers for a wide variety of applications. In the specific field of robotics or other machinery designed to interact with humans or their environment, new needs and technological solutions are often being discovered due to the existing, relatively unexplored new scenario it is. The project consisted of three main parts: Two VHDL-based systems and one short experiment on the haptic perception. Both VHDL systems are based on a Cognitive Sensorimotor Loop (CSL) which is a control loop designed by the NRL and mainly developed by Dr. Prof. Hild. The CSL is a control loop whose main characteristic is the fact that it does not use any external sensor to measure the speed or position of the motor but the motor itself. The motor always generates a voltage that is proportional to its angular speed so it does not need calibration. This method is energy efficient and simplifies control loops in complex systems. The first system, named CSL Stay In Touch (SIT), consists in a one DC motor system controller by a FPGA Board (Zynq ZYBO 7000) whose aim is to keep contact with any external object that touches its Sensing Platform in both directions. Apart from the main behavior, three features (Search Mode, Inertia Mode and Return Mode) have been designed to enhance the haptic interaction experience. Additionally, a VGA-Screen is also controlled by the FPGA Board for the monitoring of the whole system. This system has been completely developed, tested and improved; analyzing its timing and consumption properties. The second system, named CSL Fingerlike Mechanism (FM), consists in a fingerlike mechanical system controlled by two DC motors (Each controlling one part of the finger). The behavior is similar to the first system but in a more complex structure. This system was optional and not part of the original objectives of the thesis and it could not be properly finished and tested due to the lack of time. The haptic perception experiment was an experiment conducted to have an insight into the complexity of human haptic perception in order to implement this knowledge into technological applications. The experiment consisted in testing the capability of the subjects to recognize different objects and shapes while being blindfolded and with their ears covered. Two groups were done, one had full haptic perception while the other had to explore the environment with a plastic piece attached to their finger to create a haptic handicap. The conclusion of the thesis was that a haptic system based only on a CSL-based system is not enough to retrieve valuable information from the environment and that other sensors are needed (temperature, pressure, etc.) but that a CSL-based system is very useful to control the force applied by the system to interact with haptic sensible surfaces such as skin or tactile screens. RESUMEN. Este documento es un resumen del proyecto fin de grado titulado ���VHDL-Based System Design of a Cognitive Sensorimotor Loop (CSL) for Haptic Human-Machine Interaction (HMI)��� escrito por Pablo de Miguel, estudiante de Ingenier��a Electr��nica de Comunicaciones en la Universidad Polit��cnica de Madrid (UPM Madrid, Espa��a) durante un programa de intercambio Erasmus+ en la Beuth Hochschule f��r Technik (BHT Berlin, Alemania). El tutor de este proyecto ha sido Dr. Prof. Hild. Este proyecto se ha desarrollado dentro del Neurorobotics Research Laboratory (NRL) en estrecha colaboraci��n con Benjamin Panreck (un miembro del NRL) y con Pablo Lezcano (Otro estudiante de intercambio de la UPM). Para una comprensi��n completa del trabajo es necesaria una lectura detenida de todo el documento y el visionado de los videos y an��lisis del dise��o VHDL incluidos en el CD adjunto. En el creciente sector de la automatizaci��n, una gran cantidad de esfuerzo est�� dedicada a mejorar, adaptar y dise��ar controladores de motor para un gran rango de aplicaciones. En el campo espec��fico de la rob��tica u otra maquinaria dise��ada para interactuar con los humanos o con su entorno, nuevas necesidades y soluciones tecnol��gicas se siguen desarrollado debido al relativamente inexplorado y nuevo escenario que supone. El proyecto consta de tres partes principales: Dos sistemas basados en VHDL y un peque��o experimento sobre la percepci��n h��ptica. Ambos sistemas VHDL est��n basados en el Cognitive Sesnorimotor Loop (CSL) que es un lazo de control creado por el NRL y cuyo desarrollador principal ha sido Dr. Prof. Hild. El CSL es un lazo de control cuya principal caracter��stica es la ausencia de sensores externos para medir la velocidad o la posici��n del motor, usando el propio motor como sensor. El motor siempre genera un voltaje proporcional a su velocidad angular de modo que no es necesaria calibraci��n. Este m��todo es eficiente en t��rminos energ��ticos y simplifica los lazos de control en sistemas complejos. El primer sistema, llamado CSL Stay In Touch (SIT), consiste en un sistema formado por un motor DC controlado por una FPGA Board (Zynq ZYBO 7000) cuyo objetivo es mantener contacto con cualquier objeto externo que toque su plataforma sensible en ambas direcciones. Aparte del funcionamiento b��sico, tres modos (Search Mode, Inertia Mode y Return Mode) han sido dise��ados para mejorar la interacci��n. Adicionalmente, se ha dise��ado el control a trav��s de la FPGA Board de una pantalla VGA para la monitorizaci��n de todo el sistema. El sistema ha sido totalmente desarrollado, testeado y mejorado; analizando su propiedades de timing y consumo energ��tico. El segundo sistema, llamado CSL Fingerlike Mechanism (FM), consiste en un mecanismo similar a un dedo controlado por dos motores DC (Cada uno controlando una falange). Su comportamiento es similar al del primer sistema pero con una estructura m��s compleja. Este sistema no formaba parte de los objetivos iniciales del proyecto y por lo tanto era opcional. No pudo ser plenamente desarrollado debido a la falta de tiempo. El experimento de percepci��n h��ptica fue dise��ado para profundizar en la percepci��n h��ptica humana con el objetivo de aplicar este conocimiento en aplicaciones tecnol��gicas. El experimento consist��a en testear la capacidad de los sujetos para reconocer diferentes objetos, formas y texturas en condiciones de privaci��n del sentido del o��do y la vista. Se crearon dos grupos, en uno los sujetos ten��an plena percepci��n h��ptica mientras que en el otro deb��an interactuar con los objetos a trav��s de una pieza de pl��stico para generar un h��ndicap h��ptico. La conclusi��n del proyecto fue que un sistema h��ptico basado solo en sistemas CSL no es suficiente para recopilar informaci��n valiosa del entorno y que debe hacer uso de otros sensores (temperatura, presi��n, etc.). En cambio, un sistema basado en CSL es id��neo para el control de la fuerza aplicada por el sistema durante la interacci��n con superficies h��pticas sensibles tales como la piel o pantallas t��ctiles.

The role of voice input for human-machine communication.

Optimism is growing that the near future will witness rapid growth in human-computer interaction using voice. System prototypes have recently been built that demonstrate speaker-independent real-time speech recognition, and understanding of naturally spoken utterances with vocabularies of 1000 to 2000 words, and larger. Already, computer manufacturers are building speech recognition subsystems into their new product lines. However, before this technology can be broadly useful, a substantial knowledge base is needed about human spoken language and performance during computer-based spoken interaction. This paper reviews application areas in which spoken interaction can play a significant role, assesses potential benefits of spoken interaction with machines, and compares voice with other modalities of human-computer interaction. It also discusses information that will be needed to build a firm empirical foundation for the design of future spoken and multimodal interfaces. Finally, it argues for a more systematic and scientific approach to investigating spoken input and performance with future language technology.

A Virtual Joystick as a Novel Interaction Modality for Surgeon-Computer Interaction During Computer-Assisted Surgical Procedures

Interacting with a computer system in the operating room (OR) can be a frustrating experience for a surgeon, who currently has to verbally delegate to an assistant every computer interaction task. This indirect mode of interaction is time consuming, error prone and can lead to poor usability of OR computer systems. This thesis describes the design and evaluation of a joystick-like device that allows direct surgeon control of the computer in the OR. The device was tested extensively in comparison to a mouse and delegated dictation with seven surgeons, eleven residents, and five graduate students. The device contains no electronic parts, is easy to use, is unobtrusive, has no physical connection to the computer and makes use of an existing tool in the OR. We performed a user study to determine its effectiveness in allowing a user to perform all the tasks they would be expected to perform on an OR computer system during a computer-assisted surgery. Dictation was found to be superior to the joystick in qualitative measures, but the joystick was preferred over dictation in user satisfaction responses. The mouse outperformed both joystick and dictation, but it is not a readily accepted modality in the OR.

A testbed for providing uniformity to user-computer interaction languages /

"Issued August 1980."