Interaction techniques with novel multimodal feedback for addressing gesture-sensing systems

Users need to be able to address in-air gesture systems, which means finding where to perform gestures and how to direct them towards the intended system. This is necessary for input to be sensed correctly and without unintentionally affecting other systems. This thesis investigates novel interaction techniques which allow users to address gesture systems properly, helping them find where and how to gesture. It also investigates audio, tactile and interactive light displays for multimodal gesture feedback; these can be used by gesture systems with limited output capabilities (like mobile phones and small household controls), allowing the interaction techniques to be used by a variety of device types. It investigates tactile and interactive light displays in greater detail, as these are not as well understood as audio displays. Experiments 1 and 2 explored tactile feedback for gesture systems, comparing an ultrasound haptic display to wearable tactile displays at different body locations and investigating feedback designs. These experiments found that tactile feedback improves the user experience of gesturing by reassuring users that their movements are being sensed. Experiment 3 investigated interactive light displays for gesture systems, finding this novel display type effective for giving feedback and presenting information. It also found that interactive light feedback is enhanced by audio and tactile feedback. These feedback modalities were then used alongside audio feedback in two interaction techniques for addressing gesture systems: sensor strength feedback and rhythmic gestures. Sensor strength feedback is multimodal feedback that tells users how well they can be sensed, encouraging them to find where to gesture through active exploration. Experiment 4 found that they can do this with 51mm accuracy, with combinations of audio and interactive light feedback leading to the best performance. Rhythmic gestures are continuously repeated gesture movements which can be used to direct input. Experiment 5 investigated the usability of this technique, finding that users can match rhythmic gestures well and with ease. Finally, these interaction techniques were combined, resulting in a new single interaction for addressing gesture systems. Using this interaction, users could direct their input with rhythmic gestures while using the sensor strength feedback to find a good location for addressing the system. Experiment 6 studied the effectiveness and usability of this technique, as well as the design space for combining the two types of feedback. It found that this interaction was successful, with users matching 99.9% of rhythmic gestures, with 80mm accuracy from target points. The findings show that gesture systems could successfully use this interaction technique to allow users to address them. Novel design recommendations for using rhythmic gestures and sensor strength feedback were created, informed by the experiment findings.

Wave impedance selection for passivity-based bilateral teleoperation

When a task must be executed in a remote or dangerous environment, teleoperation systems may be employed to extend the influence of the human operator. In the case of manipulation tasks, haptic feedback of the forces experienced by the remote (slave) system is often highly useful in improving an operator's ability to perform effectively. In many of these cases (especially teleoperation over the internet and ground-to-space teleoperation), substantial communication latency exists in the control loop and has the strong tendency to cause instability of the system. The first viable solution to this problem in the literature was based on a scattering/wave transformation from transmission line theory. This wave transformation requires the designer to select a wave impedance parameter appropriate to the teleoperation system. It is widely recognized that a small value of wave impedance is well suited to free motion and a large value is preferable for contact tasks. Beyond this basic observation, however, very little guidance exists in the literature regarding the selection of an appropriate value. Moreover, prior research on impedance selection generally fails to account for the fact that in any realistic contact task there will simultaneously exist contact considerations (perpendicular to the surface of contact) and quasi-free-motion considerations (parallel to the surface of contact). The primary contribution of the present work is to introduce an approximate linearized optimum for the choice of wave impedance and to apply this quasi-optimal choice to the Cartesian reality of such a contact task, in which it cannot be expected that a given joint will be either perfectly normal to or perfectly parallel to the motion constraint. The proposed scheme selects a wave impedance matrix that is appropriate to the conditions encountered by the manipulator. This choice may be implemented as a static wave impedance value or as a time-varying choice updated according to the instantaneous conditions encountered. A Lyapunov-like analysis is presented demonstrating that time variation in wave impedance will not violate the passivity of the system. Experimental trials, both in simulation and on a haptic feedback device, are presented validating the technique. Consideration is also given to the case of an uncertain environment, in which an a priori impedance choice may not be possible.

Interface em smartphone para supervisão de um sistema de controlo de uma instalação solar térmica

Dissertação de Mestrado, Engenharia Elétrica e Eletrónica, Especialização em Sistemas de Energia e Controlo, Instituto Superior de Engenharia, Universidade do Algarve, 2014

Diseño y uso de una paleta háptica para prácticas de teleoperación con simulink

En este artículo se presenta el uso de dispositivos de realimentación de fuerzas de un grado de libertad como herramienta para el desarrollo de aplicaciones docentes en asignaturas de teleoperación y telerrobótica, con el objetivo de disponer de una interfaz física que pueda ser utilizada como equipo de prácticas. Estos dispositivos, conocidos como haptic paddle, ya han sido probados satisfactoriamente para aplicaciones educativas de modelado y simulación de sistemas, ntroducción a los haptics e ingeniería de control en diversas Universidades. Para el desarrollo de los experimentos que se muestran en este artículo se han utilizado dos dispositivos haptic paddle con un controlador basado en Arduino y fabricados mediante tecnologías aditivas de impresión 3D. Gracias a los sensores de fuerza y posición incorporados se pueden implementar esquemas bilaterales de teleoperación de posicón-posición y fuerza-posición.

Dynamic virtual workspace for teleoperation of heterogeneous robots in non-deterministic environments

Robots are ever increasing in a variety of different workplaces providing an array of benefits such alternative solutions to traditional human labor. While developing fully autonomous robots is the ultimate goal in many robotic applications the reality is that there still exist many situationswere robots require some level of teleoperation in order to achieve assigned goals especially when deployed in non-deterministic environments. For instance teleoperation is commonly used in areas such as search and rescue, bomb disposal and exploration of inaccessible or harsh terrain. This is due to a range of factors such as the lack of ability for robots to quickly and reliably navigate unknown environments or provide high-level decision making especially intime critical tasks. To provide an adequate solution for such situations human-in-the-loop control is required. When developing human-in-the-loop control it is important to take advantage of the complimentary skill-sets that both humans and robots share. For example robots can performrapid calculations, provide accurate measurements through hardware such as sensors and store large amounts of data while humans provide experience, intuition, risk management and complex decision making capabilities. Shared autonomy is the concept of building robotic systems that take advantage of these complementary skills-sets to provide a robust an efficient robotic solution. While the requirement of human-in-the-loop control exists Human Machine Interaction (HMI) remains an important research topic especially the area of User Interface (UI) design.In order to provide operators with an effective teleoperation system it is important that the interface is intuitive and dynamic while also achieving a high level of immersion. Recent advancements in virtual and augmented reality hardware is giving rise to innovative HMI systems. Interactive hardware such as Microsoft Kinect, leap motion, Oculus Rift, Samsung Gear VR and even CAVE Automatic Virtual Environments [1] are providing vast improvements over traditional user interface designs such as the experimental web browser JanusVR [2]. This combined with the introduction of standardized robot frameworks such as ROS and Webots [3] that now support a large number of different robots provides an opportunity to develop a universal UI for teleoperation control to improve operator efficiency while reducing teleoperation training.This research introduces the concept of a dynamic virtual workspace for teleoperation of heterogeneous robots in non-deterministic environments that require human-in-the-loop control. The system first identifies the connected robots through the use kinematic information then determines its network capabilities such as latency and bandwidth. Given the robot type and network capabilities the system can then provide the operator with available teleoperation modes such as pick and place control or waypoint navigation while also allowing them to manipulate the virtual workspace layout to provide information from onboard camera’s or sensors.

Intimité et sensations dans les films d'animation en volume de Girlin Bassovskaja

Peu satisfaite des concepts généralement mentionnés lorsqu’il s’agit d’écrire sur les films réalisés en stop-motion, je propose d’analyser un corpus de quatre films réalisés par un duo estonien peu connu, les réalisatrices Jelena Girlin et Mari-Liis Bassovskaja, en ancrant mon discours dans une recherche plus large sur l’intimité et la sensorialité en art. J’effleure, par l’entremise d’une revue de littérature, le paradoxe d’animer l’inanimé et l’idée du umheimlich freudien, prégnants dans les écrits substantiels autour du cinéma d’animation en volume. Après avoir démontré que l’œuvre de Girlin Bassovskaja s’incrit dans le domaine de l’intime, j’approfondis l’analyse en m’appuyant sur les théories de la visualité haptique appliquée aux films. Je découvre le corpus à la lumière de ces théories, et évoque l’idée du regard caressant du spectateur vers le film, mais aussi de sa réversibilité. De plus, en tant que réalisatrice-animatrice de court-métrages d’animation, les théories susmentionnées outillent ma pensée afin de décrire ma volonté quasi obsessionnelle de rendre l’intimité tangible par une animation sensuelle en pâte à modeler sur verre.

Intimité et sensations dans les films d'animation en volume de Girlin Bassovskaja

Peu satisfaite des concepts généralement mentionnés lorsqu’il s’agit d’écrire sur les films réalisés en stop-motion, je propose d’analyser un corpus de quatre films réalisés par un duo estonien peu connu, les réalisatrices Jelena Girlin et Mari-Liis Bassovskaja, en ancrant mon discours dans une recherche plus large sur l’intimité et la sensorialité en art. J’effleure, par l’entremise d’une revue de littérature, le paradoxe d’animer l’inanimé et l’idée du umheimlich freudien, prégnants dans les écrits substantiels autour du cinéma d’animation en volume. Après avoir démontré que l’œuvre de Girlin Bassovskaja s’incrit dans le domaine de l’intime, j’approfondis l’analyse en m’appuyant sur les théories de la visualité haptique appliquée aux films. Je découvre le corpus à la lumière de ces théories, et évoque l’idée du regard caressant du spectateur vers le film, mais aussi de sa réversibilité. De plus, en tant que réalisatrice-animatrice de court-métrages d’animation, les théories susmentionnées outillent ma pensée afin de décrire ma volonté quasi obsessionnelle de rendre l’intimité tangible par une animation sensuelle en pâte à modeler sur verre.

The sensory features of packaging and its effects on product perception

The present study shows the influence of tactile contacts with packaging on product taste. A within-subject experiment 1 (product: grenadine syrup) x 3 (packaging’s material: plastic, aluminium, glass) was conducted. Moreover, we examine the role of the Need For Touch (NFT) as a moderator. We confirmed that the same product tasted three times by each participant is judged differently depending on the packaging. Furthermore, participants with a high NFT seem to be more influenced by sensory features of packaging than those with a low NFT. These results support previous researches about tactile effects on taste.

Virtual reality training for micro-robotic cell injection

This research was carried out to fill the gap within existing knowledge on the approaches to supplement the training for micro-robotic cell injection procedure by utilising virtual reality and haptic technologies.

Por baixo da pele outra pele: conjunto de obras artísticas. Corpo, ecrã e interface para uma visualidade háptica interativa

A tese teórico-prática de doutoramento apresentada contempla a investigação, a criação, a produção e a conceptualização da instalação artística interativa intitulada Por baixo da pele outra pele. A Obra é constituída por três objetos tridimensionais concebidos à escala humana. Recorre a materiais flexíveis, como têxteis, convidando o público (interator), à envolvência física, numa relação corporal sensorial e sensual com a obra. Os objetos contêm dispositivos técnicos interativos e sensores tácteis, que, ao serem utilizados, desencadeiam estímulos multissensoriais no espectador. A instalação interativa focaliza a experiência háptica e íntima do interator considerando os seus mecanismos sensoriais e cognitivos como um potencial aparato na construção de experiências fenomenológicas, singulares e individuais. A autora considera a interatividade enquanto elemento potenciador da experiência estética visual háptica. Na argumentação conceitual da obra, reflete-se sobre o tema da visualidade háptica interativa a partirdos conceitos de ecrã, corpo e interface, assim como de endossensorialidade. Instrumentam-se metodologias de investigação em ação, experimentais e observacionais. Apresentam-se os processos investigativos, criativos e técnicos necessários ao desenvolvimento e à materialização da instalação artística. A investigação revela-se de grande interesse para o avanço da pesquisa de novas linguagens experimentais apresentando estratégias de criação artística que, ao privilegiarem o corpo físico e fenomenológico do interator, transpõe a experiência háptica interativa para um grau interno de imersão motoro-sensorial; Underneath the skin another skin: art installation. Body, screen and interface towards an interactive haptic visuality. Abstract: The theoretical-practical doctorate dissertation presents the research and conceptual framework behind, and the processes leading to, the creation and production of the interactive installation art piece Underneath the skin, another skin. The piece is presented in the shape of three human-scale tridimentional objects. It is made from flexible materials, such as textiles, inviting the (interacting) audience, to physically engage in a bodily sensorial, and sensuous, relationship with the artwork. The objects enclose interactive devices and tactile sensors that, when used, trigger in the interactor multiple sensorial stimuli. The interactive installation focuses on the interactor's intimate haptic experience taking in consideration his or hers sensorial and cognitive mechanisms as a potential apparatus in the construction of unique individual phenomenological experiences. The author understands interactivity as a triggering element into an haptic visual aesthetical experience. The supporting conceptual reasoning deals with thought and criticism on interactive haptic visuality applied to the concepts of screen, body and interface, as well as with that of endo-sensoriality. The dissertation describes the use of experimental and observation research methodologies. It also elaborates on the research, creative and technical processes at play in the installation's development and realization. The research at hand has shown great potential for the further development of new experimental languages, as it presents art-creation strategies privileging the interactor's physical and phenomenological body, and thus able to take the interactive haptic experience onto an greater inner level of motor-sensorial immersion.

‘Crows’ vs. ‘Avatar’, or: 3D vs. total-dimensional immersion

3D film’s explicit new space depth arguably provides both an enhanced realistic quality to the image and a wealth of more acute visual and haptic sensations (a ‘montage of attractions’) to the increasingly involved spectator. But David Cronenberg’s related ironic remark that ‘cinema as such is from the outset a «special effect»’ should warn us against the geometrical naiveté of such assumptions, within a Cartesian ocularcentric tradition for long overcome by Merleau-Ponty’s embodiment of perception and Deleuze’s notion of the self-consistency of the artistic sensation and space. Indeed, ‘2D’ traditional cinema already provides the accomplished «fourth wall effect», enclosing the beholder behind his back within a space that no longer belongs to the screen (nor to ‘reality’) as such, and therefore is no longer ‘illusorily’ two-dimensional. This kind of totally absorbing, ‘dream-like’ space, metaphorical for both painting and cinema, is illustrated by the episode ‘Crows’ in Kurosawa’s Dreams. Such a space requires the actual effacement of the empirical status of spectator, screen and film as separate dimensions, and it is precisely the 3D caracteristic unfolding of merely frontal space layers (and film events) out of the screen towards us (and sometimes above the heads of the spectators before us) that reinstalls at the core of the film-viewing phenomenon a regressive struggle with reality and with different degrees of realism, originally overcome by film since the Lumière’s Arrival of a Train at Ciotat seminal demonstration. Through an analysis of crucial aspects in Avatar and the recent Cave of Forgotten Dreams, both dealing with historical and ontological deepening processes of ‘going inside’, we shall try to show how the formal and technically advanced component of those 3D-depth films impairs, on the contrary, their apparent conceptual purpose on the level of contents, and we will assume, drawing on Merleau-Ponty and Deleuze, that this technological mistake is due to a lack of recognition of the nature of perception and sensation in relation to space and human experience.