A Virtual Joystick as a Novel Interaction Modality for Surgeon-Computer Interaction During Computer-Assisted Surgical Procedures

Interacting with a computer system in the operating room (OR) can be a frustrating experience for a surgeon, who currently has to verbally delegate to an assistant every computer interaction task. This indirect mode of interaction is time consuming, error prone and can lead to poor usability of OR computer systems. This thesis describes the design and evaluation of a joystick-like device that allows direct surgeon control of the computer in the OR. The device was tested extensively in comparison to a mouse and delegated dictation with seven surgeons, eleven residents, and five graduate students. The device contains no electronic parts, is easy to use, is unobtrusive, has no physical connection to the computer and makes use of an existing tool in the OR. We performed a user study to determine its effectiveness in allowing a user to perform all the tasks they would be expected to perform on an OR computer system during a computer-assisted surgery. Dictation was found to be superior to the joystick in qualitative measures, but the joystick was preferred over dictation in user satisfaction responses. The mouse outperformed both joystick and dictation, but it is not a readily accepted modality in the OR.

A Virtual Joystick as a Novel Interaction Modality for Surgeon-Computer Interaction During Computer-Assisted Surgical Procedures

Interacting with a computer system in the operating room (OR) can be a frustrating experience for a surgeon, who currently has to verbally delegate to an assistant every computer interaction task. This indirect mode of interaction is time consuming, error prone and can lead to poor usability of OR computer systems. This thesis describes the design and evaluation of a joystick-like device that allows direct surgeon control of the computer in the OR. The device was tested extensively in comparison to a mouse and delegated dictation with seven surgeons, eleven residents, and five graduate students. The device contains no electronic parts, is easy to use, is unobtrusive, has no physical connection to the computer and makes use of an existing tool in the OR. We performed a user study to determine its effectiveness in allowing a user to perform all the tasks they would be expected to perform on an OR computer system during a computer-assisted surgery. Dictation was found to be superior to the joystick in qualitative measures, but the joystick was preferred over dictation in user satisfaction responses. The mouse outperformed both joystick and dictation, but it is not a readily accepted modality in the OR.

Sistema de treino para pilotos de parapente

Mestrado em Engenharia Electrotécnica – Sistemas Eléctricos de Energia

Evaluation of distinct input methods of an intelligent wheelchair in simulated and real environments: a performance and usability study

This paper focuses on evaluating the usability of an Intelligent Wheelchair (IW) in both real and simulated environments. The wheelchair is controlled at a high-level by a flexible multimodal interface, using voice commands, facial expressions, head movements and joystick as its main inputs. A Quasi-experimental design was applied including a deterministic sample with a questionnaire that enabled to apply the System Usability Scale. The subjects were divided in two independent samples: 46 individuals performing the experiment with an Intelligent Wheelchair in a simulated environment (28 using different commands in a sequential way and 18 with the liberty to choose the command); 12 individuals performing the experiment with a real IW. The main conclusion achieved by this study is that the usability of the Intelligent Wheelchair in a real environment is higher than in the simulated environment. However there were not statistical evidences to affirm that there are differences between the real and simulated wheelchairs in terms of safety and control. Also, most of users considered the multimodal way of driving the wheelchair very practical and satisfactory. Thus, it may be concluded that the multimodal interfaces enables very easy and safe control of the IW both in simulated and real environments.

Interação multimodal de dispositivos robóticos em ambiente simulado

Dissertação de mestrado integrado em Engenharia e Gestão de Sistemas de Informação

Etude d'administration contrôlée de cannabis et profils cinétiques des cannabinoïdes dans les fluides biologiques

Le cannabis est l'une des drogues illégales les plus consommées au monde. Le delta-9- tétrahydrocannabinol (THC) est le composé psychoactif majeur du cannabis et est fréquemment détecté dans le sang de conducteurs impliqués dans un accident de la route, alors qu'il est actuellement interdit de conduire sous l'influence du THC en Suisse comme dans de nombreux pays européens. La détection de ce composé dans le sang est suivie d'une série de mesures pénales et administratives à l'encontre du conducteur impliqué. Cette thèse intitulée « Etude d'administration contrôlée de cannabis et profils cinétiques dans les fluides biologiques » s'inscrit dans une étude toxicologique menée au Centre Hospitalier Universitaire Vaudois (CHUV). Le but de cette étude est d'évaluer les effets du cannabis sur le fonctionnement du cerveau lorsque le consommateur accompli une tâche fondamentale de la conduite automobile. Pour cela, l'imagerie médicale par résonnance magnétique fonctionnelle (IRMf) est combinée à un test informatisé simulant une tâche psychomotrice requise pour une conduite automobile sûre. Il s'agit d'un test de double tâche de poursuite d'une cible au moyen d'un curseur dirigé par un joystick, combiné à une tâche secondaire de détection de signaux routiers. Parallèlement, les profils cinétiques sanguins et salivaires des volontaires ont été déterminés grâce à des prélèvements effectués tout au long de la journée d'étude. Les objectifs principaux de ce travail de thèse sont les suivants : après une revue de la littérature existante sur les techniques d'analyse des cannabinoïdes dans les fluides biologiques, une méthode a été développée et validée pour ces composés dans la salive puis appliquée aux échantillons de l'étude. En parallèle, les échantillons sanguins et urinaires ont été analysés, en partie avec une méthode adaptée de celle développée pour la salive. Pour montrer la versatilité de cette méthode, celle-ci a été employée pour analyser des échantillons de bile qui a l'avantage de contenir des concentrations élevées de métabolites conjugués. Dans un deuxième temps, les dosages effectués sur le sang et la salive ont permis d'établir les profils cinétiques des cannabinoïdes, de déterminer leurs paramètres pharmacocinétiques et de les comparer. Les données obtenues ont montré que la concentration sanguine en acide 11-nor-9- carboxy-delta-9-tétrahydrocannabinol (THCCOOH) pouvait servir à distinguer les fumeurs occasionnels des fumeurs réguliers de cannabis, car elle est significativement plus élevée chez les consommateurs réguliers. De plus, d'autres cannabinoïdes présents dans la salive pourraient servir de marqueurs de consommation récente dans ce fluide biologique. Enfin, que ce soit dans le sang, la salive ou l'urine, des phytocannabinoïdes peuvent être utilisés comme marqueur de consommation de cannabis illégal car ils sont absents des médicaments à base de cannabis utilisant des préparations synthétiques ou purifiées du THC.

Acute Effects of Cannabis Smoking on Skills Related to Driving : an fMRI Study

Introduction : Driving is a complex everyday task requiring mechanisms of perception, attention, learning, memory, decision making and action control, thus indicating that involves numerous and varied brain networks. If many data have been accumulated over time about the effects of alcohol consumption on driving capability, much less is known about the role of other psychoactive substances, such as cannabis (Chang et al.2007, Ramaekers et al, 2006). Indeed, the solicited brain areas during safe driving which could be affected by cannabis exposure have not yet been clearly identified. Our aim is to study these brain regions during a tracking task related to driving skills and to evaluate the modulation due to the tolerance of cannabis effects. Methods : Eight non-smoker control subjects participated to an fMRI experiment based on a visuo-motor tracking task, alternating active tracking blocks with passive tracking viewing and rest condition. Half of the active tracking conditions included randomly presented traffic lights as distractors. Subjects were asked to track with a joystick with their right hand and to press a button with their left index at each appearance of a distractor. Four smoking subjects participated to the same fMRI sessions once before and once after smoking cannabis and a placebo in two independent cross-over experiments. We quantified the performance of the subjects by measuring the precision of the behavioural responses (i.e. percentage of time of correct tracking and reaction times to distractors). Functional MRI data were acquired using on a 3.0T Siemens Trio system equipped with a 32-channel head coil. BOLD signals will be obtained with a gradient-echo EPI sequence (TR=2s, TE=30ms, FoV=216mm, FA=90°, matrix size 72×72, 32 slices, thickness 3mm). Preprocessing, single subject analysis and group statistics were conducted on SPM8b. Results were thresholded at p<0.05 (FWE corrected) and at k>30 for spatial extent. Results : Behavioural results showed a significant impairment in task and cognitive test performance of the subjects after cannabis inhalation when comparing their tracking accuracy either to the controls subjects or to their performances before the inhalation or after the placebo inhalation (p<0.001 corrected). In controls, fMRI BOLD analysis of the active tracking condition compared to the passive one revealed networks of polymodal areas in superior frontal and parietal cortex dealing with attention and visuo-spatial coordination. In accordance to what is known of the visual and sensory motor networks we found activations in V4, frontal eye-field, right middle frontal gyrus, intra-parietal sulcus, temporo-parietal junction, premotor and sensory-motor cortex. The presence of distractors added a significant activation in the precuneus. Preliminary results on cannabis smokers in the acute phase, compared either to themselves before the cannabis inhalation or to control subjects, showed a decreased activation in large portions of the frontal and parietal attention network during the simple tracking task, but greater involvement of precuneus, of the superior part of intraparietal sulcus and middle frontal gyrus bilaterally when distractors were present in the task. Conclusions : Our preliminary results suggest that acute cannabis smoking alters performances and brain activity during active tracking tasks, partly reorganizing the recruitment of brain areas of the attention network.

Intracellular nanomanipulation by a photonic-force microscope with real-time acquisition of a 3D stiffness matrix.

A traditional photonic-force microscope (PFM) results in huge sets of data, which requires tedious numerical analysis. In this paper, we propose instead an analog signal processor to attain real-time capabilities while retaining the richness of the traditional PFM data. Our system is devoted to intracellular measurements and is fully interactive through the use of a haptic joystick. Using our specialized analog hardware along with a dedicated algorithm, we can extract the full 3D stiffness matrix of the optical trap in real time, including the off-diagonal cross-terms. Our system is also capable of simultaneously recording data for subsequent offline analysis. This allows us to check that a good correlation exists between the classical analysis of stiffness and our real-time measurements. We monitor the PFM beads using an optical microscope. The force-feedback mechanism of the haptic joystick helps us in interactively guiding the bead inside living cells and collecting information from its (possibly anisotropic) environment. The instantaneous stiffness measurements are also displayed in real time on a graphical user interface. The whole system has been built and is operational; here we present early results that confirm the consistency of the real-time measurements with offline computations.

Using the optical flow to implement a relative virtual mouse controlled by head movements

The following paper introduces the work conducted to create a relative virtual mouse based on the interpretation of head movements and face gesture through a low cost camera and the optical flow of the images. This virtual device is designed specifically as an alternative non-contact pointer for people with mobility impairments in the upper extremities and reduced head control. The proposed virtual device was compared with a conventional mouse, a touchpad and a digital joystick. Validation results show performances close to a digital joystick but far away from a conventional mouse.

Novel control, haptic and calibration methods for teleoperated electrohydraulic servo systems

The aim of this thesis is to propose a novel control method for teleoperated electrohydraulic servo systems that implements a reliable haptic sense between the human and manipulator interaction, and an ideal position control between the manipulator and the task environment interaction. The proposed method has the characteristics of a universal technique independent of the actual control algorithm and it can be applied with other suitable control methods as a real-time control strategy. The motivation to develop this control method is the necessity for a reliable real-time controller for teleoperated electrohydraulic servo systems that provides highly accurate position control based on joystick inputs with haptic capabilities. The contribution of the research is that the proposed control method combines a directed random search method and a real-time simulation to develop an intelligent controller in which each generation of parameters is tested on-line by the real-time simulator before being applied to the real process. The controller was evaluated on a hydraulic position servo system. The simulator of the hydraulic system was built based on Markov chain Monte Carlo (MCMC) method. A Particle Swarm Optimization algorithm combined with the foraging behavior of E. coli bacteria was utilized as the directed random search engine. The control strategy allows the operator to be plugged into the work environment dynamically and kinetically. This helps to ensure the system has haptic sense with high stability, without abstracting away the dynamics of the hydraulic system. The new control algorithm provides asymptotically exact tracking of both, the position and the contact force. In addition, this research proposes a novel method for re-calibration of multi-axis force/torque sensors. The method makes several improvements to traditional methods. It can be used without dismantling the sensor from its application and it requires smaller number of standard loads for calibration. It is also more cost efficient and faster in comparison to traditional calibration methods. The proposed method was developed in response to re-calibration issues with the force sensors utilized in teleoperated systems. The new approach aimed to avoid dismantling of the sensors from their applications for applying calibration. A major complication with many manipulators is the difficulty accessing them when they operate inside a non-accessible environment; especially if those environments are harsh; such as in radioactive areas. The proposed technique is based on design of experiment methodology. It has been successfully applied to different force/torque sensors and this research presents experimental validation of use of the calibration method with one of the force sensors which method has been applied to.

Estimation of distances in virtual environments using size constancy

It is reported in the literature that distances from the observer are underestimated more in virtual environments (VEs) than in physical world conditions. On the other hand estimation of size in VEs is quite accurate and follows a size-constancy law when rich cues are present. This study investigates how estimation of distance in a CAVETM environment is affected by poor and rich cue conditions, subject experience, and environmental learning when the position of the objects is estimated using an experimental paradigm that exploits size constancy. A group of 18 healthy participants was asked to move a virtual sphere controlled using the wand joystick to the position where they thought a previously-displayed virtual cube (stimulus) had appeared. Real-size physical models of the virtual objects were also presented to the participants as a reference of real physical distance during the trials. An accurate estimation of distance implied that the participants assessed the relative size of sphere and cube correctly. The cube appeared at depths between 0.6 m and 3 m, measured along the depth direction of the CAVE. The task was carried out in two environments: a poor cue one with limited background cues, and a rich cue one with textured background surfaces. It was found that distances were underestimated in both poor and rich cue conditions, with greater underestimation in the poor cue environment. The analysis also indicated that factors such as subject experience and environmental learning were not influential. However, least square fitting of Stevens’ power law indicated a high degree of accuracy during the estimation of object locations. This accuracy was higher than in other studies which were not based on a size-estimation paradigm. Thus as indirect result, this study appears to show that accuracy when estimating egocentric distances may be increased using an experimental method that provides information on the relative size of the objects used.

Atividades de aquisição de dados no laboratório de física em cursos de tecnologia

No presente trabalho é proposto um conjunto de atividades de aquisição de dados para um laboratório didático de Física, no contexto de ensino tecnológico de escolas técnicas. São construídas rotinas de programação com a ferramenta Visual Basic for Applications (VBA), incorporadas em planilhas Excel, sendo a aquisição de dados através da porta de joystick da placa de som do microcomputador. As atividades propostas envolvem especialmente experimentos no campo da mecânica, no qual são investigados o sistema massa-mola, pêndulo simples e movimento de rotação de um pião. A metodologia proposta para estas atividades é embasada na teórica construtivista de Vygotsky. O material foi aplicado e avaliado em uma oficina oferecida para alunos de cursos superiores de tecnologia (Automação Industrial e Manutenção Industrial e Sistemas de Informação) do CEFET/PR, unidade sudoeste do Paraná em 2005. O produto educacional deste trabalho consiste em um texto de apoio sobre medidas físicas e erros e sobre o uso da linguagem VBA em sistemas de aquisição de dados com a placa de som do microcomputador, e rotinas incorporadas a planilhas Excel que permitem a aquisição automática dos dados trabalhados nas atividades experimentais. No texto de apoio, também são sugeridas atividades para os alunos. As planilhas Excel (incluindo as macros VBA) constam o CD-ROM anexo a este trabalho, e estão disponibilizadas na web.

Development of an active orthosis prototype for lower limbs

ARAUJO, Márcio V. ; ALSINA, Pablo J. ; MEDEIROS, Adelardo A. D. ; PEREIRA, Jonathan P.P. ; DOMINGOS, Elber C. ; ARAÚJO, Fábio M.U. ; SILVA, Jáder S. . Development of an Active Orthosis Prototype for Lower Limbs. In: INTERNATIONAL CONGRESS OF MECHANICAL ENGINEERING, 20., 2009, Gramado, RS. Proceedings… Gramado, RS: [s. n.], 2009

Investigação da relação entre conteúdo onírico e aprendizado de uma tarefa cognitiva complexa

Several lines of evidence indicate that sleep is beneficial for learning, but there is no experimental evidence yet that the content of dreams is adaptive, i.e., that dreams help the dreamer to cope with challenges of the following day. Our aim here is to investigate the role of dreams in the acquisition of a complex cognitive task. We investigated electroencephalographic recordings and dream reports of adult subjects exposed to a computer game comprising perceptual, motor, spatial, emotional and higher-level cognitive aspects (Doom). Subjects slept two nights in the sleep laboratory, a completely dark room with a comfortable bed and controlled temperature. Electroencephalographic recordings with 28 channels were continuously performed throughout the experiment to identify episodes of rapid-eye-movement (REM) sleep. Behaviors were continuously recorded in audio and video with an infrared camera. Dream reports were collected upon forced awakening from late REM sleep, and again in the morning after spontaneous awakening. On day 1, subjects were habituated to the sleep laboratory, no computer game was played, and negative controls for gamerelated dream reports were collected. On day 2, subjects played the computer game before and after sleep. Each game session lasted for an hour, and sleep for 7-9 hours. 9 different measures of performance indicated significant improve overnight. 81% of the subjects experienced intrusion of elements of the game into their dreams, including potentially adaptative strategies (insights). There was a linear correlation between performance and dream intrusion as well as for game improval and quantity of reported dreaming. In the electrophysiological analysis we mapped the subjects brain activities in different stages (SWS 1, REM 1, SWS 2, REM 2, Game 1 and Game 2), and found a modest reverberation in motor areas related to the joystick control during the sleep. When separated by gender, we found a significant difference on female subjects in the channels that indicate motor learning. Analysis of dream reports showed that the amount of gamerelated elements in dreams correlated with performance gains according to an inverted-U function analogous to the Yerkes-Dodson law that governs the relationship between arousal and learning. The results indicate that dreaming is an adaptive behavior

Projeto e concepção de um quadcopter utilizando o controle PID

The present work refers to the design, conception and development of a quadcopter based on PID controllers, a widespread microcontroller on the market was being implemented, the Arduino. Also made use of LabVIEW programming tool from National Instruments company for the quadcopter control and telemetry. For the control, LabVIEW software acquired the joystick commands, making the necessary adjustments to the perfect interpretation by microcontroller on the quadcopter and adjusts the parameters of PID controllers. For telemetry, data relating to quadcopter behavior are received, interpreted and presented in an intuitive user interface. In the first part of this graduate work presents the theoretical background on the subject, with a brief history about the quadcopters, followed by the main projects in the academic and commercial matters. Also are presented the theories of communications used in the design and PID control. Then an overview of the dynamic and mathematical model is demonstrated. Having done this, the physical and computer components required to complete the project are showed and the results are achieved consequently. Finally, a conclusion is made taking into account the results obtained. In this work will be presented the PID control of quadcopter translational movements only to roll and pitch movements