Single-pulse transcranial magnetic stimulation over the frontal eye field can facilitate and inhibit saccade triggering

The aim of this study was to investigate the effect of single-pulse transcranial magnetic stimulation on the triggering of saccades. The right frontal eye field was stimulated during modified gap and overlap paradigms with flashed presentation of the lateral visual target of 80 ms. In order to examine possible facilitating or inhibitory effects on saccade triggering, three different time intervals of stimulation were chosen, i.e. simultaneously with onset of the target, during the presentation and after target end. Stimulation applied simultaneously with target onset significantly decreased the latency of contralateral saccades in the gap but not in the overlap paradigm. Stimulation after target end significantly increased saccade latency for both sides in the gap paradigm and for the contralateral side in the overlap paradigm. Stimulation during presentation had no effect in either paradigm. The results show that, depending on the time interval and the paradigm tested, a facilitation or inhibition of saccade triggering can be achieved. The results are discussed in a context of two probable transcranial magnetic stimulation effects, a direct interference with the frontal eye field on the one hand and a remote interference with the superior colliculus on the other hand.

Sight-reading of violinists: eye movements anticipate the musical flow

When sight-reading a piece of music the eyes constantly scan the score slightly ahead of music execution. This separation between reading and acting is commonly termed eye-hand span and can be expressed in two ways: as anticipation in notes or in time. Previous research, predominantly in piano players, found skill-dependent differences of eye-hand span. To date no study has explored visual anticipation in violinists. The present study investigated how structural properties of a piece of music affect the eye-hand span in a group of violinists. To this end eye movements and bow reversals were recorded synchronously while musicians sight-read a piece of music. The results suggest that structural differences of the score are reflected in the eye-hand span in a way similar to skill level. Specifically, the piece with higher complexity was associated with lower anticipation in notes, longer fixation duration and a tendency for more regressive fixations. Anticipation in time, however, remained the same (approximately 1 s) independently of the score played but was correlated with playing tempo. We conclude that the eye-hand span is not only influenced by the experience of the musician, but also by the structure of the score to be played.

Eye Movement Measures of Cognitive Control in Children with Tourette Syndrome

Tourette Syndrome begins in childhood and is characterized by uncontrollable repetitive actions like neck craning or hopping and noises such as sniffing or chirping. Worst in early adolescence, these tics wax and wane in severity and occur in bouts unpredictably, often drawing unwanted attention from bystanders. Making matters worse, over half of children with Tourette Syndrome also suffer from comorbid, or concurrent, disorders such as attention deficit hyperactivity disorder (ADHD) and obsessive-compulsive disorder (OCD). These disorders introduce anxious thoughts, impulsivity, inattention, and mood variability that further disrupt children with Tourette Syndrome from focusing and performing well at school and home. Thus, deficits in the cognitive control functions of response inhibition, response generation, and working memory have long been ascribed to Tourette Syndrome. Yet, without considering the effect of medication, age, and comorbidity, this is a premature attribution. This study used an infrared eye tracking camera and various computer tasks requiring eye movement responses to evaluate response inhibition, response generation, and working memory in Tourette Syndrome. This study, the first to control for medication, age, and comorbidity, enrolled 39 unmedicated children with Tourette Syndrome and 29 typically developing peers aged 10-16 years who completed reflexive and voluntary eye movement tasks and diagnostic rating scales to assess symptom severities of Tourette Syndrome, ADHD, and OCD. Children with Tourette Syndrome and comorbid ADHD and/or OCD, but not children with Tourette Syndrome only, took longer to respond and made more errors and distracted eye movements compared to typically-developing children, displaying cognitive control deficits. However, increasing symptom severities of Tourette Syndrome, ADHD, and OCD correlated with one another. Thus, cognitive control deficits were not specific to Tourette Syndrome patients with comorbid conditions, but rather increase with increasing tic severity, suggesting that a majority of Tourette Syndrome patients, regardless of a clinical diagnosis of ADHD and/or OCD, have symptoms of cognitive control deficits at some level. Therefore, clinicians should evaluate and counsel all families of children with Tourette Syndrome, with or without currently diagnosed ADHD and/or OCD, about the functional ramifications of comorbid symptoms and that they may wax and wane with tic severity.

Hand-held 3D light field photography and applications

We propose a method to acquire 3D light fields using a hand-held camera, and describe several computational photography applications facilitated by our approach. As our input we take an image sequence from a camera translating along an approximately linear path with limited camera rotations. Users can acquire such data easily in a few seconds by moving a hand-held camera. We include a novel approach to resample the input into regularly sampled 3D light fields by aligning them in the spatio-temporal domain, and a technique for high-quality disparity estimation from light fields. We show applications including digital refocusing and synthetic aperture blur, foreground removal, selective colorization, and others.

Fusion of fundus images and MRI data of the human eye

Purpose Ophthalmologists are confronted with a set of different image modalities to diagnose eye tumors e.g., fundus photography, CT and MRI. However, these images are often complementary and represent pathologies differently. Some aspects of tumors can only be seen in a particular modality. A fusion of modalities would improve the contextual information for diagnosis. The presented work attempts to register color fundus photography with MRI volumes. This would complement the low resolution 3D information in the MRI with high resolution 2D fundus images. Methods MRI volumes were acquired from 12 infants under the age of 5 with unilateral retinoblastoma. The contrast-enhanced T1-FLAIR sequence was performed with an isotropic resolution of less than 0.5mm. Fundus images were acquired with a RetCam camera. For healthy eyes, two landmarks were used: the optic disk and the fovea. The eyes were detected and extracted from the MRI volume using a 3D adaption of the Fast Radial Symmetry Transform (FRST). The cropped volume was automatically segmented using the Split Bregman algorithm. The optic nerve was enhanced by a Frangi vessel filter. By intersection the nerve with the retina the optic disk was found. The fovea position was estimated by constraining the position with the angle between the optic and the visual axis as well as the distance from the optic disk. The optical axis was detected automatically by fitting a parable on to the lens surface. On the fundus, the optic disk and the fovea were detected by using the method of Budai et al. Finally, the image was projected on to the segmented surface using the lens position as the camera center. In tumor affected eyes, the manually segmented tumors were used instead of the optic disk and macula for the registration. Results In all of the 12 MRI volumes that were tested the 24 eyes were found correctly, including healthy and pathological cases. In healthy eyes the optic nerve head was found in all of the tested eyes with an error of 1.08 +/- 0.37mm. A successful registration can be seen in figure 1. Conclusions The presented method is a step toward automatic fusion of modalities in ophthalmology. The combination enhances the MRI volume with higher resolution from the color fundus on the retina. Tumor treatment planning is improved by avoiding critical structures and disease progression monitoring is made easier.

Vision-based vehicle detection and tracking with a mobile camera using a statistical framework

En esta tesis se aborda la detección y el seguimiento automático de vehículos mediante técnicas de visión artificial con una cámara monocular embarcada. Este problema ha suscitado un gran interés por parte de la industria automovilística y de la comunidad científica ya que supone el primer paso en aras de la ayuda a la conducción, la prevención de accidentes y, en última instancia, la conducción automática. A pesar de que se le ha dedicado mucho esfuerzo en los últimos años, de momento no se ha encontrado ninguna solución completamente satisfactoria y por lo tanto continúa siendo un tema de investigación abierto. Los principales problemas que plantean la detección y seguimiento mediante visión artificial son la gran variabilidad entre vehículos, un fondo que cambia dinámicamente debido al movimiento de la cámara, y la necesidad de operar en tiempo real. En este contexto, esta tesis propone un marco unificado para la detección y seguimiento de vehículos que afronta los problemas descritos mediante un enfoque estadístico. El marco se compone de tres grandes bloques, i.e., generación de hipótesis, verificación de hipótesis, y seguimiento de vehículos, que se llevan a cabo de manera secuencial. No obstante, se potencia el intercambio de información entre los diferentes bloques con objeto de obtener el máximo grado posible de adaptación a cambios en el entorno y de reducir el coste computacional. Para abordar la primera tarea de generación de hipótesis, se proponen dos métodos complementarios basados respectivamente en el análisis de la apariencia y la geometría de la escena. Para ello resulta especialmente interesante el uso de un dominio transformado en el que se elimina la perspectiva de la imagen original, puesto que este dominio permite una búsqueda rápida dentro de la imagen y por tanto una generación eficiente de hipótesis de localización de los vehículos. Los candidatos finales se obtienen por medio de un marco colaborativo entre el dominio original y el dominio transformado. Para la verificación de hipótesis se adopta un método de aprendizaje supervisado. Así, se evalúan algunos de los métodos de extracción de características más populares y se proponen nuevos descriptores con arreglo al conocimiento de la apariencia de los vehículos. Para evaluar la efectividad en la tarea de clasificación de estos descriptores, y dado que no existen bases de datos públicas que se adapten al problema descrito, se ha generado una nueva base de datos sobre la que se han realizado pruebas masivas. Finalmente, se presenta una metodología para la fusión de los diferentes clasificadores y se plantea una discusión sobre las combinaciones que ofrecen los mejores resultados. El núcleo del marco propuesto está constituido por un método Bayesiano de seguimiento basado en filtros de partículas. Se plantean contribuciones en los tres elementos fundamentales de estos filtros: el algoritmo de inferencia, el modelo dinámico y el modelo de observación. En concreto, se propone el uso de un método de muestreo basado en MCMC que evita el elevado coste computacional de los filtros de partículas tradicionales y por consiguiente permite que el modelado conjunto de múltiples vehículos sea computacionalmente viable. Por otra parte, el dominio transformado mencionado anteriormente permite la definición de un modelo dinámico de velocidad constante ya que se preserva el movimiento suave de los vehículos en autopistas. Por último, se propone un modelo de observación que integra diferentes características. En particular, además de la apariencia de los vehículos, el modelo tiene en cuenta también toda la información recibida de los bloques de procesamiento previos. El método propuesto se ejecuta en tiempo real en un ordenador de propósito general y da unos resultados sobresalientes en comparación con los métodos tradicionales. ABSTRACT This thesis addresses on-road vehicle detection and tracking with a monocular vision system. This problem has attracted the attention of the automotive industry and the research community as it is the first step for driver assistance and collision avoidance systems and for eventual autonomous driving. Although many effort has been devoted to address it in recent years, no satisfactory solution has yet been devised and thus it is an active research issue. The main challenges for vision-based vehicle detection and tracking are the high variability among vehicles, the dynamically changing background due to camera motion and the real-time processing requirement. In this thesis, a unified approach using statistical methods is presented for vehicle detection and tracking that tackles these issues. The approach is divided into three primary tasks, i.e., vehicle hypothesis generation, hypothesis verification, and vehicle tracking, which are performed sequentially. Nevertheless, the exchange of information between processing blocks is fostered so that the maximum degree of adaptation to changes in the environment can be achieved and the computational cost is alleviated. Two complementary strategies are proposed to address the first task, i.e., hypothesis generation, based respectively on appearance and geometry analysis. To this end, the use of a rectified domain in which the perspective is removed from the original image is especially interesting, as it allows for fast image scanning and coarse hypothesis generation. The final vehicle candidates are produced using a collaborative framework between the original and the rectified domains. A supervised classification strategy is adopted for the verification of the hypothesized vehicle locations. In particular, state-of-the-art methods for feature extraction are evaluated and new descriptors are proposed by exploiting the knowledge on vehicle appearance. Due to the lack of appropriate public databases, a new database is generated and the classification performance of the descriptors is extensively tested on it. Finally, a methodology for the fusion of the different classifiers is presented and the best combinations are discussed. The core of the proposed approach is a Bayesian tracking framework using particle filters. Contributions are made on its three key elements: the inference algorithm, the dynamic model and the observation model. In particular, the use of a Markov chain Monte Carlo method is proposed for sampling, which circumvents the exponential complexity increase of traditional particle filters thus making joint multiple vehicle tracking affordable. On the other hand, the aforementioned rectified domain allows for the definition of a constant-velocity dynamic model since it preserves the smooth motion of vehicles in highways. Finally, a multiple-cue observation model is proposed that not only accounts for vehicle appearance but also integrates the available information from the analysis in the previous blocks. The proposed approach is proven to run near real-time in a general purpose PC and to deliver outstanding results compared to traditional methods.

Towards hand biometrics in mobile devices

The increasing demand of security oriented to mobile applications has raised the attention to biometrics, as a proper and suitable solution for providing secure environment to mobile devices. With this aim, this document presents a biometric system based on hand geometry oriented to mobile devices, involving a high degree of freedom in terms of illumination, hand rotation and distance to camera. The user takes a picture of their own hand in the free space, without requiring any flat surface to locate the hand, and without removals of rings, bracelets or watches. The proposed biometric system relies on an accurate segmentation procedure, able to isolate hands from any background; a feature extraction, invariant to orientation, illumination, distance to camera and background; and a user classification, based on k-Nearest Neighbor approach, able to provide an accurate results on individual identification. The proposed method has been evaluated with two own databases collected with a HTC mobile. First database contains 120 individuals, with 20 acquisitions of both hands. Second database is a synthetic database, containing 408000 images of hand samples in different backgrounds: tiles, grass, water, sand, soil and the like. The system is able to identify individuals properly with False Reject Rate of 5.78% and False Acceptance Rate of 0.089%, using 60 features (15 features per finger)

An approach to hand biometrics in mobile devices

This paper focuses on hand biometrics applied to images acquired from a mobile device. The system offers the possibility of identifying individuals based on features extracted from hand pictures obtained with a low-quality camera embedded on a mobile device. Furthermore, the acquisitions have been carried out regardless illumination control, orientation, distance to camera, and similar aspects. In addition, the whole system has been tested with an owned database. Finally, the results obtained (6.0% ± 0.2) and the algorithm structure are both promising in relation to a posterior mobile implementation

Biomechanics of smart wings in a bat robot: morphing wings using SMA actuators

This paper presents the design of a bat-like micro aerial vehicle with actuated morphing wings. NiTi shape memory alloys (SMAs) acting as artificial biceps and triceps muscles are used for mimicking the morphing wing mechanism of the bat flight apparatus. Our objective is twofold. Firstly, we have implemented a control architecture that allows an accurate and fast SMA actuation. This control makes use of the electrical resistance measurements of SMAs to adjust morphing wing motions. Secondly, the feasibility of using SMA actuation technology is evaluated for the application at hand. To this purpose, experiments are conducted to analyze the control performance in terms of nominal and overloaded operation modes of the SMAs. This analysis includes: (i) inertial forces regarding the stretchable wing membrane and aerodynamic loads, and (ii) uncertainties due to impact of airflow conditions over the resistance–motion relationship of SMAs. With the proposed control, morphing actuation speed can be increased up to 2.5 Hz, being sufficient to generate lift forces at a cruising speed of 5ms−1.

Design of a training tool for improving the use of hand-held detectors in humanitarian demining

Purpose – The purpose of this paper is to introduce the design of a training tool intended to improve deminers' technique during close-in detection tasks. Design/methodology/approach – Following an introduction that highlights the impact of mines and improvised explosive devices (IEDs), and the importance of training for enhancing the safety and the efficiency of the deminers, this paper considers the utilization of a sensory tracking system to study the skill of the hand-held detector expert operators. With the compiled information, some critical performance variables can be extracted, assessed, and quantified, so that they can be used afterwards as reference values for the training task. In a second stage, the sensory tracking system is used for analysing the trainee skills. The experimentation phase aims to test the effectiveness of the elements that compose the sensory system to track the hand-held detector during the training sessions. Findings – The proposed training tool will be able to evaluate the deminers' efficiency during the scanning tasks and will provide important information for improving their competences. Originality/value – This paper highlights the need of introducing emerging technologies for enhancing the current training techniques for deminers and proposes a sensory tracking system that can be successfully utilised for evaluating trainees' performance with hand-held detectors.

Bayesian scene analysis for multi-camera 3D tracking and camera positioning

Los sistemas de seguimiento mono-cámara han demostrado su notable capacidad para el análisis de trajectorias de objectos móviles y para monitorización de escenas de interés; sin embargo, tanto su robustez como sus posibilidades en cuanto a comprensión semántica de la escena están fuertemente limitadas por su naturaleza local y monocular, lo que los hace insuficientes para aplicaciones realistas de videovigilancia. El objetivo de esta tesis es la extensión de las posibilidades de los sistemas de seguimiento de objetos móviles para lograr un mayor grado de robustez y comprensión de la escena. La extensión propuesta se divide en dos direcciones separadas. La primera puede considerarse local, ya que está orientada a la mejora y enriquecimiento de las posiciones estimadas para los objetos móviles observados directamente por las cámaras del sistema; dicha extensión se logra mediante el desarrollo de un sistema multi-cámara de seguimiento 3D, capaz de proporcionar consistentemente las posiciones 3D de múltiples objetos a partir de las observaciones capturadas por un conjunto de sensores calibrados y con campos de visión solapados. La segunda extensión puede considerarse global, dado que su objetivo consiste en proporcionar un contexto global para relacionar las observaciones locales realizadas por una cámara con una escena de mucho mayor tamaño; para ello se propone un sistema automático de localización de cámaras basado en las trayectorias observadas de varios objetos móviles y en un mapa esquemático de la escena global monitorizada. Ambas líneas de investigación se tratan utilizando, como marco común, técnicas de estimación bayesiana: esta elección está justificada por la versatilidad y flexibilidad proporcionada por dicho marco estadístico, que permite la combinación natural de múltiples fuentes de información sobre los parámetros a estimar, así como un tratamiento riguroso de la incertidumbre asociada a las mismas mediante la inclusión de modelos de observación específicamente diseñados. Además, el marco seleccionado abre grandes posibilidades operacionales, puesto que permite la creación de diferentes métodos numéricos adaptados a las necesidades y características específicas de distintos problemas tratados. El sistema de seguimiento 3D con múltiples cámaras propuesto está específicamente diseñado para permitir descripciones esquemáticas de las medidas realizadas individualmente por cada una de las cámaras del sistema: esta elección de diseño, por tanto, no asume ningún algoritmo específico de detección o seguimiento 2D en ninguno de los sensores de la red, y hace que el sistema propuesto sea aplicable a redes reales de vigilancia con capacidades limitadas tanto en términos de procesamiento como de transmision. La combinación robusta de las observaciones capturadas individualmente por las cámaras, ruidosas, incompletas y probablemente contaminadas por falsas detecciones, se basa en un metodo de asociación bayesiana basado en geometría y color: los resultados de dicha asociación permiten el seguimiento 3D de los objetos de la escena mediante el uso de un filtro de partículas. El sistema de fusión de observaciones propuesto tiene, como principales características, una gran precisión en términos de localización 3D de objetos, y una destacable capacidad de recuperación tras eventuales errores debidos a un número insuficiente de datos de entrada. El sistema automático de localización de cámaras se basa en la observación de múltiples objetos móviles y un mapa esquemático de las áreas transitables del entorno monitorizado para inferir la posición absoluta de dicho sensor. Para este propósito, se propone un novedoso marco bayesiano que combina modelos dinámicos inducidos por el mapa en los objetos móviles presentes en la escena con las trayectorias observadas por la cámara, lo que representa un enfoque nunca utilizado en la literatura existente. El sistema de localización se divide en dos sub-tareas diferenciadas, debido a que cada una de estas tareas requiere del diseño de algoritmos específicos de muestreo para explotar en profundidad las características del marco desarrollado: por un lado, análisis de la ambigüedad del caso específicamente tratado y estimación aproximada de la localización de la cámara, y por otro, refinado de la localización de la cámara. El sistema completo, diseñado y probado para el caso específico de localización de cámaras en entornos de tráfico urbano, podría tener aplicación también en otros entornos y sensores de diferentes modalidades tras ciertas adaptaciones. ABSTRACT Mono-camera tracking systems have proved their capabilities for moving object trajectory analysis and scene monitoring, but their robustness and semantic possibilities are strongly limited by their local and monocular nature and are often insufficient for realistic surveillance applications. This thesis is aimed at extending the possibilities of moving object tracking systems to a higher level of scene understanding. The proposed extension comprises two separate directions. The first one is local, since is aimed at enriching the inferred positions of the moving objects within the area of the monitored scene directly covered by the cameras of the system; this task is achieved through the development of a multi-camera system for robust 3D tracking, able to provide 3D tracking information of multiple simultaneous moving objects from the observations reported by a set of calibrated cameras with semi-overlapping fields of view. The second extension is global, as is aimed at providing local observations performed within the field of view of one camera with a global context relating them to a much larger scene; to this end, an automatic camera positioning system relying only on observed object trajectories and a scene map is designed. The two lines of research in this thesis are addressed using Bayesian estimation as a general unifying framework. Its suitability for these two applications is justified by the flexibility and versatility of that stochastic framework, which allows the combination of multiple sources of information about the parameters to estimate in a natural and elegant way, addressing at the same time the uncertainty associated to those sources through the inclusion of models designed to this end. In addition, it opens multiple possibilities for the creation of different numerical methods for achieving satisfactory and efficient practical solutions to each addressed application. The proposed multi-camera 3D tracking method is specifically designed to work on schematic descriptions of the observations performed by each camera of the system: this choice allows the use of unspecific off-the-shelf 2D detection and/or tracking subsystems running independently at each sensor, and makes the proposal suitable for real surveillance networks with moderate computational and transmission capabilities. The robust combination of such noisy, incomplete and possibly unreliable schematic descriptors relies on a Bayesian association method, based on geometry and color, whose results allow the tracking of the targets in the scene with a particle filter. The main features exhibited by the proposal are, first, a remarkable accuracy in terms of target 3D positioning, and second, a great recovery ability after tracking losses due to insufficient input data. The proposed system for visual-based camera self-positioning uses the observations of moving objects and a schematic map of the passable areas of the environment to infer the absolute sensor position. To this end, a new Bayesian framework combining trajectory observations and map-induced dynamic models for moving objects is designed, which represents an approach to camera positioning never addressed before in the literature. This task is divided into two different sub-tasks, setting ambiguity analysis and approximate position estimation, on the one hand, and position refining, on the other, since they require the design of specific sampling algorithms to correctly exploit the discriminative features of the developed framework. This system, designed for camera positioning and demonstrated in urban traffic environments, can also be applied to different environments and sensors of other modalities after certain required adaptations.

Human-robot interaction for telemanipulation in large workspaces

Independientemente de la existencia de técnicas altamente sofisticadas y capacidades de cómputo cada vez más elevadas, los problemas asociados a los robots que interactúan con entornos no estructurados siguen siendo un desafío abierto en robótica. A pesar de los grandes avances de los sistemas robóticos autónomos, hay algunas situaciones en las que una persona en el bucle sigue siendo necesaria. Ejemplos de esto son, tareas en entornos de fusión nuclear, misiones espaciales, operaciones submarinas y cirugía robótica. Esta necesidad se debe a que las tecnologías actuales no pueden realizar de forma fiable y autónoma cualquier tipo de tarea. Esta tesis presenta métodos para la teleoperación de robots abarcando distintos niveles de abstracción que van desde el control supervisado, en el que un operador da instrucciones de alto nivel en la forma de acciones, hasta el control bilateral, donde los comandos toman la forma de señales de control de bajo nivel. En primer lugar, se presenta un enfoque para llevar a cabo la teleoperación supervisada de robots humanoides. El objetivo es controlar robots terrestres capaces de ejecutar tareas complejas en entornos de búsqueda y rescate utilizando enlaces de comunicación limitados. Esta propuesta incorpora comportamientos autónomos que el operador puede utilizar para realizar tareas de navegación y manipulación mientras se permite cubrir grandes áreas de entornos remotos diseñados para el acceso de personas. Los resultados experimentales demuestran la eficacia de los métodos propuestos. En segundo lugar, se investiga el uso de dispositivos rentables para telemanipulación guiada. Se presenta una aplicación que involucra un robot humanoide bimanual y un traje de captura de movimiento basado en sensores inerciales. En esta aplicación, se estudian las capacidades de adaptación introducidas por el factor humano y cómo estas pueden compensar la falta de sistemas robóticos de alta precisión. Este trabajo es el resultado de una colaboración entre investigadores del Biorobotics Laboratory de la Universidad de Harvard y el Centro de Automática y Robótica UPM-CSIC. En tercer lugar, se presenta un nuevo controlador háptico que combina velocidad y posición. Este controlador bilateral híbrido hace frente a los problemas relacionados con la teleoperación de un robot esclavo con un gran espacio de trabajo usando un dispositivo háptico pequeño como maestro. Se pueden cubrir amplias áreas de trabajo al cambiar automáticamente entre los modos de control de velocidad y posición. Este controlador háptico es ideal para sistemas maestro-esclavo con cinemáticas diferentes, donde los comandos se transmiten en el espacio de la tarea del entorno remoto. El método es validado para realizar telemanipulación hábil de objetos con un robot industrial. Por último, se introducen dos contribuciones en el campo de la manipulación robótica. Por un lado, se presenta un nuevo algoritmo de cinemática inversa, llamado método iterativo de desacoplamiento cinemático. Este método se ha desarrollado para resolver el problema cinemático inverso de un tipo de robot de seis grados de libertad donde una solución cerrada no está disponible. La eficacia del método se compara con métodos numéricos convencionales. Además, se ha diseñado una taxonomía robusta de agarres que permite controlar diferentes manos robóticas utilizando una correspondencia, basada en gestos, entre los espacios de trabajo de la mano humana y de la mano robótica. El gesto de la mano humana se identifica mediante la lectura de los movimientos relativos del índice, el pulgar y el dedo medio del usuario durante las primeras etapas del agarre. ABSTRACT Regardless of the availability of highly sophisticated techniques and ever increasing computing capabilities, the problems associated with robots interacting with unstructured environments remains an open challenge. Despite great advances in autonomous robotics, there are some situations where a humanin- the-loop is still required, such as, nuclear, space, subsea and robotic surgery operations. This is because the current technologies cannot reliably perform all kinds of task autonomously. This thesis presents methods for robot teleoperation strategies at different levels of abstraction ranging from supervisory control, where the operator gives high-level task actions, to bilateral teleoperation, where the commands take the form of low-level control inputs. These strategies contribute to improve the current human-robot interfaces specially in the case of slave robots deployed at large workspaces. First, an approach to perform supervisory teleoperation of humanoid robots is presented. The goal is to control ground robots capable of executing complex tasks in disaster relief environments under constrained communication links. This proposal incorporates autonomous behaviors that the operator can use to perform navigation and manipulation tasks which allow covering large human engineered areas of the remote environment. The experimental results demonstrate the efficiency of the proposed methods. Second, the use of cost-effective devices for guided telemanipulation is investigated. A case study involving a bimanual humanoid robot and an Inertial Measurement Unit (IMU) Motion Capture (MoCap) suit is introduced. Herein, it is corroborated how the adaptation capabilities offered by the human-in-the-loop factor can compensate for the lack of high-precision robotic systems. This work is the result of collaboration between researchers from the Harvard Biorobotics Laboratory and the Centre for Automation and Robotics UPM-CSIC. Thirdly, a new haptic rate-position controller is presented. This hybrid bilateral controller copes with the problems related to the teleoperation of a slave robot with large workspace using a small haptic device as master. Large workspaces can be covered by automatically switching between rate and position control modes. This haptic controller is ideal to couple kinematic dissimilar master-slave systems where the commands are transmitted in the task space of the remote environment. The method is validated to perform dexterous telemanipulation of objects with a robotic manipulator. Finally, two contributions for robotic manipulation are introduced. First, a new algorithm, the Iterative Kinematic Decoupling method, is presented. It is a numeric method developed to solve the Inverse Kinematics (IK) problem of a type of six-DoF robotic arms where a close-form solution is not available. The effectiveness of this IK method is compared against conventional numerical methods. Second, a robust grasp mapping has been conceived. It allows to control a wide range of different robotic hands using a gesture based correspondence between the human hand space and the robotic hand space. The human hand gesture is identified by reading the relative movements of the index, thumb and middle fingers of the user during the early stages of grasping.

BR3: a biologically inspired fish-like robot actuated by SMA-based artificial muscles

Los peces son animales, donde en la mayoría de los casos, son considerados como nadadores muy eficientes y con una alta capacidad de maniobra. En general los peces se caracterizan por su capacidad de maniobra, locomoción silencioso, giros y partidas rápidas y viajes de larga distancia. Los estudios han identificado varios tipos de locomoción que los peces usan para generar maniobras y natación constante. A bajas velocidades la mayoría de los peces utilizan sus aletas pares y / o impares para su locomoción, que ofrecen una mayor maniobrabilidad y mejor eficiencia de propulsión. A altas velocidades la locomoción implica el cuerpo y / o aleta caudal porque esto puede lograr un mayor empuje y aceleración. Estas características pueden inspirar el diseo y fabricación de una piel muy flexible, una aleta caudal mórfica y una espina dorsal no articulada con una gran capacidad de maniobra. Esta tesis presenta el desarrollo de un novedoso pez robot bio-inspirado y biomimético llamado BR3, inspirado en la capacidad de maniobra y nado constante de los peces vertebrados. Inspirado por la morfología de los peces Micropterus salmoides o también conocido como lubina negra, el robot BR3 utiliza su fundamento biológico para desarrollar modelos y métodos matemáticos precisos que permiten imitar la locomoción de los peces reales. Los peces Largemouth Bass pueden lograr un nivel increíble de maniobrabilidad y eficacia de la propulsión mediante la combinación de los movimientos ondulatorios y aletas morficas. Para imitar la locomoción de los peces reales en una contraparte artificial se necesita del análisis de tecnologías de actuación alternativos, como arreglos de fibras musculares en lugar de servo actuadores o motores DC estándar, así como un material flexible que proporciona una estructura continua sin juntas. Las aleaciones con memoria de forma (SMAs) proveen la posibilidad de construir robots livianos, que no emiten ruido, sin motores, sin juntas y sin engranajes. Asi es como un pez robot submarino se ha desarrollado y cuyos movimientos son generados mediante SMAs. Estos actuadores son los adecuados para doblar la espina dorsal continua del pez robot, que a su vez provoca un cambio en la curvatura del cuerpo. Este tipo de arreglo estructural está inspirado en los músculos rojos del pescado, que son usados principalmente durante la natación constante para la flexión de una estructura flexible pero casi incompresible como lo es la espina dorsal de pescado. Del mismo modo la aleta caudal se basa en SMAs y se modifica para llevar a cabo el trabajo necesario. La estructura flexible proporciona empuje y permite que el BR3 nade. Por otro lado la aleta caudal mórfica proporciona movimientos de balanceo y guiada. Motivado por la versatilidad del BR3 para imitar todos los modos de natación (anguilliforme, carangiforme, subcarangiforme y tunniforme) se propone un controlador de doblado y velocidad. La ley de control de doblado y velocidad incorpora la información del ángulo de curvatura y de la frecuencia para producir el modo de natación deseado y a su vez controlar la velocidad de natación. Así mismo de acuerdo con el hecho biológico de la influencia de la forma de la aleta caudal en la maniobrabilidad durante la natación constante se propone un control de actitud. Esta novedoso robot pescado es el primero de su tipo en incorporar sólo SMAs para doblar una estructura flexible continua y sin juntas y engranajes para producir empuje e imitar todos los modos de natación, así como la aleta caudal que es capaz de cambiar su forma. Este novedoso diseo mecatrónico presenta un futuro muy prometedor para el diseo de vehículos submarinos capaces de modificar su forma y nadar mas eficientemente. La nueva metodología de control propuesto en esta tesis proporcionan una forma totalmente nueva de control de robots basados en SMAs, haciéndolos energéticamente más eficientes y la incorporación de una aleta caudal mórfica permite realizar maniobras más eficientemente. En su conjunto, el proyecto BR3 consta de cinco grandes etapas de desarrollo: • Estudio y análisis biológico del nado de los peces con el propósito de definir criterios de diseño y control. • Formulación de modelos matemáticos que describan la: i) cinemática del cuerpo, ii) dinámica, iii) hidrodinámica iv) análisis de los modos de vibración y v) actuación usando SMA. Estos modelos permiten estimar la influencia de modular la aleta caudal y el doblado del cuerpo en la producción de fuerzas de empuje y fuerzas de rotación necesarias en las maniobras y optimización del consumo de energía. • Diseño y fabricación de BR3: i) estructura esquelética de la columna vertebral y el cuerpo, ii) mecanismo de actuación basado en SMAs para el cuerpo y la aleta caudal, iii) piel artificial, iv) electrónica embebida y v) fusión sensorial. Está dirigido a desarrollar la plataforma de pez robot BR3 que permite probar los métodos propuestos. • Controlador de nado: compuesto por: i) control de las SMA (modulación de la forma de la aleta caudal y regulación de la actitud) y ii) control de nado continuo (modulación de la velocidad y doblado). Está dirigido a la formulación de los métodos de control adecuados que permiten la modulación adecuada de la aleta caudal y el cuerpo del BR3. • Experimentos: está dirigido a la cuantificación de los efectos de: i) la correcta modulación de la aleta caudal en la producción de rotación y su efecto hidrodinámico durante la maniobra, ii) doblado del cuerpo para la producción de empuje y iii) efecto de la flexibilidad de la piel en la habilidad para doblarse del BR3. También tiene como objetivo demostrar y validar la hipótesis de mejora en la eficiencia de la natación y las maniobras gracias a los nuevos métodos de control presentados en esta tesis. A lo largo del desarrollo de cada una de las cinco etapas, se irán presentando los retos, problemáticas y soluciones a abordar. Los experimentos en canales de agua estarán orientados a discutir y demostrar cómo la aleta caudal y el cuerpo pueden afectar considerablemente la dinámica / hidrodinámica de natación / maniobras y cómo tomar ventaja de la modulación de curvatura que la aleta caudal mórfica y el cuerpo permiten para cambiar correctamente la geometría de la aleta caudal y del cuerpo durante la natación constante y maniobras. ABSTRACT Fishes are animals where in most cases are considered as highly manoeuvrable and effortless swimmers. In general fishes are characterized for his manoeuvring skills, noiseless locomotion, rapid turning, fast starting and long distance cruising. Studies have identified several types of locomotion that fish use to generate maneuvering and steady swimming. At low speeds most fishes uses median and/or paired fins for its locomotion, offering greater maneuverability and better propulsive efficiency At high speeds the locomotion involves the body and/or caudal fin because this can achieve greater thrust and accelerations. This can inspire the design and fabrication of a highly deformable soft artificial skins, morphing caudal fins and non articulated backbone with a significant maneuverability capacity. This thesis presents the development of a novel bio-inspired and biomimetic fishlike robot (BR3) inspired by the maneuverability and steady swimming ability of ray-finned fishes (Actinopterygii, bony fishes). Inspired by the morphology of the Largemouth Bass fish, the BR3 uses its biological foundation to develop accurate mathematical models and methods allowing to mimic fish locomotion. The Largemouth Bass fishes can achieve an amazing level of maneuverability and propulsive efficiency by combining undulatory movements and morphing fins. To mimic the locomotion of the real fishes on an artificial counterpart needs the analysis of alternative actuation technologies more likely muscle fiber arrays instead of standard servomotor actuators as well as a bendable material that provides a continuous structure without joins. The Shape Memory Alloys (SMAs) provide the possibility of building lightweight, joint-less, noise-less, motor-less and gear-less robots. Thus a swimming underwater fish-like robot has been developed whose movements are generated using SMAs. These actuators are suitable for bending the continuous backbone of the fish, which in turn causes a change in the curvature of the body. This type of structural arrangement is inspired by fish red muscles, which are mainly recruited during steady swimming for the bending of a flexible but nearly incompressible structure such as the fishbone. Likewise the caudal fin is based on SMAs and is customized to provide the necessary work out. The bendable structure provides thrust and allows the BR3 to swim. On the other hand the morphing caudal fin provides roll and yaw movements. Motivated by the versatility of the BR3 to mimic all the swimming modes (anguilliform, caranguiform, subcaranguiform and thunniform) a bending-speed controller is proposed. The bending-speed control law incorporates bend angle and frequency information to produce desired swimming mode and swimming speed. Likewise according to the biological fact about the influence of caudal fin shape in the maneuverability during steady swimming an attitude control is proposed. This novel fish robot is the first of its kind to incorporate only SMAs to bend a flexible continuous structure without joints and gears to produce thrust and mimic all the swimming modes as well as the caudal fin to be morphing. This novel mechatronic design is a promising way to design more efficient swimming/morphing underwater vehicles. The novel control methodology proposed in this thesis provide a totally new way of controlling robots based on SMAs, making them more energy efficient and the incorporation of a morphing caudal fin allows to perform more efficient maneuvers. As a whole, the BR3 project consists of five major stages of development: • Study and analysis of biological fish swimming data reported in specialized literature aimed at defining design and control criteria. • Formulation of mathematical models for: i) body kinematics, ii) dynamics, iii) hydrodynamics, iv) free vibration analysis and v) SMA muscle-like actuation. It is aimed at modelling the e ects of modulating caudal fin and body bend into the production of thrust forces for swimming, rotational forces for maneuvering and energy consumption optimisation. • Bio-inspired design and fabrication of: i) skeletal structure of backbone and body, ii) SMA muscle-like mechanisms for the body and caudal fin, iii) the artificial skin, iv) electronics onboard and v) sensor fusion. It is aimed at developing the fish-like platform (BR3) that allows for testing the methods proposed. • The swimming controller: i) control of SMA-muscles (morphing-caudal fin modulation and attitude regulation) and ii) steady swimming control (bend modulation and speed modulation). It is aimed at formulating the proper control methods that allow for the proper modulation of BR3’s caudal fin and body. • Experiments: it is aimed at quantifying the effects of: i) properly caudal fin modulation into hydrodynamics and rotation production for maneuvering, ii) body bending into thrust generation and iii) skin flexibility into BR3 bending ability. It is also aimed at demonstrating and validating the hypothesis of improving swimming and maneuvering efficiency thanks to the novel control methods presented in this thesis. This thesis introduces the challenges and methods to address these stages. Waterchannel experiments will be oriented to discuss and demonstrate how the caudal fin and body can considerably affect the dynamics/hydrodynamics of swimming/maneuvering and how to take advantage of bend modulation that the morphing-caudal fin and body enable to properly change caudal fin and body’ geometry during steady swimming and maneuvering.

Bayesian scene analysis for multi-camera 3D tracking and camera positioning

Los sistemas de seguimiento mono-cámara han demostrado su notable capacidad para el análisis de trajectorias de objectos móviles y para monitorización de escenas de interés; sin embargo, tanto su robustez como sus posibilidades en cuanto a comprensión semántica de la escena están fuertemente limitadas por su naturaleza local y monocular, lo que los hace insuficientes para aplicaciones realistas de videovigilancia. El objetivo de esta tesis es la extensión de las posibilidades de los sistemas de seguimiento de objetos móviles para lograr un mayor grado de robustez y comprensión de la escena. La extensión propuesta se divide en dos direcciones separadas. La primera puede considerarse local, ya que está orientada a la mejora y enriquecimiento de las posiciones estimadas para los objetos móviles observados directamente por las cámaras del sistema; dicha extensión se logra mediante el desarrollo de un sistema multi-cámara de seguimiento 3D, capaz de proporcionar consistentemente las posiciones 3D de múltiples objetos a partir de las observaciones capturadas por un conjunto de sensores calibrados y con campos de visión solapados. La segunda extensión puede considerarse global, dado que su objetivo consiste en proporcionar un contexto global para relacionar las observaciones locales realizadas por una cámara con una escena de mucho mayor tamaño; para ello se propone un sistema automático de localización de cámaras basado en las trayectorias observadas de varios objetos móviles y en un mapa esquemático de la escena global monitorizada. Ambas líneas de investigación se tratan utilizando, como marco común, técnicas de estimación bayesiana: esta elección está justificada por la versatilidad y flexibilidad proporcionada por dicho marco estadístico, que permite la combinación natural de múltiples fuentes de información sobre los parámetros a estimar, así como un tratamiento riguroso de la incertidumbre asociada a las mismas mediante la inclusión de modelos de observación específicamente diseñados. Además, el marco seleccionado abre grandes posibilidades operacionales, puesto que permite la creación de diferentes métodos numéricos adaptados a las necesidades y características específicas de distintos problemas tratados. El sistema de seguimiento 3D con múltiples cámaras propuesto está específicamente diseñado para permitir descripciones esquemáticas de las medidas realizadas individualmente por cada una de las cámaras del sistema: esta elección de diseño, por tanto, no asume ningún algoritmo específico de detección o seguimiento 2D en ninguno de los sensores de la red, y hace que el sistema propuesto sea aplicable a redes reales de vigilancia con capacidades limitadas tanto en términos de procesamiento como de transmision. La combinación robusta de las observaciones capturadas individualmente por las cámaras, ruidosas, incompletas y probablemente contaminadas por falsas detecciones, se basa en un metodo de asociación bayesiana basado en geometría y color: los resultados de dicha asociación permiten el seguimiento 3D de los objetos de la escena mediante el uso de un filtro de partículas. El sistema de fusión de observaciones propuesto tiene, como principales características, una gran precisión en términos de localización 3D de objetos, y una destacable capacidad de recuperación tras eventuales errores debidos a un número insuficiente de datos de entrada. El sistema automático de localización de cámaras se basa en la observación de múltiples objetos móviles y un mapa esquemático de las áreas transitables del entorno monitorizado para inferir la posición absoluta de dicho sensor. Para este propósito, se propone un novedoso marco bayesiano que combina modelos dinámicos inducidos por el mapa en los objetos móviles presentes en la escena con las trayectorias observadas por la cámara, lo que representa un enfoque nunca utilizado en la literatura existente. El sistema de localización se divide en dos sub-tareas diferenciadas, debido a que cada una de estas tareas requiere del diseño de algoritmos específicos de muestreo para explotar en profundidad las características del marco desarrollado: por un lado, análisis de la ambigüedad del caso específicamente tratado y estimación aproximada de la localización de la cámara, y por otro, refinado de la localización de la cámara. El sistema completo, diseñado y probado para el caso específico de localización de cámaras en entornos de tráfico urbano, podría tener aplicación también en otros entornos y sensores de diferentes modalidades tras ciertas adaptaciones. ABSTRACT Mono-camera tracking systems have proved their capabilities for moving object trajectory analysis and scene monitoring, but their robustness and semantic possibilities are strongly limited by their local and monocular nature and are often insufficient for realistic surveillance applications. This thesis is aimed at extending the possibilities of moving object tracking systems to a higher level of scene understanding. The proposed extension comprises two separate directions. The first one is local, since is aimed at enriching the inferred positions of the moving objects within the area of the monitored scene directly covered by the cameras of the system; this task is achieved through the development of a multi-camera system for robust 3D tracking, able to provide 3D tracking information of multiple simultaneous moving objects from the observations reported by a set of calibrated cameras with semi-overlapping fields of view. The second extension is global, as is aimed at providing local observations performed within the field of view of one camera with a global context relating them to a much larger scene; to this end, an automatic camera positioning system relying only on observed object trajectories and a scene map is designed. The two lines of research in this thesis are addressed using Bayesian estimation as a general unifying framework. Its suitability for these two applications is justified by the flexibility and versatility of that stochastic framework, which allows the combination of multiple sources of information about the parameters to estimate in a natural and elegant way, addressing at the same time the uncertainty associated to those sources through the inclusion of models designed to this end. In addition, it opens multiple possibilities for the creation of different numerical methods for achieving satisfactory and efficient practical solutions to each addressed application. The proposed multi-camera 3D tracking method is specifically designed to work on schematic descriptions of the observations performed by each camera of the system: this choice allows the use of unspecific off-the-shelf 2D detection and/or tracking subsystems running independently at each sensor, and makes the proposal suitable for real surveillance networks with moderate computational and transmission capabilities. The robust combination of such noisy, incomplete and possibly unreliable schematic descriptors relies on a Bayesian association method, based on geometry and color, whose results allow the tracking of the targets in the scene with a particle filter. The main features exhibited by the proposal are, first, a remarkable accuracy in terms of target 3D positioning, and second, a great recovery ability after tracking losses due to insufficient input data. The proposed system for visual-based camera self-positioning uses the observations of moving objects and a schematic map of the passable areas of the environment to infer the absolute sensor position. To this end, a new Bayesian framework combining trajectory observations and map-induced dynamic models for moving objects is designed, which represents an approach to camera positioning never addressed before in the literature. This task is divided into two different sub-tasks, setting ambiguity analysis and approximate position estimation, on the one hand, and position refining, on the other, since they require the design of specific sampling algorithms to correctly exploit the discriminative features of the developed framework. This system, designed for camera positioning and demonstrated in urban traffic environments, can also be applied to different environments and sensors of other modalities after certain required adaptations.

Advanced Control Strategies for a 6 DoF Hydraulic Parallel Robot Based on the Dynamic Model

Nowadays robots have made their way into real applications that were prohibitive and unthinkable thirty years ago. This is mainly due to the increase in power computations and the evolution in the theoretical field of robotics and control. Even though there is plenty of information in the current literature on this topics, it is not easy to find clear concepts of how to proceed in order to design and implement a controller for a robot. In general, the design of a controller requires of a complete understanding and knowledge of the system to be controlled. Therefore, for advanced control techniques the systems must be first identified. Once again this particular objective is cumbersome and is never straight forward requiring of great expertise and some criteria must be adopted. On the other hand, the particular problem of designing a controller is even more complex when dealing with Parallel Manipulators (PM), since their closed-loop structures give rise to a highly nonlinear system. Under this basis the current work is developed, which intends to resume and gather all the concepts and experiences involve for the control of an Hydraulic Parallel Manipulator. The main objective of this thesis is to provide a guide remarking all the steps involve in the designing of advanced control technique for PMs. The analysis of the PM under study is minced up to the core of the mechanism: the hydraulic actuators. The actuators are modeled and experimental identified. Additionally, some consideration regarding traditional PID controllers are presented and an adaptive controller is finally implemented. From a macro perspective the kinematic and dynamic model of the PM are presented. Based on the model of the system and extending the adaptive controller of the actuator, a control strategy for the PM is developed and its performance is analyzed with simulation.