Analysis of Local Search Methods for 3D Data

In questa tesi sono stati analizzati alcuni metodi di ricerca per dati 3D. Viene illustrata una panoramica generale sul campo della Computer Vision, sullo stato dell’arte dei sensori per l’acquisizione e su alcuni dei formati utilizzati per la descrizione di dati 3D. In seguito è stato fatto un approfondimento sulla 3D Object Recognition dove, oltre ad essere descritto l’intero processo di matching tra Local Features, è stata fatta una focalizzazione sulla fase di detection dei punti salienti. In particolare è stato analizzato un Learned Keypoint detector, basato su tecniche di apprendimento di machine learning. Quest ultimo viene illustrato con l’implementazione di due algoritmi di ricerca di vicini: uno esauriente (K-d tree) e uno approssimato (Radial Search). Sono state riportate infine alcune valutazioni sperimentali in termini di efficienza e velocità del detector implementato con diversi metodi di ricerca, mostrando l’effettivo miglioramento di performance senza una considerabile perdita di accuratezza con la ricerca approssimata.

Computational Foundations for Safe and Efficient Human-Robot Collaboration in Assembly Cells

Human and robots have complementary strengths in performing assembly operations. Humans are very good at perception tasks in unstructured environments. They are able to recognize and locate a part from a box of miscellaneous parts. They are also very good at complex manipulation in tight spaces. The sensory characteristics of the humans, motor abilities, knowledge and skills give the humans the ability to react to unexpected situations and resolve problems quickly. In contrast, robots are very good at pick and place operations and highly repeatable in placement tasks. Robots can perform tasks at high speeds and still maintain precision in their operations. Robots can also operate for long periods of times. Robots are also very good at applying high forces and torques. Typically, robots are used in mass production. Small batch and custom production operations predominantly use manual labor. The high labor cost is making it difficult for small and medium manufacturers to remain cost competitive in high wage markets. These manufactures are mainly involved in small batch and custom production. They need to find a way to reduce the labor cost in assembly operations. Purely robotic cells will not be able to provide them the necessary flexibility. Creating hybrid cells where humans and robots can collaborate in close physical proximities is a potential solution. The underlying idea behind such cells is to decompose assembly operations into tasks such that humans and robots can collaborate by performing sub-tasks that are suitable for them. Realizing hybrid cells that enable effective human and robot collaboration is challenging. This dissertation addresses the following three computational issues involved in developing and utilizing hybrid assembly cells: - We should be able to automatically generate plans to operate hybrid assembly cells to ensure efficient cell operation. This requires generating feasible assembly sequences and instructions for robots and human operators, respectively. Automated planning poses the following two challenges. First, generating operation plans for complex assemblies is challenging. The complexity can come due to the combinatorial explosion caused by the size of the assembly or the complex paths needed to perform the assembly. Second, generating feasible plans requires accounting for robot and human motion constraints. The first objective of the dissertation is to develop the underlying computational foundations for automatically generating plans for the operation of hybrid cells. It addresses both assembly complexity and motion constraints issues. - The collaboration between humans and robots in the assembly cell will only be practical if human safety can be ensured during the assembly tasks that require collaboration between humans and robots. The second objective of the dissertation is to evaluate different options for real-time monitoring of the state of human operator with respect to the robot and develop strategies for taking appropriate measures to ensure human safety when the planned move by the robot may compromise the safety of the human operator. In order to be competitive in the market, the developed solution will have to include considerations about cost without significantly compromising quality. - In the envisioned hybrid cell, we will be relying on human operators to bring the part into the cell. If the human operator makes an error in selecting the part or fails to place it correctly, the robot will be unable to correctly perform the task assigned to it. If the error goes undetected, it can lead to a defective product and inefficiencies in the cell operation. The reason for human error can be either confusion due to poor quality instructions or human operator not paying adequate attention to the instructions. In order to ensure smooth and error-free operation of the cell, we will need to monitor the state of the assembly operations in the cell. The third objective of the dissertation is to identify and track parts in the cell and automatically generate instructions for taking corrective actions if a human operator deviates from the selected plan. Potential corrective actions may involve re-planning if it is possible to continue assembly from the current state. Corrective actions may also involve issuing warning and generating instructions to undo the current task.

A hybrid system for upper limb movement restoration in quadriplegics

Generally, quadriplegic individuals have difficulties performing object manipulation. Toward satisfactory manipulation, reach and grasp movements must be performed with voluntary control, and for that, grasp force feedback is essential. A hybrid system aiming at partial upper limb sensory-motor restoration for quadriplegics was built. Such device is composed of an elbow dynamic orthosis that provides elbow flexion/extension (range was approximately from 20 degrees to 120 degrees, and average angular speed was approximately 15 degrees/s) with forearm support, a wrist static orthosis and neuromuscular electrical stimulation for grasping generation, and a glove with force sensors that allows grasping force feedback. The glove presents two user interface modes: visual by light emitting diodes or audio emitted by buzzer. Voice control of the entire system (elbow dynamic orthosis and electrical stimulator) is performed by the patient. The movements provided by the hybrid system, combined with the scapular and shoulder movements performed by the patient, can aid quadriplegic individuals in tasks that involve reach and grasp movements.

La reconnaissance des objets sonores chez l'humain

Abstract : This thesis investigated the spatio-temporal brain mechanisms of three processes involved in recognizing environmental sounds produced by living (animal vocalisations) and man-made (manufactured) objects: their discrimination, their plasticity, and the involvement of action representations. Results showed rapid brain discrimination between these categories beginning at ~70ms. Then, beginning at ~150ms, effects of plasticity are observed, without any influence of the categories of sounds. Both of these processes of discrimination and repetition priming involved brain structures located in temporal and frontal lobes. Activation of brain areas BA21 and BA22 suggest an access to semantic representations and/or linked to object manipulation. To investigate the involvement of action representations in sound recognition, analyses were restricted to sounds produced by man-made objects. Results suggest an access to representations linked to action functionally related to sound rather than to representations linked to action that produced sound. These effects occurred at ~300ms post-stimulus onset and involved differential activity brain regions attributed to the mirror neuron system. These data are discussed in regard to motor preparation of actions functionally linked to sounds. Collectively these data showed a sequential progression of cerebral activity underlying the recognizing of environmental sounds. The processes occurred firstly in a shared network of brain areas before propagating elsewhere and/or leading to differential activity in these structures. Cerebral responses observed in this work allowed establishing a dynamic model of discrimination of sounds produced by living and man-made objects.

Interaction between affordance and handedness recognition: a chronometric study

The visualization of tools and manipulable objects activates motor-related areas in the cortex, facilitating possible actions toward them. This pattern of activity may underlie the phenomenon of object affordance. Some cortical motor neurons are also covertly activated during the recognition of body parts such as hands. One hypothesis is that different subpopulations of motor neurons in the frontal cortex are activated in each motor program; for example, canonical neurons in the premotor cortex are responsible for the affordance of visual objects, while mirror neurons support motor imagery triggered during handedness recognition. However, the question remains whether these subpopulations work independently. This hypothesis can be tested with a manual reaction time (MRT) task with a priming paradigm to evaluate whether the view of a manipulable object interferes with the motor imagery of the subject's hand. The MRT provides a measure of the course of information processing in the brain and allows indirect evaluation of cognitive processes. Our results suggest that canonical and mirror neurons work together to create a motor plan involving hand movements to facilitate successful object manipulation.

Encodage des forces tactiles dans le cortex somatosensoriel primaire

Les deux fonctions principales de la main sont la manipulation d’objet et l’exploration tactile. La détection du glissement, rapportée par les mécanorécepteurs de la peau glabre, est essentielle pour l’exécution de ces deux fonctions. Durant la manipulation d’objet, la détection rapide du micro-glissement (incipient slip) amène la main à augmenter la force de pince pour éviter que l’objet ne tombe. À l’opposé, le glissement est un aspect essentiel à l’exploration tactile puisqu’il favorise une plus grande acuité tactile. Pour ces deux actions, les forces normale et tangentielle exercées sur la peau permettent de décrire le glissement mais également ce qui arrive juste avant qu’il y ait glissement. Toutefois, on ignore comment ces forces contrôlées par le sujet pourraient être encodées au niveau cortical. C’est pourquoi nous avons enregistré l’activité unitaire des neurones du cortex somatosensoriel primaire (S1) durant l’exécution de deux tâches haptiques chez les primates. Dans la première tâche, deux singes devaient saisir une pastille de métal fixe et y exercer des forces de cisaillement sans glissement dans une de quatre directions orthogonales. Des 144 neurones enregistrés, 111 (77%) étaient modulés à la direction de la force de cisaillement. L’ensemble de ces vecteurs préférés s’étendait dans toutes les directions avec un arc variant de 50° à 170°. Plus de 21 de ces neurones (19%) étaient également modulés à l’intensité de la force de cisaillement. Bien que 66 neurones (59%) montraient clairement une réponse à adaptation lente et 45 autres (41%) une réponse à adaptation rapide, cette classification ne semblait pas expliquer la modulation à l’intensité et à la direction de la force de cisaillement. Ces résultats montrent que les neurones de S1 encodent simultanément la direction et l’intensité des forces même en l’absence de glissement. Dans la seconde tâche, deux singes ont parcouru différentes surfaces avec le bout des doigts à la recherche d’une cible tactile, sans feedback visuel. Durant l’exploration, les singes, comme les humains, contrôlaient les forces et la vitesse de leurs doigts dans une plage de valeurs réduite. Les surfaces à haut coefficient de friction offraient une plus grande résistance tangentielle à la peau et amenaient les singes à alléger la force de contact, normale à la peau. Par conséquent, la somme scalaire des composantes normale et tangentielle demeurait constante entre les surfaces. Ces observations démontrent que les singes contrôlent les forces normale et tangentielle qu’ils appliquent durant l’exploration tactile. Celles-ci sont également ajustées selon les propriétés de surfaces telles que la texture et la friction. Des 230 neurones enregistrés durant la tâche d’exploration tactile, 96 (42%) ont montré une fréquence de décharge instantanée reliée aux forces exercées par les doigts sur la surface. De ces neurones, 52 (54%) étaient modulés avec la force normale ou la force tangentielle bien que l’autre composante orthogonale avait peu ou pas d’influence sur la fréquence de décharge. Une autre sous-population de 44 (46%) neurones répondait au ratio entre la force normale et la force tangentielle indépendamment de l’intensité. Plus précisément, 29 (30%) neurones augmentaient et 15 (16%) autres diminuaient leur fréquence de décharge en relation avec ce ratio. Par ailleurs, environ la moitié de tous les neurones (112) étaient significativement modulés à la direction de la force tangentielle. De ces neurones, 59 (53%) répondaient à la fois à la direction et à l’intensité des forces. L’exploration de trois ou quatre différentes surfaces a permis d’évaluer l’impact du coefficient de friction sur la modulation de 102 neurones de S1. En fait, 17 (17%) neurones ont montré une augmentation de leur fréquence de décharge avec l’augmentation du coefficient de friction alors que 8 (8%) autres ont montré le comportement inverse. Par contre, 37 (36%) neurones présentaient une décharge maximale sur une surface en particulier, sans relation linéaire avec le coefficient de friction des surfaces. La classification d’adaptation rapide ou lente des neurones de S1 n’a pu être mise en relation avec la modulation aux forces et à la friction. Ces résultats montrent que la fréquence de décharge des neurones de S1 encode l’intensité des forces normale et tangentielle, le ratio entre les deux composantes et la direction du mouvement. Ces résultats montrent que le comportement d’une importante sous-population des neurones de S1 est déterminé par les forces normale et tangentielle sur la peau. La modulation aux forces présentée ici fait le pont entre les travaux évaluant les propriétés de surfaces telles que la rugosité et les études touchant à la manipulation d’objets. Ce système de référence s’applique en présence ou en absence de glissement entre la peau et la surface. Nos résultats quant à la modulation des neurones à adaptation rapide ou lente nous amènent à suggérer que cette classification découle de la manière que la peau est stimulée. Nous discuterons aussi de la possibilité que l’activité des neurones de S1 puisse inclure une composante motrice durant ces tâches sensorimotrices. Finalement, un nouveau cadre de référence tridimensionnel sera proposé pour décrire et rassembler, dans un même continuum, les différentes modulations aux forces normale et tangentielle observées dans S1 durant l’exploration tactile.

An Encoded Infrared Sheet Of Light Navigational Beacon System For Precise Localization Of Indoor Mobile Robot Vehicles

A new localization approach to increase the navigational capabilities and object manipulation of autonomous mobile robots, based on an encoded infrared sheet of light beacon system, which provides position errors smaller than 0.02m is presented in this paper. To achieve this minimal position error, a resolution enhancement technique has been developed by utilising an inbuilt odometric/optical flow sensor information. This system respects strong low cost constraints by using an innovative assembly for the digitally encoded infrared transmitter. For better guidance of mobile robot vehicles, an online traffic signalling capability is also incorporated. Other added features are its less computational complexity and online localization capability all these without any estimation uncertainty. The constructional details, experimental results and computational methodologies of the system are also described

Wahrnehmungspsychologische Evaluation eines dreidimensionalen Visualisierungssystems

Diese Arbeit beschreibt den Evaluationsprozess einer dreidimensionalen Visualisierungstechnik, die am Institut für periphere Mikroelektronik der Universität Kassel entwickelt wurde. Hinter der dreidimensionalen Darstellung mittels Linsenrasterscheibe verbirgt sich eine neue Dimension der Interaktion mit dem Computer. Im Vergleich zu gewöhnlichen dreidimensionalen Darstellungen, bei denen ein 3D-Objekt auf einer 2D-Fläche abgebildet wird und somit nach wie vor nicht aus der Bildschirmebene heraus kann, können bei der stereoskopen Darstellung die Objekte dreidimensional visualisiert werden. Die Objekte tauchen vor, beziehungsweise hinter der Darstellungsebene auf. Da die Linsenrasterscheibe bisher noch nicht wahrnehmungspsychologisch untersucht wurde und auch allgemein auf dem Gebiet der Evaluation von 3D-Systemen nur wenige Untersuchungen mit quantitativen Ergebnissen verfügbar sind (Vollbracht, 1997), besteht hier ein zentrales Forschungsinteresse. Um eine Evaluation dieses 3D-Systems durchzuführen, wird im Theorieteil der Arbeit zunächst der Begriff der Evaluation definiert. Des Weiteren werden die wahrnehmungspsychologischen Grundlagen der monokularen und binokularen Raumwahrnehmung erörtert. Anschließend werden Techniken zur Erzeugung von Tiefe in Bildern und auf Bildschirmen erläutert und die Unterschiede zwischen der technisch erzeugten und der natürlichen Tiefenwahrnehmung näher beleuchtet. Nach der Vorstellung verschiedener stereoskoper Systeme wird näher auf die autostereoskope Linsenrasterscheibe eingegangen. Zum Abschluss des theoretischen Teils dieser Arbeit wird die Theorie des eingesetzten Befindlichkeitsfragebogens veranschaulicht. Gegenstand des empirischen Teils dieser Arbeit sind zwei zentrale Fragestellungen. Erstens soll untersucht werden, ob durch den höheren Informationsgehalt grundlegende Wahrnehmungsleistungen in bestimmten Bereichen positiv beeinflusst werden können. Zweitens soll untersucht werden, ob sich die höhere visuelle Natürlichkeit und die Neuartigkeit der Bildpräsentation auch auf die subjektive Befindlichkeit der Probanden auswirkt. Die empirische Überprüfung dieser Hypothesen erfolgt mittels dreier Experimente. Bei den ersten beiden Experimenten stehen grundlegende wahrnehmungspsychologische Leistungen im Vordergrund, während in der dritten Untersuchung der Bereich der subjektiven Befindlichkeit gemessen wird. Abschließend werden die Ergebnisse der Untersuchungen vorgestellt und diskutiert. Des Weiteren werden konkrete Einsatzmöglichkeiten für die Linsenrasterscheibe aufgezeigt und denkbare nachfolgende experimentelle Vorgehensweisen skizziert.

Alignment by Maximization of Mutual Information

A new information-theoretic approach is presented for finding the pose of an object in an image. The technique does not require information about the surface properties of the object, besides its shape, and is robust with respect to variations of illumination. In our derivation, few assumptions are made about the nature of the imaging process. As a result the algorithms are quite general and can foreseeably be used in a wide variety of imaging situations. Experiments are presented that demonstrate the approach registering magnetic resonance (MR) images with computed tomography (CT) images, aligning a complex 3D object model to real scenes including clutter and occlusion, tracking a human head in a video sequence and aligning a view-based 2D object model to real images. The method is based on a formulation of the mutual information between the model and the image called EMMA. As applied here the technique is intensity-based, rather than feature-based. It works well in domains where edge or gradient-magnitude based methods have difficulty, yet it is more robust than traditional correlation. Additionally, it has an efficient implementation that is based on stochastic approximation. Finally, we will describe a number of additional real-world applications that can be solved efficiently and reliably using EMMA. EMMA can be used in machine learning to find maximally informative projections of high-dimensional data. EMMA can also be used to detect and correct corruption in magnetic resonance images (MRI).

An approach to coded structured light to obtain three dimensional information

The human visual ability to perceive depth looks like a puzzle. We perceive three-dimensional spatial information quickly and efficiently by using the binocular stereopsis of our eyes and, what is mote important the learning of the most common objects which we achieved through living. Nowadays, modelling the behaviour of our brain is a fiction, that is why the huge problem of 3D perception and further, interpretation is split into a sequence of easier problems. A lot of research is involved in robot vision in order to obtain 3D information of the surrounded scene. Most of this research is based on modelling the stereopsis of humans by using two cameras as if they were two eyes. This method is known as stereo vision and has been widely studied in the past and is being studied at present, and a lot of work will be surely done in the future. This fact allows us to affirm that this topic is one of the most interesting ones in computer vision. The stereo vision principle is based on obtaining the three dimensional position of an object point from the position of its projective points in both camera image planes. However, before inferring 3D information, the mathematical models of both cameras have to be known. This step is known as camera calibration and is broadly describes in the thesis. Perhaps the most important problem in stereo vision is the determination of the pair of homologue points in the two images, known as the correspondence problem, and it is also one of the most difficult problems to be solved which is currently investigated by a lot of researchers. The epipolar geometry allows us to reduce the correspondence problem. An approach to the epipolar geometry is describes in the thesis. Nevertheless, it does not solve it at all as a lot of considerations have to be taken into account. As an example we have to consider points without correspondence due to a surface occlusion or simply due to a projection out of the camera scope. The interest of the thesis is focused on structured light which has been considered as one of the most frequently used techniques in order to reduce the problems related lo stereo vision. Structured light is based on the relationship between a projected light pattern its projection and an image sensor. The deformations between the pattern projected into the scene and the one captured by the camera, permits to obtain three dimensional information of the illuminated scene. This technique has been widely used in such applications as: 3D object reconstruction, robot navigation, quality control, and so on. Although the projection of regular patterns solve the problem of points without match, it does not solve the problem of multiple matching, which leads us to use hard computing algorithms in order to search the correct matches. In recent years, another structured light technique has increased in importance. This technique is based on the codification of the light projected on the scene in order to be used as a tool to obtain an unique match. Each token of light is imaged by the camera, we have to read the label (decode the pattern) in order to solve the correspondence problem. The advantages and disadvantages of stereo vision against structured light and a survey on coded structured light are related and discussed. The work carried out in the frame of this thesis has permitted to present a new coded structured light pattern which solves the correspondence problem uniquely and robust. Unique, as each token of light is coded by a different word which removes the problem of multiple matching. Robust, since the pattern has been coded using the position of each token of light with respect to both co-ordinate axis. Algorithms and experimental results are included in the thesis. The reader can see examples 3D measurement of static objects, and the more complicated measurement of moving objects. The technique can be used in both cases as the pattern is coded by a single projection shot. Then it can be used in several applications of robot vision. Our interest is focused on the mathematical study of the camera and pattern projector models. We are also interested in how these models can be obtained by calibration, and how they can be used to obtained three dimensional information from two correspondence points. Furthermore, we have studied structured light and coded structured light, and we have presented a new coded structured light pattern. However, in this thesis we started from the assumption that the correspondence points could be well-segmented from the captured image. Computer vision constitutes a huge problem and a lot of work is being done at all levels of human vision modelling, starting from a)image acquisition; b) further image enhancement, filtering and processing, c) image segmentation which involves thresholding, thinning, contour detection, texture and colour analysis, and so on. The interest of this thesis starts in the next step, usually known as depth perception or 3D measurement.

Perceptual cues for orientation in a two finger haptic grasp task

Single point interaction haptic devices do not provide the natural grasp and manipulations found in the real world, as afforded by multi-fingered haptics. The present study investigates a two-fingered grasp manipulation involving rotation with and without force feedback. There were three visual cue conditions: monocular, binocular and projective lighting. Performance metrics of time and positional accuracy were assessed. The results indicate that adding haptics to an object manipulation task increases the positional accuracy but slightly increases the overall time taken.

Extending the friction cone algorithm for arbitrary polygon based haptic objects

Most haptic environments are based on single point interactions whereas in practice, object manipulation requires multiple contact points between the object, fingers, thumb and palm. The Friction Cone Algorithm was developed specifically to work well in a multi-finger haptic environment where object manipulation would occur. However, the Friction Cone Algorithm has two shortcomings when applied to polygon meshes: there is no means of transitioning polygon boundaries or feeling non-convex edges. In order to overcome these deficiencies, Face Directed Connection Graphs have been developed as well as a robust method for applying friction to non-convex edges. Both these extensions are described herein, as well as the implementation issues associated with them.

A survey of multi-fingered robot hands: issues and grasping achievements

Recently a substantial amount of research has been done in the field of dextrous manipulation and hand manoeuvres. The main concern has been how to control robot hands so that they can execute manipulation tasks with the same dexterity and intuition as human hands. This paper surveys multi-fingered robot hand research and development topics which include robot hand design, object force distribution and control, grip transform, grasp stability and its synthesis, grasp stiffness and compliance motion and robot arm-hand coordination. Three main topics are presented in this article. The first is an introduction to the subject. The second concentrates on examples of mechanical manipulators used in research and the methods employed to control them. The third presents work which has been done on the field of object manipulation.

Using virtual worlds in distance learning environments

This paper presents two tools developed to facilitate the use and automate the process of using Virtual Worlds for educational purposes. The first tool has been developed to automatically create the classroom space, usually called region in the virtual world, which means, a region in the virtual world used to develop educational activities between professors, students and interactive objects. The second tool helps the process of creating 3D interactive objects in a virtual world. With these tools educators will be able to produce 3D interactive learning objects and use them in virtual classrooms improving the quality and appeal, for students, of their classes. © 2011 IEEE.

Forças manuais e o design de produtos: uma revisão

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)