Visual perception for the 3D recognition of geometric pieces in robotic manipulation

During grasping and intelligent robotic manipulation tasks, the camera position relative to the scene changes dramatically because the robot is moving to adapt its path and correctly grasp objects. This is because the camera is mounted at the robot effector. For this reason, in this type of environment, a visual recognition system must be implemented to recognize and “automatically and autonomously” obtain the positions of objects in the scene. Furthermore, in industrial environments, all objects that are manipulated by robots are made of the same material and cannot be differentiated by features such as texture or color. In this work, first, a study and analysis of 3D recognition descriptors has been completed for application in these environments. Second, a visual recognition system designed from specific distributed client-server architecture has been proposed to be applied in the recognition process of industrial objects without these appearance features. Our system has been implemented to overcome problems of recognition when the objects can only be recognized by geometric shape and the simplicity of shapes could create ambiguity. Finally, some real tests are performed and illustrated to verify the satisfactory performance of the proposed system.

3D Visual Data-Driven Spatiotemporal Deformations for Non-Rigid Object Grasping Using Robot Hands

Sensing techniques are important for solving problems of uncertainty inherent to intelligent grasping tasks. The main goal here is to present a visual sensing system based on range imaging technology for robot manipulation of non-rigid objects. Our proposal provides a suitable visual perception system of complex grasping tasks to support a robot controller when other sensor systems, such as tactile and force, are not able to obtain useful data relevant to the grasping manipulation task. In particular, a new visual approach based on RGBD data was implemented to help a robot controller carry out intelligent manipulation tasks with flexible objects. The proposed method supervises the interaction between the grasped object and the robot hand in order to avoid poor contact between the fingertips and an object when there is neither force nor pressure data. This new approach is also used to measure changes to the shape of an object’s surfaces and so allows us to find deformations caused by inappropriate pressure being applied by the hand’s fingers. Test was carried out for grasping tasks involving several flexible household objects with a multi-fingered robot hand working in real time. Our approach generates pulses from the deformation detection method and sends an event message to the robot controller when surface deformation is detected. In comparison with other methods, the obtained results reveal that our visual pipeline does not use deformations models of objects and materials, as well as the approach works well both planar and 3D household objects in real time. In addition, our method does not depend on the pose of the robot hand because the location of the reference system is computed from a recognition process of a pattern located place at the robot forearm. The presented experiments demonstrate that the proposed method accomplishes a good monitoring of grasping task with several objects and different grasping configurations in indoor environments.

Developmental trajectories of part-based and configural object recognition in adolescence.

Three experiments assessed the development of children's part and configural (part-relational) processing in object recognition during adolescence. In total, 312 school children aged 7-16 years and 80 adults were tested in 3-alternative forced choice (3-AFC) tasks. They judged the correct appearance of upright and inverted presented familiar animals, artifacts, and newly learned multipart objects, which had been manipulated either in terms of individual parts or part relations. Manipulation of part relations was constrained to either metric (animals, artifacts, and multipart objects) or categorical (multipart objects only) changes. For animals and artifacts, even the youngest children were close to adult levels for the correct recognition of an individual part change. By contrast, it was not until 11-12 years of age that they achieved similar levels of performance with regard to altered metric part relations. For the newly learned multipart objects, performance was equivalent throughout the tested age range for upright presented stimuli in the case of categorical part-specific and part-relational changes. In the case of metric manipulations, the results confirmed the data pattern observed for animals and artifacts. Together, the results provide converging evidence, with studies of face recognition, for a surprisingly late consolidation of configural-metric relative to part-based object recognition.

Waveguide microgripper for identification, sensing and manipulation

A Waveguide Microgripper utilizes flexible optical waveguides as gripping arms, which provide the physical means for grasping a microobject, while simultaneously enabling light to be delivered and collected. This unique capability allows extensive optical characterization of the structure being held such as transmission, reflection or fluorescence. One of the simplest capabilities of the waveguide microgripper is to be able to detect the presence of a microobject between the microgripper facets by monitoring the transmitted intensity of light coupled through the facets. The intensity of coupled light is expected to drop when there is an object obstructing the path of light. The optical sensing and characterization function of the microgripper is a strong function of the optical power incident on the structure of interest. Hence it is important to understand the factors affecting the power distribution across the facet. The microgripper is also capable of detecting the fluorescence. This capability of microgripper is expected to have applications in medical, bio-medical and related fields.

Creating reusable well-structured pdf as a sequence of component object graphic (cog) elements

Portable Document Format (PDF) is a page-oriented, graphically rich format based on PostScript semantics and it is also the format interpreted by the Adobe Acrobat viewers. Although each of the pages in a PDF document is an independent graphic object this property does not necessarily extend to the components (headings, diagrams, paragraphs etc.) within a page. This, in turn, makes the manipulation and extraction of graphic objects on a PDF page into a very difficult and uncertain process. The work described here investigates the advantages of a model wherein PDF pages are created from assemblies of COGs (Component Object Graphics) each with a clearly defined graphic state. The relative positioning of COGs on a PDF page is determined by appropriate "spacer" objects and a traversal of the tree of COGs and spacers determines the rendering order. The enhanced revisability of PDF documents within the COG model is discussed, together with the application of the model in those contexts which require easy revisability coupled with the ability to maintain and amend PDF document structure.

Integrated visual perception architecture for robotic clothes perception and manipulation

This thesis proposes a generic visual perception architecture for robotic clothes perception and manipulation. This proposed architecture is fully integrated with a stereo vision system and a dual-arm robot and is able to perform a number of autonomous laundering tasks. Clothes perception and manipulation is a novel research topic in robotics and has experienced rapid development in recent years. Compared to the task of perceiving and manipulating rigid objects, clothes perception and manipulation poses a greater challenge. This can be attributed to two reasons: firstly, deformable clothing requires precise (high-acuity) visual perception and dexterous manipulation; secondly, as clothing approximates a non-rigid 2-manifold in 3-space, that can adopt a quasi-infinite configuration space, the potential variability in the appearance of clothing items makes them difficult to understand, identify uniquely, and interact with by machine. From an applications perspective, and as part of EU CloPeMa project, the integrated visual perception architecture refines a pre-existing clothing manipulation pipeline by completing pre-wash clothes (category) sorting (using single-shot or interactive perception for garment categorisation and manipulation) and post-wash dual-arm flattening. To the best of the author’s knowledge, as investigated in this thesis, the autonomous clothing perception and manipulation solutions presented here were first proposed and reported by the author. All of the reported robot demonstrations in this work follow a perception-manipulation method- ology where visual and tactile feedback (in the form of surface wrinkledness captured by the high accuracy depth sensor i.e. CloPeMa stereo head or the predictive confidence modelled by Gaussian Processing) serve as the halting criteria in the flattening and sorting tasks, respectively. From scientific perspective, the proposed visual perception architecture addresses the above challenges by parsing and grouping 3D clothing configurations hierarchically from low-level curvatures, through mid-level surface shape representations (providing topological descriptions and 3D texture representations), to high-level semantic structures and statistical descriptions. A range of visual features such as Shape Index, Surface Topologies Analysis and Local Binary Patterns have been adapted within this work to parse clothing surfaces and textures and several novel features have been devised, including B-Spline Patches with Locality-Constrained Linear coding, and Topology Spatial Distance to describe and quantify generic landmarks (wrinkles and folds). The essence of this proposed architecture comprises 3D generic surface parsing and interpretation, which is critical to underpinning a number of laundering tasks and has the potential to be extended to other rigid and non-rigid object perception and manipulation tasks. The experimental results presented in this thesis demonstrate that: firstly, the proposed grasp- ing approach achieves on-average 84.7% accuracy; secondly, the proposed flattening approach is able to flatten towels, t-shirts and pants (shorts) within 9 iterations on-average; thirdly, the proposed clothes recognition pipeline can recognise clothes categories from highly wrinkled configurations and advances the state-of-the-art by 36% in terms of classification accuracy, achieving an 83.2% true-positive classification rate when discriminating between five categories of clothes; finally the Gaussian Process based interactive perception approach exhibits a substantial improvement over single-shot perception. Accordingly, this thesis has advanced the state-of-the-art of robot clothes perception and manipulation.

Robotic Sensing and Manipulation of Deformable Linear Objects with Learning-based methods

Nowadays robotic applications are widespread and most of the manipulation tasks are efficiently solved. However, Deformable-Objects (DOs) still represent a huge limitation for robots. The main difficulty in DOs manipulation is dealing with the shape and dynamics uncertainties, which prevents the use of model-based approaches (since they are excessively computationally complex) and makes sensory data difficult to interpret. This thesis reports the research activities aimed to address some applications in robotic manipulation and sensing of Deformable-Linear-Objects (DLOs), with particular focus to electric wires. In all the works, a significant effort was made in the study of an effective strategy for analyzing sensory signals with various machine learning algorithms. In the former part of the document, the main focus concerns the wire terminals, i.e. detection, grasping, and insertion. First, a pipeline that integrates vision and tactile sensing is developed, then further improvements are proposed for each module. A novel procedure is proposed to gather and label massive amounts of training images for object detection with minimal human intervention. Together with this strategy, we extend a generic object detector based on Convolutional-Neural-Networks for orientation prediction. The insertion task is also extended by developing a closed-loop control capable to guide the insertion of a longer and curved segment of wire through a hole, where the contact forces are estimated by means of a Recurrent-Neural-Network. In the latter part of the thesis, the interest shifts to the DLO shape. Robotic reshaping of a DLO is addressed by means of a sequence of pick-and-place primitives, while a decision making process driven by visual data learns the optimal grasping locations exploiting Deep Q-learning and finds the best releasing point. The success of the solution leverages on a reliable interpretation of the DLO shape. For this reason, further developments are made on the visual segmentation.

Robotic Manipulation of DLOs for Wiring Harness Assembly: a Machine Learning Approach

In recent times, a significant research effort has been focused on how deformable linear objects (DLOs) can be manipulated for real world applications such as assembly of wiring harnesses for the automotive and aerospace sector. This represents an open topic because of the difficulties in modelling accurately the behaviour of these objects and simulate a task involving their manipulation, considering a variety of different scenarios. These problems have led to the development of data-driven techniques in which machine learning techniques are exploited to obtain reliable solutions. However, this approach makes the solution difficult to be extended, since the learning must be replicated almost from scratch as the scenario changes. It follows that some model-based methodology must be introduced to generalize the results and reduce the training effort accordingly. The objective of this thesis is to develop a solution for the DLOs manipulation to assemble a wiring harness for the automotive sector based on adaptation of a base trajectory set by means of reinforcement learning methods. The idea is to create a trajectory planning software capable of solving the proposed task, reducing where possible the learning time, which is done in real time, but at the same time presenting suitable performance and reliability. The solution has been implemented on a collaborative 7-DOFs Panda robot at the Laboratory of Automation and Robotics of the University of Bologna. Experimental results are reported showing how the robot is capable of optimizing the manipulation of the DLOs gaining experience along the task repetition, but showing at the same time a high success rate from the very beginning of the learning phase.

Virtualization of Wiring Harness Manipulation Tasks through the PyChrono Simulation Engine

The purpose of this thesis work is the study and creation of a harness modelling system. The model needs to simulate faithfully the physical behaviour of the harness, without any instability or incorrect movements. Since there are various simulation engines that try to model wiring's systems, this thesis work focused on the creation and test of a 3D environment with wiring and other objects through the PyChrono Simulation Engine. Fine-tuning of the simulation parameters were done during the test to achieve the most stable and correct simulation possible, but tests showed the intrinsic limits of the Engine regarding the collisions' detection between the various part of the cables, while collisions between cables and other physical objects such as pavement, walls and others are well managed by the simulator. Finally, the main purpose of the model is to be used to train Artificial Intelligence through Reinforcement Learnings techniques, so we designed, using OpenAI Gym APIs, the general structure of the learning environment, defining its basic functions and an initial framework.

Manipulation of quantum state transfer in cold Rydberg atom collisions

We investigate the role of the dc Stark effect in multilevel pairwise interactions between cold Rydberg atoms. We have observed the decay of nD + nD quasi-molecules by detecting the products in the (n + 2) P state after pulsed excitation for 29 <= n <= 41. The decay rate can be manipulated with a dc electric field and requires a consideration of the multilevel nature of the process to explain the observations. The time dependence of the (n + 2) P signal is found to support a time-dependent picture of the dynamics.

Museum, object and information

This text aims to approach museums` role in the production of knowledge and how objects are transformed into documents when museums incorporate them. On accepting the effects of such transformation, museums start working not only with material goods, but also symbolic goods. The collection manager or exhibition curator communicate through documents rather than bringing into light its intrinsic content. In this sense, every process involving museum documents, from the selection of collections to exhibitions, has a rhetoric and ideological nature which is given. Museums must search for meanings through correlations established in the process of producing information. Exhibitions should present objects in multiple contexts, giving visitors the opportunity to participate and attribute their own meanings to them.

Manipulation of rest period length induces different causes of fatigue in vertical jumping

The aim of this study was to directly compare the causes of fatigue after a short- and a long-rest interval between consecutive stretch-shortening cycle exercises. Eleven healthy males jumped with different resting period lengths (short = 6.1 +/- 1 s, long = 8.6 +/- 0.9 s), performing countermovement jumps at 95% of their maximal jump height until they were unable to sustain the target height. After short- and long-rest, the maximal voluntary isometric contraction knee extension torque decreased (-7%; p = 0.04), comparing to values obtained before exercise protocols. No change was seen from pre- to post-exercise, for either short- or long-rest, in biceps femoris coactivation (-1%; p = 0.95), peak-to-peak amplitude (1%; p = 0.95) and duration (-8%; p = 0.92) of the compound muscle action potential of the vastus lateralis. Evoked peak twitch torque reduced after both exercise protocols (short = -26%, long = -32%; p = 0.003) indicating peripheral fatigue. However, central fatigue occurred only after short-rest evidenced by a reduction in voluntary activation of the quadriceps muscle (-14%; p = 0.013) measured using the interpolated twitch technique. In conclusion, after Stretch-shortening cycle exercise using short rest period length, the cause of fatigue was central and peripheral, while after using long rest period length, the cause of fatigue was peripheral.

Monocular vision and increased distance reducing the effects of visual manipulation on body sway

The goal of this study was to examine the coupling between visual information and body sway with binocular and monocular vision at two distances from the front wall of a moving room. Ten participants stood as still as possible inside of a moving room facing the front wall in conditions that combined room movement with monocular/binocular vision and distance from the front wall (75 and 150cm). Visual information effect on body sway decreased with monocular vision and with increased distance from the front wall. In addition, the combination of monocular vision with the farther distance resulted in the smallest body sway response to the driving stimulus provided by the moving room. These results suggest that binocularvision near the front wall provides visual information of a better quality than the monocular vision far from the front wall. We discuss the results with respect to two modes of visual detection of body sway: ocular and extraocular. (C) 2009 Elsevier Ireland Ltd. All rights reserved.

Methods of Fourier optics in digital holographic microscopy

In this paper, processing methods of Fourier optics implemented in a digital holographic microscopy system are presented. The proposed methodology is based on the possibility of the digital holography in carrying out the whole reconstruction of the recorded wave front and consequently, the determination of the phase and intensity distribution in any arbitrary plane located between the object and the recording plane. In this way, in digital holographic microscopy the field produced by the objective lens can be reconstructed along its propagation, allowing the reconstruction of the back focal plane of the lens, so that the complex amplitudes of the Fraunhofer diffraction, or equivalently the Fourier transform, of the light distribution across the object can be known. The manipulation of Fourier transform plane makes possible the design of digital methods of optical processing and image analysis. The proposed method has a great practical utility and represents a powerful tool in image analysis and data processing. The theoretical aspects of the method are presented, and its validity has been demonstrated using computer generated holograms and images simulations of microscopic objects. (c) 2007 Elsevier B.V. All rights reserved.

Colloidal Processing of Glass-Ceramics for Laminated Object Manufacturing

Green tapes of Li(2)O-ZrO(2)-SiO(2)-Al(2)O(3) (LZSA) parent glass were produced by aqueous tape casting as the starting material for the laminated object manufacturing (LOM) process. The rheological behavior of the powder suspensions in aqueous media, as well as the mechanical properties of the cast tapes, was evaluated. According to xi potential measurements, the LZSA glass powder particles showed acid surface characteristics and an IEP of around 4 when in aqueous media. The critical volume fraction of solids was about 72 wt% (27 vol%), which hindered the processability of more concentrated slurries. The glass particles also showed an anisometric profile, which contributed to an increase in the interactions between particles during flow. Therefore, the suspensions could not be processed at high solids loadings. Aqueous-based glass suspensions were also characterized by shear thickening after the addition of dispersants. Three slurry compositions were formulated, suitable green tapes were cast, and tapes were successfully laminated by LOM to a gear wheel geometry. A higher tensile strength of the green tapes corresponded to a higher tensile strength of the laminates. Thermal treatment was then applied to the laminates: pyrolysis at 525 degrees C, sintering at 700 degrees C for 1 h, and crystallization at 850 degrees C for 30 min. A 20% volumetric shrinkage was observed, but no surface flaws or inhomogeneous areas were detected. The sintered part maintained the curved edges and internal profile after heat treatment.