Integrated visual perception architecture for robotic clothes perception and manipulation

This thesis proposes a generic visual perception architecture for robotic clothes perception and manipulation. This proposed architecture is fully integrated with a stereo vision system and a dual-arm robot and is able to perform a number of autonomous laundering tasks. Clothes perception and manipulation is a novel research topic in robotics and has experienced rapid development in recent years. Compared to the task of perceiving and manipulating rigid objects, clothes perception and manipulation poses a greater challenge. This can be attributed to two reasons: firstly, deformable clothing requires precise (high-acuity) visual perception and dexterous manipulation; secondly, as clothing approximates a non-rigid 2-manifold in 3-space, that can adopt a quasi-infinite configuration space, the potential variability in the appearance of clothing items makes them difficult to understand, identify uniquely, and interact with by machine. From an applications perspective, and as part of EU CloPeMa project, the integrated visual perception architecture refines a pre-existing clothing manipulation pipeline by completing pre-wash clothes (category) sorting (using single-shot or interactive perception for garment categorisation and manipulation) and post-wash dual-arm flattening. To the best of the author’s knowledge, as investigated in this thesis, the autonomous clothing perception and manipulation solutions presented here were first proposed and reported by the author. All of the reported robot demonstrations in this work follow a perception-manipulation method- ology where visual and tactile feedback (in the form of surface wrinkledness captured by the high accuracy depth sensor i.e. CloPeMa stereo head or the predictive confidence modelled by Gaussian Processing) serve as the halting criteria in the flattening and sorting tasks, respectively. From scientific perspective, the proposed visual perception architecture addresses the above challenges by parsing and grouping 3D clothing configurations hierarchically from low-level curvatures, through mid-level surface shape representations (providing topological descriptions and 3D texture representations), to high-level semantic structures and statistical descriptions. A range of visual features such as Shape Index, Surface Topologies Analysis and Local Binary Patterns have been adapted within this work to parse clothing surfaces and textures and several novel features have been devised, including B-Spline Patches with Locality-Constrained Linear coding, and Topology Spatial Distance to describe and quantify generic landmarks (wrinkles and folds). The essence of this proposed architecture comprises 3D generic surface parsing and interpretation, which is critical to underpinning a number of laundering tasks and has the potential to be extended to other rigid and non-rigid object perception and manipulation tasks. The experimental results presented in this thesis demonstrate that: firstly, the proposed grasp- ing approach achieves on-average 84.7% accuracy; secondly, the proposed flattening approach is able to flatten towels, t-shirts and pants (shorts) within 9 iterations on-average; thirdly, the proposed clothes recognition pipeline can recognise clothes categories from highly wrinkled configurations and advances the state-of-the-art by 36% in terms of classification accuracy, achieving an 83.2% true-positive classification rate when discriminating between five categories of clothes; finally the Gaussian Process based interactive perception approach exhibits a substantial improvement over single-shot perception. Accordingly, this thesis has advanced the state-of-the-art of robot clothes perception and manipulation.

Memory-based reasoning approach for pattern recognition of binary images

Template matching is concerned with measuring the similarity between patterns of two objects. This paper proposes a memory-based reasoning approach for pattern recognition of binary images with a large template set. It seems that memory-based reasoning intrinsically requires a large database. Moreover, some binary image recognition problems inherently need large template sets, such as the recognition of Chinese characters which needs thousands of templates. The proposed algorithm is based on the Connection Machine, which is the most massively parallel machine to date, using a multiresolution method to search for the matching template. The approach uses the pyramid data structure for the multiresolution representation of templates and the input image pattern. For a given binary image it scans the template pyramid searching the match. A binary image of N × N pixels can be matched in O(log N) time complexity by our algorithm and is independent of the number of templates. Implementation of the proposed scheme is described in detail.

Inherent structures of phase-separating binary mixtures: Nucleation, spinodal decomposition, and pattern formation

An energy landscape view of phase separation and nonideality in binary mixtures is developed by exploring their potential energy landscape (PEL) as functions of temperature and composition. We employ molecular dynamics simulations to study a model that promotes structure breaking in the solute-solvent parent binary liquid, at low temperatures. The PEL of the system captures the potential energy distribution of the inherent structures (IS) of the system and is obtained by removing the kinetic energy (including that of intermolecular vibrations). The broader distribution of the inherent structure energy for structure breaking liquid than that of the structure making liquid demonstrates the larger role of entropy in stabilizing the parent liquid of the structure breaking type of binary mixtures. At high temperature, although the parent structure of the structure breaking binary mixture is homogenous, the corresponding inherent structure is found to be always phase separated, with a density pattern that exhibits marked correlation with the energy of its inherent structure. Over a broad range of intermediate inherent structure energy, bicontinuous phase separation prevails with interpenetrating stripes as signatures of spinodal decomposition. At low inherent structure energy, the structure is largely phase separated with one interface where as at high inherent structure energy we find nucleation type growth. Interestingly, at low temperature, the average inherent structure energy (< EIS >) exhibits a drop with temperature which signals the onset of crystallization in one of the phases while the other remains in the liquid state. The nonideal composition dependence of viscosity is anticorrelated with average inherent structure energy.

Local order and magnetic field effects on the electronic properties of disordered binary alloys in the quantum site percolation limit

Electronic properties of disordered binary alloys are studied via the calculation of the average Density of States (DOS) in two and three dimensions. We propose a new approximate scheme that allows for the inclusion of local order effects in finite geometries and extrapolates the behavior of infinite systems following finite-size scaling ideas. We particularly investigate the limit of the Quantum Site Percolation regime described by a tight-binding Hamiltonian. This limit was chosen to probe the role of short range order (SRO) properties under extreme conditions. The method is numerically highly efficient and asymptotically exact in important limits, predicting the correct DOS structure as a function of the SRO parameters. Magnetic field effects can also be included in our model to study the interplay of local order and the shifted quantum interference driven by the field. The average DOS is highly sensitive to changes in the SRO properties and striking effects are observed when a magnetic field is applied near the segregated regime. The new effects observed are twofold: there is a reduction of the band width and the formation of a gap in the middle of the band, both as a consequence of destructive interference of electronic paths and the loss of coherence for particular values of the magnetic field. The above phenomena are periodic in the magnetic flux. For other limits that imply strong localization, the magnetic field produces minor changes in the structure of the average DOS. © World Scientific Publishing Company.

Studies on the life history of Eleutherodactylus ricordii planirostris (Cope) in Florida, with special reference to the local distribution of an allelomorphic color pattern.

"Literature cited": p. 55-56.

Control of pattern formation by time-delay feedback with global and local contributions

We consider the suppression of spatiotemporal chaos in the complex GinzburgLandau equation by a combined global and local time-delay feedback. Feedback terms are implemented as a control scheme, i.e., they are proportional to the difference between the time-delayed state of the system and its current state. We perform a linear stability analysis of uniform oscillations with respect to space-dependent perturbations and compare with numerical simulations. Similarly, for the fixed-point solution that corresponds to amplitude death in the spatially extended system, a linear stability analysis with respect to space-dependent perturbations is performed and complemented by numerical simulations. © 2010 Elsevier B.V. All rights reserved.

The effectiveness of using a simple ARIA based geographical classification to identify road crash patterns in rural and urban areas of Queensland

Research has noted a ‘pronounced pattern of increase with increasing remoteness' of death rates in road crashes. However, crash characteristics by remoteness are not commonly or consistently reported, with definitions of rural and urban often relying on proxy representations such as prevailing speed limit. The current paper seeks to evaluate the efficacy of the Accessibility / Remoteness Index of Australia (ARIA+) to identifying trends in road crashes. ARIA+ does not rely on road-specific measures and uses distances to populated centres to attribute a score to an area, which can in turn be grouped into 5 classifications of increasing remoteness. The current paper uses applications of these classifications at the broad level of Australian Bureau of Statistics' Statistical Local Areas, thus avoiding precise crash locating or dedicated mapping software. Analyses used Queensland road crash database details for all 31,346 crashes resulting in a fatality or hospitalisation occurring between 1st July, 2001 and 30th June 2006 inclusive. Results showed that this simplified application of ARIA+ aligned with previous definitions such as speed limit, while also providing further delineation. Differences in crash contributing factors were noted with increasing remoteness such as a greater representation of alcohol and ‘excessive speed for circumstances.' Other factors such as the predominance of younger drivers in crashes differed little by remoteness classification. The results are discussed in terms of the utility of remoteness as a graduated rather than binary (rural/urban) construct and the potential for combining ARIA crash data with census and hospital datasets.

Power law distributions in pattern dynamics of attacker-defender dyads in the team sport of Rugby Union : phenomena in a region of self-organized criticality?

In the region of self-organized criticality (SOC) interdependency between multi-agent system components exists and slight changes in near-neighbor interactions can break the balance of equally poised options leading to transitions in system order. In this region, frequency of events of differing magnitudes exhibits a power law distribution. The aim of this paper was to investigate whether a power law distribution characterized attacker-defender interactions in team sports. For this purpose we observed attacker and defender in a dyadic sub-phase of rugby union near the try line. Videogrammetry was used to capture players’ motion over time as player locations were digitized. Power laws were calculated for the rate of change of players’ relative position. Data revealed that three emergent patterns from dyadic system interactions (i.e., try; unsuccessful tackle; effective tackle) displayed a power law distribution. Results suggested that pattern forming dynamics dyads in rugby union exhibited SOC. It was concluded that rugby union dyads evolve in SOC regions suggesting that players’ decisions and actions are governed by local interactions rules.