Designing a Computer Vision System for Underwater Robotic Interventions

This thesis deals with the challenging problem of designing systems able to perceive objects in underwater environments. In the last few decades research activities in robotics have advanced the state of art regarding intervention capabilities of autonomous systems. State of art in fields such as localization and navigation, real time perception and cognition, safe action and manipulation capabilities, applied to ground environments (both indoor and outdoor) has now reached such a readiness level that it allows high level autonomous operations. On the opposite side, the underwater environment remains a very difficult one for autonomous robots. Water influences the mechanical and electrical design of systems, interferes with sensors by limiting their capabilities, heavily impacts on data transmissions, and generally requires systems with low power consumption in order to enable reasonable mission duration. Interest in underwater applications is driven by needs of exploring and intervening in environments in which human capabilities are very limited. Nowadays, most underwater field operations are carried out by manned or remotely operated vehicles, deployed for explorations and limited intervention missions. Manned vehicles, directly on-board controlled, expose human operators to risks related to the stay in field of the mission, within a hostile environment. Remotely Operated Vehicles (ROV) currently represent the most advanced technology for underwater intervention services available on the market. These vehicles can be remotely operated for long time but they need support from an oceanographic vessel with multiple teams of highly specialized pilots. Vehicles equipped with multiple state-of-art sensors and capable to autonomously plan missions have been deployed in the last ten years and exploited as observers for underwater fauna, seabed, ship wrecks, and so on. On the other hand, underwater operations like object recovery and equipment maintenance are still challenging tasks to be conducted without human supervision since they require object perception and localization with much higher accuracy and robustness, to a degree seldom available in Autonomous Underwater Vehicles (AUV). This thesis reports the study, from design to deployment and evaluation, of a general purpose and configurable platform dedicated to stereo-vision perception in underwater environments. Several aspects related to the peculiar environment characteristics have been taken into account during all stages of system design and evaluation: depth of operation and light conditions, together with water turbidity and external weather, heavily impact on perception capabilities. The vision platform proposed in this work is a modular system comprising off-the-shelf components for both the imaging sensors and the computational unit, linked by a high performance ethernet network bus. The adopted design philosophy aims at achieving high flexibility in terms of feasible perception applications, that should not be as limited as in case of a special-purpose and dedicated hardware. Flexibility is required by the variability of underwater environments, with water conditions ranging from clear to turbid, light backscattering varying with daylight and depth, strong color distortion, and other environmental factors. Furthermore, the proposed modular design ensures an easier maintenance and update of the system over time. Performance of the proposed system, in terms of perception capabilities, has been evaluated in several underwater contexts taking advantage of the opportunity offered by the MARIS national project. Design issues like energy power consumption, heat dissipation and network capabilities have been evaluated in different scenarios. Finally, real-world experiments, conducted in multiple and variable underwater contexts, including open sea waters, have led to the collection of several datasets that have been publicly released to the scientific community. The vision system has been integrated in a state of the art AUV equipped with a robotic arm and gripper, and has been exploited in the robot control loop to successfully perform underwater grasping operations.

The role of local luminance amplitude in the interpretation of shape-from-shading in textured surfaces

Luminance changes within a scene are ambiguous; they can indicate reflectance changes, shadows, or shading due to surface undulations. How does vision distinguish between these possibilities? When a surface painted with an albedo texture is shaded, the change in local mean luminance (LM) is accompanied by a similar modulation of the local luminance amplitude (AM) of the texture. This relationship does not necessarily hold for reflectance changes or for shading of a relief texture. Here we concentrate on the role of AM in shape-from-shading. Observers were presented with a noise texture onto which sinusoidal LM and AM signals were superimposed, and were asked to indicate which of two marked locations was closer to them. Shape-from-shading was enhanced when LM and AM co-varied (in-phase), and was disrupted when they were out-of-phase. The perceptual differences between cue types (in-phase vs out-of-phase) were enhanced when the two cues were present at different orientations within a single image. Similar results were found with a haptic matching task. We conclude that vision can use AM to disambiguate luminance changes. LM and AM have a positive relationship for rendered, undulating, albedo textures, and we assess the degree to which this relationship holds in natural images. [Supported by EPSRC grants to AJS and MAG].

A developmental dissociation of view-dependent and view-invariant object recognition in adolescence

Spatial generalization skills in school children aged 8-16 were studied with regard to unfamiliar objects that had been previously learned in a cross-modal priming and learning paradigm. We observed a developmental dissociation with younger children recognizing objects only from previously learnt perspectives whereas older children generalized acquired object knowledge to new viewpoints as well. Haptic and - to a lesser extent - visual priming improved spatial generalization in all but the youngest children. The data supports the idea of dissociable, view-dependent and view-invariant object representations with different developmental trajectories that are subject to modulatory effects of priming. Late-developing areas in the parietal or the prefrontal cortex may account for the retarded onset of view-invariant object recognition. © 2006 Elsevier B.V. All rights reserved.

The spatial distribution and temporal dynamics of brain regions activated during the perception of object and non-object patterns

Both animal and human studies suggest that the efficiency with which we are able to grasp objects is attributable to a repertoire of motor signals derived directly from vision. This is in general agreement with the long-held belief that the automatic generation of motor signals by the perception of objects is based on the actions they afford. In this study, we used magnetoencephalography (MEG) to determine the spatial distribution and temporal dynamics of brain regions activated during passive viewing of object and non-object targets that varied in the extent to which they afforded a grasping action. Synthetic Aperture Magnetometry (SAM) was used to localize task-related oscillatory power changes within specific frequency bands, and the time course of activity within given regions-of-interest was determined by calculating time-frequency plots using a Morlet wavelet transform. Both single subject and group-averaged data on the spatial distribution of brain activity are presented. We show that: (i) significant reductions in 10-25 Hz activity within extrastriate cortex, occipito-temporal cortex, sensori-motor cortex and cerebellum were evident with passive viewing of both objects and non-objects; and (ii) reductions in oscillatory activity within the posterior part of the superior parietal cortex (area Ba7) were only evident with the perception of objects. Assuming that focal reductions in low-frequency oscillations (< 30 Hz) reflect areas of heightened neural activity, we conclude that: (i) activity within a network of brain areas, including the sensori-motor cortex, is not critically dependent on stimulus type and may reflect general changes in visual attention; and (ii) the posterior part of the superior parietal cortex, area Ba7, is activated preferentially by objects and may play a role in computations related to grasping. © 2006 Elsevier Inc. All rights reserved.

What is second-order vision for? Discriminating illumination versus material changes

The human visual system is sensitive to second-order modulations of the local contrast (CM) or amplitude (AM) of a carrier signal. Second-order cues are detected independently of first-order luminance signals; however, it is not clear why vision should benet from second-order sensitivity. Analysis of the first-and second-order contents of natural images suggests that these cues tend to occur together, but their phase relationship varies. We have shown that in-phase combinations of LM and AM are perceived as a shaded corrugated surface whereas the anti-phase combination can be seen as corrugated when presented alone or as a flat material change when presented in a plaid containing the in-phase cue. We now extend these findings using new stimulus types and a novel haptic matching task. We also introduce a computational model based on initially separate first-and second-order channels that are combined within orientation and subsequently across orientation to produce a shading signal. Contrast gain control allows the LM + AM cue to suppress responses to the LM-AM when presented in a plaid. Thus, the model sees LM -AM as flat in these circumstances. We conclude that second-order vision plays a key role in disambiguating the origin of luminance changes within an image. © ARVO.

Cellular interaction with novel biomaterials

The objective of this thesis is to report the behaviour of mammalian cells with biocompatible synthetic polymers with potential for applications to the human body. Composite hydrogel materials were tested as possible keratoprosthetic devices. It was found that surface topography is an important consideration, pores, channels and fibres exposed on the surface of the hydrogels tested can have significant effects on the extent of cell adheson and proliferation. It is recommended that the core component is fabricated out of one of the following to provide a non cell adhesive base; A8, A11, A13, A22, A23. The haptic periphery fabricated out of one of the following would provide a cell adhesive composite; A16, A30, A33, A37, A38, A42, A43, A44. The presence of vitronectin in the ocular tissue appears to lead to higher cell adhesion to the posterior surface of a contact lens when compared to the anterior surface. Group IV contact lenses adhere more cells than Group II contact lenses - this may indicate that more protein (including vitronectin) is able to adhere to the contact lens due to the Group IV contact lenses high water content and ionic hydrogel matrix. Artificial lung surfactant analogues were found to be non cytotoxic but also decreased cell proliferation when tested at higher concentrations. Poly(lysine ethyl ester adipamide) [PLETESA] had the most favourable response on cell proliferation and commercial styrene/maleic anhydride (pMA/STY sp2) the most pronounced inhibitory response. The mode of action that decreases cell proliferation appears to be through membrane destabilization. Tissue culture well plates coated with PLETESA allowed cells to adhere in a concentration dependent manner, multilaminar liposomes possibly of PLETESA were observed in solution in PLETESA coated wells. Polyhydroxybutryate (PHB) and polyhydroxyvalerate (PHV) blends that contained hydroxyapatite were found to be the most cell adhesive material of those materials tested. The blends that were most susceptible to degradation adhered the most cells in initial stages of degradation. The initial slight increase in cell adhesion may be due to the increased rugosity of the material. As the degradation continued the number of cells adhering to the samples decreased, this may indicate that the polarity was inhibitory to cell adhesion during the later stages of degradation.

Evaluation of modern intraocular lenses

Accommodating Intraocular Lenses (IOLs), multifocal IOLs (MIOLs) and toric IOLs are designed to provide a greater level of spectacle independency post cataract surgery. All of these IOLs are reliant on the accurate calculation of intraocular lens power determined through reliable ocular biometry. A standardised defocus area metric and reading performance index metric were devised for the evaluation of the range of focus and the reading ability of subjects implanted with presbyopic correcting IOLs. The range of clear vision after implantation of an MIOL is extended by a second focal point; however, this results in the prevalence of dysphotopsia. A bespoke halometer was designed and validated to assess this photopic phenomenon. There is a lack of standardisation in the methods used for determining IOL orientation and thus rotation. A repeatable, objective method was developed to allow the accurate assessment of IOL rotation, which was used to determine the rotational and positional stability of a closed loop haptic IOL. A new commercially available biometry device was validated for use with subjects prior to cataract surgery. The optical low coherence reflectometry instrument proved to be a valid method for assessing ocular biometry and covered a wider range of ocular parameters in comparison with previous instruments. The advantages of MIOLs were shown to include an extended range of clear vision translating into greater reading ability. However, an increased prevalence of dysphotopsia was shown with a bespoke halometer, which was dependent on the MIOL optic design. Implantation of a single optic accommodating IOL did not improve reading ability but achieved high subjective ratings of near vision. The closed-loop haptic IOL displayed excellent rotational stability in the late period but relatively poor rotational stability in the early period post implantation. The orientation error was compounded by the high frequency of positional misalignment leading to an extensive overall misalignment of the IOL. This thesis demonstrates the functionality of new IOL lens designs and the importance of standardised testing methods, thus providing a greater understanding of the consequences of implanting these IOLs. Consequently, the findings of the thesis will influence future designs of IOLs and testing methods.

Imagery in multi-modal object learning

Spatial objects may not only be perceived visually but also by touch. We report recent experiments investigating to what extent prior object knowledge acquired in either the haptic or visual sensory modality transfers to a subsequent visual learning task. Results indicate that even mental object representations learnt in one sensory modality may attain a multi-modal quality. These findings seem incompatible with picture-based reasoning schemas but leave open the possibility of modality-specific reasoning mechanisms.

Mobile assistive technologies for the visually impaired

There are around 285 million visually impaired people worldwide, and around 370,000 people are registered as blind or partially sighted in the UK. Ongoing advances in information technology (IT) are increasing the scope for IT-based mobile assistive technologies to facilitate the independence, safety, and improved quality of life of the visually impaired. Research is being directed at making mobile phones and other handheld devices accessible via our haptic (touch) and audio sensory channels. We review research and innovation within the field of mobile assistive technology for the visually impaired and, in so doing, highlight the need for successful collaboration between clinical expertise, computer science, and domain users to realize fully the potential benefits of such technologies. We initially reflect on research that has been conducted to make mobile phones more accessible to people with vision loss. We then discuss innovative assistive applications designed for the visually impaired that are either delivered via mainstream devices and can be used while in motion (e.g., mobile phones) or are embedded within an environment that may be in motion (e.g., public transport) or within which the user may be in motion (e.g., smart homes). © 2013 Elsevier Inc.

A micro-opto-electro-mechanical system (MOEMS) for microstructure manipulation

Microstructure manipulation is a fundamental process to the study of biology and medicine, as well as to advance micro- and nano-system applications. Manipulation of microstructures has been achieved through various microgripper devices developed recently, which lead to advances in micromachine assembly, and single cell manipulation, among others. Only two kinds of integrated feedback have been demonstrated so far, force sensing and optical binary feedback. As a result, the physical, mechanical, optical, and chemical information about the microstructure under study must be extracted from macroscopic instrumentation, such as confocal fluorescence microscopy and Raman spectroscopy. In this research work, novel Micro-Opto-Electro-Mechanical-System (MOEMS) microgrippers are presented. These devices utilize flexible optical waveguides as gripping arms, which provide the physical means for grasping a microobject, while simultaneously enabling light to be delivered and collected. This unique capability allows extensive optical characterization of the structure being held such as transmission, reflection, or fluorescence. The microgrippers require external actuation which was accomplished by two methods: initially with a micrometer screw, and later with a piezoelectric actuator. Thanks to a novel actuation mechanism, the "fishbone", the gripping facets remain parallel within 1 degree. The design, simulation, fabrication, and characterization are systematically presented. The devices mechanical operation was verified by means of 3D finite element analysis simulations. Also, the optical performance and losses were simulated by the 3D-to-2D effective index (finite difference time domain FDTD) method as well as 3D Beam Propagation Method (3D-BPM). The microgrippers were designed to manipulate structures from submicron dimensions up to approximately 100 μm. The devices were implemented in SU-8 due to its suitable optical and mechanical properties. This work demonstrates two practical applications: the manipulation of single SKOV-3 human ovarian carcinoma cells, and the detection and identification of microparts tagged with a fluorescent "barcode" implemented with quantum dots. The novel devices presented open up new possibilities in the field of micromanipulation at the microscale, scalable to the nano-domain.

Challenging ESL Students to Avoid Plagiarism and Properly Summarize and Cite Articles

Research shows that plagiarism is a problem not only for English language learners but also for students whose first language is English. With the Internet and ease of copying and pasting information into a word document, plagiarism in on the rise (Maslen, 2003). Oftentimes, students are not aware they are doing something wrong. American students come into college with the cultural conditioning of knowing (perhaps not fully grasping) American academic standards (Gu & Brooks, 2007). International students have the additional disadvantage of not knowing the conventions, traditions, and values held in academic discourse (Gu & Brooks, 2007). Within American academic circles, plagiarism is considered “one of the worst crimes” a student can commit (Wheeler, 2008). However, outside the United States, plagiarism is culturally acceptable; in fact a moral transgression would be to not copy and paste the words of an expert (Wheeler, 2008). Most of the students in English for Academic Purposes (EAP) at Miami-Dade College are planning on continuing their education once they finish the EAP program so it is essential that they are exposed to the issue of plagiarism. A number of faculty who teach in subject areas have complained that incoming students do not have the skills needed to succeed; these skills include how to cite sources and reference material. As a result of this, the focus of this action research project was on incorporating and explaining plagiarism and providing a number of writing opportunities throughout the semester.

Waveguide microgripper for identification, sensing and manipulation

A Waveguide Microgripper utilizes flexible optical waveguides as gripping arms, which provide the physical means for grasping a microobject, while simultaneously enabling light to be delivered and collected. This unique capability allows extensive optical characterization of the structure being held such as transmission, reflection or fluorescence. One of the simplest capabilities of the waveguide microgripper is to be able to detect the presence of a microobject between the microgripper facets by monitoring the transmitted intensity of light coupled through the facets. The intensity of coupled light is expected to drop when there is an object obstructing the path of light. The optical sensing and characterization function of the microgripper is a strong function of the optical power incident on the structure of interest. Hence it is important to understand the factors affecting the power distribution across the facet. The microgripper is also capable of detecting the fluorescence. This capability of microgripper is expected to have applications in medical, bio-medical and related fields.

Investigando o uso do sentido háptico na visualização de dados multivariados em gráficos de dispersão bidimensionais

Data Visualization is widely used to facilitate the comprehension of information and find relationships between data. One of the most widely used techniques for multivariate data (4 or more variables) visualization is the 2D scatterplot. This technique associates each data item to a visual mark in the following way: two variables are mapped to Cartesian coordinates so that a visual mark can be placed on the Cartesian plane; the others variables are mapped gradually to visual properties of the mark, such as size, color, shape, among others. As the number of variables to be visualized increases, the amount of visual properties associated to the mark increases as well. As a result, the complexity of the final visualization is higher. However, increasing the complexity of the visualization does not necessarily implies a better visualization and, sometimes, it provides an inverse situation, producing a visually polluted and confusing visualization—this problem is called visual properties overload. This work aims to investigate whether it is possible to work around the overload of the visual channel and improve insight about multivariate data visualized through a modification in the 2D scatterplot technique. In this modification, we map the variables from data items to multisensoriy marks. These marks are composed not only by visual properties, but haptic properties, such as vibration, viscosity and elastic resistance, as well. We believed that this approach could ease the insight process, through the transposition of properties from the visual channel to the haptic channel. The hypothesis was verified through experiments, in which we have analyzed (a) the accuracy of the answers; (b) response time; and (c) the grade of personal satisfaction with the proposed approach. However, the hypothesis was not validated. The results suggest that there is an equivalence between the investigated visual and haptic properties in all analyzed aspects, though in strictly numeric terms the multisensory visualization achieved better results in response time and personal satisfaction.

A produção textual no ensino fundamental: processo de retextualização com o gênero memórias

Teaching Portuguese language in Brazilian public schools is still limited mostly to studying decontextualized text fragments, memorizing classifications and cult of grammar rules. Considering the language as a social, cultural practice which emerges from the intersubjective interaction, we sought to propose an educational intervention that prioritizes the retextualization processes from speech to the writing of memoirs as a textual genre, so as to contribute for improving learner’s discursive performances. Therefore, paying attention to these concerns and in attempt to contribute for improving the teaching of Portuguese language in elementary school, we chose as privileged locus a 9th grade class from a state school in Bento Fernandes, RN. The corpus is formed by texts produced and retextualized by students from the elders’ oral reports within local community. We sought thus to understand what memory is, its importance for registering local spoken language and culture, as much as to carry out didactic actions that favor students’ learning in the activities of textual production. In light of the theoretical overviews about linguistic-discursive relations, based on Marcuschi’s (1993, 1997, 2001, 2002, 2006, 2008, 2010) conception of oralitiy-writing continuum and the debates proposed by Antunes (2003, 2014), Alves Filho (2011), Koch (2012) and Bakhtin (1992, 2011), we aimed to understand, by analyzing the retextualized memoirs, how these practices complement each other within the process of orality and writing. As for the proposal of didactic sequences, the study has been oriented by Dolz and Scheneuely (2004); as for the memoirs, by the guidelines of Coracine and Ghiraldelo (2011) and Le Goff (2010, 2013). In this way, this work followed the action-research methodology in a qualitative approach, considering the teacher (researcher) as an active agent involved in the process of knowledge production in his own educational practice, so as to interfere in the mediation, knowledge production and its dissemination in classroom context, which is the privileged locus for constructing and transforming process. There is much to be research within the area of retextualization. Yet we verified that this educational intervention, based on discursive operators of retextualization, has been proven viable as an efficient path so that we teachers can work the peculiarities of usages and functions of textual genres in oral and written modalities of a language, without grasping both as a dichotomy. This accredited us to strengthen a discourse that undoes many myths still present in that order, especially the one that causes more damage for the learners of Portuguese language – that writing is a representation of speech.

The functional subdivision of the visual brain : Is there a real illusion effect on action? A multi-lab replication study

Acknowledgements We thank Brian Roberts and Mike Harris for responding to our questions regarding their paper; Zoltan Dienes for advice on Bayes factors; Denise Fischer, Melanie Römer, Ioana Stanciu, Aleksandra Romanczuk, Stefano Uccelli, Nuria Martos Sánchez, and Rosa María Beño Ruiz de la Sierra for help collecting data; Eva Viviani for managing data collection in Parma. We thank Maurizio Gentilucci for letting us use his lab, and the Centro Intradipartimentale Mente e Cervello (CIMeC), University of Trento, and especially Francesco Pavani for lending us his motion tracking equipment. We thank Rachel Foster for proofreading. KKK was supported by a Ph.D. scholarship as part of a grant to VHF within the International Graduate Research Training Group on Cross-Modal Interaction in Natural and Artificial Cognitive Systems (CINACS; DFG IKG-1247) and TS by a grant (DFG – SCHE 735/3-1); both from the German Research Council.