Designing a Computer Vision System for Underwater Robotic Interventions

This thesis deals with the challenging problem of designing systems able to perceive objects in underwater environments. In the last few decades research activities in robotics have advanced the state of art regarding intervention capabilities of autonomous systems. State of art in fields such as localization and navigation, real time perception and cognition, safe action and manipulation capabilities, applied to ground environments (both indoor and outdoor) has now reached such a readiness level that it allows high level autonomous operations. On the opposite side, the underwater environment remains a very difficult one for autonomous robots. Water influences the mechanical and electrical design of systems, interferes with sensors by limiting their capabilities, heavily impacts on data transmissions, and generally requires systems with low power consumption in order to enable reasonable mission duration. Interest in underwater applications is driven by needs of exploring and intervening in environments in which human capabilities are very limited. Nowadays, most underwater field operations are carried out by manned or remotely operated vehicles, deployed for explorations and limited intervention missions. Manned vehicles, directly on-board controlled, expose human operators to risks related to the stay in field of the mission, within a hostile environment. Remotely Operated Vehicles (ROV) currently represent the most advanced technology for underwater intervention services available on the market. These vehicles can be remotely operated for long time but they need support from an oceanographic vessel with multiple teams of highly specialized pilots. Vehicles equipped with multiple state-of-art sensors and capable to autonomously plan missions have been deployed in the last ten years and exploited as observers for underwater fauna, seabed, ship wrecks, and so on. On the other hand, underwater operations like object recovery and equipment maintenance are still challenging tasks to be conducted without human supervision since they require object perception and localization with much higher accuracy and robustness, to a degree seldom available in Autonomous Underwater Vehicles (AUV). This thesis reports the study, from design to deployment and evaluation, of a general purpose and configurable platform dedicated to stereo-vision perception in underwater environments. Several aspects related to the peculiar environment characteristics have been taken into account during all stages of system design and evaluation: depth of operation and light conditions, together with water turbidity and external weather, heavily impact on perception capabilities. The vision platform proposed in this work is a modular system comprising off-the-shelf components for both the imaging sensors and the computational unit, linked by a high performance ethernet network bus. The adopted design philosophy aims at achieving high flexibility in terms of feasible perception applications, that should not be as limited as in case of a special-purpose and dedicated hardware. Flexibility is required by the variability of underwater environments, with water conditions ranging from clear to turbid, light backscattering varying with daylight and depth, strong color distortion, and other environmental factors. Furthermore, the proposed modular design ensures an easier maintenance and update of the system over time. Performance of the proposed system, in terms of perception capabilities, has been evaluated in several underwater contexts taking advantage of the opportunity offered by the MARIS national project. Design issues like energy power consumption, heat dissipation and network capabilities have been evaluated in different scenarios. Finally, real-world experiments, conducted in multiple and variable underwater contexts, including open sea waters, have led to the collection of several datasets that have been publicly released to the scientific community. The vision system has been integrated in a state of the art AUV equipped with a robotic arm and gripper, and has been exploited in the robot control loop to successfully perform underwater grasping operations.

Effect of uncorrected astigmatism on vision

PURPOSE: To examine the effect of uncorrected astigmatism in older adults. SETTING: University Vision Clinic METHOD: Twenty-one healthy presbyopes, aged 58.9±2.8 years, had astigmatism of 0.0 to -4.0 x 90?DC and -3.0DC of cylinder at 90?, 180? and 45? induced with spectacle lenses, with the mean spherical equivalent compensated to plano, in random order. Visual acuity was assessed binocularly using a computerised test chart at 95%, 50% and 10% contrast. Near acuity and reading speed were measured using standardised reading texts. Light scatter was quantified with the cQuant and driving reaction times with a computer simulator. Finally visual clarity of a mobile phone and computer screen was subjectively rated. RESULTS: Distance visual acuity decreased with increasing uncorrected astigmatic power (F=174.50, p<0.001) and was reduced at lower contrasts (F=170.77, p<0.001). Near visual acuity and reading speed also decreased with increasing uncorrected astigmatism power (p<0.001). Light scatter was not significantly affected by uncorrected astigmatism (p>0.05), but the reliability and variability of measurements decreased with increasing uncorrected astigmatic power (p<0.05). Driving simulator performance was also unaffected by uncorrected astigmatism (p>0.05), but subjective rating of clarity decreased with increasing uncorrected astigmatic power (p<0.001). Uncorrected astigmatism at 45? or 180? orientation resulted in a worse distance and near visual acuity, and subjective rated clarity than 90? orientation (p<0.05). CONCLUSION: Uncorrected astigmatism, even as low as 1.0DC, causes a significant burden on a patient’s vision. If left uncorrected, this could impact significantly on their independence, quality of life and wellbeing.

The role of oscillatory brain activity in object processing and figure-ground segmentation in human vision

The perception of an object as a single entity within a visual scene requires that its features are bound together and segregated from the background and/or other objects. Here, we used magnetoencephalography (MEG) to assess the hypothesis that coherent percepts may arise from the synchronized high frequency (gamma) activity between neurons that code features of the same object. We also assessed the role of low frequency (alpha, beta) activity in object processing. The target stimulus (i.e. object) was a small patch of a concentric grating of 3c/°, viewed eccentrically. The background stimulus was either a blank field or a concentric grating of 3c/° periodicity, viewed centrally. With patterned backgrounds, the target stimulus emerged--through rotation about its own centre--as a circular subsection of the background. Data were acquired using a 275-channel whole-head MEG system and analyzed using Synthetic Aperture Magnetometry (SAM), which allows one to generate images of task-related cortical oscillatory power changes within specific frequency bands. Significant oscillatory activity across a broad range of frequencies was evident at the V1/V2 border, and subsequent analyses were based on a virtual electrode at this location. When the target was presented in isolation, we observed that: (i) contralateral stimulation yielded a sustained power increase in gamma activity; and (ii) both contra- and ipsilateral stimulation yielded near identical transient power changes in alpha (and beta) activity. When the target was presented against a patterned background, we observed that: (i) contralateral stimulation yielded an increase in high-gamma (>55 Hz) power together with a decrease in low-gamma (40-55 Hz) power; and (ii) both contra- and ipsilateral stimulation yielded a transient decrease in alpha (and beta) activity, though the reduction tended to be greatest for contralateral stimulation. The opposing power changes across different regions of the gamma spectrum with 'figure/ground' stimulation suggest a possible dual role for gamma rhythms in visual object coding, and provide general support of the binding-by-synchronization hypothesis. As the power changes in alpha and beta activity were largely independent of the spatial location of the target, however, we conclude that their role in object processing may relate principally to changes in visual attention.

Binocular contrast vision at and above threshold

A fundamental problem for any visual system with binocular overlap is the combination of information from the two eyes. Electrophysiology shows that binocular integration of luminance contrast occurs early in visual cortex, but a specific systems architecture has not been established for human vision. Here, we address this by performing binocular summation and monocular, binocular, and dichoptic masking experiments for horizontal 1 cycle per degree test and masking gratings. These data reject three previously published proposals, each of which predict too little binocular summation and insufficient dichoptic facilitation. However, a simple development of one of the rejected models (the twin summation model) and a completely new model (the two-stage model) provide very good fits to the data. Two features common to both models are gently accelerating (almost linear) contrast transduction prior to binocular summation and suppressive ocular interactions that contribute to contrast gain control. With all model parameters fixed, both models correctly predict (1) systematic variation in psychometric slopes, (2) dichoptic contrast matching, and (3) high levels of binocular summation for various levels of binocular pedestal contrast. A review of evidence from elsewhere leads us to favor the two-stage model. © 2006 ARVO.

Psychophysical evidence for two routes to suppression before binocular summation of signals in human vision

Visual mechanisms in primary visual cortex are suppressed by the superposition of gratings perpendicular to their preferred orientations. A clear picture of this process is needed to (i) inform functional architecture of image-processing models, (ii) identify the pathways available to support binocular rivalry, and (iii) generally advance our understanding of early vision. Here we use monoptic sine-wave gratings and cross-orientation masking (XOM) to reveal two cross-oriented suppressive pathways in humans, both of which occur before full binocular summation of signals. One is a within-eye (ipsiocular) pathway that is spatially broadband, immune to contrast adaptation and has a suppressive weight that tends to decrease with stimulus duration. The other pathway operates between the eyes (interocular), is spatially tuned, desensitizes with contrast adaptation and has a suppressive weight that increases with stimulus duration. When cross-oriented masks are presented to both eyes, masking is enhanced or diminished for conditions in which either ipsiocular or interocular pathways dominate masking, respectively. We propose that ipsiocular suppression precedes the influence of interocular suppression and tentatively associate the two effects with the lateral geniculate nucleus (or retina) and the visual cortex respectively. The interocular route is a good candidate for the initial pathway involved in binocular rivalry and predicts that interocular cross-orientation suppression should be found in cortical cells with predominantly ipsiocular drive. © 2007 IBRO.

Binocular interaction:Contrast matching and contrast discrimination are predicted by the same model

How do signals from the 2 eyes combine and interact? Our recent work has challenged earlier schemes in which monocular contrast signals are subject to square-law transduction followed by summation across eyes and binocular gain control. Much more successful was a new 'two-stage' model in which the initial transducer was almost linear and contrast gain control occurred both pre- and post-binocular summation. Here we extend that work by: (i) exploring the two-dimensional stimulus space (defined by left- and right-eye contrasts) more thoroughly, and (ii) performing contrast discrimination and contrast matching tasks for the same stimuli. Twenty-five base-stimuli made from 1 c/deg patches of horizontal grating, were defined by the factorial combination of 5 contrasts for the left eye (0.3-32%) with five contrasts for the right eye (0.3-32%). Other than in contrast, the gratings in the two eyes were identical. In a 2IFC discrimination task, the base-stimuli were masks (pedestals), where the contrast increment was presented to one eye only. In a matching task, the base-stimuli were standards to which observers matched the contrast of either a monocular or binocular test grating. In the model, discrimination depends on the local gradient of the observer's internal contrast-response function, while matching equates the magnitude (rather than gradient) of response to the test and standard. With all model parameters fixed by previous work, the two-stage model successfully predicted both the discrimination and the matching data and was much more successful than linear or quadratic binocular summation models. These results show that performance measures and perception (contrast discrimination and contrast matching) can be understood in the same theoretical framework for binocular contrast vision. © 2007 VSP.

Creutzfeldt-Jakob disease and vision

This review describes a group of diseases known as the transmissible spongiform encephalopathies (TSEs), which affect animals and humans. Examination of affected brain tissue suggests that these diseases are caused by the acquisition and deposition of prion protein (PrP). Creutzfeldt-Jakob disease (CJD) is the most important form of TSE in humans with at least four different varieties of the disease. Variant CJD (vCJD), a new form of the disease found in the UK, has several features that differ from the classical forms including early age of onset, longer duration of disease, psychiatric presentation (for example, depression) and extensive florid plaque development in the brain. About 10 per cent of patients with CJD exhibit visual symptoms at disease presentation and approximately 50 per cent during the course of the disease. The most commonly reported visual symptoms include diplopia, supranuclear palsies, complex visual disturbances, homonymous visual field defects, hallucinations and cortical blindness. Saccadic and smooth pursuit movements appear to be more rarely affected. The agent causing vCJD accumulates in lymphoid tissue such as the spleen and tonsils. The cornea has lymphoid tissue in the form of corneal dendritic cells that are important in the regulation of the immune response in the anterior segment of the eye. The presence of these cells in the cornea has raised the possibility of transmission between patients via optical devices that contact the eye. Although such transmission is theoretically possible it remains highly improbable.

An integrated scheduling problem of PCB components on sequential pick-and-place machines:mathematical models and heuristic solutions

This paper formulates several mathematical models for determining the optimal sequence of component placements and assignment of component types to feeders simultaneously or the integrated scheduling problem for a type of surface mount technology placement machines, called the sequential pick-andplace (PAP) machine. A PAP machine has multiple stationary feeders storing components, a stationary working table holding a printed circuit board (PCB), and a movable placement head to pick up components from feeders and place them to a board. The objective of integrated problem is to minimize the total distance traveled by the placement head. Two integer nonlinear programming models are formulated first. Then, each of them is equivalently converted into an integer linear type. The models for the integrated problem are verified by two commercial packages. In addition, a hybrid genetic algorithm previously developed by the authors is adopted to solve the models. The algorithm not only generates the optimal solutions quickly for small-sized problems, but also outperforms the genetic algorithms developed by other researchers in terms of total traveling distance.

The relative contributions of colour and luminance signals towards the visuomotor localisation of targets in human peripheral vision

We sought to determine the extent to which colour (and luminance) signals contribute towards the visuomotor localization of targets. To do so we exploited the movement-related illusory displacement a small stationary window undergoes when it has a continuously moving carrier grating behind it. We used drifting (1.0-4.2 Hz) red/green-modulated isoluminant gratings or yellow/black luminance-modulated gratings as carriers, each curtailed in space by a stationary, two-dimensional window. After each trial, the perceived location of the window was recorded with reference to an on-screen ruler (perceptual task) or the on-screen touch of a ballistic pointing movement made without visual feedback (visuomotor task). Our results showed that the perceptual displacement measures were similar for each stimulus type and weakly dependent on stimulus drift rate. However, while the visuomotor displacement measures were similar for each stimulus type at low drift rates (<4 Hz), they were significantly larger for luminance than colour stimuli at high drift rates (>4 Hz). We show that the latter cannot be attributed to differences in perceived speed between stimulus types. We assume, therefore, that our visuomotor localization judgements were more susceptible to the (carrier) motion of luminance patterns than colour patterns. We suggest that, far from being detrimental, this susceptibility may indicate the operation of mechanisms designed to counter the temporal asynchrony between perceptual experiences and the physical changes in the environment that give rise to them. We propose that perceptual localisation is equally supported by both colour and luminance signals but that visuomotor localisation is predominantly supported by luminance signals. We discuss the neural pathways that may be involved with visuomotor localization. © 2007 Springer-Verlag.

Mach edges: local features predicted by 3rd derivative spatial filtering

Edges are key points of information in visual scenes. One important class of models supposes that edges correspond to the steepest parts of the luminance profile, implying that they can be found as peaks and troughs in the response of a gradient (1st derivative) filter, or as zero-crossings in the 2nd derivative (ZCs). We tested those ideas using a stimulus that has no local peaks of gradient and no ZCs, at any scale. The stimulus profile is analogous to the Mach ramp, but it is the luminance gradient (not the absolute luminance) that increases as a linear ramp between two plateaux; the luminance profile is a blurred triangle-wave. For all image-blurs tested, observers marked edges at or close to the corner points in the gradient profile, even though these were not gradient maxima. These Mach edges correspond to peaks and troughs in the 3rd derivative. Thus Mach edges are inconsistent with many standard edge-detection schemes, but are nicely predicted by a recent model that finds edge points with a 2-stage sequence of 1st then 2nd derivative operators, each followed by a half-wave rectifier.

Blur adaptation explained by a norm-based model of contrast adaptation

Adapting to blurred images makes in-focus images look too sharp, and vice-versa (Webster et al, 2002 Nature Neuroscience 5 839 - 840). We asked how such blur adaptation is related to contrast adaptation. Georgeson (1985 Spatial Vision 1 103 - 112) found that grating contrast adaptation followed a subtractive rule: perceived (matched) contrast of a grating was fairly well predicted by subtracting some fraction k(~0.3) of the adapting contrast from the test contrast. Here we apply that rule to the responses of a set of spatial filters at different scales and orientations. Blur is encoded by the pattern of filter response magnitudes over scale. We tested two versions - the 'norm model' and 'fatigue model' - against blur-matching data obtained after adaptation to sharpened, in-focus or blurred images. In the fatigue model, filter responses are simply reduced by exposure to the adapter. In the norm model, (a) the visual system is pre-adapted to a focused world and (b) discrepancy between observed and expected responses to the experimental adapter leads to additional reduction (or enhancement) of filter responses during experimental adaptation. The two models are closely related, but only the norm model gave a satisfactory account of results across the four experiments analysed, with one free parameter k. This model implies that the visual system is pre-adapted to focused images, that adapting to in-focus or blank images produces no change in adaptation, and that adapting to sharpened or blurred images changes the state of adaptation, leading to changes in perceived blur or sharpness.

Suppression and summation in contrast gain control for human vision

Over the last ten years our understanding of early spatial vision has improved enormously. The long-standing model of probability summation amongst multiple independent mechanisms with static output nonlinearities responsible for masking is obsolete. It has been replaced by a much more complex network of additive, suppressive, and facilitatory interactions and nonlinearities across eyes, area, spatial frequency, and orientation that extend well beyond the classical recep-tive field (CRF). A review of a substantial body of psychophysical work performed by ourselves (20 papers), and others, leads us to the following tentative account of the processing path for signal contrast. The first suppression stage is monocular, isotropic, non-adaptable, accelerates with RMS contrast, most potent for low spatial and high temporal frequencies, and extends slightly beyond the CRF. Second and third stages of suppression are difficult to disentangle but are possibly pre- and post-binocular summation, and involve components that are scale invariant, isotropic, anisotropic, chromatic, achromatic, adaptable, interocular, substantially larger than the CRF, and saturated by contrast. The monocular excitatory pathways begin with half-wave rectification, followed by a preliminary stage of half-binocular summation, a square-law transducer, full binocular summation, pooling over phase, cross-mechanism facilitatory interactions, additive noise, linear summation over area, and a slightly uncertain decision-maker. The purpose of each of these interactions is far from clear, but the system benefits from area and binocular summation of weak contrast signals as well as area and ocularity invariances above threshold (a herd of zebras doesn't change its contrast when it increases in number or when you close one eye). One of many remaining challenges is to determine the stage or stages of spatial tuning in the excitatory pathway.

Depth propagation and surface construction in 3-D vision

In stereo vision, regions with ambiguous or unspecified disparity can acquire perceived depth from unambiguous regions. This has been called stereo capture, depth interpolation or surface completion. We studied some striking induced depth effects suggesting that depth interpolation and surface completion are distinct stages of visual processing. An inducing texture (2-D Gaussian noise) had sinusoidal modulation of disparity, creating a smooth horizontal corrugation. The central region of this surface was replaced by various test patterns whose perceived corrugation was measured. When the test image was horizontal 1-D noise, shown to one eye or to both eyes without disparity, it appeared corrugated in much the same way as the disparity-modulated (DM) flanking regions. But when the test image was 2-D noise, or vertical 1-D noise, little or no depth was induced. This suggests that horizontal orientation was a key factor. For a horizontal sine-wave luminance grating, strong depth was induced, but for a square-wave grating, depth was induced only when its edges were aligned with the peaks and troughs of the DM flanking surface. These and related results suggest that disparity (or local depth) propagates along horizontal 1-D features, and then a 3-D surface is constructed from the depth samples acquired. The shape of the constructed surface can be different from the inducer, and so surface construction appears to operate on the results of a more local depth propagation process.

Discriminating depth in corrugated stereo surfaces: facilitation by a pedestal is explained by removal of uncertainty

With luminance gratings, psychophysical thresholds for detecting a small increase in the contrast of a weak ‘pedestal’ grating are 2–3 times lower than for detection of a grating when the pedestal is absent. This is the ‘dipper effect’ – a reliable improvement whose interpretation remains controversial. Analogies between luminance and depth (disparity) processing have attracted interest in the existence of a ‘disparity dipper’. Are thresholds for disparity modulation (corrugated surfaces), facilitated by the presence of a weak disparity-modulated pedestal? We used a 14-bit greyscale to render small disparities accurately, and measured 2AFC discrimination thresholds for disparity modulation (0.3 or 0.6 c/deg) of a random texture at various pedestal levels. In the first experiment, a clear dipper was found. Thresholds were about 2× lower with weak pedestals than without. But here the phase of modulation (0 or 180 deg) was varied from trial to trial. In a noisy signal-detection framework, this creates uncertainty that is reduced by the pedestal, which thus improves performance. When the uncertainty was eliminated by keeping phase constant within sessions, the dipper effect was weak or absent. Monte Carlo simulations showed that the influence of uncertainty could account well for the results of both experiments. A corollary is that the visual depth response to small disparities is probably linear, with no threshold-like nonlinearity.