The role of texture amplitude in shape from shading: evidence from a haptic matching task

The pattern of illumination on an undulating surface can be used to infer its 3-D form (shape from shading). But the recovery of shape would be invalid if the shading actually arose from reflectance variation. When a corrugated surface is painted with an albedo texture, the variation in local mean luminance (LM) due to shading is accompanied by a similar modulation in texture amplitude (AM). This is not so for reflectance variation, nor for roughly textured surfaces. We used a haptic matching technique to show that modulations of texture amplitude play a role in the interpretation of shape from shading. Observers were shown plaid stimuli comprising LM and AM combined in-phase (LM+AM) on one oblique and in anti-phase (LM-AM) on the other. Stimuli were presented via a modified ReachIN workstation allowing the co-registration of visual and haptic stimuli. In the first experiment, observers were asked to adjust the phase of a haptic surface, which had the same orientation as the LM+AM combination, until its peak in depth aligned with the visually perceived peak. The resulting alignments were consistent with the use of a lighting-from-above prior. In the second experiment, observers were asked to adjust the amplitude of the haptic surface to match that of the visually perceived surface. Observers chose relatively large amplitude settings when the haptic surface was oriented and phase-aligned with the LM+AM cue. When the haptic surface was aligned with the LM-AM cue, amplitude settings were close to zero. Thus the LM/AM phase relation is a significant visual depth cue, and is used to discriminate between shading and reflectance variations. [Supported by the Engineering and Physical Sciences Research Council, EPSRC].

The role of texture in shape from shading: are humans biased towards seeing relief textures?

When a textured surface is modulated in depth and illuminated, the level of illumination varies across the surface, producing coarse-scale luminance modulations (LM) and amplitude modulation (AM) of the fine-scale texture. If the surface has an albedo texture (reflectance variation) then the LM and AM components are always in-phase, but if the surface has a relief texture the phase relation between LM and AM varies with the direction and nature of the illuminant. We showed observers sinusoidal luminance and amplitude modulations of a binary noise texture, in various phase relationships, in a paired-comparisons design. In the first experiment, the combinations under test were presented in different temporal intervals. Observers indicated which interval contained the more depthy stimulus. LM and AM in-phase were seen as more depthy than LM alone which was in turn more depthy than LM and AM in anti-phase, but the differences were weak. In the second experiment the combinations under test were presented in a single interval on opposite obliques of a plaid pattern. Observers were asked to indicate the more depthy oblique. Observers produced the same depth rankings as before, but now the effects were more robust and significant. Intermediate LM/AM phase relationships were also tested: phase differences less than 90 deg were seen as more depthy than LM-only, while those greater than 90 deg were seen as less depthy. We conjecture that the visual system construes phase offsets between LM and AM as indicating relief texture and thus perceives these combinations as depthy even when their phase relationship is other than zero. However, when different LM/AM pairs are combined in a plaid, the signals on the obliques are unlikely to indicate corrugations of the same texture, and in this case the out-of-phase pairing is seen as flat. [Supported by the Engineering and Physical Sciences Research Council (EPSRC)].

The role of texture amplitude in shape from shading

When a textured surface is modulated in depth and illuminated, parts of the surface receive different levels of illumination; the resulting variations in luminance can be used to infer the shape of the depth modulations-shape from shading. The changes in illumination also produce changes in the amplitude of the texture, although local contrast remains constant. We investigated the role of texture amplitude in supporting shape from shading. If a luminance plaid is added to a binary noise texture (LM), shape from shading produces perception of corrugations in two directions. If the amplitude of the noise is also modulated (AM) such that it is in-phase with one of the luminance sinusoids and out-of-phase with the other, the resulting surface is seen as corrugated in only one directionöthat supported by the in-phase pairing. We confirmed this subjective report experimentally, using a depth-mapping technique. Further, we asked naïve observers to indicate the direction of corrugations in plaids made up of various combinations of LM and AM. LM+AM was seen as having most depth, then LM-only, then LM-AM, and then AM-only. Our results suggest that while LM is required to see depth from shading, its phase relative to any AM component is also important.

The role of local luminance amplitude in the interpretation of shape-from-shading in textured surfaces

Luminance changes within a scene are ambiguous; they can indicate reflectance changes, shadows, or shading due to surface undulations. How does vision distinguish between these possibilities? When a surface painted with an albedo texture is shaded, the change in local mean luminance (LM) is accompanied by a similar modulation of the local luminance amplitude (AM) of the texture. This relationship does not necessarily hold for reflectance changes or for shading of a relief texture. Here we concentrate on the role of AM in shape-from-shading. Observers were presented with a noise texture onto which sinusoidal LM and AM signals were superimposed, and were asked to indicate which of two marked locations was closer to them. Shape-from-shading was enhanced when LM and AM co-varied (in-phase), and was disrupted when they were out-of-phase. The perceptual differences between cue types (in-phase vs out-of-phase) were enhanced when the two cues were present at different orientations within a single image. Similar results were found with a haptic matching task. We conclude that vision can use AM to disambiguate luminance changes. LM and AM have a positive relationship for rendered, undulating, albedo textures, and we assess the degree to which this relationship holds in natural images. [Supported by EPSRC grants to AJS and MAG].

Local luminance amplitude modulates the interpretation of shape-from-shading in textured surfaces

The pattern of illumination on an undulating surface can be used to infer its 3-D form (shape-from-shading). But the recovery of shape would be invalid if the luminance changes actually arose from changes in reflectance. So how does vision distinguish variation in illumination from variation in reflectance to avoid illusory depth? When a corrugated surface is painted with an albedo texture, the variation in local mean luminance (LM) due to shading is accompanied by a similar modulation in local luminance amplitude (AM). This is not so for reflectance variation, nor for roughly textured surfaces. We used depth mapping and paired comparison methods to show that modulations of local luminance amplitude play a role in the interpretation of shape-from-shading. The shape-from-shading percept was enhanced when LM and AM co-varied (in-phase) and was disrupted when they were out of phase or (to a lesser degree) when AM was absent. The perceptual differences between cue types (in-phase vs out-of-phase) were enhanced when the two cues were present at different orientations within a single image. Our results suggest that when LM and AM co-vary (in-phase) this indicates that the source of variation is illumination (caused by undulations of the surface), rather than surface reflectance. Hence, the congruence of LM and AM is a cue that supports a shape-from-shading interpretation. © 2006 Elsevier Ltd. All rights reserved.

Combining Leaf Shape and Texture for Costa Rican Plant Species Identification

In the last decade, research in Computer Vision has developed several algorithms to help botanists and non-experts to classify plants based on images of their leaves. LeafSnap is a mobile application that uses a multiscale curvature model of the leaf margin to classify leaf images into species. It has achieved high levels of accuracy on 184 tree species from Northeast US. We extend the research that led to the development of LeafSnap along two lines. First, LeafSnap’s underlying algorithms are applied to a set of 66 tree species from Costa Rica. Then, texture is used as an additional criterion to measure the level of improvement achieved in the automatic identification of Costa Rica tree species. A 25.6% improvement was achieved for a Costa Rican clean image dataset and 42.5% for a Costa Rican noisy image dataset. In both cases, our results show this increment as statistically significant. Further statistical analysis of visual noise impact, best algorithm combinations per species, and best value of , the minimal cardinality of the set of candidate species that the tested algorithms render as best matches is also presented in this research

Absence of flash-lag when judging global shape from local positions

When a flash is presented aligned with a moving stimulus, the former is perceived to lag behind the latter (the flash-lag effect). We study whether this mislocalization occurs when a positional judgment is not required, but a veridical spatial relationship between moving and flashed stimuli is needed to perceive a global shape. To do this, we used Glass patterns that are formed by pairs of correlated dots. One dot of each pair was presented moving and, at a given moment, the other dot of each pair was flashed in order to build the Glass pattern. If a flash-lag effect occurs between each pair of dots, we expect the best perception of the global shape to occur when the flashed dots are presented before the moving dots arrive at the position that physically builds the Glass pattern. Contrary to this, we found that the best detection of Glass patterns occurred for the situation of physical alignment. This result is not consistent with a low-level contribution to the flash-lag effect.

The Design of Shape from Motion Constraints

This report presents a set of representations methodologies and tools for the purpose of visualizing, analyzing and designing functional shapes in terms of constraints on motion. The core of the research is an interactive computational environment that provides an explicit visual representation of motion constraints produced by shape interactions, and a series of tools that allow for the manipulation of motion constraints and their underlying shapes for the purpose of design.

Sun and sky:does human vision assume a mixture of point and diffuse illumination when interpreting shape-from-shading?

People readily perceive smooth luminance variations as being due to the shading produced by undulations of a 3-D surface (shape-from-shading). In doing so, the visual system must simultaneously estimate the shape of the surface and the nature of the illumination. Remarkably, shape-from-shading operates even when both these properties are unknown and neither can be estimated directly from the image. In such circumstances humans are thought to adopt a default illumination model. A widely held view is that the default illuminant is a point source located above the observer's head. However, some have argued instead that the default illuminant is a diffuse source. We now present evidence that humans may adopt a flexible illumination model that includes both diffuse and point source elements. Our model estimates a direction for the point source and then weights the contribution of this source according to a bias function. For most people the preferred illuminant direction is overhead with a strong diffuse component.

Shape from photographs:a multi-view stereo pipeline

Acquiring 3D shape from images is a classic problem in Computer Vision occupying researchers for at least 20 years. Only recently however have these ideas matured enough to provide highly accurate results. We present a complete algorithm to reconstruct 3D objects from images using the stereo correspondence cue. The technique can be described as a pipeline of four basic building blocks: camera calibration, image segmentation, photo-consistency estimation from images, and surface extraction from photo-consistency. In this Chapter we will put more emphasis on the latter two: namely how to extract geometric information from a set of photographs without explicit camera visibility, and how to combine different geometry estimates in an optimal way. © 2010 Springer-Verlag Berlin Heidelberg.

Lumbar spine texture enhances 10-year fracture probability assessment.

We found that lumbar spine texture analysis using trabecular bone score (TBS) is a risk factor for MOF and a risk factor for death in a retrospective cohort study from a large clinical registry for the province of Manitoba, Canada. INTRODUCTION: FRAX® estimates the 10-year probability of major osteoporotic fracture (MOF) using clinical risk factors and femoral neck bone mineral density (BMD). Trabecular bone score (TBS), derived from texture in the spine dual X-ray absorptiometry (DXA) image, is related to bone microarchitecture and fracture risk independently of BMD. Our objective was to determine whether TBS provides information on MOF probability beyond that provided by the FRAX variables. METHODS: We included 33,352 women aged 40-100 years (mean 63 years) with baseline DXA measurements of lumbar spine TBS and femoral neck BMD. The association between TBS, the FRAX variables, and the risk of MOF or death was examined using an extension of the Poisson regression model. RESULTS: During the mean of 4.7 years, 1,754 women died and 1,872 sustained one or more MOF. For each standard deviation reduction in TBS, there was a 36 % increase in MOF risk (HR 1.36, 95 % CI 1.30-1.42, p < 0.001) and a 32 % increase in death (HR 1.32, 95 % CI 1.26-1.39, p < 0.001). When adjusted for significant clinical risk factors and femoral neck BMD, lumbar spine TBS was still a significant predictor of MOF (HR 1.18, 95 % CI 1.12-1.23) and death (HR 1.20, 95 % CI 1.14-1.26). Models for estimating MOF probability, accounting for competing mortality, showed that low TBS (10th percentile) increased risk by 1.5-1.6-fold compared with high TBS (90th percentile) across a broad range of ages and femoral neck T-scores. CONCLUSIONS: Lumbar spine TBS is able to predict incident MOF independent of FRAX clinical risk factors and femoral neck BMD even after accounting for the increased death hazard.

A Laboratory Investigation into the Effects of the Aggregated Related Facts of Critical VMA in Asphalt Paving Mixtures, June 2000

This report summarizes research conducted at Iowa State University on behalf of the Iowa Department of Transportation, focusing on the volumetric state of hot-mix asphalt (HMA) mixtures as they transition from stable to unstable configurations. This has raditionally been addressed during mix design by meeting a minimum voids in the mineral aggregate (VMA) requirement, based solely upon the nominal maximum aggregate size without regard to other significant aggregate-related properties. The goal was to expand the current specification to include additional aggregate properties, e.g., fineness modulus, percent crushed fine and coarse aggregate, and their interactions. The work was accomplished in three phases: a literature review, extensive laboratory testing, and statistical analysis of test results. The literature review focused on the history and development of the current specification, laboratory methods of identifying critical mixtures, and the effects of other aggregate-related factors on critical mixtures. The laboratory testing involved three maximum aggregate sizes (19.0, 12.5, and 9.5 millimeters), three gradations (coarse, fine, and dense), and combinations of natural and manufactured coarse and fine aggregates. Specimens were compacted using the Superpave Gyratory Compactor (SGC), conventionally tested for bulk and maximum theoretical specific gravities and physically tested using the Nottingham Asphalt Tester (NAT) under a repeated load confined configuration to identify the transition state from sound to unsound. The statistical analysis involved using ANOVA and linear regression to examine the effects of identified aggregate factors on critical state transitions in asphalt paving mixtures and to develop predictive equations. The results clearly demonstrate that the volumetric conditions of an HMA mixture at the stable unstable threshold are influenced by a composite measure of the maximum aggregate size and gradation and by aggregate shape and texture. The currently defined VMA criterion, while significant, is seen to be insufficient by itself to correctly differentiate sound from unsound mixtures. Under current specifications, many otherwise sound mixtures are subject to rejection solely on the basis of failing to meet the VMA requirement. Based on the laboratory data and statistical analysis, a new paradigm to volumetric mix design is proposed that explicitly accounts for aggregate factors (gradation, shape, and texture).

Surface reconstruction from microscopic images in optical lithography.

This paper presents a method to reconstruct 3D surfaces of silicon wafers from 2D images of printed circuits taken with a scanning electron microscope. Our reconstruction method combines the physical model of the optical acquisition system with prior knowledge about the shapes of the patterns in the circuit; the result is a shape-from-shading technique with a shape prior. The reconstruction of the surface is formulated as an optimization problem with an objective functional that combines a data-fidelity term on the microscopic image with two prior terms on the surface. The data term models the acquisition system through the irradiance equation characteristic of the microscope; the first prior is a smoothness penalty on the reconstructed surface, and the second prior constrains the shape of the surface to agree with the expected shape of the pattern in the circuit. In order to account for the variability of the manufacturing process, this second prior includes a deformation field that allows a nonlinear elastic deformation between the expected pattern and the reconstructed surface. As a result, the minimization problem has two unknowns, and the reconstruction method provides two outputs: 1) a reconstructed surface and 2) a deformation field. The reconstructed surface is derived from the shading observed in the image and the prior knowledge about the pattern in the circuit, while the deformation field produces a mapping between the expected shape and the reconstructed surface that provides a measure of deviation between the circuit design models and the real manufacturing process.

Model-based objects recognition in industrial environments for autonomous vehicles control

Behavior-based navigation of autonomous vehicles requires the recognition of the navigable areas and the potential obstacles. In this paper we describe a model-based objects recognition system which is part of an image interpretation system intended to assist the navigation of autonomous vehicles that operate in industrial environments. The recognition system integrates color, shape and texture information together with the location of the vanishing point. The recognition process starts from some prior scene knowledge, that is, a generic model of the expected scene and the potential objects. The recognition system constitutes an approach where different low-level vision techniques extract a multitude of image descriptors which are then analyzed using a rule-based reasoning system to interpret the image content. This system has been implemented using a rule-based cooperative expert system