Surface Reflectance Estimation and Natural Illumination Statistics

Humans recognize optical reflectance properties of surfaces such as metal, plastic, or paper from a single image without knowledge of illumination. We develop a machine vision system to perform similar recognition tasks automatically. Reflectance estimation under unknown, arbitrary illumination proves highly underconstrained due to the variety of potential illumination distributions and surface reflectance properties. We have found that the spatial structure of real-world illumination possesses some of the statistical regularities observed in the natural image statistics literature. A human or computer vision system may be able to exploit this prior information to determine the most likely surface reflectance given an observed image. We develop an algorithm for reflectance classification under unknown real-world illumination, which learns relationships between surface reflectance and certain features (statistics) computed from a single observed image. We also develop an automatic feature selection method.

How do Humans Determine Reflectance Properties under Unknown Illumination?

Under normal viewing conditions, humans find it easy to distinguish between objects made out of different materials such as plastic, metal, or paper. Untextured materials such as these have different surface reflectance properties, including lightness and gloss. With single isolated images and unknown illumination conditions, the task of estimating surface reflectance is highly underconstrained, because many combinations of reflection and illumination are consistent with a given image. In order to work out how humans estimate surface reflectance properties, we asked subjects to match the appearance of isolated spheres taken out of their original contexts. We found that subjects were able to perform the task accurately and reliably without contextual information to specify the illumination. The spheres were rendered under a variety of artificial illuminations, such as a single point light source, and a number of photographically-captured real-world illuminations from both indoor and outdoor scenes. Subjects performed more accurately for stimuli viewed under real-world patterns of illumination than under artificial illuminations, suggesting that subjects use stored assumptions about the regularities of real-world illuminations to solve the ill-posed problem.

Recognition of Surface Reflectance Properties from a Single Image under Unknown Real-World Illumination

This paper describes a machine vision system that classifies reflectance properties of surfaces such as metal, plastic, or paper, under unknown real-world illumination. We demonstrate performance of our algorithm for surfaces of arbitrary geometry. Reflectance estimation under arbitrary omnidirectional illumination proves highly underconstrained. Our reflectance estimation algorithm succeeds by learning relationships between surface reflectance and certain statistics computed from an observed image, which depend on statistical regularities in the spatial structure of real-world illumination. Although the algorithm assumes known geometry, its statistical nature makes it robust to inaccurate geometry estimates.

Reflectance Map Techniques for Analyzing Surface Defects in Metal Castings

This report explores the relation between image intensity and object shape. It is shown that image intensity is related to surface orientation and that a variation in image intensity is related to surface curvature. Computational methods are developed which use the measured intensity variation across surfaces of smooth objects to determine surface orientation. In general, surface orientation is not determined locally by the intensity value recorded at each image point. Tools are needed to explore the problem of determining surface orientation from image intensity. The notion of gradient space , popularized by Huffman and Mackworth, is used to represent surface orientation. The notion of a reflectance map, originated by Horn, is used to represent the relation between surface orientation image intensity. The image Hessian is defined and used to represent surface curvature. Properties of surface curvature are expressed as constraints on possible surface orientations corresponding to a given image point. Methods are presented which embed assumptions about surface curvature in algorithms for determining surface orientation from the intensities recorded in a single view. If additional images of the same object are obtained by varying the direction of incident illumination, then surface orientation is determined locally by the intensity values recorded at each image point. This fact is exploited in a new technique called photometric stereo. The visual inspection of surface defects in metal castings is considered. Two casting applications are discussed. The first is the precision investment casting of turbine blades and vanes for aircraft jet engines. In this application, grain size is an important process variable. The existing industry standard for estimating the average grain size of metals is implemented and demonstrated on a sample turbine vane. Grain size can be computed form the measurements obtained in an image, once the foreshortening effects of surface curvature are accounted for. The second is the green sand mold casting of shuttle eyes for textile looms. Here, physical constraints inherent to the casting process translate into these constraints, it is necessary to interpret features of intensity as features of object shape. Both applications demonstrate that successful visual inspection requires the ability to interpret observed changes in intensity in the context of surface topography. The theoretical tools developed in this report provide a framework for this interpretation.

Noninvasive monitoring of tissue hemoglobin using UV-VIS diffuse reflectance spectroscopy: a pilot study.

We conducted a pilot study on 10 patients undergoing general surgery to test the feasibility of diffuse reflectance spectroscopy in the visible wavelength range as a noninvasive monitoring tool for blood loss during surgery. Ratios of raw diffuse reflectance at wavelength pairs were tested as a first-pass for estimating hemoglobin concentration. Ratios can be calculated easily and rapidly with limited post-processing, and so this can be considered a near real-time monitoring device. We found the best hemoglobin correlations were when ratios at isosbestic points of oxy- and deoxyhemoglobin were used, specifically 529/500 nm. Baseline subtraction improved correlations, specifically at 520/509 nm. These results demonstrate proof-of-concept for the ability of this noninvasive device to monitor hemoglobin concentration changes due to surgical blood loss. The 529/500 nm ratio also appears to account for variations in probe pressure, as determined from measurements on two volunteers.

Rapid ratiometric determination of hemoglobin concentration using UV-VIS diffuse reflectance at isosbestic wavelengths.

We developed a ratiometric method capable of estimating total hemoglobin concentration from optically measured diffuse reflectance spectra. The three isosbestic wavelength ratio pairs that best correlated to total hemoglobin concentration independent of saturation and scattering were 545/390, 452/390, and 529/390 nm. These wavelength pairs were selected using forward Monte Carlo simulations which were used to extract hemoglobin concentration from experimental phantom measurements. Linear regression coefficients from the simulated data were directly applied to the phantom data, by calibrating for instrument throughput using a single phantom. Phantoms with variable scattering and hemoglobin saturation were tested with two different instruments, and the average percent errors between the expected and ratiometrically-extracted hemoglobin concentration were as low as 6.3%. A correlation of r = 0.88 between hemoglobin concentration extracted using the 529/390 nm isosbestic ratio and a scalable inverse Monte Carlo model was achieved for in vivo dysplastic cervical measurements (hemoglobin concentrations have been shown to be diagnostic for the detection of cervical pre-cancer by our group). These results indicate that use of such a simple ratiometric method has the potential to be used in clinical applications where tissue hemoglobin concentrations need to be rapidly quantified in vivo.

Instrument independent diffuse reflectance spectroscopy.

Diffuse reflectance spectroscopy with a fiber optic probe is a powerful tool for quantitative tissue characterization and disease diagnosis. Significant systematic errors can arise in the measured reflectance spectra and thus in the derived tissue physiological and morphological parameters due to real-time instrument fluctuations. We demonstrate a novel fiber optic probe with real-time, self-calibration capability that can be used for UV-visible diffuse reflectance spectroscopy in biological tissue in clinical settings. The probe is tested in a number of synthetic liquid phantoms over a wide range of tissue optical properties for significant variations in source intensity fluctuations caused by instrument warm up and day-to-day drift. While the accuracy for extraction of absorber concentrations is comparable to that achieved with the traditional calibration (with a reflectance standard), the accuracy for extraction of reduced scattering coefficients is significantly improved with the self-calibration probe compared to traditional calibration. This technology could be used to achieve instrument-independent diffuse reflectance spectroscopy in vivo and obviate the need for instrument warm up and post∕premeasurement calibration, thus saving up to an hour of precious clinical time.