Estimation and inference for grasping and manipulation tasks using vision and kinesthetic sensors

This thesis presents a novel framework for state estimation in the context of robotic grasping and manipulation. The overall estimation approach is based on fusing various visual cues for manipulator tracking, namely appearance and feature-based, shape-based, and silhouette-based visual cues. Similarly, a framework is developed to fuse the above visual cues, but also kinesthetic cues such as force-torque and tactile measurements, for in-hand object pose estimation. The cues are extracted from multiple sensor modalities and are fused in a variety of Kalman filters.

A hybrid estimator is developed to estimate both a continuous state (robot and object states) and discrete states, called contact modes, which specify how each finger contacts a particular object surface. A static multiple model estimator is used to compute and maintain this mode probability. The thesis also develops an estimation framework for estimating model parameters associated with object grasping. Dual and joint state-parameter estimation is explored for parameter estimation of a grasped object's mass and center of mass. Experimental results demonstrate simultaneous object localization and center of mass estimation.

Dual-arm estimation is developed for two arm robotic manipulation tasks. Two types of filters are explored; the first is an augmented filter that contains both arms in the state vector while the second runs two filters in parallel, one for each arm. These two frameworks and their performance is compared in a dual-arm task of removing a wheel from a hub.

This thesis also presents a new method for action selection involving touch. This next best touch method selects an available action for interacting with an object that will gain the most information. The algorithm employs information theory to compute an information gain metric that is based on a probabilistic belief suitable for the task. An estimation framework is used to maintain this belief over time. Kinesthetic measurements such as contact and tactile measurements are used to update the state belief after every interactive action. Simulation and experimental results are demonstrated using next best touch for object localization, specifically a door handle on a door. The next best touch theory is extended for model parameter determination. Since many objects within a particular object category share the same rough shape, principle component analysis may be used to parametrize the object mesh models. These parameters can be estimated using the action selection technique that selects the touching action which best both localizes and estimates these parameters. Simulation results are then presented involving localizing and determining a parameter of a screwdriver.

Lastly, the next best touch theory is further extended to model classes. Instead of estimating parameters, object class determination is incorporated into the information gain metric calculation. The best touching action is selected in order to best discern between the possible model classes. Simulation results are presented to validate the theory.

Genetically engineered sensors of cell signaling

Measuring electrical activity in large numbers of cells with high spatial and temporal resolution is a fundamental problem for the study of neural development and information processing. To address this problem, we have constructed FlaSh: a novel, genetically-encoded probe that can be used to measure trans-membrane voltage in single cells. We fused a modified green fluorescent protein (GFP) into a voltage-sensitive potassium channel so that voltage dependent rearrangements in the potassium channel induce changes in the fluorescence of GFP. A voltage sensor encoded into DNA has the advantage that it may be introduced into an organism non-invasively and targeted to specific developmental stages, brain regions, cell types, and sub-cellular compartments.

We also describe modifications to FlaSh that shift its color, kinetics, and dynamic range. We used multiple green fluorescent proteins to produce variants of the FlaSh sensor that generate ratiometric signal output via fluorescence resonance energy transfer (FRET). Finally, we describe initial work toward FlaSh variants that are sensitive to G-protein coupled receptor (GPCR) activation. These sensors can be used to design functional assays for receptor activation in living cells.

1. Astronomical photometry at wavelengths of 8.5, 10.5, and 11.6 µm. 2. Infrared emission from asteroids at wavelengths of 8.5, 10.5, and 11.6 µm

The purpose of this thesis is to present new observations of thermal-infrared radiation from asteroids. Stellar photometry was performed to provide standards for comparison with the asteroid data. The details of the photometry and the data reduction are discussed in Part 1. A system of standard stars is derived for wavelengths of 8.5, 10.5 and 11.6 µm and a new calibration is adopted. Sources of error are evaluated and comparisons are made with the data of other observers.

The observations and analysis of the thermal-emission observations of asteroids are presented in Part 2. Thermal-emission lightcurve and phase effect data are considered. Special color diagrams are introduced to display the observational data. These diagrams are free of any model-dependent assumptions and show that asteroids differ in their surface properties.

On the basis of photometric models, (4) Vesta is thought to have a bolometric Bond albedo of about 0.1, an emissivity greater than 0.7 and a true radius that is close to the model value of 300^(+50)_(-30)km. Model albedos and model radii are given for asteroids 1, 2, 4, 5, 6, 7, 15, 19, 20, 27, 39, 44, 68, 80, 324 and 674. The asteroid (324) Bamberga is extremely dark with a model (~bolometric Bond) albedo in the 0.01 - 0.02 range, which is thought to be the lowest albedo yet measured for any solar-system body. The crucial question about such low-albedo asteroids is their number and the distribution of their orbits.

Computational modeling and psychophysics in low- and mid-level vision

This thesis addresses a series of topics related to the question of how people find the foreground objects from complex scenes. With both computer vision modeling, as well as psychophysical analyses, we explore the computational principles for low- and mid-level vision.

We first explore the computational methods of generating saliency maps from images and image sequences. We propose an extremely fast algorithm called Image Signature that detects the locations in the image that attract human eye gazes. With a series of experimental validations based on human behavioral data collected from various psychophysical experiments, we conclude that the Image Signature and its spatial-temporal extension, the Phase Discrepancy, are among the most accurate algorithms for saliency detection under various conditions.

In the second part, we bridge the gap between fixation prediction and salient object segmentation with two efforts. First, we propose a new dataset that contains both fixation and object segmentation information. By simultaneously presenting the two types of human data in the same dataset, we are able to analyze their intrinsic connection, as well as understanding the drawbacks of today’s “standard” but inappropriately labeled salient object segmentation dataset. Second, we also propose an algorithm of salient object segmentation. Based on our novel discoveries on the connections of fixation data and salient object segmentation data, our model significantly outperforms all existing models on all 3 datasets with large margins.

In the third part of the thesis, we discuss topics around the human factors of boundary analysis. Closely related to salient object segmentation, boundary analysis focuses on delimiting the local contours of an object. We identify the potential pitfalls of algorithm evaluation for the problem of boundary detection. Our analysis indicates that today’s popular boundary detection datasets contain significant level of noise, which may severely influence the benchmarking results. To give further insights on the labeling process, we propose a model to characterize the principles of the human factors during the labeling process.

The analyses reported in this thesis offer new perspectives to a series of interrelating issues in low- and mid-level vision. It gives warning signs to some of today’s “standard” procedures, while proposing new directions to encourage future research.

I. Spectroscopic observation of discrete sites in simple liquids. II. Infrared intensity perturbations of hydrogen halide fundamentals in liquid xenon. III. Rotation of HCl in solid rare gases.

Part I

The spectrum of dissolved mercury atoms in simple liquids has been shown to be capable of revealing information concerning local structures in these liquids.

Part II

Infrared intensity perturbations in simple solutions have been shown to involve more detailed interaction than just dielectric polarization. No correlation has been found between frequency shifts and intensity enhancements.

Part III

Evidence for perturbed rotation of HCl in rare gas matrices has been found. The magnitude of the barrier to rotation is concluded to be of order of 30 cm^(-1).

On Ultrafast Time-Domain TeraHertz Spectroscopy in the Condensed Phase: Linear Spectroscopic Measurements of Hydrogen-Bond Dynamics of Astrochemical Ice Analogs and Nonlinear TeraHertz Kerr Effect Measurements of Vibrational Quantum Beats

Much of the chemistry that affects life on planet Earth occurs in the condensed phase. The TeraHertz (THz) or far-infrared (far-IR) region of the electromagnetic spectrum (from 0.1 THz to 10 THz, 3 cm^-1 to 300 cm^-1, or 3000 μm to 30 μm) has been shown to provide unique possibilities in the study of condensed-phase processes. The goal of this work is to expand the possibilities available in the THz region and undertake new investigations of fundamental interest to chemistry. Since we are fundamentally interested in condensed-phase processes, this thesis focuses on two areas where THz spectroscopy can provide new understanding: astrochemistry and solvation science. To advance these fields, we had to develop new instrumentation that would enable the experiments necessary to answer new questions in either astrochemistry or solvation science. We first developed a new experimental setup capable of studying astrochemical ice analogs in both the TeraHertz (THz), or far-Infrared (far-IR), region (0.3 - 7.5 THz; 10 - 250 cm^-1) and the mid-IR (400 - 4000 cm^-1). The importance of astrochemical ices lies in their key role in the formation of complex organic molecules, such as amino acids and sugars in space. Thus, the instruments are capable of performing variety of spectroscopic studies that can provide especially relevant laboratory data to support astronomical observations from telescopes such as the Herschel Space Telescope, the Stratospheric Observatory for Infrared Astronomy (SOFIA), and the Atacama Large Millimeter Array (ALMA). The experimental apparatus uses a THz time-domain spectrometer, with a 1750/875 nm plasma source and a GaP detector crystal, to cover the bandwidth mentioned above with ~10 GHz (~0.3 cm^-1) resolution.

Using the above instrumentation, experimental spectra of astrochemical ice analogs of water and carbon dioxide in pure, mixed, and layered ices were collected at different temperatures under high vacuum conditions with the goal of investigating the structure of the ice. We tentatively observe a new feature in both amorphous solid water and crystalline water at 33 cm^-1 (1 THz). In addition, our studies of mixed and layered ices show how it is possible to identify the location of carbon dioxide as it segregates within the ice by observing its effect on the THz spectrum of water ice. The THz spectra of mixed and layered ices are further analyzed by fitting their spectra features to those of pure amorphous solid water and crystalline water ice to quantify the effects of temperature changes on structure. From the results of this work, it appears that THz spectroscopy is potentially well suited to study thermal transformations within the ice.

To advance the study of liquids with THz spectroscopy, we developed a new ultrafast nonlinear THz spectroscopic technique: heterodyne-detected, ultrafast THz Kerr effect (TKE) spectroscopy. We implemented a heterodyne-detection scheme into a TKE spectrometer that uses a stilbazoiumbased THz emitter, 4-N,N-dimethylamino-4-N-methyl-stilbazolium 2,4,6-trimethylbenzenesulfonate (DSTMS), and high numerical aperture optics which generates THz electric field in excess of 300 kV/cm, in the sample. This allows us to report the first measurement of quantum beats at terahertz (THz) frequencies that result from vibrational coherences initiated by the nonlinear, dipolar interaction of a broadband, high-energy, (sub)picosecond THz pulse with the sample. Our instrument improves on both the frequency coverage, and sensitivity previously reported; it also ensures a backgroundless measurement of the THz Kerr effect in pure liquids. For liquid diiodomethane, we observe a quantum beat at 3.66 THz (122 cm^-1), in exact agreement with the fundamental transition frequency of the υ4 vibration of the molecule. This result provides new insight into dipolar vs. Raman selection rules at terahertz frequencies.

To conclude we discuss future directions for the nonlinear THz spectroscopy in the Blake lab. We report the first results from an experiment using a plasma-based THz source for nonlinear spectroscopy that has the potential to enable nonlinear THz spectra with a sub-100 fs temporal resolution, and how the optics involved in the plasma mechanism can enable THz pulse shaping. Finally, we discuss how a single-shot THz detection scheme could improve the acquisition of THz data and how such a scheme could be implemented in the Blake lab. The instruments developed herein will hopefully remain a part of the groups core competencies and serve as building blocks for the next generation of THz instrumentation that pushes the frontiers of both chemistry and the scientific enterprise as a whole.

Deep near-infrared spectroscopy of high-redshift galaxies: the physical growth of passive systems

The assembly history of massive galaxies is one of the most important aspects of galaxy formation and evolution. Although we have a broad idea of what physical processes govern the early phases of galaxy evolution, there are still many open questions. In this thesis I demonstrate the crucial role that spectroscopy can play in a physical understanding of galaxy evolution. I present deep near-infrared spectroscopy for a sample of high-redshift galaxies, from which I derive important physical properties and their evolution with cosmic time. I take advantage of the recent arrival of efficient near-infrared detectors to target the rest-frame optical spectra of z > 1 galaxies, from which many physical quantities can be derived. After illustrating the applications of near-infrared deep spectroscopy with a study of star-forming galaxies, I focus on the evolution of massive quiescent systems.

Most of this thesis is based on two samples collected at the W. M. Keck Observatory that represent a significant step forward in the spectroscopic study of z > 1 quiescent galaxies. All previous spectroscopic samples at this redshift were either limited to a few objects, or much shallower in terms of depth. Our first sample is composed of 56 quiescent galaxies at 1 < z < 1.6 collected using the upgraded red arm of the Low Resolution Imaging Spectrometer (LRIS). The second consists of 24 deep spectra of 1.5 < z < 2.5 quiescent objects observed with the Multi-Object Spectrometer For Infra-Red Exploration (MOSFIRE). Together, these spectra span the critical epoch 1 < z < 2.5, where most of the red sequence is formed, and where the sizes of quiescent systems are observed to increase significantly.

We measure stellar velocity dispersions and dynamical masses for the largest number of z > 1 quiescent galaxies to date. By assuming that the velocity dispersion of a massive galaxy does not change throughout its lifetime, as suggested by theoretical studies, we match galaxies in the local universe with their high-redshift progenitors. This allows us to derive the physical growth in mass and size experienced by individual systems, which represents a substantial advance over photometric inferences based on the overall galaxy population. We find a significant physical growth among quiescent galaxies over 0 < z < 2.5 and, by comparing the slope of growth in the mass-size plane dlogR_e/dlogM_∗ with the results of numerical simulations, we can constrain the physical process responsible for the evolution. Our results show that the slope of growth becomes steeper at higher redshifts, yet is broadly consistent with minor mergers being the main process by which individual objects evolve in mass and size.

By fitting stellar population models to the observed spectroscopy and photometry we derive reliable ages and other stellar population properties. We show that the addition of the spectroscopic data helps break the degeneracy between age and dust extinction, and yields significantly more robust results compared to fitting models to the photometry alone. We detect a clear relation between size and age, where larger galaxies are younger. Therefore, over time the average size of the quiescent population will increase because of the contribution of large galaxies recently arrived to the red sequence. This effect, called progenitor bias, is different from the physical size growth discussed above, but represents another contribution to the observed difference between the typical sizes of low- and high-redshift quiescent galaxies. By reconstructing the evolution of the red sequence starting at z ∼ 1.25 and using our stellar population histories to infer the past behavior to z ∼ 2, we demonstrate that progenitor bias accounts for only half of the observed growth of the population. The remaining size evolution must be due to physical growth of individual systems, in agreement with our dynamical study.

Finally, we use the stellar population properties to explore the earliest periods which led to the formation of massive quiescent galaxies. We find tentative evidence for two channels of star formation quenching, which suggests the existence of two independent physical mechanisms. We also detect a mass downsizing, where more massive galaxies form at higher redshift, and then evolve passively. By analyzing in depth the star formation history of the brightest object at z > 2 in our sample, we are able to put constraints on the quenching timescale and on the properties of its progenitor.

A consistent picture emerges from our analyses: massive galaxies form at very early epochs, are quenched on short timescales, and then evolve passively. The evolution is passive in the sense that no new stars are formed, but significant mass and size growth is achieved by accreting smaller, gas-poor systems. At the same time the population of quiescent galaxies grows in number due to the quenching of larger star-forming galaxies. This picture is in agreement with other observational studies, such as measurements of the merger rate and analyses of galaxy evolution at fixed number density.

Optomechanical Inertial Sensors and Feedback Cooling

The optomechanical interaction is an extremely powerful tool with which to measure mechanical motion. The displacement resolution of chip-scale optomechanical systems has been measured on the order of 1⁄10th of a proton radius. So strong is this optomechanical interaction that it has recently been used to remove almost all thermal noise from a mechanical resonator and observe its quantum ground-state of motion starting from cryogenic temperatures.

In this work, chapter 1 describes the basic physics of the canonical optomechanical system, optical measurement techniques, and how the optomechanical interaction affects the coupled mechanical resonator. In chapter 2, we describe our techniques for realizing this canonical optomechanical system in a chip-scale form factor.

In chapter 3, we describe an experiment where we used radiation pressure feedback to cool a mesoscopic mechanical resonator near its quantum ground-state from room-temperature. We cooled the resonator from a room temperature phonon occupation of <n> = 6.5 million to an occupation of <n> = 66, which means the resonator is in its ground state approximately 2% of the time, while being coupled to a room-temperature thermal environment. At the time of this work, this is the closest a mesoscopic mechanical resonator has been to its ground-state of motion at room temperature, and this work begins to open the door to room-temperature quantum control of mechanical objects.

Chapter 4 begins with the realization that the displacement resolutions achieved by optomechanical systems can surpass those of conventional MEMS sensors by an order of magnitude or more. This provides the motivation to develop and calibrate an optomechanical accelerometer with a resolution of approximately 10 micro-g/rt-Hz over a bandwidth of approximately 30 kHz. In chapter 5, we improve upon the performance and practicality of this sensor by greatly increasing the test mass size, investigating and reducing low-frequency noise, and incorporating more robust optical coupling techniques and capacitive wavelength tuning. Finally, in chapter 6 we present our progress towards developing another optomechanical inertial sensor - a gyroscope.