Estimation and inference for grasping and manipulation tasks using vision and kinesthetic sensors

This thesis presents a novel framework for state estimation in the context of robotic grasping and manipulation. The overall estimation approach is based on fusing various visual cues for manipulator tracking, namely appearance and feature-based, shape-based, and silhouette-based visual cues. Similarly, a framework is developed to fuse the above visual cues, but also kinesthetic cues such as force-torque and tactile measurements, for in-hand object pose estimation. The cues are extracted from multiple sensor modalities and are fused in a variety of Kalman filters.

A hybrid estimator is developed to estimate both a continuous state (robot and object states) and discrete states, called contact modes, which specify how each finger contacts a particular object surface. A static multiple model estimator is used to compute and maintain this mode probability. The thesis also develops an estimation framework for estimating model parameters associated with object grasping. Dual and joint state-parameter estimation is explored for parameter estimation of a grasped object's mass and center of mass. Experimental results demonstrate simultaneous object localization and center of mass estimation.

Dual-arm estimation is developed for two arm robotic manipulation tasks. Two types of filters are explored; the first is an augmented filter that contains both arms in the state vector while the second runs two filters in parallel, one for each arm. These two frameworks and their performance is compared in a dual-arm task of removing a wheel from a hub.

This thesis also presents a new method for action selection involving touch. This next best touch method selects an available action for interacting with an object that will gain the most information. The algorithm employs information theory to compute an information gain metric that is based on a probabilistic belief suitable for the task. An estimation framework is used to maintain this belief over time. Kinesthetic measurements such as contact and tactile measurements are used to update the state belief after every interactive action. Simulation and experimental results are demonstrated using next best touch for object localization, specifically a door handle on a door. The next best touch theory is extended for model parameter determination. Since many objects within a particular object category share the same rough shape, principle component analysis may be used to parametrize the object mesh models. These parameters can be estimated using the action selection technique that selects the touching action which best both localizes and estimates these parameters. Simulation results are then presented involving localizing and determining a parameter of a screwdriver.

Lastly, the next best touch theory is further extended to model classes. Instead of estimating parameters, object class determination is incorporated into the information gain metric calculation. The best touching action is selected in order to best discern between the possible model classes. Simulation results are presented to validate the theory.

Protein-protein recognition: The neonatal Fe receptor and immunoglobulin G

The neonatal Fe receptor (FeRn) binds the Fe portion of immunoglobulin G (IgG) at the acidic pH of endosomes or the gut and releases IgG at the alkaline pH of blood. FeRn is responsible for the maternofetal transfer of IgG and for rescuing endocytosed IgG from a default degradative pathway. We investigated how FeRn interacts with IgG by constructing a heterodimeric form of the Fe (hdFc) that contains one FeRn binding site. This molecule was used to characterize the interaction between one FeRn molecule and one Fe and to determine under what conditions FeRn forms a dimer. The hdFc binds one FeRn molecule at pH 6.0 with a K_d of 80 nM. In solution and with FeRn anchored to solid supports, the heterodimeric Fe does not induce a dimer of FeRn molecules. FcRnhdFc complex crystals were obtained and the complex structure was solved to 2.8 Å resolution. Analysis of this structure refined the understanding of the mechanism of the pH-dependent binding, shed light on the role played by carbohydrates in the Fe binding, and provided insights on how to design therapeutic IgG antibodies with longer serum half-lives. The FcRn-hdFc complex in the crystal did not contain the FeRn dimer. To characterize the tendency of FeRn to form a dimer in a membrane we analyzed the tendency of the hdFc to induce cross-phosphorylation of FeRn-tyrosine kinase chimeras. We also constructed FeRn-cyan and FeRn-yellow fluorescent proteins and have analyzed the tendency of these molecules to exhibit fluorescence resonance energy transfer. As of now, neither of these analyses have lead to conclusive results. In the process of acquiring the context to appreciate the structure of the FcRn-hdFc interface, we developed a study of 171 other nonobligate protein-protein interfaces that includes an original principal component analysis of the quantifiable aspects of these interfaces.