Estimation and inference for grasping and manipulation tasks using vision and kinesthetic sensors

This thesis presents a novel framework for state estimation in the context of robotic grasping and manipulation. The overall estimation approach is based on fusing various visual cues for manipulator tracking, namely appearance and feature-based, shape-based, and silhouette-based visual cues. Similarly, a framework is developed to fuse the above visual cues, but also kinesthetic cues such as force-torque and tactile measurements, for in-hand object pose estimation. The cues are extracted from multiple sensor modalities and are fused in a variety of Kalman filters.

A hybrid estimator is developed to estimate both a continuous state (robot and object states) and discrete states, called contact modes, which specify how each finger contacts a particular object surface. A static multiple model estimator is used to compute and maintain this mode probability. The thesis also develops an estimation framework for estimating model parameters associated with object grasping. Dual and joint state-parameter estimation is explored for parameter estimation of a grasped object's mass and center of mass. Experimental results demonstrate simultaneous object localization and center of mass estimation.

Dual-arm estimation is developed for two arm robotic manipulation tasks. Two types of filters are explored; the first is an augmented filter that contains both arms in the state vector while the second runs two filters in parallel, one for each arm. These two frameworks and their performance is compared in a dual-arm task of removing a wheel from a hub.

This thesis also presents a new method for action selection involving touch. This next best touch method selects an available action for interacting with an object that will gain the most information. The algorithm employs information theory to compute an information gain metric that is based on a probabilistic belief suitable for the task. An estimation framework is used to maintain this belief over time. Kinesthetic measurements such as contact and tactile measurements are used to update the state belief after every interactive action. Simulation and experimental results are demonstrated using next best touch for object localization, specifically a door handle on a door. The next best touch theory is extended for model parameter determination. Since many objects within a particular object category share the same rough shape, principle component analysis may be used to parametrize the object mesh models. These parameters can be estimated using the action selection technique that selects the touching action which best both localizes and estimates these parameters. Simulation results are then presented involving localizing and determining a parameter of a screwdriver.

Lastly, the next best touch theory is further extended to model classes. Instead of estimating parameters, object class determination is incorporated into the information gain metric calculation. The best touching action is selected in order to best discern between the possible model classes. Simulation results are presented to validate the theory.

Genetically engineered sensors of cell signaling

Measuring electrical activity in large numbers of cells with high spatial and temporal resolution is a fundamental problem for the study of neural development and information processing. To address this problem, we have constructed FlaSh: a novel, genetically-encoded probe that can be used to measure trans-membrane voltage in single cells. We fused a modified green fluorescent protein (GFP) into a voltage-sensitive potassium channel so that voltage dependent rearrangements in the potassium channel induce changes in the fluorescence of GFP. A voltage sensor encoded into DNA has the advantage that it may be introduced into an organism non-invasively and targeted to specific developmental stages, brain regions, cell types, and sub-cellular compartments.

We also describe modifications to FlaSh that shift its color, kinetics, and dynamic range. We used multiple green fluorescent proteins to produce variants of the FlaSh sensor that generate ratiometric signal output via fluorescence resonance energy transfer (FRET). Finally, we describe initial work toward FlaSh variants that are sensitive to G-protein coupled receptor (GPCR) activation. These sensors can be used to design functional assays for receptor activation in living cells.

Insights into the mechanism of human erythrocyte hexose transport : a transferred NOE study of glucose binding to GLUT1

This study examines binding of α- and β-D-glucose in their equilibrium mixture to the glucose transporter (GLUT1) in human erythrocyte membrane preparations by an ^1H NMR method, the transferred NOE (TRNOE). This method is shown theoretically and experimentally to be a sensitive probe of weak ligand-macromolecule interactions. The TRNOEs observed are shown to arise solely from glucose binding to GLUT1. Sites at both membrane faces contribute to the TRNOEs. Binding curves obtained are consistent with a homogeneous class of sugar sites, with an apparent K_D which varies (from ~30 mM to ~70 mM for both anomers) depending on the membrane preparation examined. Preparations with a higher proportion of the cytoplasmic membrane face exposed to bulk solution yield higher apparent KK_Ds. The glucose transport inhibitor cytochalasin B essentially eliminates the TRNOE. Nonlinearity was found in the dependence on sugar concentration of the apparent inhibition constant for cytochalasin B reversal of the TRNOE observed in the α anomer (and probably the β anomer); such nonlinearity implies the existence of ternary complexes of sugar, inhibitor and transporter. The inhibition results furthermore imply the presence of a class of relatively high-affinity (K_D < 2mM) sugar sites specific for the α anomer which do not contribute to NMR-observable binding. The presence of two classes of sugar-sensitive cytochalasin B sites is also indicated. These results are compared with predictions of the alternating conformer model of glucose transport. Variation of apparent K_D in the NMR-observable sites, the formation of ternary complexes and the presence of an anomer-specific site are shown to be inconsistent with this model. An alternate model is developed which reconciles these results with the known transport behavior of GLUT1. In this model, the transporter possesses (at minimum) three classes of sugar sites: (i) transport sites, which are alternately exposed to the cytoplasmic or the extracellular compartment, but never to both simultaneously, (ii) a class of sites (probably relatively low-affinity) which are confined to one compartment, and (iii) the high-affinity α anomer-specific sites, which are confined to the cytoplasmic compartment.

Reaction of glucose catalyzed by framework and extraframework tin sites in zeolite beta

The isomerization of glucose into fructose is a large-scale reaction for the production of high-fructose corn syrup, and is now being considered as an intermediate step in the possible route of biomass conversion into fuels and chemicals. Recently, it has been shown that a hydrophobic, large pore, silica molecular sieve having the zeolite beta structure and containing framework Sn⁴⁺ (Sn-Beta) is able to isomerize glucose into fructose in aqueous media. Here, I have investigated how this catalyst converts glucose to fructose and show that it is analogous to that achieved with metalloenzymes. Specifically, glucose partitions into the molecular sieve in the pyranose form, ring opens to the acyclic form in the presence of the Lewis acid center (framework Sn⁴⁺), isomerizes into the acyclic form of fructose and finally ring closes to yield the furanose product. Akin to the metalloenzyme, the isomerization step proceeds by intramolecular hydride transfer from C2 to C1. Extraframework tin oxides located within hydrophobic channels of the molecular sieve that exclude liquid water can also isomerize glucose to fructose in aqueous media, but do so through a base-catalyzed proton abstraction mechanism. Extraframework tin oxide particles located at the external surface of the molecular sieve crystals or on amorphous silica supports are not active in aqueous media but are able to perform the isomerization in methanol by a base-catalyzed proton abstraction mechanism. Post-synthetic exchange of Na⁺ with Sn-Beta alters the glucose reaction pathway from the 1,2 intramolecular hydrogen shift (isomerization) to produce fructose towards the 1,2 intramolecular carbon shift (epimerization) that forms mannose. Na⁺ remains exchanged onto silanol groups during reaction in methanol solvent, leading to a near complete shift in selectivity towards glucose epimerization to mannose. In contrast, decationation occurs during reaction in aqueous solutions and gradually increases the reaction selectivity to isomerization at the expense of epimerization. Decationation and concomitant changes in selectivity can be eliminated by addition of NaCl to the aqueous reaction solution. Thus, framework tin sites with a proximal silanol group are the active sites for the 1, 2 intramolecular hydride shift in the isomerization of glucose to fructose, while these sites with Na-exchanged silanol group are the active sites for the 1, 2 intramolecular carbon shift in epimerization of glucose to mannose.

MEMS for Diabetic Retinopathy

As the worldwide prevalence of diabetes mellitus continues to increase, diabetic retinopathy remains the leading cause of visual impairment and blindness in many developed countries. Between 32 to 40 percent of about 246 million people with diabetes develop diabetic retinopathy. Approximately 4.1 million American adults 40 years and older are affected by diabetic retinopathy. This glucose-induced microvascular disease progressively damages the tiny blood vessels that nourish the retina, the light-sensitive tissue at the back of the eye, leading to retinal ischemia (i.e., inadequate blood flow), retinal hypoxia (i.e., oxygen deprivation), and retinal nerve cell degeneration or death. It is a most serious sight-threatening complication of diabetes, resulting in significant irreversible vision loss, and even total blindness.

Unfortunately, although current treatments of diabetic retinopathy (i.e., laser therapy, vitrectomy surgery and anti-VEGF therapy) can reduce vision loss, they only slow down but cannot stop the degradation of the retina. Patients require repeated treatment to protect their sight. The current treatments also have significant drawbacks. Laser therapy is focused on preserving the macula, the area of the retina that is responsible for sharp, clear, central vision, by sacrificing the peripheral retina since there is only limited oxygen supply. Therefore, laser therapy results in a constricted peripheral visual field, reduced color vision, delayed dark adaptation, and weakened night vision. Vitrectomy surgery increases the risk of neovascular glaucoma, another devastating ocular disease, characterized by the proliferation of fibrovascular tissue in the anterior chamber angle. Anti-VEGF agents have potential adverse effects, and currently there is insufficient evidence to recommend their routine use.

In this work, for the first time, a paradigm shift in the treatment of diabetic retinopathy is proposed: providing localized, supplemental oxygen to the ischemic tissue via an implantable MEMS device. The retinal architecture (e.g., thickness, cell densities, layered structure, etc.) of the rabbit eye exposed to ischemic hypoxic injuries was well preserved after targeted oxygen delivery to the hypoxic tissue, showing that the use of an external source of oxygen could improve the retinal oxygenation and prevent the progression of the ischemic cascade.

The proposed MEMS device transports oxygen from an oxygen-rich space to the oxygen-deficient vitreous, the gel-like fluid that fills the inside of the eye, and then to the ischemic retina. This oxygen transport process is purely passive and completely driven by the gradient of oxygen partial pressure (pO₂). Two types of devices were designed. For the first type, the oxygen-rich space is underneath the conjunctiva, a membrane covering the sclera (white part of the eye), beneath the eyelids and highly permeable to oxygen in the atmosphere when the eye is open. Therefore, sub-conjunctival pO₂ is very high during the daytime. For the second type, the oxygen-rich space is inside the device since pure oxygen is needle-injected into the device on a regular basis.

To prevent too fast or too slow permeation of oxygen through the device that is made of parylene and silicone (two widely used biocompatible polymers in medical devices), the material properties of the hybrid parylene/silicone were investigated, including mechanical behaviors, permeation rates, and adhesive forces. Then the thicknesses of parylene and silicone became important design parameters that were fine-tuned to reach the optimal oxygen permeation rate.

The passive MEMS oxygen transporter devices were designed, built, and tested in both bench-top artificial eye models and in-vitro porcine cadaver eyes. The 3D unsteady saccade-induced laminar flow of water inside the eye model was modeled by computational fluid dynamics to study the convective transport of oxygen inside the eye induced by saccade (rapid eye movement). The saccade-enhanced transport effect was also demonstrated experimentally. Acute in-vivo animal experiments were performed in rabbits and dogs to verify the surgical procedure and the device functionality. Various hypotheses were confirmed both experimentally and computationally, suggesting that both the two types of devices are very promising to cure diabetic retinopathy. The chronic implantation of devices in ischemic dog eyes is still underway.

The proposed MEMS oxygen transporter devices can be also applied to treat other ocular and systemic diseases accompanied by retinal ischemia, such as central retinal artery occlusion, carotid artery disease, and some form of glaucoma.

Optomechanical Inertial Sensors and Feedback Cooling

The optomechanical interaction is an extremely powerful tool with which to measure mechanical motion. The displacement resolution of chip-scale optomechanical systems has been measured on the order of 1⁄10th of a proton radius. So strong is this optomechanical interaction that it has recently been used to remove almost all thermal noise from a mechanical resonator and observe its quantum ground-state of motion starting from cryogenic temperatures.

In this work, chapter 1 describes the basic physics of the canonical optomechanical system, optical measurement techniques, and how the optomechanical interaction affects the coupled mechanical resonator. In chapter 2, we describe our techniques for realizing this canonical optomechanical system in a chip-scale form factor.

In chapter 3, we describe an experiment where we used radiation pressure feedback to cool a mesoscopic mechanical resonator near its quantum ground-state from room-temperature. We cooled the resonator from a room temperature phonon occupation of <n> = 6.5 million to an occupation of <n> = 66, which means the resonator is in its ground state approximately 2% of the time, while being coupled to a room-temperature thermal environment. At the time of this work, this is the closest a mesoscopic mechanical resonator has been to its ground-state of motion at room temperature, and this work begins to open the door to room-temperature quantum control of mechanical objects.

Chapter 4 begins with the realization that the displacement resolutions achieved by optomechanical systems can surpass those of conventional MEMS sensors by an order of magnitude or more. This provides the motivation to develop and calibrate an optomechanical accelerometer with a resolution of approximately 10 micro-g/rt-Hz over a bandwidth of approximately 30 kHz. In chapter 5, we improve upon the performance and practicality of this sensor by greatly increasing the test mass size, investigating and reducing low-frequency noise, and incorporating more robust optical coupling techniques and capacitive wavelength tuning. Finally, in chapter 6 we present our progress towards developing another optomechanical inertial sensor - a gyroscope.

A nonlinear systems model for the control mechanism of free fatty acid-glucose metabolosm in normal humans

A mathematical model is proposed in this thesis for the control mechanism of free fatty acid-glucose metabolism in healthy individuals under resting conditions. The objective is to explain in a consistent manner some clinical laboratory observations such as glucose, insulin and free fatty acid responses to intravenous injection of glucose, insulin, etc. Responses up to only about two hours from the beginning of infusion are considered. The model is an extension of the one for glucose homeostasis proposed by Charette, Kadish and Sridhar (Modeling and Control Aspects of Glucose Homeostasis. Mathematical Biosciences, 1969). It is based upon a systems approach and agrees with the current theories of glucose and free fatty acid metabolism. The description is in terms of ordinary differential equations. Validation of the model is based on clinical laboratory data available at the present time. Finally procedures are suggested for systematically identifying the parameters associated with the free fatty acid portion of the model.