Estimation and inference for grasping and manipulation tasks using vision and kinesthetic sensors

This thesis presents a novel framework for state estimation in the context of robotic grasping and manipulation. The overall estimation approach is based on fusing various visual cues for manipulator tracking, namely appearance and feature-based, shape-based, and silhouette-based visual cues. Similarly, a framework is developed to fuse the above visual cues, but also kinesthetic cues such as force-torque and tactile measurements, for in-hand object pose estimation. The cues are extracted from multiple sensor modalities and are fused in a variety of Kalman filters.

A hybrid estimator is developed to estimate both a continuous state (robot and object states) and discrete states, called contact modes, which specify how each finger contacts a particular object surface. A static multiple model estimator is used to compute and maintain this mode probability. The thesis also develops an estimation framework for estimating model parameters associated with object grasping. Dual and joint state-parameter estimation is explored for parameter estimation of a grasped object's mass and center of mass. Experimental results demonstrate simultaneous object localization and center of mass estimation.

Dual-arm estimation is developed for two arm robotic manipulation tasks. Two types of filters are explored; the first is an augmented filter that contains both arms in the state vector while the second runs two filters in parallel, one for each arm. These two frameworks and their performance is compared in a dual-arm task of removing a wheel from a hub.

This thesis also presents a new method for action selection involving touch. This next best touch method selects an available action for interacting with an object that will gain the most information. The algorithm employs information theory to compute an information gain metric that is based on a probabilistic belief suitable for the task. An estimation framework is used to maintain this belief over time. Kinesthetic measurements such as contact and tactile measurements are used to update the state belief after every interactive action. Simulation and experimental results are demonstrated using next best touch for object localization, specifically a door handle on a door. The next best touch theory is extended for model parameter determination. Since many objects within a particular object category share the same rough shape, principle component analysis may be used to parametrize the object mesh models. These parameters can be estimated using the action selection technique that selects the touching action which best both localizes and estimates these parameters. Simulation results are then presented involving localizing and determining a parameter of a screwdriver.

Lastly, the next best touch theory is further extended to model classes. Instead of estimating parameters, object class determination is incorporated into the information gain metric calculation. The best touching action is selected in order to best discern between the possible model classes. Simulation results are presented to validate the theory.

Design, specification, and synthesis of aircraft electric power systems control logic

Cyber-physical systems integrate computation, networking, and physical processes. Substantial research challenges exist in the design and verification of such large-scale, distributed sensing, ac- tuation, and control systems. Rapidly improving technology and recent advances in control theory, networked systems, and computer science give us the opportunity to drastically improve our approach to integrated flow of information and cooperative behavior. Current systems rely on text-based spec- ifications and manual design. Using new technology advances, we can create easier, more efficient, and cheaper ways of developing these control systems. This thesis will focus on design considera- tions for system topologies, ways to formally and automatically specify requirements, and methods to synthesize reactive control protocols, all within the context of an aircraft electric power system as a representative application area.

This thesis consists of three complementary parts: synthesis, specification, and design. The first section focuses on the synthesis of central and distributed reactive controllers for an aircraft elec- tric power system. This approach incorporates methodologies from computer science and control. The resulting controllers are correct by construction with respect to system requirements, which are formulated using the specification language of linear temporal logic (LTL). The second section addresses how to formally specify requirements and introduces a domain-specific language for electric power systems. A software tool automatically converts high-level requirements into LTL and synthesizes a controller.

The final sections focus on design space exploration. A design methodology is proposed that uses mixed-integer linear programming to obtain candidate topologies, which are then used to synthesize controllers. The discrete-time control logic is then verified in real-time by two methods: hardware and simulation. Finally, the problem of partial observability and dynamic state estimation is ex- plored. Given a set placement of sensors on an electric power system, measurements from these sensors can be used in conjunction with control logic to infer the state of the system.

Neural and computational representations of decision variables

These studies explore how, where, and when representations of variables critical to decision-making are represented in the brain. In order to produce a decision, humans must first determine the relevant stimuli, actions, and possible outcomes before applying an algorithm that will select an action from those available. When choosing amongst alternative stimuli, the framework of value-based decision-making proposes that values are assigned to the stimuli and that these values are then compared in an abstract “value space” in order to produce a decision. Despite much progress, in particular regarding the pinpointing of ventromedial prefrontal cortex (vmPFC) as a region that encodes the value, many basic questions remain. In Chapter 2, I show that distributed BOLD signaling in vmPFC represents the value of stimuli under consideration in a manner that is independent of the type of stimulus it is. Thus the open question of whether value is represented in abstraction, a key tenet of value-based decision-making, is confirmed. However, I also show that stimulus-dependent value representations are also present in the brain during decision-making and suggest a potential neural pathway for stimulus-to-value transformations that integrates these two results.

More broadly speaking, there is both neural and behavioral evidence that two distinct control systems are at work during action selection. These two systems compose the “goal-directed system”, which selects actions based on an internal model of the environment, and the “habitual” system, which generates responses based on antecedent stimuli only. Computational characterizations of these two systems imply that they have different informational requirements in terms of input stimuli, actions, and possible outcomes. Associative learning theory predicts that the habitual system should utilize stimulus and action information only, while goal-directed behavior requires that outcomes as well as stimuli and actions be processed. In Chapter 3, I test whether areas of the brain hypothesized to be involved in habitual versus goal-directed control represent the corresponding theorized variables.

The question of whether one or both of these neural systems drives Pavlovian conditioning is less well-studied. Chapter 4 describes an experiment in which subjects were scanned while engaged in a Pavlovian task with a simple non-trivial structure. After comparing a variety of model-based and model-free learning algorithms (thought to underpin goal-directed and habitual decision-making, respectively), it was found that subjects’ reaction times were better explained by a model-based system. In addition, neural signaling of precision, a variable based on a representation of a world model, was found in the amygdala. These data indicate that the influence of model-based representations of the environment can extend even to the most basic learning processes.

Knowledge of the state of hidden variables in an environment is required for optimal inference regarding the abstract decision structure of a given environment and therefore can be crucial to decision-making in a wide range of situations. Inferring the state of an abstract variable requires the generation and manipulation of an internal representation of beliefs over the values of the hidden variable. In Chapter 5, I describe behavioral and neural results regarding the learning strategies employed by human subjects in a hierarchical state-estimation task. In particular, a comprehensive model fit and comparison process pointed to the use of "belief thresholding". This implies that subjects tended to eliminate low-probability hypotheses regarding the state of the environment from their internal model and ceased to update the corresponding variables. Thus, in concert with incremental Bayesian learning, humans explicitly manipulate their internal model of the generative process during hierarchical inference consistent with a serial hypothesis testing strategy.

Neuronal mechanism of state control in Drosophila melanogaster

The changes in internal states, such as fear, hunger and sleep affect behavioral responses in animals. In most of the cases, these state-dependent influences are “pleiotropic”: one state affects multiple sensory modalities and behaviors; “scalable”: the strengths and choices of such modulations differ depending on the imminence of demands; and “persistent”: once the state is switched on the effects last even after the internal demands are off. These prominent features of state-control enable animals to adjust their behavioral responses depending on their internal demands. Here, we studied the neuronal mechanisms of state-controls by investigating energy-deprived state (hunger state) and social-deprived state of fruit flies, Drosophila melanogaster, as prototypic models. To approach these questions, we developed two novel methods: a genetically based method to map sites of neuromodulation in the brain and optogenetic tools in Drosophila.

These methods, and genetic perturbations, reveal that the effect of hunger to alter behavioral sensitivity to gustatory cues is mediate by two distinct neuromodulatory pathways. The neuropeptide F (NPF) – dopamine (DA) pathway increases sugar sensitivity under mild starvation, while the adipokinetic hormone (AKH)- short neuropeptide F (sNPF) pathway decreases bitter sensitivity under severe starvation. These two pathways are recruited under different levels of energy demands without any cross interaction. Effects of both of the pathways are mediated by modulation of the gustatory sensory neurons, which reinforce the concept that sensory neurons constitute an important locus for state-dependent control of behaviors. Our data suggests that multiple independent neuromodulatory pathways are underlying pleiotropic and scalable effects of the hunger state.

In addition, using optogenetic tool, we show that the neural control of male courtship song can be separated into probabilistic/biasing, and deterministic/command-like components. The former, but not the latter, neurons are subject to functional modulation by social experience, supporting the idea that they constitute a locus of state-dependent influence. Interestingly, moreover, brief activation of the former, but not the latter, neurons trigger persistent behavioral response for more than 10 min. Altogether, these findings and new tools described in this dissertation offer new entry points for future researchers to understand the neuronal mechanism of state control.

Functional evaluation of noncovalent interactions in neuroreceptors and progress toward the expansion of unnatural amino acid methodology

This dissertation primarily describes chemical-scale studies of G protein-coupled receptors and Cys-loop ligand-gated ion channels to better understand ligand binding interactions and the mechanism of channel activation using recently published crystal structures as a guide. These studies employ the use of unnatural amino acid mutagenesis and electrophysiology to measure subtle changes in receptor function.

In chapter 2, the role of a conserved aromatic microdomain predicted in the D3 dopamine receptor is probed in the closely related D2 and D4 dopamine receptors. This domain was found to act as a structural unit near the ligand binding site that is important for receptor function. The domain consists of several functionally important noncovalent interactions including hydrogen bond, aromatic-aromatic, and sulfur-π interactions that show strong couplings by mutant cycle analysis. We also assign an alternate interpretation for the linear fluorination plot observed at W6.48, a residue previously thought to participate in a cation-π interaction with dopamine.

Chapter 3 outlines attempts to incorporate chemically synthesized and in vitro acylated unnatural amino acids into mammalian cells. While our attempts were not successful, method optimizations and data for nonsense suppression with an in vivo acylated tRNA are included. This chapter is aimed to aid future researchers attempting unnatural amino acid mutagenesis in mammalian cells.

Chapter 4 identifies a cation-π interaction between glutamate and a tyrosine residue on loop C in the GluClβ receptor. Using the recently published crystal structure of the homologous GluClα receptor, other ligand-binding and protein-protein interactions are probed to determine the similarity between this invertebrate receptor and other more distantly related vertebrate Cys-loop receptors. We find that many of the interactions previously observed are conserved in the GluCl receptors, however care must be taken when extrapolating structural data.

Chapter 5 examines inherent properties of the GluClα receptor that are responsible for the observed glutamate insensitivity of the receptor. Chimera synthesis and mutagenesis reveal the C-terminal portion of the M4 helix and the C-terminus as contributing to formation of the decoupled state, where ligand binding is incapable of triggering channel gating. Receptor mutagenesis was unable to identify single residue mismatches or impaired protein-protein interactions within this domain. We conclude that M4 helix structure and/or membrane dynamics are likely the cause of ligand insensitivity in this receptor and that the M4 helix has an role important in the activation process.

Advanced density functional theory methods for materials science

In this work we chiefly deal with two broad classes of problems in computational materials science, determining the doping mechanism in a semiconductor and developing an extreme condition equation of state. While solving certain aspects of these questions is well-trodden ground, both require extending the reach of existing methods to fully answer them. Here we choose to build upon the framework of density functional theory (DFT) which provides an efficient means to investigate a system from a quantum mechanics description.

Zinc Phosphide (Zn₃P₂) could be the basis for cheap and highly efficient solar cells. Its use in this regard is limited by the difficulty in n-type doping the material. In an effort to understand the mechanism behind this, the energetics and electronic structure of intrinsic point defects in zinc phosphide are studied using generalized Kohn-Sham theory and utilizing the Heyd, Scuseria, and Ernzerhof (HSE) hybrid functional for exchange and correlation. Novel 'perturbation extrapolation' is utilized to extend the use of the computationally expensive HSE functional to this large-scale defect system. According to calculations, the formation energy of charged phosphorus interstitial defects are very low in n-type Zn₃P₂ and act as 'electron sinks', nullifying the desired doping and lowering the fermi-level back towards the p-type regime. Going forward, this insight provides clues to fabricating useful zinc phosphide based devices. In addition, the methodology developed for this work can be applied to further doping studies in other systems.

Accurate determination of high pressure and temperature equations of state is fundamental in a variety of fields. However, it is often very difficult to cover a wide range of temperatures and pressures in an laboratory setting. Here we develop methods to determine a multi-phase equation of state for Ta through computation. The typical means of investigating thermodynamic properties is via ’classical’ molecular dynamics where the atomic motion is calculated from Newtonian mechanics with the electronic effects abstracted away into an interatomic potential function. For our purposes, a ’first principles’ approach such as DFT is useful as a classical potential is typically valid for only a portion of the phase diagram (i.e. whatever part it has been fit to). Furthermore, for extremes of temperature and pressure quantum effects become critical to accurately capture an equation of state and are very hard to capture in even complex model potentials. This requires extending the inherently zero temperature DFT to predict the finite temperature response of the system. Statistical modelling and thermodynamic integration is used to extend our results over all phases, as well as phase-coexistence regions which are at the limits of typical DFT validity. We deliver the most comprehensive and accurate equation of state that has been done for Ta. This work also lends insights that can be applied to further equation of state work in many other materials.

High pressure states in condensed matter: I. High pressure behavior of the iron-sulphur system with applications to the earth's core. II. Empirical equation of state for organic compounds at high pressures

Part I:

The earth's core is generally accepted to be composed primarily of iron, with an admixture of other elements. Because the outer core is observed not to transmit shear waves at seismic frequencies, it is known to be liquid or primarily liquid. A new equation of state is presented for liquid iron, in the form of parameters for the 4th order Birch-Murnaghan and Mie-Grüneisen equations of state. The parameters were constrained by a set of values for numerous properties compiled from the literature. A detailed theoretical model is used to constrain the P-T behavior of the heat capacity, based on recent advances in the understanding of the interatomic potentials for transition metals. At the reference pressure of 10⁵ Pa and temperature of 1811 K (the normal melting point of Fe), the parameters are: ρ = 7037 kg/m³, K_S0 = 110 GPa, K_S' = 4.53, K_S" = -.0337 GPa-1, and γ = 2.8, with γ α ρ^-1.17. Comparison of the properties predicted by this model with the earth model PREM indicates that the outer core is 8 to 10 % less dense than pure liquid Fe at the same conditions. The inner core is also found to be 3 to 5% less dense than pure liquid Fe, supporting the idea of a partially molten inner core. The density deficit of the outer core implies that the elements dissolved in the liquid Fe are predominantly of lower atomic weight than Fe. Of the candidate light elements favored by researchers, only sulfur readily dissolves into Fe at low pressure, which means that this element was almost certainly concentrated in the core at early times. New melting data are presented for FeS and FeS₂ which indicate that the FeS₂ is the S-hearing liquidus solid phase at inner core pressures. Consideration of the requirement that the inner core boundary be observable by seismological means and the freezing behavior of solutions leads to the possibility that the outer core may contain a significant fraction of solid material. It is found that convection in the outer core is not hindered if the solid particles are entrained in the fluid flow. This model for a core of Fe and S admits temperatures in the range 3450K to 4200K at the top of the core. An all liquid Fe-S outer core would require a temperature of about 4900 K at the top of the core.

Part II.

The abundance of uses for organic compounds in the modern world results in many applications in which these materials are subjected to high pressures. This leads to the desire to be able to describe the behavior of these materials under such conditions. Unfortunately, the number of compounds is much greater than the number of experimental data available for many of the important properties. In the past, one approach that has worked well is the calculation of appropriate properties by summing the contributions from the organic functional groups making up molecules of the compounds in question. A new set of group contributions for the molar volume, volume thermal expansivity, heat capacity, and the Rao function is presented for functional groups containing C, H, and O. This set is, in most cases, limited in application to low molecular liquids. A new technique for the calculation of the pressure derivative of the bulk modulus is also presented. Comparison with data indicates that the presented technique works very well for most low molecular hydrocarbon liquids and somewhat less well for oxygen-bearing compounds. A similar comparison of previous results for polymers indicates that the existing tabulations of group contributions for this class of materials is in need of revision. There is also evidence that the Rao function contributions for polymers and low molecular compounds are somewhat different.

Value Estimation and Comparison in Multi-Attribute Choice

The following work explores the processes individuals utilize when making multi-attribute choices. With the exception of extremely simple or familiar choices, most decisions we face can be classified as multi-attribute choices. In order to evaluate and make choices in such an environment, we must be able to estimate and weight the particular attributes of an option. Hence, better understanding the mechanisms involved in this process is an important step for economists and psychologists. For example, when choosing between two meals that differ in taste and nutrition, what are the mechanisms that allow us to estimate and then weight attributes when constructing value? Furthermore, how can these mechanisms be influenced by variables such as attention or common physiological states, like hunger?

In order to investigate these and similar questions, we use a combination of choice and attentional data, where the attentional data was collected by recording eye movements as individuals made decisions. Chapter 1 designs and tests a neuroeconomic model of multi-attribute choice that makes predictions about choices, response time, and how these variables are correlated with attention. Chapter 2 applies the ideas in this model to intertemporal decision-making, and finds that attention causally affects discount rates. Chapter 3 explores how hunger, a common physiological state, alters the mechanisms we utilize as we make simple decisions about foods.

Synthesis, Oxidation and Photophysics of Perfluoroborated Tetrakis(pyrophosphito)diplatinate (II) and Density Functional Theory (DFT) Study of Electrochemical CO2 Reduction by Mn Catalysts

In the first part of this thesis (Chapters I and II), the synthesis, characterization, reactivity and photophysics of per(difluoroborated) tetrakis(pyrophosphito)diplatinate(II) (Pt(POPBF₂)) are discussed. Pt(POP-BF₂) was obtained by reaction of [Pt₂(POP)₄]^4- with neat boron trifluoride diethyl etherate (BF₃·Et₂O). While Pt(POP-BF₂) and [Pt₂(POP)₄]^4- have similar structures and absorption spectra, they differ in significant ways. Firstly, as discussed in Chapter I, the former is less susceptible to oxidation, as evidenced by the reversibility of its oxidation by I2. Secondly, while the first excited triplet states (T₁) of both Pt(POP-BF₂) and [Pt₂(POP)₄]^4- exhibit long lifetimes (ca. 0.01 ms at room temperature) and substantial zero-field splitting (40 cm^-1), Pt(POP-BF₂) also has a remarkably long-lived (1.6 ns at room temperature) singlet excited state (S₁), indicating slow intersystem crossing (ISC). Fluorescence lifetime and quantum yield (QY) of Pt(POP-BF₂) were measured over a range of temperatures, providing insight into the slow ISC process. The remarkable spectroscopic and photophysical properties of Pt(POP-BF₂), both in solution and as a microcrystalline powder, form the theme of Chapter II.

In the second part of the thesis (Chapters III and IV), the electrochemical reduction of CO₂ to CO by [(L)Mn(CO)₃]^- catalysts is investigated using density functional theory (DFT). As discussed in Chapter III, the turnover frequency (TOF)-limiting step is the dehydroxylation of [(bpy)Mn(CO)₃(CO₂H)]^0/- (bpy = bipyridine) by trifluoroethanol (TFEH) to form [(bpy)Mn(CO)₄]^+/0. Because the dehydroxylation of [(bpy)Mn(CO)₃(CO₂H)]^- is faster, maximum TOF (TOF_max) is achieved at potentials sufficient to completely reduce [(bpy)Mn(CO)₃(CO₂H)]⁰ to [(bpy)Mn(CO)₃(CO₂H)]^-. Substitution of bipyridine with bipyrimidine reduces the overpotential needed, but at the expense of TOF_max. In Chapter IV, the decoration of the bipyrimidine ligand with a pendant alcohol is discussed as a strategy to increase CO₂ reduction activity. Our calculations predict that the pendant alcohol acts in concert with an external TFEH molecule, the latter acidifying the former, resulting in a ~ 80,000-fold improvement in the rate of TOF-limiting dehydroxylation of [(L)Mn(CO)₃(CO₂H)]^-.

An interesting strategy for the co-upgrading of light olefins and alkanes into heavier alkanes is the subject of Appendix B. The proposed scheme involves dimerization of the light olefin, operating in tandem with transfer hydrogenation between the olefin dimer and the light alkane. The work presented therein involved a Ta olefin dimerization catalyst and a silica-supported Ir transfer hydrogenation catalyst. Olefin dimer was formed under reaction conditions; however, this did not undergo transfer hydrogenation with the light alkane. A significant challenge is that the Ta catalyst selectively produces highly branched dimers, which are unable to undergo transfer hydrogenation.

Estimation of catalyst activity profiles and deactivation parameters from reactor operating data with applications to naphtha reforming

Techniques are developed for estimating activity profiles in fixed bed reactors and catalyst deactivation parameters from operating reactor data. These techniques are applicable, in general, to most industrial catalytic processes. The catalytic reforming of naphthas is taken as a broad example to illustrate the estimation schemes and to signify the physical meaning of the kinetic parameters of the estimation equations. The work is described in two parts. Part I deals with the modeling of kinetic rate expressions and the derivation of the working equations for estimation. Part II concentrates on developing various estimation techniques.

Part I: The reactions used to describe naphtha reforming are dehydrogenation and dehydroisomerization of cycloparaffins; isomerization, dehydrocyclization and hydrocracking of paraffins; and the catalyst deactivation reactions, namely coking on alumina sites and sintering of platinum crystallites. The rate expressions for the above reactions are formulated, and the effects of transport limitations on the overall reaction rates are discussed in the appendices. Moreover, various types of interaction between the metallic and acidic active centers of reforming catalysts are discussed as characterizing the different types of reforming reactions.

Part II: In catalytic reactor operation, the activity distribution along the reactor determines the kinetics of the main reaction and is needed for predicting the effect of changes in the feed state and the operating conditions on the reactor output. In the case of a monofunctional catalyst and of bifunctional catalysts in limiting conditions, the cumulative activity is sufficient for predicting steady reactor output. The estimation of this cumulative activity can be carried out easily from measurements at the reactor exit. For a general bifunctional catalytic system, the detailed activity distribution is needed for describing the reactor operation, and some approximation must be made to obtain practicable estimation schemes. This is accomplished by parametrization techniques using measurements at a few points along the reactor. Such parametrization techniques are illustrated numerically with a simplified model of naphtha reforming.

To determine long term catalyst utilization and regeneration policies, it is necessary to estimate catalyst deactivation parameters from the the current operating data. For a first order deactivation model with a monofunctional catalyst or with a bifunctional catalyst in special limiting circumstances, analytical techniques are presented to transform the partial differential equations to ordinary differential equations which admit more feasible estimation schemes. Numerical examples include the catalytic oxidation of butene to butadiene and a simplified model of naphtha reforming. For a general bifunctional system or in the case of a monofunctional catalyst subject to general power law deactivation, the estimation can only be accomplished approximately. The basic feature of an appropriate estimation scheme involves approximating the activity profile by certain polynomials and then estimating the deactivation parameters from the integrated form of the deactivation equation by regression techniques. Different bifunctional systems must be treated by different estimation algorithms, which are illustrated by several cases of naphtha reforming with different feed or catalyst composition.