Formal methods for control synthesis in partially observed environments : application to autonomous robotic manipulation

Modern robots are increasingly expected to function in uncertain and dynamically challenging environments, often in proximity with humans. In addition, wide scale adoption of robots requires on-the-fly adaptability of software for diverse application. These requirements strongly suggest the need to adopt formal representations of high level goals and safety specifications, especially as temporal logic formulas. This approach allows for the use of formal verification techniques for controller synthesis that can give guarantees for safety and performance. Robots operating in unstructured environments also face limited sensing capability. Correctly inferring a robot's progress toward high level goal can be challenging.

This thesis develops new algorithms for synthesizing discrete controllers in partially known environments under specifications represented as linear temporal logic (LTL) formulas. It is inspired by recent developments in finite abstraction techniques for hybrid systems and motion planning problems. The robot and its environment is assumed to have a finite abstraction as a Partially Observable Markov Decision Process (POMDP), which is a powerful model class capable of representing a wide variety of problems. However, synthesizing controllers that satisfy LTL goals over POMDPs is a challenging problem which has received only limited attention.

This thesis proposes tractable, approximate algorithms for the control synthesis problem using Finite State Controllers (FSCs). The use of FSCs to control finite POMDPs allows for the closed system to be analyzed as finite global Markov chain. The thesis explicitly shows how transient and steady state behavior of the global Markov chains can be related to two different criteria with respect to satisfaction of LTL formulas. First, the maximization of the probability of LTL satisfaction is related to an optimization problem over a parametrization of the FSC. Analytic computation of gradients are derived which allows the use of first order optimization techniques.

The second criterion encourages rapid and frequent visits to a restricted set of states over infinite executions. It is formulated as a constrained optimization problem with a discounted long term reward objective by the novel utilization of a fundamental equation for Markov chains - the Poisson equation. A new constrained policy iteration technique is proposed to solve the resulting dynamic program, which also provides a way to escape local maxima.

The algorithms proposed in the thesis are applied to the task planning and execution challenges faced during the DARPA Autonomous Robotic Manipulation - Software challenge.

From lakes to lungs : assessing microbial activity in diverse environments

All major geochemical cycles on the Earth’s surface are mediated by microorganisms. Our understanding of how these microbes have interacted with their environments (and vice versa) throughout Earth's history, and how they will respond to changes in the future, is primarily based on studying their activity in different environments today. The overarching questions that motivate the research presented in the two parts of this thesis -- how do microorganisms shape their environment (and vice versa)? and how can we best study microbial activity in situ? -- have arisen from the ultimate goal of being able to predict microbial activity in response to changes within their environments both past and future.

Part one focuses on work related to microbial processes in iron-rich Lake Matano and, more broadly, microbial interactions with the biogeochemical cycling of iron. Primarily, we find that the chelation of ferrous iron by organic ligands can affect the role of iron in anoxic environmental systems, enabling photomixotrophic growth of anoxygenic microorganisms with ferrous iron, as well as catalyzing the oxidation of ferrous iron by denitrification intermediates. These results imply that the ability to grow photomixotrophically on ferrous iron might be more widespread than previously assumed, and that the co-occurrence of chemical and biological processes involved in the coupled biogeochemical cycling of iron and nitrogen likely dominate organic-rich environmental systems.

Part two switches focus to in situ measurements of growth activity and comprises work related to microbial processes in the Cystic Fibrosis lung, and more broadly, the physiology of slow growth. We introduce stable isotope labeling of microbial membrane fatty acids and whole cells with heavy water as a new technique to measure microbial activity in a wide range of environments, demonstrate its application in continuous culture in the laboratory at the population and single cell level, and apply the tool to measure the in situ activity of the opportunistic pathogen Staphylococcus aureus within the environment of expectorated mucus from cystic fibrosis patients. We find that the average in situ growth rates of S. aureus fall into a range of generation times between ~12 hours and ~4 days, with substantial heterogeneity at the single-cell level. These data illustrate the use of heavy water as a universal environmental tracer for microbial activity, and highlight the crucial importance of studying the physiology of slow growth in representative laboratory systems in order to understand the role of these microorganisms in their native environments.

Programming chemical kinetics: engineering dynamic reaction networks with DNA strand displacement

Over the last century, the silicon revolution has enabled us to build faster, smaller and more sophisticated computers. Today, these computers control phones, cars, satellites, assembly lines, and other electromechanical devices. Just as electrical wiring controls electromechanical devices, living organisms employ "chemical wiring" to make decisions about their environment and control physical processes. Currently, the big difference between these two substrates is that while we have the abstractions, design principles, verification and fabrication techniques in place for programming with silicon, we have no comparable understanding or expertise for programming chemistry.

In this thesis we take a small step towards the goal of learning how to systematically engineer prescribed non-equilibrium dynamical behaviors in chemical systems. We use the formalism of chemical reaction networks (CRNs), combined with mass-action kinetics, as our programming language for specifying dynamical behaviors. Leveraging the tools of nucleic acid nanotechnology (introduced in Chapter 1), we employ synthetic DNA molecules as our molecular architecture and toehold-mediated DNA strand displacement as our reaction primitive.

Abstraction, modular design and systematic fabrication can work only with well-understood and quantitatively characterized tools. Therefore, we embark on a detailed study of the "device physics" of DNA strand displacement (Chapter 2). We present a unified view of strand displacement biophysics and kinetics by studying the process at multiple levels of detail, using an intuitive model of a random walk on a 1-dimensional energy landscape, a secondary structure kinetics model with single base-pair steps, and a coarse-grained molecular model that incorporates three-dimensional geometric and steric effects. Further, we experimentally investigate the thermodynamics of three-way branch migration. Our findings are consistent with previously measured or inferred rates for hybridization, fraying, and branch migration, and provide a biophysical explanation of strand displacement kinetics. Our work paves the way for accurate modeling of strand displacement cascades, which would facilitate the simulation and construction of more complex molecular systems.

In Chapters 3 and 4, we identify and overcome the crucial experimental challenges involved in using our general DNA-based technology for engineering dynamical behaviors in the test tube. In this process, we identify important design rules that inform our choice of molecular motifs and our algorithms for designing and verifying DNA sequences for our molecular implementation. We also develop flexible molecular strategies for "tuning" our reaction rates and stoichiometries in order to compensate for unavoidable non-idealities in the molecular implementation, such as imperfectly synthesized molecules and spurious "leak" pathways that compete with desired pathways.

We successfully implement three distinct autocatalytic reactions, which we then combine into a de novo chemical oscillator. Unlike biological networks, which use sophisticated evolved molecules (like proteins) to realize such behavior, our test tube realization is the first to demonstrate that Watson-Crick base pairing interactions alone suffice for oscillatory dynamics. Since our design pipeline is general and applicable to any CRN, our experimental demonstration of a de novo chemical oscillator could enable the systematic construction of CRNs with other dynamic behaviors.

Biological environments of larger UK rivers: progress report 1987

This interim progress report for the 9 months from January 1987 to September 1987 aims to provide insights into the mechanisms by which populations of particles (in this instance, phytoplankton) behave in relation to fluvial flow and, thus, to better model the dispersive properties of rivers and the ecological principles governing the distribution of potamoplankton generally. The author has been able to show with dye-tracers that significant water retention in pool reaches occurs within the range of (low) discharges obtaining, in accord with the Aggregated Dead Zone model and to an extent comparable with streams and small rivers investigated previously.

Mussel eggs as indicators of mutagen exposure in coastal and estuarine environments

The aim of this study was to develop a short-term genotoxicity assay for monitoring the marine environment for mutagens. Based on the developing eggs and embryos of the marine mussel Mytilus edulis, an important pollution indicator species, the test employs the sensitive sister chromatid exchange (SCE) technique as its end-point, and exploits the potential of mussel eggs to accumulate mutagenic pollutants from the surrounding sea water. Mussel eggs take up to 6 months to develop while in the gonad, which provides scope for DNA damage to be accumulated over an extended time interval; chromosome damage is subsequently visualised as SCEs in 2-cell-stage embryos after these have been spawned in the laboratory. Methods which measure biological responses to pollutant exposure are able to integrate all the factors (internal and external) which contribute to the exposure. The new cytogenetic assay allows the effects of adult exposure to be interpreted in cells destined to become part of the next generation.

CONTRIBUTION TO THE DEVELOPMENT OF INTELLIGENT CONVERSATIONAL ASSISTANTS IN AUTOMOTIVE ENVIRONMENTS

Esta tesis presenta el diseño y desarrollo del Asistente Conversacional del Automóvil, Voice User Help en inglés, un sistema conversacional basado en el paradigma pregunta-respuesta y diseñado para consultar documentación del vehículo duartne la conducción. Este trabajo recoge investigaciones realizadas en los campos de la documentación técnica, recuperación de información, procesamineto de lenguaje natural, diseñode interfaces de usuario para vehículos, experiencia del usuario y computación afectiva con el fin de crear un asistente adaptativo y dinámico que modifica su comportamiento conversacional dependiendo del estado emocional del usuario.