Formal methods for control synthesis in partially observed environments : application to autonomous robotic manipulation

Modern robots are increasingly expected to function in uncertain and dynamically challenging environments, often in proximity with humans. In addition, wide scale adoption of robots requires on-the-fly adaptability of software for diverse application. These requirements strongly suggest the need to adopt formal representations of high level goals and safety specifications, especially as temporal logic formulas. This approach allows for the use of formal verification techniques for controller synthesis that can give guarantees for safety and performance. Robots operating in unstructured environments also face limited sensing capability. Correctly inferring a robot's progress toward high level goal can be challenging.

This thesis develops new algorithms for synthesizing discrete controllers in partially known environments under specifications represented as linear temporal logic (LTL) formulas. It is inspired by recent developments in finite abstraction techniques for hybrid systems and motion planning problems. The robot and its environment is assumed to have a finite abstraction as a Partially Observable Markov Decision Process (POMDP), which is a powerful model class capable of representing a wide variety of problems. However, synthesizing controllers that satisfy LTL goals over POMDPs is a challenging problem which has received only limited attention.

This thesis proposes tractable, approximate algorithms for the control synthesis problem using Finite State Controllers (FSCs). The use of FSCs to control finite POMDPs allows for the closed system to be analyzed as finite global Markov chain. The thesis explicitly shows how transient and steady state behavior of the global Markov chains can be related to two different criteria with respect to satisfaction of LTL formulas. First, the maximization of the probability of LTL satisfaction is related to an optimization problem over a parametrization of the FSC. Analytic computation of gradients are derived which allows the use of first order optimization techniques.

The second criterion encourages rapid and frequent visits to a restricted set of states over infinite executions. It is formulated as a constrained optimization problem with a discounted long term reward objective by the novel utilization of a fundamental equation for Markov chains - the Poisson equation. A new constrained policy iteration technique is proposed to solve the resulting dynamic program, which also provides a way to escape local maxima.

The algorithms proposed in the thesis are applied to the task planning and execution challenges faced during the DARPA Autonomous Robotic Manipulation - Software challenge.

From lakes to lungs : assessing microbial activity in diverse environments

All major geochemical cycles on the Earth’s surface are mediated by microorganisms. Our understanding of how these microbes have interacted with their environments (and vice versa) throughout Earth's history, and how they will respond to changes in the future, is primarily based on studying their activity in different environments today. The overarching questions that motivate the research presented in the two parts of this thesis -- how do microorganisms shape their environment (and vice versa)? and how can we best study microbial activity in situ? -- have arisen from the ultimate goal of being able to predict microbial activity in response to changes within their environments both past and future.

Part one focuses on work related to microbial processes in iron-rich Lake Matano and, more broadly, microbial interactions with the biogeochemical cycling of iron. Primarily, we find that the chelation of ferrous iron by organic ligands can affect the role of iron in anoxic environmental systems, enabling photomixotrophic growth of anoxygenic microorganisms with ferrous iron, as well as catalyzing the oxidation of ferrous iron by denitrification intermediates. These results imply that the ability to grow photomixotrophically on ferrous iron might be more widespread than previously assumed, and that the co-occurrence of chemical and biological processes involved in the coupled biogeochemical cycling of iron and nitrogen likely dominate organic-rich environmental systems.

Part two switches focus to in situ measurements of growth activity and comprises work related to microbial processes in the Cystic Fibrosis lung, and more broadly, the physiology of slow growth. We introduce stable isotope labeling of microbial membrane fatty acids and whole cells with heavy water as a new technique to measure microbial activity in a wide range of environments, demonstrate its application in continuous culture in the laboratory at the population and single cell level, and apply the tool to measure the in situ activity of the opportunistic pathogen Staphylococcus aureus within the environment of expectorated mucus from cystic fibrosis patients. We find that the average in situ growth rates of S. aureus fall into a range of generation times between ~12 hours and ~4 days, with substantial heterogeneity at the single-cell level. These data illustrate the use of heavy water as a universal environmental tracer for microbial activity, and highlight the crucial importance of studying the physiology of slow growth in representative laboratory systems in order to understand the role of these microorganisms in their native environments.

Field and Laboratory Studies of Atmospheric Organic Aerosol

This thesis is the culmination of field and laboratory studies aimed at assessing processes that affect the composition and distribution of atmospheric organic aerosol. An emphasis is placed on measurements conducted using compact and high-resolution Aerodyne Aerosol Mass Spectrometers (AMS). The first three chapters summarize results from aircraft campaigns designed to evaluate anthropogenic and biogenic impacts on marine aerosol and clouds off the coast of California. Subsequent chapters describe laboratory studies intended to evaluate gas and particle-phase mechanisms of organic aerosol oxidation.

The 2013 Nucleation in California Experiment (NiCE) was a campaign designed to study environments impacted by nucleated and/or freshly formed aerosol particles. Terrestrial biogenic aerosol with > 85% organic mass was observed to reside in the free troposphere above marine stratocumulus. This biogenic organic aerosol (BOA) originated from the Northwestern United States and was transported to the marine atmosphere during periodic cloud-clearing events. Spectra recorded by a cloud condensation nuclei counter demonstrated that BOA is CCN active. BOA enhancements at latitudes north of San Francisco, CA coincided with enhanced cloud water concentrations of organic species such as acetate and formate.

Airborne measurements conducted during the 2011 Eastern Pacific Emitted Aerosol Cloud Experiment (E-PEACE) were aimed at evaluating the contribution of ship emissions to the properties of marine aerosol and clouds off the coast of central California. In one study, analysis of organic aerosol mass spectra during periods of enhanced shipping activity yielded unique tracers indicative of cloud-processed ship emissions (m/z 42 and 99). The variation of their organic fraction (f₄₂ and f₉₉) was found to coincide with periods of heavy (f₄₂ > 0.15; f₉₉ > 0.04), moderate (0.05 < f₄₂ < 0.15; 0.01 < f₉₉ < 0.04), and negligible (f₄₂ < 0.05; f₉₉ < 0.01) ship influence. Application of these conditions to all measurements conducted during E-PEACE demonstrated that a large fraction of cloud droplet (72%) and dry aerosol mass (12%) sampled in the California coastal study region was heavily or moderately influenced by ship emissions. Another study investigated the chemical and physical evolution of a controlled organic plume emitted from the R/V Point Sur. Under sunny conditions, nucleated particles composed of oxidized organic compounds contributed nearly an order of magnitude more cloud condensation nuclei (CCN) than less oxidized particles formed under cloudy conditions. The processing time necessary for particles to become CCN active was short ( < 1 hr) compared to the time needed for particles to become hygroscopic at sub-saturated humidity ( > 4 hr).

Laboratory chamber experiments were also conducted to evaluate particle-phase processes influencing aerosol phase and composition. In one study, ammonium sulfate seed was coated with a layer of secondary organic aerosol (SOA) from toluene oxidation followed by a layer of SOA from α-pinene oxidation. The system exhibited different evaporative properties than ammonium sulfate seed initially coated with α-pinene SOA followed by a layer of toluene SOA. This behavior is consistent with a shell-and-core model and suggests limited mixing among different SOA types. Another study investigated the reactive uptake of isoprene epoxy diols (IEPOX) onto non-acidified aerosol. It was demonstrated that particle acidity has limited influence on organic aerosol formation onto ammonium sulfate seed, and that the chemical system is limited by the availability of nucleophiles such as sulfate.

Flow tube experiments were conducted to examine the role of iron in the reactive uptake and chemical oxidation of glycolaldehyde. Aerosol particles doped with iron and hydrogen peroxide were mixed with gas-phase glycolaldehyde and photochemically aged in a custom-built flow reactor. Compared to particles free of iron, iron-doped aerosols significantly enhanced the oxygen to carbon (O/C) ratio of accumulated organic mass. The primary oxidation mechanism is suggested to be a combination of Fenton and photo-Fenton reactions which enhance particle-phase OH radical concentrations.