Formal methods for control synthesis in partially observed environments : application to autonomous robotic manipulation

Modern robots are increasingly expected to function in uncertain and dynamically challenging environments, often in proximity with humans. In addition, wide scale adoption of robots requires on-the-fly adaptability of software for diverse application. These requirements strongly suggest the need to adopt formal representations of high level goals and safety specifications, especially as temporal logic formulas. This approach allows for the use of formal verification techniques for controller synthesis that can give guarantees for safety and performance. Robots operating in unstructured environments also face limited sensing capability. Correctly inferring a robot's progress toward high level goal can be challenging.

This thesis develops new algorithms for synthesizing discrete controllers in partially known environments under specifications represented as linear temporal logic (LTL) formulas. It is inspired by recent developments in finite abstraction techniques for hybrid systems and motion planning problems. The robot and its environment is assumed to have a finite abstraction as a Partially Observable Markov Decision Process (POMDP), which is a powerful model class capable of representing a wide variety of problems. However, synthesizing controllers that satisfy LTL goals over POMDPs is a challenging problem which has received only limited attention.

This thesis proposes tractable, approximate algorithms for the control synthesis problem using Finite State Controllers (FSCs). The use of FSCs to control finite POMDPs allows for the closed system to be analyzed as finite global Markov chain. The thesis explicitly shows how transient and steady state behavior of the global Markov chains can be related to two different criteria with respect to satisfaction of LTL formulas. First, the maximization of the probability of LTL satisfaction is related to an optimization problem over a parametrization of the FSC. Analytic computation of gradients are derived which allows the use of first order optimization techniques.

The second criterion encourages rapid and frequent visits to a restricted set of states over infinite executions. It is formulated as a constrained optimization problem with a discounted long term reward objective by the novel utilization of a fundamental equation for Markov chains - the Poisson equation. A new constrained policy iteration technique is proposed to solve the resulting dynamic program, which also provides a way to escape local maxima.

The algorithms proposed in the thesis are applied to the task planning and execution challenges faced during the DARPA Autonomous Robotic Manipulation - Software challenge.

Chains of Non-regular de Branges Spaces

We consider canonical systems with singular left endpoints, and discuss the concept of a scalar spectral measure and the corresponding generalized Fourier transform associated with a canonical system with a singular left endpoint. We use the framework of de Branges’ theory of Hilbert spaces of entire functions to study the correspondence between chains of non-regular de Branges spaces, canonical systems with singular left endpoints, and spectral measures.

We find sufficient integrability conditions on a Hamiltonian H which ensure the existence of a chain of de Branges functions in the first generalized Pólya class with Hamiltonian H. This result generalizes de Branges’ Theorem 41, which showed the sufficiency of stronger integrability conditions on H for the existence of a chain in the Pólya class. We show the conditions that de Branges came up with are also necessary. In the case of Krein’s strings, namely when the Hamiltonian is diagonal, we show our proposed conditions are also necessary.

We also investigate the asymptotic conditions on chains of de Branges functions as t approaches its left endpoint. We show there is a one-to-one correspondence between chains of de Branges functions satisfying certain asymptotic conditions and chains in the Pólya class. In the case of Krein’s strings, we also establish the one-to-one correspondence between chains satisfying certain asymptotic conditions and chains in the generalized Pólya class.

Formal methods in the foundations of science

The intent of this study is to provide formal apparatus which facilitates the investigation of problems in the methodology of science. The introduction contains several examples of such problems and motivates the subsequent formalism.

A general definition of a formal language is presented, and this definition is used to characterize an individual’s view of the world around him. A notion of empirical observation is developed which is independent of language. The interplay of formal language and observation is taken as the central theme. The process of science is conceived as the finding of that formal language that best expresses the available experimental evidence.

To characterize the manner in which a formal language imposes structure on its universe of discourse, the fundamental concepts of elements and states of a formal language are introduced. Using these, the notion of a basis for a formal language is developed as a collection of minimal states distinguishable within the language. The relation of these concepts to those of model theory is discussed.

An a priori probability defined on sets of observations is postulated as a reflection of an individual’s ontology. This probability, in conjunction with a formal language and a basis for that language, induces a subjective probability describing an individual’s conceptual view of admissible configurations of the universe. As a function of this subjective probability, and consequently of language, a measure of the informativeness of empirical observations is introduced and is shown to be intuitively plausible – particularly in the case of scientific experimentation.

The developed formalism is then systematically applied to the general problems presented in the introduction. The relationship of scientific theories to empirical observations is discussed and the need for certain tacit, unstatable knowledge is shown to be necessary to fully comprehend the meaning of realistic theories. The idea that many common concepts can be specified only by drawing on knowledge obtained from an infinite number of observations is presented, and the problems of reductionism are examined in this context.

A definition of when one formal language can be considered to be more expressive than another is presented, and the change in the informativeness of an observation as language changes is investigated. In this regard it is shown that the information inherent in an observation may decrease for a more expressive language.

The general problem of induction and its relation to the scientific method are discussed. Two hypotheses concerning an individual’s selection of an optimal language for a particular domain of discourse are presented and specific examples from the introduction are examined.

Locally convex Riesz spaces and Archimedean quotient spaces

A Riesz space with a Hausdorff, locally convex topology determined by Riesz seminorms is called a locally convex Riesz space. A sequence {x_n} in a locally convex Riesz space L is said to converge locally to x ϵ L if for some topologically bounded set B and every real r ˃ 0 there exists N (r) and n ≥ N (r) implies x – x_n ϵ r^b. Local Cauchy sequences are defined analogously, and L is said to be locally complete if every local Cauchy sequence converges locally. Then L is locally complete if and only if every monotone local Cauchy sequence has a least upper bound. This is a somewhat more general form of the completeness criterion for Riesz – normed Riesz spaces given by Luxemburg and Zaanen. Locally complete, bound, locally convex Riesz spaces are barrelled. If the space is metrizable, local completeness and topological completeness are equivalent.

Two measures of the non-archimedean character of a non-archimedean Riesz space L are the smallest ideal A_o (L) such that quotient space is Archimedean and the ideal I (L) = { x ϵ L: for some 0 ≤ v ϵ L, n |x| ≤ v for n = 1, 2, …}. In general A_o (L) ᴝ I (L). If L is itself a quotient space, a necessary and sufficient condition that A_o (L) = I (L) is given. There is an example where A_o (L) ≠ I (L).

A necessary and sufficient condition that a Riesz space L have every quotient space Archimedean is that for every 0 ≤ u, v ϵ L there exist u₁ = sup (inf (n v, u): n = 1, 2, …), and real numbers m₁ and m₂ such that m₁ u₁ ≥ v₁ and m₂ v₁ ≥ u₁. If, in addition, L is Dedekind σ – complete, then L may be represented as the space of all functions which vanish off finite subsets of some non-empty set.

Multiplication in Riesz spaces

A.G. Vulih has shown how an essentially unique intrinsic multiplication can be defined in certain types of Riesz spaces (vector lattices) L. In general, the multiplication is not universally defined in L, but L can always be imbedded in a large space L^# in which multiplication is universally defined.

If ф is a normal integral in L, then ф can be extended to a normal integral on a large space L₁(ф) in L^#, and L₁(ф) may be regarded as an abstract integral space. A very general form of the Radon-Nikodym theorem can be proved in L₁(ф), and this can be used to give a relatively simple proof of a theorem of Segal giving a necessary and sufficient condition that the Radon-Nikodym theorem hold in a measure space.

In another application, the multiplication is used to give a representation of certain Riesz spaces as rings of operators on a Hilbert space.

Smooth Banach spaces and approximations

If E and F are real Banach spaces let C^p,q(E, F) O ≤ q ≤ p ≤ ∞, denote those maps from E to F which have p continuous Frechet derivatives of which the first q derivatives are bounded. A Banach space E is defined to be C^p,q smooth if C^p,q(E,R) contains a nonzero function with bounded support. This generalizes the standard C^p smoothness classification.

If an L^p space, p ≥ 1, is C^q smooth then it is also C^q,q smooth so that in particular L^p for p an even integer is C^∞,∞ smooth and L^p for p an odd integer is C^p-1,p-1 smooth. In general, however, a C^p smooth B-space need not be C^p,p smooth. C_o is shown to be a non-C^2,2 smooth B-space although it is known to be C^∞ smooth. It is proved that if E is C^p,1 smooth then C_o(E) is C^p,1 smooth and if E has an equivalent C^p norm then c_o(E) has an equivalent C^p norm.

Various consequences of C^p,q smoothness are studied. If f ϵ C^p,q(E,F), if F is C^p,q smooth and if E is non-C^p,q smooth, then the image under f of the boundary of any bounded open subset U of E is dense in the image of U. If E is separable then E is C^p,q smooth if and only if E admits C^p,q partitions of unity; E is C^p,psmooth, p ˂∞, if and only if every closed subset of E is the zero set of some C^P function.

f ϵ C^q(E,F), 0 ≤ q ≤ p ≤ ∞, is said to be C_p,q approximable on a subset U of E if for any ϵ ˃ 0 there exists a g ϵ C^p(E,F) satisfying

sup/xϵU, O≤k≤q ‖ D^k f(x) - D^k g(x) ‖ ≤ ϵ.

It is shown that if E is separable and C^p,q smooth and if f ϵ C^q(E,F) is C_p,q approximable on some neighborhood of every point of E, then F is C_p,q approximable on all of E.

In general it is unknown whether an arbitrary function in C¹(l², R) is C_2,1 approximable and an example of a function in C¹(l², R) which may not be C_2,1 approximable is given. A weak form of C_∞,q, q≥1, to functions in C^q(l², R) is proved: Let {U_α} be a locally finite cover of l² and let {T_α} be a corresponding collection of Hilbert-Schmidt operators on l². Then for any f ϵ C^q(l²,F) such that for all α

sup ‖ D^k(f(x)-g(x))[T_αh]‖ ≤ 1.

xϵU_α,‖h‖≤1, 0≤k≤q

Formal languages, part theory, and change

A general definition of interpreted formal language is presented. The notion “is a part of" is formally developed and models of the resulting part theory are used as universes of discourse of the formal languages. It is shown that certain Boolean algebras are models of part theory.

With this development, the structure imposed upon the universe of discourse by a formal language is characterized by a group of automorphisms of the model of part theory. If the model of part theory is thought of as a static world, the automorphisms become the changes which take place in the world. Using this formalism, we discuss a notion of abstraction and the concept of definability. A Galois connection between the groups characterizing formal languages and a language-like closure over the groups is determined.

It is shown that a set theory can be developed within models of part theory such that certain strong formal languages can be said to determine their own set theory. This development is such that for a given formal language whose universe of discourse is a model of part theory, a set theory can be imbedded as a submodel of part theory so that the formal language has parts which are sets as its discursive entities.