Efficient methods for stochastic optimal control

The Hamilton Jacobi Bellman (HJB) equation is central to stochastic optimal control (SOC) theory, yielding the optimal solution to general problems specified by known dynamics and a specified cost functional. Given the assumption of quadratic cost on the control input, it is well known that the HJB reduces to a particular partial differential equation (PDE). While powerful, this reduction is not commonly used as the PDE is of second order, is nonlinear, and examples exist where the problem may not have a solution in a classical sense. Furthermore, each state of the system appears as another dimension of the PDE, giving rise to the curse of dimensionality. Since the number of degrees of freedom required to solve the optimal control problem grows exponentially with dimension, the problem becomes intractable for systems with all but modest dimension.

In the last decade researchers have found that under certain, fairly non-restrictive structural assumptions, the HJB may be transformed into a linear PDE, with an interesting analogue in the discretized domain of Markov Decision Processes (MDP). The work presented in this thesis uses the linearity of this particular form of the HJB PDE to push the computational boundaries of stochastic optimal control.

This is done by crafting together previously disjoint lines of research in computation. The first of these is the use of Sum of Squares (SOS) techniques for synthesis of control policies. A candidate polynomial with variable coefficients is proposed as the solution to the stochastic optimal control problem. An SOS relaxation is then taken to the partial differential constraints, leading to a hierarchy of semidefinite relaxations with improving sub-optimality gap. The resulting approximate solutions are shown to be guaranteed over- and under-approximations for the optimal value function. It is shown that these results extend to arbitrary parabolic and elliptic PDEs, yielding a novel method for Uncertainty Quantification (UQ) of systems governed by partial differential constraints. Domain decomposition techniques are also made available, allowing for such problems to be solved via parallelization and low-order polynomials.

The optimization-based SOS technique is then contrasted with the Separated Representation (SR) approach from the applied mathematics community. The technique allows for systems of equations to be solved through a low-rank decomposition that results in algorithms that scale linearly with dimensionality. Its application in stochastic optimal control allows for previously uncomputable problems to be solved quickly, scaling to such complex systems as the Quadcopter and VTOL aircraft. This technique may be combined with the SOS approach, yielding not only a numerical technique, but also an analytical one that allows for entirely new classes of systems to be studied and for stability properties to be guaranteed.

The analysis of the linear HJB is completed by the study of its implications in application. It is shown that the HJB and a popular technique in robotics, the use of navigation functions, sit on opposite ends of a spectrum of optimization problems, upon which tradeoffs may be made in problem complexity. Analytical solutions to the HJB in these settings are available in simplified domains, yielding guidance towards optimality for approximation schemes. Finally, the use of HJB equations in temporal multi-task planning problems is investigated. It is demonstrated that such problems are reducible to a sequence of SOC problems linked via boundary conditions. The linearity of the PDE allows us to pre-compute control policy primitives and then compose them, at essentially zero cost, to satisfy a complex temporal logic specification.

Study of the effect of alcohol on single human red blood cells using near-infrared laser tweezers Raman spectroscopy

The effect of alcohol solution on single human red blood Cells (RBCs) was investigated using near-infrared laser tweezers Raman spectroscopy (LTRS). In our system, a low-power diode laser at 785 nm was applied for the trapping of a living cell and the excitation of its Raman spectrum. Such a design could simultaneously reduce the photo-damage to the cell and suppress the interference from the fluorescence on the Raman signal. The denaturation process of single RBCs in 20% alcohol solution was investigated by detecting the time evolution of the Raman spectra at the single-cell level. The vitality of RBCs was characterized by the Raman band at 752 cm(-1), which corresponds to the porphyrin breathing mode. We found that the intensity of this band decreased by 34.1% over a period of 25 min after the administration of alcohol. In a further study of the dependence of denaturation on alcohol concentration, we discovered that the decrease in the intensity of the 752 cm(-1) band became more rapid and more prominent as the alcohol concentration increased. The present LTRS technique may have several potential applications in cell biology and medicine, including probing dynamic cellular processes at the single cell level and diagnosing cell disorders in real time. Copyright (c) 2005 John Wiley T Sons, Ltd.

Diseño y despliegue de una red inalámbrica segura y control de acceso de sensores IP

[ES]El proyecto está orientado a conseguir una comunicación inalámbrica y segura de una red de sensores IP. Por un lado, mediante el protocolo 6LoWPAN se consigue que los datos se transmitan mediante IPv6 y, por otro lado, gracias al protocolo LADON se establecen los servicios de seguridad de autenticación, integridad de datos, autorización y control de acceso.

Computationally Guided Monomerization of Red Fluorescent Proteins of the Class Anthozoa

Red fluorescent proteins (RFPs) have attracted significant engineering focus because of the promise of near infrared fluorescent proteins, whose light penetrates biological tissue, and which would allow imaging inside of vertebrate animals. The RFP landscape, which numbers ~200 members, is mostly populated by engineered variants of four native RFPs, leaving the vast majority of native RFP biodiversity untouched. This is largely due to the fact that native RFPs are obligate tetramers, limiting their usefulness as fusion proteins. Monomerization has imposed critical costs on these evolved tetramers, however, as it has invariably led to loss of brightness, and often to many other adverse effects on the fluorescent properties of the derived monomeric variants. Here we have attempted to understand why monomerization has taken such a large toll on Anthozoa class RFPs, and to outline a clear strategy for their monomerization. We begin with a structural study of the far-red fluorescence of AQ143, one of the furthest red emitting RFPs. We then try to separate the problem of stable and bright fluorescence from the design of a soluble monomeric β-barrel surface by engineering a hybrid protein (DsRmCh) with an oligomeric parent that had been previously monomerized, DsRed, and a pre-stabilized monomeric core from mCherry. This allows us to use computational design to successfully design a stable, soluble, fluorescent monomer. Next we took HcRed, which is a previously unmonomerized RFP that has far-red fluorescence (λemission = 633 nm) and attempted to monomerize it making use of lessons learned from DsRmCh. We engineered two monomeric proteins by pre-stabilizing HcRed’s core, then monomerizing in stages, making use of computational design and directed evolution techniques such as error-prone mutagenesis and DNA shuffling. We call these proteins mGinger0.1 (λem = 637 nm / Φ = 0.02) and mGinger0.2 (λem = 631 nm Φ = 0.04). They are the furthest red first generation monomeric RFPs ever developed, are significantly thermostabilized, and add diversity to a small field of far-red monomeric FPs. We anticipate that the techniques we describe will be facilitate future RFP monomerization, and that further core optimization of the mGingers may allow significant improvements in brightness.

Fiber-optic integration and efficient detection schemes for optomechanical resonators

With the advent of the laser in the year 1960, the field of optics experienced a renaissance from what was considered to be a dull, solved subject to an active area of development, with applications and discoveries which are yet to be exhausted 55 years later. Light is now nearly ubiquitous not only in cutting-edge research in physics, chemistry, and biology, but also in modern technology and infrastructure. One quality of light, that of the imparted radiation pressure force upon reflection from an object, has attracted intense interest from researchers seeking to precisely monitor and control the motional degrees of freedom of an object using light. These optomechanical interactions have inspired myriad proposals, ranging from quantum memories and transducers in quantum information networks to precision metrology of classical forces. Alongside advances in micro- and nano-fabrication, the burgeoning field of optomechanics has yielded a class of highly engineered systems designed to produce strong interactions between light and motion.

Optomechanical crystals are one such system in which the patterning of periodic holes in thin dielectric films traps both light and sound waves to a micro-scale volume. These devices feature strong radiation pressure coupling between high-quality optical cavity modes and internal nanomechanical resonances. Whether for applications in the quantum or classical domain, the utility of optomechanical crystals hinges on the degree to which light radiating from the device, having interacted with mechanical motion, can be collected and detected in an experimental apparatus consisting of conventional optical components such as lenses and optical fibers. While several efficient methods of optical coupling exist to meet this task, most are unsuitable for the cryogenic or vacuum integration required for many applications. The first portion of this dissertation will detail the development of robust and efficient methods of optically coupling optomechanical resonators to optical fibers, with an emphasis on fabrication processes and optical characterization.

I will then proceed to describe a few experiments enabled by the fiber couplers. The first studies the performance of an optomechanical resonator as a precise sensor for continuous position measurement. The sensitivity of the measurement, limited by the detection efficiency of intracavity photons, is compared to the standard quantum limit imposed by the quantum properties of the laser probe light. The added noise of the measurement is seen to fall within a factor of 3 of the standard quantum limit, representing an order of magnitude improvement over previous experiments utilizing optomechanical crystals, and matching the performance of similar measurements in the microwave domain.

The next experiment uses single photon counting to detect individual phonon emission and absorption events within the nanomechanical oscillator. The scattering of laser light from mechanical motion produces correlated photon-phonon pairs, and detection of the emitted photon corresponds to an effective phonon counting scheme. In the process of scattering, the coherence properties of the mechanical oscillation are mapped onto the reflected light. Intensity interferometry of the reflected light then allows measurement of the temporal coherence of the acoustic field. These correlations are measured for a range of experimental conditions, including the optomechanical amplification of the mechanics to a self-oscillation regime, and comparisons are drawn to a laser system for phonons. Finally, prospects for using phonon counting and intensity interferometry to produce non-classical mechanical states are detailed following recent proposals in literature.

Mechanism of red colouration in Tovel-See (”Conclusions” only) [Translation from: Archiv fur Hydrobiologie 38 299-302, 1941]

Cases of red colouration in small lake basins, due to the abundant appearance of microorganisms have long been known. Usually it is caused by a fast, sudden, intensive propagation (so called ”bloom”) of Cyanophycae and bacteria. (e.g. Oscillatoracae, thiobacteria etc.). An exception to this is the red colouration of Tovel-See, an alpine lake basin in the Dolomites of the Brenta group (Trentino), lying at a height of 1178 m and hidden in the woodland of a valley. Here the red bloom has a double rhythm: a daily and a yearly rhythm. The colouration of one part of the lake takes place in the warmest months of the year (i.e. July, August, September) and in the middle hours of the day. The immediate origin of the bloom has been known for a long time: it is caused by the Peridinacae Glenodinium sanguineum. This paper describes the phenomenon of red colouration of the lake and discusses its conditions.

Efficient two-step digit-set-restricted modified signed-digit algorithm based on optoelectronic shared content-addressable memory

A two-step digit-set-restricted modified signed-digit (MSD) adder based on symbolic substitution is presented. In the proposed addition algorithm, carry propagation is avoided by using reference digits to restrict the intermediate MSD carry and sum digits into {(1) over bar ,0} and {0, 1}, respectively. The algorithm requires only 12 minterms to generate the final results, and no complementarity operations for nonzero outputs are involved, which simplifies the system complexity significantly. An optoelectronic shared content-addressable memory based on an incoherent correlator is used for experimental demonstration. (c) 2005 Society of Photo-Optical Instrumentation Engineers.

Efficient two-step digit-set-restricted modified signed-digit algorithm based on optoelectronic shared content-addressable memory

A two-step digit-set-restricted modified signed-digit (MSD) adder based on symbolic substitution is presented. In the proposed addition algorithm, carry propagation is avoided by using reference digits to restrict the intermediate MSD carry and sum digits into {(1) over bar ,0} and {0, 1}, respectively. The algorithm requires only 12 minterms to generate the final results, and no complementarity operations for nonzero outputs are involved, which simplifies the system complexity significantly. An optoelectronic shared content-addressable memory based on an incoherent correlator is used for experimental demonstration. (c) 2005 Society of Photo-Optical Instrumentation Engineers.

Diseño de un detector para una red de vigilancia radiológica ambiental

[ES]El objetivo de este trabajo es analizar el montaje de un contador proporcional de radiación para su uso en una red de vigilancia radiológica ambiental desde su diseño básico hasta su puesta en marcha. Se realizará un análisis de las razones que nos han llevado a elegir este tipo de contadores de ionización gaseosa (frente a otros tipos), para llegar a la conclusión de que son los más adecuados para la vigilancia radiológica ambiental. Este análisis describirá la solución a la que se ha llegado para construir un contador con los medios al alcance y el porqué de su diseño concreto, así como la metodología de trabajo que se ha seguido para la obtención del mismo. Basándonos en suposiciones teóricas derivadas del análisis geométrico de la instrumentación para la obtención de resultados previsibles, se realizará una verificación mediante ensayos técnicos en el laboratorio. Se presupuestará el material utilizado y las horas requeridas, para llevar una cuenta de lo que le ha costado al departamento y lo que le habría costado en caso de no poseer la instrumentación utilizada. Después de realizar diversos test relacionados con la instrumentación electrónica, se procederá a obtener unas conclusiones derivadas de los ensayos en el laboratorio para su posterior comparación con las estimaciones teóricas. En el anexo I se encuentran los planos de diseño que se han realizado y en el anexo II los resultados finales obtenidos de los ensayos.

Diseño y desarrollo de una interfaz de configuración de un generador de tráfico de red

[ES]Se trata de un software de gobierno para un inyector de tráfico de altas prestaciones capaz de adaptar el tráfico de red a distintos parámetros. Además, se deberán integrar el conjunto en la arquitectura de pruebas del laboratorio.