Efficient methods for stochastic optimal control

The Hamilton Jacobi Bellman (HJB) equation is central to stochastic optimal control (SOC) theory, yielding the optimal solution to general problems specified by known dynamics and a specified cost functional. Given the assumption of quadratic cost on the control input, it is well known that the HJB reduces to a particular partial differential equation (PDE). While powerful, this reduction is not commonly used as the PDE is of second order, is nonlinear, and examples exist where the problem may not have a solution in a classical sense. Furthermore, each state of the system appears as another dimension of the PDE, giving rise to the curse of dimensionality. Since the number of degrees of freedom required to solve the optimal control problem grows exponentially with dimension, the problem becomes intractable for systems with all but modest dimension.

In the last decade researchers have found that under certain, fairly non-restrictive structural assumptions, the HJB may be transformed into a linear PDE, with an interesting analogue in the discretized domain of Markov Decision Processes (MDP). The work presented in this thesis uses the linearity of this particular form of the HJB PDE to push the computational boundaries of stochastic optimal control.

This is done by crafting together previously disjoint lines of research in computation. The first of these is the use of Sum of Squares (SOS) techniques for synthesis of control policies. A candidate polynomial with variable coefficients is proposed as the solution to the stochastic optimal control problem. An SOS relaxation is then taken to the partial differential constraints, leading to a hierarchy of semidefinite relaxations with improving sub-optimality gap. The resulting approximate solutions are shown to be guaranteed over- and under-approximations for the optimal value function. It is shown that these results extend to arbitrary parabolic and elliptic PDEs, yielding a novel method for Uncertainty Quantification (UQ) of systems governed by partial differential constraints. Domain decomposition techniques are also made available, allowing for such problems to be solved via parallelization and low-order polynomials.

The optimization-based SOS technique is then contrasted with the Separated Representation (SR) approach from the applied mathematics community. The technique allows for systems of equations to be solved through a low-rank decomposition that results in algorithms that scale linearly with dimensionality. Its application in stochastic optimal control allows for previously uncomputable problems to be solved quickly, scaling to such complex systems as the Quadcopter and VTOL aircraft. This technique may be combined with the SOS approach, yielding not only a numerical technique, but also an analytical one that allows for entirely new classes of systems to be studied and for stability properties to be guaranteed.

The analysis of the linear HJB is completed by the study of its implications in application. It is shown that the HJB and a popular technique in robotics, the use of navigation functions, sit on opposite ends of a spectrum of optimization problems, upon which tradeoffs may be made in problem complexity. Analytical solutions to the HJB in these settings are available in simplified domains, yielding guidance towards optimality for approximation schemes. Finally, the use of HJB equations in temporal multi-task planning problems is investigated. It is demonstrated that such problems are reducible to a sequence of SOC problems linked via boundary conditions. The linearity of the PDE allows us to pre-compute control policy primitives and then compose them, at essentially zero cost, to satisfy a complex temporal logic specification.

Convex Relaxation for Low-Dimensional Representation: Phase Transitions and Limitations

There is a growing interest in taking advantage of possible patterns and structures in data so as to extract the desired information and overcome the curse of dimensionality. In a wide range of applications, including computer vision, machine learning, medical imaging, and social networks, the signal that gives rise to the observations can be modeled to be approximately sparse and exploiting this fact can be very beneficial. This has led to an immense interest in the problem of efficiently reconstructing a sparse signal from limited linear observations. More recently, low-rank approximation techniques have become prominent tools to approach problems arising in machine learning, system identification and quantum tomography.

In sparse and low-rank estimation problems, the challenge is the inherent intractability of the objective function, and one needs efficient methods to capture the low-dimensionality of these models. Convex optimization is often a promising tool to attack such problems. An intractable problem with a combinatorial objective can often be "relaxed" to obtain a tractable but almost as powerful convex optimization problem. This dissertation studies convex optimization techniques that can take advantage of low-dimensional representations of the underlying high-dimensional data. We provide provable guarantees that ensure that the proposed algorithms will succeed under reasonable conditions, and answer questions of the following flavor:

More specifically, i) Focusing on linear inverse problems, we generalize the classical error bounds known for the least-squares technique to the lasso formulation, which incorporates the signal model. ii) We show that intuitive convex approaches do not perform as well as expected when it comes to signals that have multiple low-dimensional structures simultaneously. iii) Finally, we propose convex relaxations for the graph clustering problem and give sharp performance guarantees for a family of graphs arising from the so-called stochastic block model. We pay particular attention to the following aspects. For i) and ii), we aim to provide a general geometric framework, in which the results on sparse and low-rank estimation can be obtained as special cases. For i) and iii), we investigate the precise performance characterization, which yields the right constants in our bounds and the true dependence between the problem parameters.

Learning Multirobot Hose Transportation and Deployment by Distributed Round-Robin Q-Learning

Multi-Agent Reinforcement Learning (MARL) algorithms face two main difficulties: the curse of dimensionality, and environment non-stationarity due to the independent learning processes carried out by the agents concurrently. In this paper we formalize and prove the convergence of a Distributed Round Robin Q-learning (D-RR-QL) algorithm for cooperative systems. The computational complexity of this algorithm increases linearly with the number of agents. Moreover, it eliminates environment non sta tionarity by carrying a round-robin scheduling of the action selection and execution. That this learning scheme allows the implementation of Modular State-Action Vetoes (MSAV) in cooperative multi-agent systems, which speeds up learning convergence in over-constrained systems by vetoing state-action pairs which lead to undesired termination states (UTS) in the relevant state-action subspace. Each agent's local state-action value function learning is an independent process, including the MSAV policies. Coordination of locally optimal policies to obtain the global optimal joint policy is achieved by a greedy selection procedure using message passing. We show that D-RR-QL improves over state-of-the-art approaches, such as Distributed Q-Learning, Team Q-Learning and Coordinated Reinforcement Learning in a paradigmatic Linked Multi-Component Robotic System (L-MCRS) control problem: the hose transportation task. L-MCRS are over-constrained systems with many UTS induced by the interaction of the passive linking element and the active mobile robots.

Catalytic reduction of NO by CO with hydrotalcite derived mixed oxides

Catalysts with spinel structure derived from Hydrotalcite-like Compounds (HTLcs) containing cobalt have been investigated in NO catalytic reduction by Co. It was found that catalysts with spinel structures derived from HTLcs had obviously higher activity than that prepared from general methods. A two-step reaction was observed during the reaction curse: NO was first reduced to N2O by Co, and with the increase of temperature, the N2O was reduced to N-2. The reactivity of the catalysts studied increased with the amount of cobalt-content in the catalyst, and decreased with the calcination temperature. The crystal defect would play an important role in the reaction.

Clergy, crusified within the body of Christ - a challenge to pastoral care

M.A. Thesis / University of Pretoria / Department of Practical Theology / Advised by Prof M Masango

Tracking Human Body Pose on a Learned Smooth Space

Particle filtering is a popular method used in systems for tracking human body pose in video. One key difficulty in using particle filtering is caused by the curse of dimensionality: generally a very large number of particles is required to adequately approximate the underlying pose distribution in a high-dimensional state space. Although the number of degrees of freedom in the human body is quite large, in reality, the subset of allowable configurations in state space is generally restricted by human biomechanics, and the trajectories in this allowable subspace tend to be smooth. Therefore, a framework is proposed to learn a low-dimensional representation of the high-dimensional human poses state space. This mapping can be learned using a Gaussian Process Latent Variable Model (GPLVM) framework. One important advantage of the GPLVM framework is that both the mapping to, and mapping from the embedded space are smooth; this facilitates sampling in the low-dimensional space, and samples generated in the low-dimensional embedded space are easily mapped back into the original highdimensional space. Moreover, human body poses that are similar in the original space tend to be mapped close to each other in the embedded space; this property can be exploited when sampling in the embedded space. The proposed framework is tested in tracking 2D human body pose using a Scaled Prismatic Model. Experiments on real life video sequences demonstrate the strength of the approach. In comparison with the Multiple Hypothesis Tracking and the standard Condensation algorithm, the proposed algorithm is able to maintain tracking reliably throughout the long test sequences. It also handles singularity and self occlusion robustly.

The Harsh Reality of Pursuing Innovations: Emerging Market Perspectives

This short conference paper serves as a distillation of a keynote address delivered at the the Second National Conference on Management and Higher Education Trends & Strategies for Management & Administration hosted by Bangkok-based Stamford International University (Thailand) on November 1, 2014.Innovation is discussed as the heart of entrepreneurial processes occurring in today's capitalist economic systems, including transition economies like China and Vietnam, which underscores economic competitiveness of firms and economies. But the innovation effort and process also face dilemma of "entrepreneurial curse of innovation". Advantages and disadvantages are weighed for a more balanced view, especially in the context of outnumbering SMEs and given existence of pitfalls and traps along the innovation path of development. Toward the end, the value of the market is once again stressed amid the concern of subjective assumption and illusion about availability of market opportunities in the mind of innovators, which may contrast totally with the dismal outcome the actual market realities may show ex post.

A taxonomy for wavelet neural networks applied to nonlinear modelling

This article presents a novel classification of wavelet neural networks based on the orthogonality/non-orthogonality of neurons and the type of nonlinearity employed. On the basis of this classification different network types are studied and their characteristics illustrated by means of simple one-dimensional nonlinear examples. For multidimensional problems, which are affected by the curse of dimensionality, the idea of spherical wavelet functions is considered. The behaviour of these networks is also studied for modelling of a low-dimension map.

A fast method for fuzzy neural modeling and refinement

In the identification of complex dynamic systems using fuzzy neural networks, one of the main issues is the curse of dimensionality, which makes it difficult to retain a large number of system inputs or to consider a large number of fuzzy sets. Moreover, due to the correlations, not all possible network inputs or regression vectors in the network are necessary and adding them simply increases the model complexity and deteriorates the network generalisation performance. In this paper, the problem is solved by first proposing a fast algorithm for selection of network terms, and then introducing a refinement procedure to tackle the correlation issue. Simulation results show the efficacy of the method.

Rao-Blackwellised particle filter for colour-based tracking

Colour-based particle filters have been used exhaustively in the literature given rise to multiple applications However tracking coloured objects through time has an important drawback since the way in which the camera perceives the colour of the object can change Simple updates are often used to address this problem which imply a risk of distorting the model and losing the target In this paper a joint image characteristic-space tracking is proposed which updates the model simultaneously to the object location In order to avoid the curse of dimensionality a Rao-Blackwellised particle filter has been used Using this technique the hypotheses are evaluated depending on the difference between the model and the current target appearance during the updating stage Convincing results have been obtained in sequences under both sudden and gradual illumination condition changes Crown Copyright (C) 2010 Published by Elsevier B V All rights reserved

Romanticism and Blackwood's Magazine:'An Unprecedented Phenomenon'

Romanticism and Blackwood's Magazine is inspired by the ongoing critical fascination with Blackwood's Edinburgh Magazine, and the burgeoning recognition of its centrality to the Romantic age. Though the magazine itself was published continuously for well over a century and a half, this volume concentrates specifically on those years when William Blackwood was at the helm, beginning with his founding of the magazine in 1817 and closing with his death in 1834. These were the years when, as Samuel Taylor Coleridge put it in 1832, Blackwood's reigned as 'an unprecedented Phenomenon in the world of letters.' The magazine placed itself at the centre of the emerging mass media, commented decisively on all the major political and cultural issues that shaped the Romantic movement, and published some of the leading writers of the day, including Coleridge, Thomas De Quincey, John Galt, Felicia Hemans, James Hogg, Walter Scott, and Mary Shelley.

'This much-needed volume reminds us not only why Blackwood's was the most influential periodical publication of the time, but also how its writers, writings, and critical agendas continue to shape so many of the scholarly concerns of Romantic studies in the twenty-first century.' - Charles Mahoney, Associate Professor, University of Connecticut, USA

List of Illustrations
Acknowledgements
Abbreviations
Notes on Contributors
'A character so various, and yet so indisputably its own': A Passage to Blackwood's Edinburgh Magazine; R.Morrison & D.S.Roberts
PART I: BLACKWOOD'S AND THE PERIODICAL PRESS
Beginning Blackwood's: The Right Mix of Dulce and Ùtile; P.Flynn
John Gibson Lockhart and Blackwood's: Shaping the Romantic Periodical Press; T.Richardson
From Gluttony to Justified Sinning: Confessional Writing in Blackwood's and the London Magazine; D.Higgins
Camaraderie and Conflict: De Quincey and Wilson on Enemy Lines; R.Morrison
Selling Blackwood's Magazine, 1817-1834; D.Finkelstein
PART II: BLACKWOOD'S CULTURE AND CRITICISM
Blackwood's 'Personalities'; T.Mole
Communal Reception, Mary Shelley, and the 'Blackwood's School' of Criticism; N.Mason
Blackwoodian Allusion and the Culture of Miscellaneity; D.Stewart
Blackwood's Edinburgh Magazine in the Scientific Culture of Early Nineteenth-Century Edinburgh; W.Christie
The Art and Science of Politics in Blackwood's Edinburgh Magazine, c. 1817-1841; D.Kelly
Prosing Poetry: Blackwood's and Generic Transposition, 1820-1840; J.Camlot
PART III: BLACKWOOD'S FICTIONS
Blackwood's and the Boundaries of the Short Story; T.Killick
The Edinburgh of Blackwood's Edinburgh Magazine and James Hogg's Fiction; G.Hughes
'The Taste for Violence in Blackwood's Magazine'; M.Schoenfield
PART IV: BLACKWOOD'S AT HOME
John Wilson and Regency Authorship; R.Cronin
John Wilson and Sport; J.Strachan
William Maginn and the Blackwood's 'Preface' of 1826; D.E.Latané, Jr.
All Work and All Play: Felicia Hemans's Edinburgh Noctes; N.Sweet
PART V: BLACKWOOD'S ABROAD
Imagining India in Early Blackwood's; D.S.Roberts
Tales of the Colonies: Blackwood's, Provincialism, and British Interests Abroad; A.Jarrells
Selected Bibliography
Index

ROBERT MORRISON is Queen's National Scholar at Queen's University, Kingston, Ontario, Canada. His book, The English Opium-Eater: A Biography of Thomas De Quincey was a finalist for the James Tait Black Prize. He has edited writings by Jane Austen, Leigh Hunt, Thomas De Quincey, and John Polidori.
DANIEL SANJIV ROBERTS is Reader in English at Queen's University Belfast, UK. His publications include a monograph, Revisionary Gleam: De Quincey, Coleridge, and the High Romantic Argument (2000), and major critical editions of Thomas De Quincey's Autobiographic Sketches and Robert Southey's The Curse of Kehama; the latter was cited as a Distinguished Scholarly Edition by the MLA. He is currently working on an edition of Charles Johnstone's novel The History of Arsaces, Prince of Betlis for the Early Irish Fiction series.

Stigmatised Identity and Service Usage in Disadvantaged Communities: Residents’, community workers’ and service providers’ perspectives.

The impact of community stigmatisation upon service usage has been largely overlooked from a social identity perspective. Specifically, the social identity-mediated mechanisms by which stigmatisation hinders service use remain unspecified. The present study examines how service providers, community workers and residents recount their experience of the stigmatisation of local community identity and how this shapes residents’ uptake of welfare, education and community support services. Twenty individual and group interviews with 10 residents, 16 community workers and six statutory service providers in economically disadvantaged communities in Limerick, Ireland, were thematically analysed.Analysis indicates that statutory service providers endorsed negative stereotypes of disadvantaged areas as separate and anti-social. The awareness of this perceived division and the experience of ‘stigma consciousness’ was reported by residents and community workers to undermine trust, leading to under-utilisation of community and government services. We argue that stigmatisation acts as a ‘social curse’ by undermining shared identity between service users and providers and so turning a potentially cooperative intragroup relationship into a fraught intergroup one. We suggest that tackling stigma in order to foster a sense of shared identity is important in creating positive and cooperative service interactions for both service users and providers.

Covariance Averaging for Improved Estimation and Portfolio Allocation

We propose a new method for estimating the covariance matrix of a multivariate time series of nancial returns. The method is based on estimating sample covariances from overlapping windows of observations which are then appropriately weighted to obtain the nal covariance estimate. We extend the idea of (model) covariance averaging o ered in the covariance shrinkage approach by means of greater ease of use, exibility and robustness in averaging information over different data segments. The suggested approach does not su er from the curse of dimensionality and can be used without problems of either approximation or any demand for numerical optimization.

Varieties of resource nationalism in sub-Saharan Africa’s energy and minerals markets

This article examines resource nationalism in sub-Saharan Africa's energy and minerals markets. It does so by exploring economic and political developments in three cases: Nigeria as an example of a petro-state established by means of expropriation in the wake of decolonisation; South Africa, a mature mining industry shaped by its settler colonial history; and Mozambique, a new and therefore highly-dependent entrant into the league of significant natural gas producers. Extractive industries have played a controversial role in sub-Saharan Africa due in particular to the prevalence of the resource curse. Nevertheless, energy exports will continue to play an important role in fuelling economic growth and, potentially, also development as new deposits of natural gas and oil are discovered across the region. Resource nationalism has, moreover, increasingly constrained operations of the traditionally dominant Western energy companies, in particular as competition from state-owned energy companies in sub-Saharan Africa and from emerging powers such as China is increasing.