Single network adaptive critic aided dynamic inversion for optimal regulation and command tracking with online adaptation for enhanced robustness

To combine the advantages of both stability and optimality-based designs, a single network adaptive critic (SNAC) aided nonlinear dynamic inversion approach is presented in this paper. Here, the gains of a dynamic inversion controller are selected in such a way that the resulting controller behaves very close to a pre-synthesized SNAC controller in the output regulation sense. Because SNAC is based on optimal control theory, it makes the dynamic inversion controller operate nearly optimal. More important, it retains the two major benefits of dynamic inversion, namely (i) a closed-form expression of the controller and (ii) easy scalability to command tracking applications without knowing the reference commands a priori. An extended architecture is also presented in this paper that adapts online to system modeling and inversion errors, as well as reduced control effectiveness, thereby leading to enhanced robustness. The strengths of this hybrid method of applying SNAC to optimize an nonlinear dynamic inversion controller is demonstrated by considering a benchmark problem in robotics, that is, a two-link robotic manipulator system. Copyright (C) 2013 John Wiley & Sons, Ltd.

Reinforcement learning using a continuous time actor-critic framework with spiking neurons.

Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modulated spike-timing-dependent plasticity have made first steps towards bridging the gap between the two approaches, but faced two problems. First, reinforcement learning is typically formulated in a discrete framework, ill-adapted to the description of natural situations. Second, biologically plausible models of reward-modulated spike-timing-dependent plasticity require precise calculation of the reward prediction error, yet it remains to be shown how this can be computed by neurons. Here we propose a solution to these problems by extending the continuous temporal difference (TD) learning of Doya (2000) to the case of spiking neurons in an actor-critic network operating in continuous time, and with continuous state and action representations. In our model, the critic learns to predict expected future rewards in real time. Its activity, together with actual rewards, conditions the delivery of a neuromodulatory TD signal to itself and to the actor, which is responsible for action choice. In simulations, we show that such an architecture can solve a Morris water-maze-like navigation task, in a number of trials consistent with reported animal performance. We also use our model to solve the acrobot and the cartpole problems, two complex motor control tasks. Our model provides a plausible way of computing reward prediction error in the brain. Moreover, the analytically derived learning rule is consistent with experimental evidence for dopamine-modulated spike-timing-dependent plasticity.

An Actor-Critic based controller for glucose regulation in type 1 diabetes

A novel adaptive approach for glucose control in individuals with type 1 diabetes under sensor-augmented pump therapy is proposed. The controller, is based on Actor-Critic (AC) learning and is inspired by the principles of reinforcement learning and optimal control theory. The main characteristics of the proposed controller are (i) simultaneous adjustment of both the insulin basal rate and the bolus dose, (ii) initialization based on clinical procedures, and (iii) real-time personalization. The effectiveness of the proposed algorithm in terms of glycemic control has been investigated in silico in adults, adolescents and children under open-loop and closed-loop approaches, using announced meals with uncertainties in the order of ±25% in the estimation of carbohydrates. The results show that glucose regulation is efficient in all three groups of patients, even with uncertainties in the level of carbohydrates in the meal. The percentages in the A+B zones of the Control Variability Grid Analysis (CVGA) were 100% for adults, and 93% for both adolescents and children. The AC based controller seems to be a promising approach for the automatic adjustment of insulin infusion in order to improve glycemic control. After optimization of the algorithm, the controller will be tested in a clinical trial.

Harry Potter and the Theory of Things

This thesis investigates the boundaries between body and object in J.K. Rowling’s Harry Potter series, seven children’s literature novels published between 1997 and 2007. Lord Voldemort, Rowling’s villain, creates Horcruxes—objects that contain fragments of his soul—in order to ensure his immortality. As vessels for human soul, these objects rupture the boundaries between body and object and become “things.” Using contemporary thing theorists including John Plotz and materialists Jean Baudrillard and Walter Benjamin, I look at Voldemort’s Horcruxes as transgressive, liminal, unclassifiable entities in the first chapter. If objects can occupy the juncture between body and object, then bodies can as well. Dementors and Inferi, dark creatures that Rowling introduces throughout the series, live devoid of soul. Voldemort, too, becomes a thing as he splits his soul and creates Horcruxes. These soulless bodies are uncanny entities, provoking fear, revulsion, nausea, and the loss of language. In the second chapter, I use Sigmund Freud’s theorization of the uncanny as well as literary critic Kelly Hurley to investigate how Dementors, Inferi, and Voldemort exist as body-turned-object things at the juncture between life and death. As Voldemort increasingly invests his immaterial soul into material objects, he physically and spiritually degenerates, transforming from the young, handsome Tom Marvolo Riddle into the snake-like villain that murdered Harry’s parents and countless others. During his quest to find and destroy Voldemort’s Horcruxes, Harry encounters a different type of object, the Deathly Hallows. Although similarly accessing boundaries between body/object, life/death, and materiality/immateriality, the three Deathly Hallows do not transgress these boundaries. Through the Deathly Hallows, Rowling provides an alternative to thingification: objects that enable boundaries to fluctuate, but not breakdown. In the third chapter, I return to thing theorists, Baudrillard, and Benjamin to study how the Deathly Hallows resist thingification by not transgressing the boundaries between body and object.

The history of theory

Do you see now why it feels so good to be a critical mind? Why critique, this most ambiguous pharmakon, has become such a potent euphoric drug? You are always right! When naïve believers are clinging forcefully to their objects ... you can turn all of those attachments into so many fetishes and humiliate all the believers by showing that it is nothing but their own projection, that you, yes you alone, can see. But as soon as naïve believers are thus inflated by some belief in their own importance, in their own projective capacity, you strike them by a second uppercut and humiliate them again, this time by showing that, whatever they think, their behavior is entirely determined by the action of powerful causalities coming from objective reality they don't see, but that you, yes you, the never sleeping critic, alone can see. Isn't this fabulous? Isn't it really worth going to graduate school to study critique?

A linguistic and semantic approach to the theory and practice of literary translation

This work explores the relevance of semantic and linguistic description to translation, theory and practice. It is aimed towards a practical model of approach to texts to translate. As literary texts [poetry mainly] are the focus of attention, so are stylistic matters. Note, however, that 'style', and, to some extent, the conclusions of the work, are not limited to so-called literary texts. The study of semantic description reveals that most translation problems do not stem from the cognitive (langue-related), but rather from the contextual (parole-related) aspects of meaning. Thus, any linguistic model that fails to account for the latter is bound to fall short. T.G.G. does, whereas Systemics, concerned with both the 'Iangue' and 'parole' (stylistic and sociolinguistic mainly) aspects of meaning, provides a useful framework of approach to texts to translate. Two essential semantic principles for translation are: that meaning is the property of a language (Firth); and the 'relativity of meaning assignments' (Tymoczko). Both imply that meaning can only be assessed, correctly, in the relevant socio-cultural background. Translation is seen as a restricted creation, and the translator's encroach as a three-dimensional critical one. To encompass the most technical to the most literary text, and account for variations in emphasis in any text, translation theory must be based on typology of function Halliday's ideational, interpersonal and textual, or, Buhler's symbol, signal, symptom, Functions3. Function Coverall and specific] will dictate aims and method, and also provide the critic with criteria to assess translation Faithfulness. Translation can never be reduced to purely objective methods, however. Intuitive procedures intervene, in textual interpretation and analysis, in the choice of equivalents, and in the reception of a translation. Ultimately, translation, theory and practice, may perhaps constitute the touchstone as regards the validity of linguistic and semantic theories.

Fully probabilistic control design in an adaptive critic framework

Optimal stochastic controller pushes the closed-loop behavior as close as possible to the desired one. The fully probabilistic design (FPD) uses probabilistic description of the desired closed loop and minimizes Kullback-Leibler divergence of the closed-loop description to the desired one. Practical exploitation of the fully probabilistic design control theory continues to be hindered by the computational complexities involved in numerically solving the associated stochastic dynamic programming problem. In particular very hard multivariate integration and an approximate interpolation of the involved multivariate functions. This paper proposes a new fully probabilistic contro algorithm that uses the adaptive critic methods to circumvent the need for explicitly evaluating the optimal value function, thereby dramatically reducing computational requirements. This is a main contribution of this short paper.

Feminist Disability Theory: Domestic Violence Against Women with a Disability

Women with a disability continue to experience social oppression and domestic violence as a consequence of gender and disability dimensions. Current explanations of domestic violence and disability inadequately explain several features that lead women who have a disability to experience violent situations. This article incorporates both disability and material feminist theory as an alternative explanation to the dominant approaches (psychological and sociological traditions) of conceptualising domestic violence. This paper is informed by a study which was concerned with examining the nature and perceptions of violence against women with a physical impairment. The emerging analytical framework integrating material feminist interpretations and disability theory provided a basis for exploring gender and disability dimensions. Insight was also provided by the women who identified as having a disability in the study and who explained domestic violence in terms of a gendered and disabling experience. The article argues that material feminist interpretations and disability theory, with their emphasis on gender relations, disablism and poverty, should be used as an alternative tool for exploring the nature and consequences of violence against women with a disability.