An anticipative reactive control strategy to deal with unforeseen obstacles during a multi-sensor-based navigation task

We address the problem of the rangefinder-based avoidance of unforeseen static obstacles during a visual navigation task. We extend previous strategies which are efficient in most cases but remain still hampered by some drawbacks (e.g., risks of collisions or of local minima in some particular cases, etc.). The key idea is to complete the control strategy by adding a controller providing the robot some anticipative skills to guarantee non collision and by defining more general transition conditions to deal with local minima. Simulation results show the proposed strategy efficiency.

Reactive Control for Transmission Overload Relief Based on Sensitivity Analysis and Cooperative Game Theory

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Hybrid robot control and SLAM for persistent navigation and mapping

For a mobile robot to operate autonomously in real-world environments, it must have an effective control system and a navigation system capable of providing robust localization, path planning and path execution. In this paper we describe the work investigating synergies between mapping and control systems. We have integrated development of a control system for navigating mobile robots and a robot SLAM system. The control system is hybrid in nature and tightly coupled with the SLAM system; it uses a combination of high and low level deliberative and reactive control processes to perform obstacle avoidance, exploration, global navigation and recharging, and draws upon the map learning and localization capabilities of the SLAM system. The effectiveness of this hybrid, multi-level approach was evaluated in the context of a delivery robot scenario. Over a period of two weeks the robot performed 1143 delivery tasks to 11 different locations with only one delivery failure (from which it recovered), travelled a total distance of more than 40km, and recharged autonomously a total of 23 times. In this paper we describe the combined control and SLAM system and discuss insights gained from its successful application in a real-world context.

Reactive reaching and grasping on a humanoid: Towards closing the action-perception loop on the iCub

We propose a system incorporating a tight integration between computer vision and robot control modules on a complex, high-DOF humanoid robot. Its functionality is showcased by having our iCub humanoid robot pick-up objects from a table in front of it. An important feature is that the system can avoid obstacles - other objects detected in the visual stream - while reaching for the intended target object. Our integration also allows for non-static environments, i.e. the reaching is adapted on-the-fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. Furthermore we show that this system can be used both in autonomous and tele-operation scenarios.

Design, specification, and synthesis of aircraft electric power systems control logic

Cyber-physical systems integrate computation, networking, and physical processes. Substantial research challenges exist in the design and verification of such large-scale, distributed sensing, ac- tuation, and control systems. Rapidly improving technology and recent advances in control theory, networked systems, and computer science give us the opportunity to drastically improve our approach to integrated flow of information and cooperative behavior. Current systems rely on text-based spec- ifications and manual design. Using new technology advances, we can create easier, more efficient, and cheaper ways of developing these control systems. This thesis will focus on design considera- tions for system topologies, ways to formally and automatically specify requirements, and methods to synthesize reactive control protocols, all within the context of an aircraft electric power system as a representative application area.

This thesis consists of three complementary parts: synthesis, specification, and design. The first section focuses on the synthesis of central and distributed reactive controllers for an aircraft elec- tric power system. This approach incorporates methodologies from computer science and control. The resulting controllers are correct by construction with respect to system requirements, which are formulated using the specification language of linear temporal logic (LTL). The second section addresses how to formally specify requirements and introduces a domain-specific language for electric power systems. A software tool automatically converts high-level requirements into LTL and synthesizes a controller.

The final sections focus on design space exploration. A design methodology is proposed that uses mixed-integer linear programming to obtain candidate topologies, which are then used to synthesize controllers. The discrete-time control logic is then verified in real-time by two methods: hardware and simulation. Finally, the problem of partial observability and dynamic state estimation is ex- plored. Given a set placement of sensors on an electric power system, measurements from these sensors can be used in conjunction with control logic to infer the state of the system.

Design and control of a pendulum driven hopping robot

In this paper a new kind of hopping robot has been designed which uses inverse pendulum dynamics to induce bipedal hopping gaits. Its mechanical structure consists of a rigid inverted T-shape mounted on four compliant feet. An upright "T" structure is connected to this by a rotary joint. The horizontal beam of the upright "T" is connected to the vertical beam by a second rotary joint. Using this two degree of freedom mechanical structure, with simple reactive control, the robot is able to perform hopping, walking and running gaits. During walking, it is experimentally shown that the robot can move in a straight line, reverse direction and control its turning radius. The results show that such a simple but versatile robot displays stable locomotion and can be viable for practical applications on uneven terrain.

Adaptive Neural Networks for Control of Movement Trajectories Invariant under Speed and Force Rescaling

This article describes two neural network modules that form part of an emerging theory of how adaptive control of goal-directed sensory-motor skills is achieved by humans and other animals. The Vector-Integration-To-Endpoint (VITE) model suggests how synchronous multi-joint trajectories are generated and performed at variable speeds. The Factorization-of-LEngth-and-TEnsion (FLETE) model suggests how outflow movement commands from a VITE model may be performed at variable force levels without a loss of positional accuracy. The invariance of positional control under speed and force rescaling sheds new light upon a familiar strategy of motor skill development: Skill learning begins with performance at low speed and low limb compliance and proceeds to higher speeds and compliances. The VITE model helps to explain many neural and behavioral data about trajectory formation, including data about neural coding within the posterior parietal cortex, motor cortex, and globus pallidus, and behavioral properties such as Woodworth's Law, Fitts Law, peak acceleration as a function of movement amplitude and duration, isotonic arm movement properties before and after arm-deafferentation, central error correction properties of isometric contractions, motor priming without overt action, velocity amplification during target switching, velocity profile invariance across different movement distances, changes in velocity profile asymmetry across different movement durations, staggered onset times for controlling linear trajectories with synchronous offset times, changes in the ratio of maximum to average velocity during discrete versus serial movements, and shared properties of arm and speech articulator movements. The FLETE model provides new insights into how spina-muscular circuits process variable forces without a loss of positional control. These results explicate the size principle of motor neuron recruitment, descending co-contractive compliance signals, Renshaw cells, Ia interneurons, fast automatic reactive control by ascending feedback from muscle spindles, slow adaptive predictive control via cerebellar learning using muscle spindle error signals to train adaptive movement gains, fractured somatotopy in the opponent organization of cerebellar learning, adaptive compensation for variable moment-arms, and force feedback from Golgi tendon organs. More generally, the models provide a computational rationale for the use of nonspecific control signals in volitional control, or "acts of will", and of efference copies and opponent processing in both reactive and adaptive motor control tasks.

Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Control reactivo de vehículo aéreo no tripulado para la competición internacional IARC

El trabajo está centrado en la construcción de una simulación y en el desarrollo de un control reactivo para un vehículo aéreo no tripulado con fin de participar en la séptima edición de la competición internacional IARC. Para cumplir los objetivos de la competición se van a estudiar técnicas existentes de inteligencia artificial aplicadas al control de vehículos aéreos no tripulados, así como las técnicas para la elaboración de un modelo de simulación realista sobre el que realizar las distintas pruebas. Por último, se explica el trabajo realizado para crear un controlador reactivo que satisface las reglas de la competición y permite al vehículo aéreo no tripulado operar de forma autónoma en el ambiente de la simulación. Para validar el comportamiento, se realizan casos de prueba y un estudio de los resultados.---ABSTRACT---This report is focused on the construction of a simulation and the development of a reactive control for an unmanned aerial vehicle in order to participate in the seventh edition of the international competition IARC. Artificial intelligence techniques applied to the control of unmanned aerial vehicles are going to be studied to meet the objectives of the competition, as well as techniques for developing a realistic simulation model on which to perform the different tests. Finally, the last part of the report explains the work accomplished to create a reactive controller that meets the rules of the competition and allows the unmanned aerial vehicle to operate autonomously in the simulation environment. Test cases and a study of the results is performed to validate the behavior.

The impact of cognitive control on children's goal monitoring in a time-based prospective memory task

Peer reviewed

Proactive and reactive inhibition during overt and covert actions. An electrical neuroimaging study.

Response inhibition is the ability to suppress inadequate but automatically activated, prepotent or ongoing response tendencies. In the framework of motor inhibition, two distinct operating strategies have been described: “proactive” and “reactive” control modes. In the proactive modality, inhibition is recruited in advance by predictive signals, and actively maintained before its enactment. Conversely, in the reactive control mode, inhibition is phasically enacted after the detection of the inhibitory signal. To date, ample evidence points to a core cerebral network for reactive inhibition comprising the right inferior frontal gyrus (rIFG), the presupplementary motor area (pre-SMA) and the basal ganglia (BG). Moreover, fMRI studies showed that cerebral activations during proactive and reactive inhibition largely overlap. These findings suggest that at least part of the neural network for reactive inhibition is recruited in advance, priming cortical regions in preparation for the upcoming inhibition. So far, proactive and reactive inhibitory mechanisms have been investigated during tasks in which the requested response to be stopped or withheld was an “overt” action execution (AE) (i.e., a movement effectively performed). Nevertheless, inhibitory mechanisms are also relevant for motor control during “covert actions” (i.e., potential motor acts not overtly performed), such as motor imagery (MI). MI is the conscious, voluntary mental rehearsal of action representations without any overt movement. Previous studies revealed a substantial overlap of activated motor-related brain networks in premotor, parietal and subcortical regions during overtly executed and imagined movements. Notwithstanding this evidence for a shared set of cerebral regions involved in encoding actions, whether or not those actions are effectively executed, the neural bases of motor inhibition during MI, preventing covert action from being overtly performed, in spite of the activation of the motor system, remain to be fully clarified. Taking into account this background, we performed a high density EEG study evaluating cerebral mechanisms and their related sources elicited during two types of cued Go/NoGo task, requiring the execution or withholding of an overt (Go) or a covert (MI) action, respectively. The EEG analyses were performed in two steps, with different aims: 1) Analysis of the “response phase” of the cued overt and covert Go/NoGo tasks, for the evaluation of reactive inhibitory control of overt and covert actions. 2) Analysis of the “preparatory phase” of the cued overt and covert Go/NoGo EEG datasets, focusing on cerebral activities time-locked to the preparatory signals, for the evaluation of proactive inhibitory mechanisms and their related neural sources. For these purposes, a spatiotemporal analysis of the scalp electric fields was applied on the EEG data recorded during the overt and covert Go/NoGo tasks. The spatiotemporal approach provide an objective definition of time windows for source analysis, relying on the statistical proof that the electric fields are different and thus generated by different neural sources. The analysis of the “response phase” revealed that key nodes of the inhibitory circuit, underpinning inhibition of the overt movement during the NoGo response, were also activated during the MI enactment. In both cases, inhibition relied on the activation of pre-SMA and rIFG, but with different temporal patterns of activation in accord with the intended “covert” or “overt” modality of motor performance. During the NoGo condition, the pre-SMA and rIFG were sequentially activated, pointing to an early decisional role of pre-SMA and to a later role of rIFG in the enactment of inhibitory control of the overt action. Conversely, a concomitant activation of pre-SMA and rIFG emerged during the imagined motor response. This latter finding suggested that an inhibitory mechanism (likely underpinned by the rIFG), could be prewired into a prepared “covert modality” of motor response, as an intrinsic component of the MI enactment. This mechanism would allow the rehearsal of the imagined motor representations, without any overt movement. The analyses of the “preparatory phase”, confirmed in both overt and covert Go/NoGo tasks the priming of cerebral regions pertaining to putative inhibitory network, reactively triggered in the following response phase. Nonetheless, differences in the preparatory strategies between the two tasks emerged, depending on the intended “overt” or “covert” modality of the possible incoming motor response. During the preparation of the overt Go/NoGo task, the cue primed the possible overt response programs in motor and premotor cortex. At the same time, through preactivation of a pre-SMA-related decisional mechanism, it triggered a parallel preparation for the successful response selection and/or inhibition during the subsequent response phase. Conversely, the preparatory strategy for the covert Go/NoGo task was centred on the goal-oriented priming of an inhibitory mechanism related to the rIFG that, being tuned to the instructed covert modality of the motor performance and instantiated during the subsequent MI enactment, allowed the imagined response to remain a potential motor act. Taken together, the results of the present study demonstrate a substantial overlap of cerebral networks activated during proactive recruitment and subsequent reactive enactment of motor inhibition in both overt and covert actions. At the same time, our data show that preparatory cues predisposed ab initio a different organization of the cerebral areas (in particular of the pre-SMA and rIFG) involved with sensorimotor transformations and motor inhibitory control for executed and imagined actions. During the preparatory phases of our cued overt and covert Go/NoGo tasks, the different adopted strategies were tuned to the “how” of the motor performance, reflecting the intended overt and covert modality of the possible incoming action.

Doubly Fed Induction Generator for wind energy generation using nine-switch power converter

A Three-Phase Nine-Switch Converter (NSC) topology for Doubly Fed Induction Generator in wind energy generation is proposed in this paper. This converter topology was used in various applications such as Hybrid Electric Vehicles and Uninterruptable Power Supplies. In this paper, Nine-Switch Converter is introduced in Doubly Fed Induction Generator in renewable energy application for the first time. It replaces the conventional Back-to-Back Pulse Width Modulated voltage source converter (VSC) which composed of twelve switches in many DFIG applications. Reduction in number of switches is the most beneficial in terms of cost and power switching losses. The operation principle of Nine-Switch Converter using SPWM method is discussed. The resulting NSC performance of rotor side current control, active power and reactive control are compared with Back-to Back voltage source converter performance. DC link voltage regulation using front end converter is also presented. Finally the simulation results of DFIG performances using NSC and Back-to-Back VSC are analyzed and compared.

Improving voltage profile of residential distribution systems using rooftop PVs and Battery Energy Storage systems

Large number of rooftop Photovoltaics (PVs) have turned traditional passive networks into active networks with intermittent and bidirectional power flow. A community based distribution network grid reinforcement process is proposed to address technical challenges associated with large integration of rooftop PVs. Probabilistic estimation of intermittent PV generation is considered. Depending on the network parameters such as the R/X ratio of distribution feeder, either reactive control from PVs or coordinated control of PVs and Battery Energy Storage (BES) has been proposed. Determination of BES capacity is one of the significant outcomes from the proposed method and several factors such as variation in PV installed capacity as well as participation from community members are analyzed. The proposed approach is convenient for the community members providing them flexibility of managing their integrated PV and BES systems

Curiosity driven reinforcement learning for motion planning on humanoids

Most previous work on artificial curiosity (AC) and intrinsic motivation focuses on basic concepts and theory. Experimental results are generally limited to toy scenarios, such as navigation in a simulated maze, or control of a simple mechanical system with one or two degrees of freedom. To study AC in a more realistic setting, we embody a curious agent in the complex iCub humanoid robot. Our novel reinforcement learning (RL) framework consists of a state-of-the-art, low-level, reactive control layer, which controls the iCub while respecting constraints, and a high-level curious agent, which explores the iCub's state-action space through information gain maximization, learning a world model from experience, controlling the actual iCub hardware in real-time. To the best of our knowledge, this is the first ever embodied, curious agent for real-time motion planning on a humanoid. We demonstrate that it can learn compact Markov models to represent large regions of the iCub's configuration space, and that the iCub explores intelligently, showing interest in its physical constraints as well as in objects it finds in its environment.

Motor learning.

Although learning a motor skill, such as a tennis stroke, feels like a unitary experience, researchers who study motor control and learning break the processes involved into a number of interacting components. These components can be organized into four main groups. First, skilled performance requires the effective and efficient gathering of sensory information, such as deciding where and when to direct one's gaze around the court, and thus an important component of skill acquisition involves learning how best to extract task-relevant information. Second, the performer must learn key features of the task such as the geometry and mechanics of the tennis racket and ball, the properties of the court surface, and how the wind affects the ball's flight. Third, the player needs to set up different classes of control that include predictive and reactive control mechanisms that generate appropriate motor commands to achieve the task goals, as well as compliance control that specifies, for example, the stiffness with which the arm holds the racket. Finally, the successful performer can learn higher-level skills such as anticipating and countering the opponent's strategy and making effective decisions about shot selection. In this Primer we shall consider these components of motor learning using as an example how we learn to play tennis.