Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Where do these bugs come from? Phenotypic structure ofTriatoma infestans populations after control interventions in the Argentine Chaco

House re-invasion by native triatomines after insecticide-based control campaigns represents a major threat for Chagas disease vector control. We conducted a longitudinal intervention study in a rural section (Area III, 407 houses) of Pampa del Indio, northeastern Argentina, and used wing geometric morphometry to compare pre-spray and post-spray (re-infestant bugs) Triatoma infestanspopulations. The community-wide spraying with pyrethroids reduced the prevalence of house infestation by T. infestans from 31.9% to < 1% during a four-year follow-up, unlike our previous studies in the neighbouring Area I. Two groups of bug collection sites differing in wing shape variables before interventions (including 221 adults from 11 domiciles) were used as a reference for assigning 44 post-spray adults. Wing shape variables from post-spray, high-density bug colonies and pre-spray groups were significantly different, suggesting that re-infestant insects had an external origin. Insects from one house differed strongly in wing shape variables from all other specimens. A further comparison between insects from both areas supported the existence of independent re-infestation processes within the same district. These results point to local heterogeneities in house re-infestation dynamics and emphasise the need to expand the geographic coverage of vector surveillance and control operations to the affected region.

Modeling and control of electromechanical systems

The material presented in the these notes covers the sessions Modelling of electromechanical systems, Passive control theory I and Passive control theory II of the II EURON/GEOPLEX Summer School on Modelling and Control of Complex Dynamical Systems.We start with a general description of what an electromechanical system is from a network modelling point of view. Next, a general formulation in terms of PHDS is introduced, and some of the previous electromechanical systems are rewritten in this formalism. Power converters, which are variable structure systems (VSS), can also be given a PHDS form.We conclude the modelling part of these lectures with a rather complex example, showing the interconnection of subsystems from several domains, namely an arrangement to temporally store the surplus energy in a section of a metropolitan transportation system based on dc motor vehicles, using either arrays of supercapacitors or an electric poweredflywheel. The second part of the lectures addresses control of PHD systems. We first present the idea of control as power connection of a plant and a controller. Next we discuss how to circumvent this obstacle and present the basic ideas of Interconnection and Damping Assignment (IDA) passivity-based control of PHD systems.

Contextual factors multiplex to control multisensory processes.

This study analyzed high-density event-related potentials (ERPs) within an electrical neuroimaging framework to provide insights regarding the interaction between multisensory processes and stimulus probabilities. Specifically, we identified the spatiotemporal brain mechanisms by which the proportion of temporally congruent and task-irrelevant auditory information influences stimulus processing during a visual duration discrimination task. The spatial position (top/bottom) of the visual stimulus was indicative of how frequently the visual and auditory stimuli would be congruent in their duration (i.e., context of congruence). Stronger influences of irrelevant sound were observed when contexts associated with a high proportion of auditory-visual congruence repeated and also when contexts associated with a low proportion of congruence switched. Context of congruence and context transition resulted in weaker brain responses at 228 to 257 ms poststimulus to conditions giving rise to larger behavioral cross-modal interactions. Importantly, a control oddball task revealed that both congruent and incongruent audiovisual stimuli triggered equivalent non-linear multisensory interactions when congruence was not a relevant dimension. Collectively, these results are well explained by statistical learning, which links a particular context (here: a spatial location) with a certain level of top-down attentional control that further modulates cross-modal interactions based on whether a particular context repeated or changed. The current findings shed new light on the importance of context-based control over multisensory processing, whose influences multiplex across finer and broader time scales.

Novel force control methods in a teleoperation system of a hydraulic slider

The objective of the this research project is to develop a novel force control scheme for the teleoperation of a hydraulically driven manipulator, and to implement an ideal transparent mapping between human and machine interaction, and machine and task environment interaction. This master‘s thesis provides a preparatory study for the present research project. The research is limited into a single degree of freedom hydraulic slider with 6-DOF Phantom haptic device. The key contribution of the thesis is to set up the experimental rig including electromechanical haptic device, hydraulic servo and 6-DOF force sensor. The slider is firstly tested as a position servo by using previously developed intelligent switching control algorithm. Subsequently the teleoperated system is set up and the preliminary experiments are carried out. In addition to development of the single DOF experimental set up, methods such as passivity control in teleoperation are reviewed. The thesis also contains review of modeling of the servo slider in particular reference to the servo valve. Markov Chain Monte Carlo method is utilized in developing the robustness of the model in presence of noise.

Doubly fed wind turbine performance in variable grid conditions

Wind turbines based on doubly fed induction generators (DFIG) become the most popular solution in high power wind generation industry. While this topology provides great performance with the reduced power rating of power converter, it has more complicated structure in comparison with full-rated topologies, and therefore leads to complexity of control algorithms and electromechanical processes in the system. The purpose of presented study is to present a proper vector control scheme for the DFIG and overall control for the WT to investigate its behavior at different wind speeds and in different grid voltage conditions: voltage sags, magnitude and frequency variations. The key principles of variable-speed wind turbine were implemented in simulation model and demonstrated during the study. Then, based on developed control scheme and mathematical model, the set of simulation is made to analyze reactive power capabilities of the DFIG wind turbine. Further, the rating of rotor-side converter is modified to not only generate active rated active power, but also to fulfill Grid Codes. Results of modelling and analyzing of the DFIG WT behavior under different speeds and different voltage conditions are presented in the work.

Problem solving and computer-assisted instruction in science education : analysis of research findings and the research process /

The quantitative component of this study examined the effect of computerassisted instruction (CAI) on science problem-solving performance, as well as the significance of logical reasoning ability to this relationship. I had the dual role of researcher and teacher, as I conducted the study with 84 grade seven students to whom I simultaneously taught science on a rotary-basis. A two-treatment research design using this sample of convenience allowed for a comparison between the problem-solving performance of a CAI treatment group (n = 46) versus a laboratory-based control group (n = 38). Science problem-solving performance was measured by a pretest and posttest that I developed for this study. The validity of these tests was addressed through critical discussions with faculty members, colleagues, as well as through feedback gained in a pilot study. High reliability was revealed between the pretest and the posttest; in this way, students who tended to score high on the pretest also tended to score high on the posttest. Interrater reliability was found to be high for 30 randomly-selected test responses which were scored independently by two raters (i.e., myself and my faculty advisor). Results indicated that the form of computer-assisted instruction (CAI) used in this study did not significantly improve students' problem-solving performance. Logical reasoning ability was measured by an abbreviated version of the Group Assessment of Lx)gical Thinking (GALT). Logical reasoning ability was found to be correlated to problem-solving performance in that, students with high logical reasoning ability tended to do better on the problem-solving tests and vice versa. However, no significant difference was observed in problem-solving improvement, in the laboratory-based instruction group versus the CAI group, for students varying in level of logical reasoning ability.Insignificant trends were noted in results obtained from students of high logical reasoning ability, but require further study. It was acknowledged that conclusions drawn from the quantitative component of this study were limited, as further modifications of the tests were recommended, as well as the use of a larger sample size. The purpose of the qualitative component of the study was to provide a detailed description ofmy thesis research process as a Brock University Master of Education student. My research journal notes served as the data base for open coding analysis. This analysis revealed six main themes which best described my research experience: research interests, practical considerations, research design, research analysis, development of the problem-solving tests, and scoring scheme development. These important areas ofmy thesis research experience were recounted in the form of a personal narrative. It was noted that the research process was a form of problem solving in itself, as I made use of several problem-solving strategies to achieve desired thesis outcomes.

Gestion adaptative des ressources dans les réseaux maillés sans fil à multiples-radios multiples-canaux

Depuis quelques années, la recherche dans le domaine des réseaux maillés sans fil ("Wireless Mesh Network (WMN)" en anglais) suscite un grand intérêt auprès de la communauté des chercheurs en télécommunications. Ceci est dû aux nombreux avantages que la technologie WMN offre, telles que l'installation facile et peu coûteuse, la connectivité fiable et l'interopérabilité flexible avec d'autres réseaux existants (réseaux Wi-Fi, réseaux WiMax, réseaux cellulaires, réseaux de capteurs, etc.). Cependant, plusieurs problèmes restent encore à résoudre comme le passage à l'échelle, la sécurité, la qualité de service (QdS), la gestion des ressources, etc. Ces problèmes persistent pour les WMNs, d'autant plus que le nombre des utilisateurs va en se multipliant. Il faut donc penser à améliorer les protocoles existants ou à en concevoir de nouveaux. L'objectif de notre recherche est de résoudre certaines des limitations rencontrées à l'heure actuelle dans les WMNs et d'améliorer la QdS des applications multimédia temps-réel (par exemple, la voix). Le travail de recherche de cette thèse sera divisé essentiellement en trois principaux volets: le contrôle d‟admission du trafic, la différentiation du trafic et la réaffectation adaptative des canaux lors de la présence du trafic en relève ("handoff" en anglais). Dans le premier volet, nous proposons un mécanisme distribué de contrôle d'admission se basant sur le concept des cliques (une clique correspond à un sous-ensemble de liens logiques qui interfèrent les uns avec les autres) dans un réseau à multiples-sauts, multiples-radios et multiples-canaux, appelé RCAC. Nous proposons en particulier un modèle analytique qui calcule le ratio approprié d'admission du trafic et qui garantit une probabilité de perte de paquets dans le réseau n'excédant pas un seuil prédéfini. Le mécanisme RCAC permet d‟assurer la QdS requise pour les flux entrants, sans dégrader la QdS des flux existants. Il permet aussi d‟assurer la QdS en termes de longueur du délai de bout en bout pour les divers flux. Le deuxième volet traite de la différentiation de services dans le protocole IEEE 802.11s afin de permettre une meilleure QdS, notamment pour les applications avec des contraintes temporelles (par exemple, voix, visioconférence). À cet égard, nous proposons un mécanisme d'ajustement de tranches de temps ("time-slots"), selon la classe de service, ED-MDA (Enhanced Differentiated-Mesh Deterministic Access), combiné à un algorithme efficace de contrôle d'admission EAC (Efficient Admission Control), afin de permettre une utilisation élevée et efficace des ressources. Le mécanisme EAC prend en compte le trafic en relève et lui attribue une priorité supérieure par rapport au nouveau trafic pour minimiser les interruptions de communications en cours. Dans le troisième volet, nous nous intéressons à minimiser le surcoût et le délai de re-routage des utilisateurs mobiles et/ou des applications multimédia en réaffectant les canaux dans les WMNs à Multiples-Radios (MR-WMNs). En premier lieu, nous proposons un modèle d'optimisation qui maximise le débit, améliore l'équité entre utilisateurs et minimise le surcoût dû à la relève des appels. Ce modèle a été résolu par le logiciel CPLEX pour un nombre limité de noeuds. En second lieu, nous élaborons des heuristiques/méta-heuristiques centralisées pour permettre de résoudre ce modèle pour des réseaux de taille réelle. Finalement, nous proposons un algorithme pour réaffecter en temps-réel et de façon prudente les canaux aux interfaces. Cet algorithme a pour objectif de minimiser le surcoût et le délai du re-routage spécialement du trafic dynamique généré par les appels en relève. Ensuite, ce mécanisme est amélioré en prenant en compte l‟équilibrage de la charge entre cliques.

Gestion des ressources dans les réseaux cellulaires sans fil

L’émergence de nouvelles applications et de nouveaux services (tels que les applications multimédias, la voix-sur-IP, la télévision-sur-IP, la vidéo-sur-demande, etc.) et le besoin croissant de mobilité des utilisateurs entrainent une demande de bande passante de plus en plus croissante et une difficulté dans sa gestion dans les réseaux cellulaires sans fil (WCNs), causant une dégradation de la qualité de service. Ainsi, dans cette thèse, nous nous intéressons à la gestion des ressources, plus précisément à la bande passante, dans les WCNs. Dans une première partie de la thèse, nous nous concentrons sur la prédiction de la mobilité des utilisateurs des WCNs. Dans ce contexte, nous proposons un modèle de prédiction de la mobilité, relativement précis qui permet de prédire la destination finale ou intermédiaire et, par la suite, les chemins des utilisateurs mobiles vers leur destination prédite. Ce modèle se base sur : (a) les habitudes de l’utilisateur en terme de déplacements (filtrées selon le type de jour et le moment de la journée) ; (b) le déplacement courant de l’utilisateur ; (c) la connaissance de l’utilisateur ; (d) la direction vers une destination estimée ; et (e) la structure spatiale de la zone de déplacement. Les résultats de simulation montrent que ce modèle donne une précision largement meilleure aux approches existantes. Dans la deuxième partie de cette thèse, nous nous intéressons au contrôle d’admission et à la gestion de la bande passante dans les WCNs. En effet, nous proposons une approche de gestion de la bande passante comprenant : (1) une approche d’estimation du temps de transfert intercellulaire prenant en compte la densité de la zone de déplacement en terme d’utilisateurs, les caractéristiques de mobilité des utilisateurs et les feux tricolores ; (2) une approche d’estimation de la bande passante disponible à l’avance dans les cellules prenant en compte les exigences en bande passante et la durée de vie des sessions en cours ; et (3) une approche de réservation passive de bande passante dans les cellules qui seront visitées pour les sessions en cours et de contrôle d’admission des demandes de nouvelles sessions prenant en compte la mobilité des utilisateurs et le comportement des cellules. Les résultats de simulation indiquent que cette approche réduit largement les ruptures abruptes de sessions en cours, offre un taux de refus de nouvelles demandes de connexion acceptable et un taux élevé d’utilisation de la bande passante. Dans la troisième partie de la thèse, nous nous penchons sur la principale limite de la première et deuxième parties de la thèse, à savoir l’évolutivité (selon le nombre d’utilisateurs) et proposons une plateforme qui intègre des modèles de prédiction de mobilité avec des modèles de prédiction de la bande passante disponible. En effet, dans les deux parties précédentes de la thèse, les prédictions de la mobilité sont effectuées pour chaque utilisateur. Ainsi, pour rendre notre proposition de plateforme évolutive, nous proposons des modèles de prédiction de mobilité par groupe d’utilisateurs en nous basant sur : (a) les profils des utilisateurs (c’est-à-dire leur préférence en termes de caractéristiques de route) ; (b) l’état du trafic routier et le comportement des utilisateurs ; et (c) la structure spatiale de la zone de déplacement. Les résultats de simulation montrent que la plateforme proposée améliore la performance du réseau comparée aux plateformes existantes qui proposent des modèles de prédiction de la mobilité par groupe d’utilisateurs pour la réservation de bande passante.

A Framework for TEI-Based Scholarly Text Editions

In this paper, we describe an interdisciplinary project in which visualization techniques were developed for and applied to scholarly work from literary studies. The aim was to bring Christof Schöch's electronic edition of Bérardier de Bataut's Essai sur le récit (1776) to the web. This edition is based on the Text Encoding Initiative's XML-based encoding scheme (TEI P5, subset TEI-Lite). This now de facto standard applies to machine-readable texts used chiefly in the humanities and social sciences. The intention of this edition is to make the edited text freely available on the web, to allow for alternative text views (here original and modern/corrected text), to ensure reader-friendly annotation and navigation, to permit on-line collaboration in encoding and annotation as well as user comments, all in an open source, generically usable, lightweight package. These aims were attained by relying on a GPL-based, public domain CMS (Drupal) and combining it with XSL-Stylesheets and Java Script.

Virtual Model Control of a Biped Walking Robot

The transformation from high level task specification to low level motion control is a fundamental issue in sensorimotor control in animals and robots. This thesis develops a control scheme called virtual model control which addresses this issue. Virtual model control is a motion control language which uses simulations of imagined mechanical components to create forces, which are applied through joint torques, thereby creating the illusion that the components are connected to the robot. Due to the intuitive nature of this technique, designing a virtual model controller requires the same skills as designing the mechanism itself. A high level control system can be cascaded with the low level virtual model controller to modulate the parameters of the virtual mechanisms. Discrete commands from the high level controller would then result in fluid motion. An extension of Gardner's Partitioned Actuator Set Control method is developed. This method allows for the specification of constraints on the generalized forces which each serial path of a parallel mechanism can apply. Virtual model control has been applied to a bipedal walking robot. A simple algorithm utilizing a simple set of virtual components has successfully compelled the robot to walk eight consecutive steps.

Virtual Model Control of a Hexapod Walking Robot

Since robots are typically designed with an individual actuator at each joint, the control of these systems is often difficult and non-intuitive. This thesis explains a more intuitive control scheme called Virtual Model Control. This thesis also demonstrates the simplicity and ease of this control method by using it to control a simulated walking hexapod. Virtual Model Control uses imagined mechanical components to create virtual forces, which are applied through the joint torques of real actuators. This method produces a straightforward means of controlling joint torques to produce a desired robot behavior. Due to the intuitive nature of this control scheme, the design of a virtual model controller is similar to the design of a controller with basic mechanical components. The ease of this control scheme facilitates the use of a high level control system which can be used above the low level virtual model controllers to modulate the parameters of the imaginary mechanical components. In order to apply Virtual Model Control to parallel mechanisms, a solution to the force distribution problem is required. This thesis uses an extension of Gardner`s Partitioned Force Control method which allows for the specification of constrained degrees of freedom. This virtual model control technique was applied to a simulated hexapod robot. Although the hexapod is a highly non-linear, parallel mechanism, the virtual models allowed text-book control solutions to be used while the robot was walking. Using a simple linear control law, the robot walked while simultaneously balancing a pendulum and tracking an object.

Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Can wavelets improve the representation of forecast error covariances in variational data assimilation?

Two wavelet-based control variable transform schemes are described and are used to model some important features of forecast error statistics for use in variational data assimilation. The first is a conventional wavelet scheme and the other is an approximation of it. Their ability to capture the position and scale-dependent aspects of covariance structures is tested in a two-dimensional latitude-height context. This is done by comparing the covariance structures implied by the wavelet schemes with those found from the explicit forecast error covariance matrix, and with a non-wavelet- based covariance scheme used currently in an operational assimilation scheme. Qualitatively, the wavelet-based schemes show potential at modeling forecast error statistics well without giving preference to either position or scale-dependent aspects. The degree of spectral representation can be controlled by changing the number of spectral bands in the schemes, and the least number of bands that achieves adequate results is found for the model domain used. Evidence is found of a trade-off between the localization of features in positional and spectral spaces when the number of bands is changed. By examining implied covariance diagnostics, the wavelet-based schemes are found, on the whole, to give results that are closer to diagnostics found from the explicit matrix than from the nonwavelet scheme. Even though the nature of the covariances has the right qualities in spectral space, variances are found to be too low at some wavenumbers and vertical correlation length scales are found to be too long at most scales. The wavelet schemes are found to be good at resolving variations in position and scale-dependent horizontal length scales, although the length scales reproduced are usually too short. The second of the wavelet-based schemes is often found to be better than the first in some important respects, but, unlike the first, it has no exact inverse transform.

Impedance control of redundant drive joints with double actuation

This paper proposes impedance control of redundant drive joints with double actuation (RDJ-DA) to produce compliant motions with the future goal of higher bandwidth. First, to reduce joint inertia, a double-input-single-output mechanism with one internal degree of freedom (DOF) is presented as part of the basic structure of the RDJ-DA. Next, the basic structure of RDJ-DA is further explained and its dynamics and statics are derived. Then, the impedance control scheme of RDJ-DA to produce compliant motions is proposed and the validity of the proposed controller is investigated using numerical examples.