Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Comprehensive evaluation of the IEEE 802.15.4 MAC layer performance with retransmissions

Supported by IEEE 802.15.4 standardization activities, embedded networks have been gaining popularity in recent years. The focus of this paper is to quantify the behavior of key networking metrics of IEEE 802.15.4 beacon-enabled nodes under typical operating conditions, with the inclusion of packet retransmissions. We corrected and extended previous analyses by scrutinizing the assumptions on which the prevalent Markovian modeling is generally based. By means of a comparative study, we singled out which of the assumptions impact each of the performance metrics (throughput, delay, power consumption, collision probability, and packet-discard probability). In particular, we showed that - unlike what is usually assumed - the probability that a node senses the channel busy is not constant for all the stages of the backoff procedure and that these differences have a noticeable impact on backoff delay, packet-discard probability, and power consumption. Similarly, we showed that - again contrary to common assumption - the probability of obtaining transmission access to the channel depends on the number of nodes that is simultaneously sensing it. We evidenced that ignoring this dependence has a significant impact on the calculated values of throughput and collision probability. Circumventing these and other assumptions, we rigorously characterize, through a semianalytical approach, the key metrics in a beacon-enabled IEEE 802.15.4 system with retransmissions.

Control de dispositivos con Android

Este proyecto de final de carrera de la titulación de Ingeniería en Informática de Gestión consiste en el diseño y desarrollo de un sistema de control de dispositivos conectados a Raspberry Pi y gestionados desde un dispositivo Android. Se podrán llevar a cabo diferentes acciones como encender, apagar, programar la calefacción y consultar la temperatura actual remotamente desde un dispostivo conectado a la red wifi doméstica o desde de cualquier red que esté conectada a internet.

Implementing theLean Manufacturing System

Tuottavuuden ja tehokkuuden parantaminen on keskeisimpiä menestystekijöitä glo-baaleilla markkinoilla. Yksi kehittämisen toimintamalli on lean manufacturing -ajattelutapa, joka perustuu imuohjaukseen. Sen lähtökohtana on asiakasarvo ja keskit-tyminen arvoa lisäävien toimintojen maksimointiin. Tutkimuksen tavoitteena oli parantaa Strömfors Electric Oy:n materiaalivirtauksia yhdellä tuotantolinjalla. Projektin oli tarkoitus olla pilotti lean-implementointiin koko tehtaalla. Projektissa otettiin käyttöön kanban-imuohjausjärjestelmä, jota varten teh-tiin analyysejä ja laskelmia arvovirtoihin, varastoihin ja ohjaukseen. Lean-ajattelutapaan ja -työkaluihin perustuvaa kirjallisuutta käytettiin pohjatietona pilotti-projektissa sekätulevien kehitysprojektien suunnittelussa. Pilottiprojektin tuloksena materiaalinohjaus tehostui ja keskeneräisen tuotannon va-rasto tasoittui käyttötarpeen mukaiseksi. Työntekijät ja työnjohtajat tutustuivat leanin perusteisiin osallistuessaan pilottiprojektiin. Pidemmän aikavälin suunnitelmissa ontoimitusvarmuuden ja laadun parantuminen ensin pilottiprojektia koskevalla linjalla ja myöhemmin koko tehtaalla imuohjauksen kokonaisvaltaisen käyttöönoton myötä.

Field Sales Management Control: Towards a Multi-Level Theory

The purpose of this dissertation is to increase the understanding and knowledge of field sales management control systems (i.e. sales managers monitoring, directing, evaluating and rewarding activities) and their potential consequences on salespeople. This topic is important because research conducted in the past has indicated that the choice of control system type can on the other hand have desirable consequences, such as high levels of motivation and performance, and on the other hand leadto harmful unintended consequences, such as opportunistic or unethical behaviors. Despite the fact that marketing and sales management control systems have been under rigorous research for over two decades, it still is at a very early stage of development, and several inconsistencies can be found in the research results. This dissertation argues that these inconsistencies are mainly derived from misspecification of the level of analysis in the past research. These different levels of analysis (i.e. strategic, tactical, and operational levels) involve very different decision-making situations regarding the control and motivation of sales force, which should be taken into consideration when conceptualizing the control. Moreover, the study of salesperson consequences of a field sales management control system is actually a cross-level phenomenon, which means that at least two levels of analysis are simultaneously involved. The results of this dissertation confirm the need to re-conceptualize the field sales management control system concept. It provides empirical evidence for the assertion that control should be conceptualized with more details atthe tactical/operational level of analysis than at the strategic levelof analysis. Moreover, the results show that some controls are more efficiently communicated to field salespeople than others. It is proposed that this difference is due to different purposes of control; some controls aredesigned for influencing salespersons' behavior (aim at motivating) whereas some controls are designed to aid decision-making (aim at providing information). According to the empirical results of this dissertation, the both types of controls have an impact to the sales force, but this impactis not as strong as expected. The results obtained in this dissertation shed some light to the nature of field sales management control systems, and their consequences on salespeopl

Converter-flux-based current control of voltage source PWM rectifiers - analysis and implementation

Pulsewidth-modulated (PWM) rectifier technology is increasingly used in industrial applications like variable-speed motor drives, since it offers several desired features such as sinusoidal input currents, controllable power factor, bidirectional power flow and high quality DC output voltage. To achieve these features,however, an effective control system with fast and accurate current and DC voltage responses is required. From various control strategies proposed to meet these control objectives, in most cases the commonly known principle of the synchronous-frame current vector control along with some space-vector PWM scheme have been applied. Recently, however, new control approaches analogous to the well-established direct torque control (DTC) method for electrical machines have also emerged to implement a high-performance PWM rectifier. In this thesis the concepts of classical synchronous-frame current control and DTC-based PWM rectifier control are combined and a new converter-flux-based current control (CFCC) scheme is introduced. To achieve sufficient dynamic performance and to ensure a stable operation, the proposed control system is thoroughly analysed and simple rules for the controller design are suggested. Special attention is paid to the estimationof the converter flux, which is the key element of converter-flux-based control. Discrete-time implementation is also discussed. Line-voltage-sensorless reactive reactive power control methods for the L- and LCL-type line filters are presented. For the L-filter an open-loop control law for the d-axis current referenceis proposed. In the case of the LCL-filter the combined open-loop control and feedback control is proposed. The influence of the erroneous filter parameter estimates on the accuracy of the developed control schemes is also discussed. A newzero vector selection rule for suppressing the zero-sequence current in parallel-connected PWM rectifiers is proposed. With this method a truly standalone and independent control of the converter units is allowed and traditional transformer isolation and synchronised-control-based solutions are avoided. The implementation requires only one additional current sensor. The proposed schemes are evaluated by the simulations and laboratory experiments. A satisfactory performance and good agreement between the theory and practice are demonstrated.

Controlling Access to Personal Information

Tässä diplomityössä käsitellään henkilökohtaisen tiedon saannin kontrollointia ja tiedon kuvaamista. Työn käytännön osuudessa suunniteltiin XML –malli henkilökohtaisen tiedon kuvaamiseen. Henkilökohtaisten tietojen käyttäminen mahdollistaa henkilökohtaisen palvelun tarjoamisen ja myös palvelun automatisoinnin käyttäjälle. Henkilökohtaisen tiedon kuvaaminen on hyvin oleellista, jotta palvelut voivat kysellä ja ymmärtää tietoja. Henkilökohtaiseen tietoon vaikuttaa erilaisia tekijöitä, jotka on myös otettava huomioon tietoa kuvattaessa. Henkilökohtaisen tiedon leviäminen eri palveluiden tarjoajille tuo mukanaan myös riskejä. Henkilökohtaisen tiedon joutuminen väärän henkilön käsiin saattaa aiheuttaa vakaviakin ongelmia tiedon omistajalle. Henkilökohtaisen tiedon turvallisen ja luotettavan käytettävyyden kannalta onkin hyvin oleellista, että käyttäjällä on mahdollisuus kontrolloida kenelle hän haluaa luovuttaa mitäkin tietoa.

Production Control Principles in Printed Wiring Board Fabrication

Työn tavoitteena oli kuvata piirilevyvalmistaja Aspocomp Oy:n Espoon tehtaan tämän hetkinen tuotannonohjausperiaate ja tunnistaa siinä esiintyvät puutteet sekä kehittää vaihtoehtoinen tuotannonohjausperiaate piirilevyvalmistukseen. Vaihtoehtoisen ohjausperiaatteen lähtökohtana oli tuotannonohjauksen sopeuttaminen vaativaan ja jatkuvasti muuttuvaan liiketoimintaympäristöön. Työn teoreettinen osa keskittyi tuotannonohjauksen eri lähestymistapoihin. Kirjallisuuskatsauksessa esitetään eri tuotannonohjausperiaatteiden keskeiset sisällöt, jotka muodostavat rungon toimivalle tuotannonohjauskäytännölle. Työn kokeellinen osa keskittyi Espoon piirilevytehtaan tuotannonohjausperiaatteen selvittämiseen. Espoon piirilevytehtaan nykyisessä tuotannonohjausperiaatteessa havaittujen ongelmakohtien ja liiketoimintaympäristön vaatimusten perusteella kehitettiin vaihtoehtoinen tuotannonohjaustapa. Vaihtoehtoisen tuotannonohjaustavan päämääränä oli läpimenoajan lyhentäminen sekä tuotannon parempi hallittavuus. Vaihtoehtoinen toimintamalli tavoitteiden saavuttamiseksi perustuu pullonkaulateoriaan, jossa keskeisin muutos nykyiseen toimintamalliin oli puolivalmisteiden varastointi toimitusajan lyhentämiseksi sekä tuotantovolyymin heilahdusten vaikutusten vähentämiseksi. Työn kokeellisessa osassa ilmeni, että kysynnän muutokset ja kapasiteetin suunnittelun puute aiheuttivat ongelmia piirilevytehtaan tuotannonohjauksessa.

Digitalización del sistema de control de nivel en calentadores y tanques de drenaje de una Central Nuclear

Contexto Una central nuclear, al igual que cualquier otro tipo de central generadora de energía eléctrica, mediante turbinas de vapor, está basada en un proceso termodinámico. El rendimiento de las mismas es función del salto entálpico del vapor, para mejorarlo las centrales están constituidas por un ciclo compound formado por turbina de alta presión y turbinas de baja presión, y un ciclo regenerativo consistente en calentar el agua de alimentación antes de su introducción a los generadores de vapor. Un ciclo regenerativo está basado en etapas de calentadores o cambiadores de calor para aprovechar al máximo la energía térmica del vapor, este proyecto está basado en la mejora y optimización del proceso de control de estos para contribuir a mejorar el rendimiento de la central. Objetivo Implementar un sistema de control que nos permita modernizar los clásicos sistemas basados en controles locales y comunicaciones analógicas. Mejorar el rendimiento del ciclo regenerativo de la central, aprovechando las mejoras tecnológicas que ofrece el mercado, tanto en el hardware como en el software de los sistemas de instrumentación y control. Optimizar el rendimiento de los lazos de control de cada uno de los elementos del ciclo regenerativo mediante estrategias de control. Procedimiento Desarrollo de un sistema de control actualizado considerando, como premisa principal, la fiabilidad del sistema, el análisis de fallos y la jerarquización del riesgo. Análisis y cálculo de los lazos de control considerando las premisas establecidas. Configuración de los lazos mediante estrategias de control que nos permitan optimizar y minimizar los efectos del fallo. Para ello se han utilizado parámetros y datos extraídos de la Central Nuclear de Ascó. Conclusiones Se ha modernizado y optimizado el sistema de control mejorando el rendimiento del ciclo regenerativo. Se ha conseguido un sistema más fiable, reduciendo el riesgo del fallo y disminuyendo los efectos de los mismos. El coste de un proyecto de estas características es inferior al de un sistema convencional y ofrece más posibilidades. Es un sistema abierto que permite utilizar e interconectar equipos de diferentes fabricantes, lo que favorece tanto el mantenimiento como las posibles ampliaciones futuras del sistema.

On the Analysis and Control of a Linear Synchronous Servomotor with a Flexible Load

Industry's growing need for higher productivity is placing new demands on mechanisms connected with electrical motors, because these can easily lead to vibration problems due to fast dynamics. Furthermore, the nonlinear effects caused by a motor frequently reduce servo stability, which diminishes the controller's ability to predict and maintain speed. Hence, the flexibility of a mechanism and its control has become an important area of research. The basic approach in control system engineering is to assume that the mechanism connected to a motor is rigid, so that vibrations in the tool mechanism, reel, gripper or any apparatus connected to the motor are not taken into account. This might reduce the ability of the machine system to carry out its assignment and shorten the lifetime of the equipment. Nonetheless, it is usually more important to know how the mechanism, or in other words the load on the motor, behaves. A nonlinear load control method for a permanent magnet linear synchronous motor is developed and implemented in the thesis. The purpose of the controller is to track a flexible load to the desired velocity reference as fast as possible and without awkward oscillations. The control method is based on an adaptive backstepping algorithm with its stability ensured by the Lyapunov stability theorem. As a reference controller for the backstepping method, a hybrid neural controller is introduced in which the linear motor itself is controlled by a conventional PI velocity controller and the vibration of the associated flexible mechanism is suppressed from an outer control loop using a compensation signal from a multilayer perceptron network. To avoid the local minimum problem entailed in neural networks, the initial weights are searched for offline by means of a differential evolution algorithm. The states of a mechanical system for controllers are estimated using the Kalman filter. The theoretical results obtained from the control design are validated with the lumped mass model for a mechanism. Generalization of the mechanism allows the methods derived here to be widely implemented in machine automation. The control algorithms are first designed in a specially introduced nonlinear simulation model and then implemented in the physical linear motor using a DSP (Digital Signal Processor) application. The measurements prove that both controllers are capable of suppressing vibration, but that the backstepping method is superior to others due to its accuracy of response and stability properties.

System Investigation for Hybrid Electric Vehicle

A hybrid electric vehicle is a fast-growing concept in the field of vehicle industry. Nowadays two global problems make manufactures to develop such systems. These problems are: the growing cost of a fuel and environmental pollution. Also development of controlled electric drive with high control accuracy and reliability allows improving of vehicle drive characteristics. The objective of this Diploma Thesis is to investigate the possibilities of electrical drive application for new principle of parallel hybrid vehicle system. Electric motor calculations, selection of most suitable control system and other calculations are needed. This work is not final work for such topic. Further investigation with more precise calculations, modeling, measurements and cost calculations are needed to answer the question if such system is efficient.

The effects of real time control of welding parameters on weld quality in plasma arc keyhole welding

Joints intended for welding frequently show variations in geometry and position, for which it is unfortunately not possible to apply a single set of operating parameters to ensure constant quality. The cause of this difficulty lies in a number of factors, including inaccurate joint preparation and joint fit up, tack welds, as well as thermal distortion of the workpiece. In plasma arc keyhole welding of butt joints, deviations in the gap width may cause weld defects such as an incomplete weld bead, excessive penetration and burn through. Manual adjustment of welding parameters to compensate for variations in the gap width is very difficult, and unsatisfactory weld quality is often obtained. In this study a control system for plasma arc keyhole welding has been developed and used to study the effects of the real time control of welding parameters on gap tolerance during welding of austenitic stainless steel AISI 304L. The welding tests demonstrated the beneficial effect of real time control on weld quality. Compared with welding using constant parameters, the maximum tolerable gap width with an acceptable weld quality was 47% higher when using the real time controlled parameters for a plate thickness of 5 mm. In addition, burn through occurred with significantly larger gap widths when parameters were controlled in real time. Increased gap tolerance enables joints to be prepared and fit up less accurately, saving time and preparation costs for welding. In addition to the control system, a novel technique for back face monitoring is described in this study. The test results showed that the technique could be successfully applied for penetration monitoring when welding non magnetic materials. The results also imply that it is possible to measure the dimensions of the plasma efflux or weld root, and use this information in a feedback control system and, thus, maintain the required weld quality.

The Detector Control Systems for the CMS Resistive Plate Chamber at LHC

The RPC Detector Control System (RCS) is the main subject of this PhD work. The project, involving the Lappeenranta University of Technology, the Warsaw University and INFN of Naples, is aimed to integrate the different subsystems for the RPC detector and its trigger chain in order to develop a common framework to control and monitoring the different parts. In this project, I have been strongly involved during the last three years on the hardware and software development, construction and commissioning as main responsible and coordinator. The CMS Resistive Plate Chambers (RPC) system consists of 912 double-gap chambers at its start-up in middle of 2008. A continuous control and monitoring of the detector, the trigger and all the ancillary sub-systems (high voltages, low voltages, environmental, gas, and cooling), is required to achieve the operational stability and reliability of a so large and complex detector and trigger system. Role of the RPC Detector Control System is to monitor the detector conditions and performance, control and monitor all subsystems related to RPC and their electronics and store all the information in a dedicated database, called Condition DB. Therefore the RPC DCS system has to assure the safe and correct operation of the sub-detectors during all CMS life time (more than 10 year), detect abnormal and harmful situations and take protective and automatic actions to minimize consequential damages. The analysis of the requirements and project challenges, the architecture design and its development as well as the calibration and commissioning phases represent themain tasks of the work developed for this PhD thesis. Different technologies, middleware and solutions has been studied and adopted in the design and development of the different components and a big challenging consisted in the integration of these different parts each other and in the general CMS control system and data acquisition framework. Therefore, the RCS installation and commissioning phase as well as its performance and the first results, obtained during the last three years CMS cosmic runs, will be

Antenna Design for Contactless IC Card Applications

Contactless integrated circuit cards are one form of application of radio frequency identification. They are used in applications such as access control, identification, and payment in public transport. The contactless IC cards are passive which means that both the data and the energy are transferred to the card without contact using inductive coupling. Antenna design and optimization of the design for contactless IC cards defined by ISO/IEC14443 is studied. The basic operation principles of contactless system are presented and the structure of contactless IC card is illustrated. The structure was divided between the contactless chip and the antenna. The operation of the antenna was covered in depth and the parameters affecting to the performance of the antenna were presented. Also the different antenna technologies and connection technologies were provided. The antenna design process with the parameters and the design tools isillustrated and optimization of the design is studied. To make the design process more ideal a target of development was discovered, which was the implementation of test application. The optimization of the antenna design was presented based on the optimization criteria defined in this study. The solution for the implementation of these criteria and the effect of each criterion was found. For enhancing the performance of the antenna a focus for future study was proposed.