Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Meta-analysis of studies analyzing the role of human papillomavirus in the development of bladder carcinoma

PURPOSE We aimed to ascertain the degree of association between bladder cancer and human papillomavirus (HPV) infection. MATERIALS AND METHODS We performed a meta-analysis of observational studies with cases and controls with publication dates up to January 2011. The PubMed electronic database was searched by using the key words "bladder cancer and virus." Twenty-one articles were selected that met the required methodological criteria. We implemented an internal quality control system to verify the selected search method. We analyzed the pooled effect of all the studies and also analyzed the techniques used as follows: 1) studies with DNA-based techniques, among which we found studies with polymerase chain reaction (PCR)-based techniques and 2) studies with non-PCR-based techniques, and studies with non-DNA-based techniques. RESULTS Taking into account the 21 studies that were included in the meta-analysis, we obtained a heterogeneity chi-squared value of Q(exp)=26.45 (p=0.383). The pooled odds ratio (OR) was 2.13 (95% confidence interval [CI], 1.54 to 2.95), which points to a significant effect between HPV and bladder cancer. Twenty studies assessed the presence of DNA. The overall effect showed a significant relationship between virus presence and bladder cancer, with a pooled OR of 2.19 (95% CI, 1.40 to 3.43). Of the other six studies, four examined the virus's capsid antigen and two detected antibodies in serum by Western blot. The estimated pooled OR in this group was 2.11 (95% CI, 1.27 to 3.51), which confirmed the relationship between the presence of virus and cancer. CONCLUSIONS The pooled OR value showed a moderate relationship between viral infection and bladder tumors.

Web-based Implementation of Winter Maintenance Decision Support System Using GIS and Remote Sensing, May 2005

Winter maintenance, particularly snow removal and the stress of snow removal materials on public structures, is an enormous budgetary burden on municipalities and nongovernmental maintenance organizations in cold climates. Lately, geospatial technologies such as remote sensing, geographic information systems (GIS), and decision support tools are roviding a valuable tool for planning snow removal operations. A few researchers recently used geospatial technologies to develop winter maintenance tools. However, most of these winter maintenance tools, while having the potential to address some of these information needs, are not typically placed in the hands of planners and other interested stakeholders. Most tools are not constructed with a nontechnical user in mind and lack an easyto-use, easily understood interface. A major goal of this project was to implement a web-based Winter Maintenance Decision Support System (WMDSS) that enhances the capacity of stakeholders (city/county planners, resource managers, transportation personnel, citizens, and policy makers) to evaluate different procedures for managing snow removal assets optimally. This was accomplished by integrating geospatial analytical techniques (GIS and remote sensing), the existing snow removal asset management system, and webbased spatial decision support systems. The web-based system was implemented using the ESRI ArcIMS ActiveX Connector and related web technologies, such as Active Server Pages, JavaScript, HTML, and XML. The expert knowledge on snow removal procedures is gathered and integrated into the system in the form of encoded business rules using Visual Rule Studio. The system developed not only manages the resources but also provides expert advice to assist complex decision making, such as routing, optimal resource allocation, and monitoring live weather information. This system was developed in collaboration with Black Hawk County, IA, the city of Columbia, MO, and the Iowa Department of transportation. This product was also demonstrated for these agencies to improve the usability and applicability of the system.

Development of a specific control banding tool for nanomaterials : report

[Table des matières] 1. Introduction to the control banding method : Nanomaterials and occupational risk assessment; Alternative method known as control banding; Scope and limits of control banding. - 2. Control banding process applied to manufactured nanomaterials: General points; Operating principle. - 3. Implementation of control banding: Gathering of information; Hazard bands; Exposure bands; Allocation of risk control bands. - 4. Bibliography: Publications; Books, reports, opinions, bulletins; Standards and references; Legislation and regulations; Websites. - Annexes

Contribución al diseño de GNSS-SDR, un receptor GNSS de código abierto

[ANGLÈS] This project introduces GNSS-SDR, an open source Global Navigation Satellite System software-defined receiver. The lack of reconfigurability of current commercial-of-the-shelf receivers and the advent of new radionavigation signals and systems make software receivers an appealing approach to design new architectures and signal processing algorithms. With the aim of exploring the full potential of this forthcoming scenario with a plurality of new signal structures and frequency bands available for positioning, this paper describes the software architecture design and provides details about its implementation, targeting a multiband, multisystem GNSS receiver. The result is a testbed for GNSS signal processing that allows any kind of customization, including interchangeability of signal sources, signal processing algorithms, interoperability with other systems, output formats, and the offering of interfaces to all the intermediate signals, parameters and variables. The source code release under the GNU General Public License (GPL) secures practical usability, inspection, and continuous improvement by the research community, allowing the discussion based on tangible code and the analysis of results obtained with real signals. The source code is complemented by a development ecosystem, consisting of a website (http://gnss-sdr.org), as well as a revision control system, instructions for users and developers, and communication tools. The project shows in detail the design of the initial blocks of the Signal Processing Plane of the receiver: signal conditioner, the acquisition block and the receiver channel, the project also extends the functionality of the acquisition and tracking modules of the GNSS-SDR receiver to track the new Galileo E1 signals available. Each section provides a theoretical analysis, implementation details of each block and subsequent testing to confirm the calculations with both synthetically generated signals and with real signals from satellites in space.

A randomized trial of spiritual assessment of outpatients with schizophrenia: Patients' and clinicians' experience

Objective: Recovery-oriented care for patients with schizophrenia involves consideration of cultural issues, such as religion and spirituality. However, there is evidence that psychiatrists rarely address such topics. This study examined acceptance of a spiritual assessment by patients and clinicians, suggestions for treatment that arose from the assessment, and patient outcomes-in terms of treatment compliance and satisfaction with care (as measured by treatment alliance). Methods: Outpatients with psychosis were randomly assigned to two groups: an intervention group that received traditional treatment and a religious and spiritual assessment (N=40) and a control group that received only traditional treatment (N=38). Eight psychiatrists were trained to administer the assessment to their established and stable patients. After each administration, the psychiatrist attended a supervision session with a psychiatrist and a psychologist of religion. Baseline and three-month data were collected. Results: The spiritual assessment was well accepted by patients. During supervision, psychiatrists reported potential clinical uses for the assessment information for 67% of patients. No between-group differences in medication adherence and satisfaction with care were found at three months, although patients in the in- tervention group had significantly better appointment attendance dur- ing the follow-up period. Their interest in discussing religion and spirituality with their psychiatrists remained high. The process was not as well accepted by psychiatrists. Conclusions: Spiritual assessment can raise important clinical issues in the treatment of patients with chronic schizophrenia. Cultural factors, such as religion and spirituality, should be considered early in clinical training, because many clinicians are not at ease addressing such topics with patients.

Control de dispositivos con Android

Este proyecto de final de carrera de la titulación de Ingeniería en Informática de Gestión consiste en el diseño y desarrollo de un sistema de control de dispositivos conectados a Raspberry Pi y gestionados desde un dispositivo Android. Se podrán llevar a cabo diferentes acciones como encender, apagar, programar la calefacción y consultar la temperatura actual remotamente desde un dispostivo conectado a la red wifi doméstica o desde de cualquier red que esté conectada a internet.

Implementation of a Pilot Continuous Monitoring System: Iowa Falls Arch Bridge, HR-1088, 2015

The goal of this work was to move structural health monitoring (SHM) one step closer to being ready for mainstream use by the Iowa Department of Transportation (DOT) Office of Bridges and Structures. To meet this goal, the objective of this project was to implement a pilot multi-sensor continuous monitoring system on the Iowa Falls Arch Bridge such that autonomous data analysis, storage, and retrieval can be demonstrated. The challenge with this work was to develop the open channels for communication, coordination, and cooperation of various Iowa DOT offices that could make use of the data. In a way, the end product was to be something akin to a control system that would allow for real-time evaluation of the operational condition of a monitored bridge. Development and finalization of general hardware and software components for a bridge SHM system were investigated and completed. This development and finalization was framed around the demonstration installation on the Iowa Falls Arch Bridge. The hardware system focused on using off-the-shelf sensors that could be read in either “fast” or “slow” modes depending on the desired monitoring metric. As hoped, the installed system operated with very few problems. In terms of communications—in part due to the anticipated installation on the I-74 bridge over the Mississippi River—a hardline digital subscriber line (DSL) internet connection and grid power were used. During operation, this system would transmit data to a central server location where the data would be processed and then archived for future retrieval and use. The pilot monitoring system was developed for general performance evaluation purposes (construction, structural, environmental, etc.) such that it could be easily adapted to the Iowa DOT’s bridges and other monitoring needs. The system was developed allowing easy access to near real-time data in a format usable to Iowa DOT engineers.

Developing models for online academic advising

Peer-reviewed

Implementing theLean Manufacturing System

Tuottavuuden ja tehokkuuden parantaminen on keskeisimpiä menestystekijöitä glo-baaleilla markkinoilla. Yksi kehittämisen toimintamalli on lean manufacturing -ajattelutapa, joka perustuu imuohjaukseen. Sen lähtökohtana on asiakasarvo ja keskit-tyminen arvoa lisäävien toimintojen maksimointiin. Tutkimuksen tavoitteena oli parantaa Strömfors Electric Oy:n materiaalivirtauksia yhdellä tuotantolinjalla. Projektin oli tarkoitus olla pilotti lean-implementointiin koko tehtaalla. Projektissa otettiin käyttöön kanban-imuohjausjärjestelmä, jota varten teh-tiin analyysejä ja laskelmia arvovirtoihin, varastoihin ja ohjaukseen. Lean-ajattelutapaan ja -työkaluihin perustuvaa kirjallisuutta käytettiin pohjatietona pilotti-projektissa sekätulevien kehitysprojektien suunnittelussa. Pilottiprojektin tuloksena materiaalinohjaus tehostui ja keskeneräisen tuotannon va-rasto tasoittui käyttötarpeen mukaiseksi. Työntekijät ja työnjohtajat tutustuivat leanin perusteisiin osallistuessaan pilottiprojektiin. Pidemmän aikavälin suunnitelmissa ontoimitusvarmuuden ja laadun parantuminen ensin pilottiprojektia koskevalla linjalla ja myöhemmin koko tehtaalla imuohjauksen kokonaisvaltaisen käyttöönoton myötä.

Field Sales Management Control: Towards a Multi-Level Theory

The purpose of this dissertation is to increase the understanding and knowledge of field sales management control systems (i.e. sales managers monitoring, directing, evaluating and rewarding activities) and their potential consequences on salespeople. This topic is important because research conducted in the past has indicated that the choice of control system type can on the other hand have desirable consequences, such as high levels of motivation and performance, and on the other hand leadto harmful unintended consequences, such as opportunistic or unethical behaviors. Despite the fact that marketing and sales management control systems have been under rigorous research for over two decades, it still is at a very early stage of development, and several inconsistencies can be found in the research results. This dissertation argues that these inconsistencies are mainly derived from misspecification of the level of analysis in the past research. These different levels of analysis (i.e. strategic, tactical, and operational levels) involve very different decision-making situations regarding the control and motivation of sales force, which should be taken into consideration when conceptualizing the control. Moreover, the study of salesperson consequences of a field sales management control system is actually a cross-level phenomenon, which means that at least two levels of analysis are simultaneously involved. The results of this dissertation confirm the need to re-conceptualize the field sales management control system concept. It provides empirical evidence for the assertion that control should be conceptualized with more details atthe tactical/operational level of analysis than at the strategic levelof analysis. Moreover, the results show that some controls are more efficiently communicated to field salespeople than others. It is proposed that this difference is due to different purposes of control; some controls aredesigned for influencing salespersons' behavior (aim at motivating) whereas some controls are designed to aid decision-making (aim at providing information). According to the empirical results of this dissertation, the both types of controls have an impact to the sales force, but this impactis not as strong as expected. The results obtained in this dissertation shed some light to the nature of field sales management control systems, and their consequences on salespeopl

Converter-flux-based current control of voltage source PWM rectifiers - analysis and implementation

Pulsewidth-modulated (PWM) rectifier technology is increasingly used in industrial applications like variable-speed motor drives, since it offers several desired features such as sinusoidal input currents, controllable power factor, bidirectional power flow and high quality DC output voltage. To achieve these features,however, an effective control system with fast and accurate current and DC voltage responses is required. From various control strategies proposed to meet these control objectives, in most cases the commonly known principle of the synchronous-frame current vector control along with some space-vector PWM scheme have been applied. Recently, however, new control approaches analogous to the well-established direct torque control (DTC) method for electrical machines have also emerged to implement a high-performance PWM rectifier. In this thesis the concepts of classical synchronous-frame current control and DTC-based PWM rectifier control are combined and a new converter-flux-based current control (CFCC) scheme is introduced. To achieve sufficient dynamic performance and to ensure a stable operation, the proposed control system is thoroughly analysed and simple rules for the controller design are suggested. Special attention is paid to the estimationof the converter flux, which is the key element of converter-flux-based control. Discrete-time implementation is also discussed. Line-voltage-sensorless reactive reactive power control methods for the L- and LCL-type line filters are presented. For the L-filter an open-loop control law for the d-axis current referenceis proposed. In the case of the LCL-filter the combined open-loop control and feedback control is proposed. The influence of the erroneous filter parameter estimates on the accuracy of the developed control schemes is also discussed. A newzero vector selection rule for suppressing the zero-sequence current in parallel-connected PWM rectifiers is proposed. With this method a truly standalone and independent control of the converter units is allowed and traditional transformer isolation and synchronised-control-based solutions are avoided. The implementation requires only one additional current sensor. The proposed schemes are evaluated by the simulations and laboratory experiments. A satisfactory performance and good agreement between the theory and practice are demonstrated.

Production Control Principles in Printed Wiring Board Fabrication

Työn tavoitteena oli kuvata piirilevyvalmistaja Aspocomp Oy:n Espoon tehtaan tämän hetkinen tuotannonohjausperiaate ja tunnistaa siinä esiintyvät puutteet sekä kehittää vaihtoehtoinen tuotannonohjausperiaate piirilevyvalmistukseen. Vaihtoehtoisen ohjausperiaatteen lähtökohtana oli tuotannonohjauksen sopeuttaminen vaativaan ja jatkuvasti muuttuvaan liiketoimintaympäristöön. Työn teoreettinen osa keskittyi tuotannonohjauksen eri lähestymistapoihin. Kirjallisuuskatsauksessa esitetään eri tuotannonohjausperiaatteiden keskeiset sisällöt, jotka muodostavat rungon toimivalle tuotannonohjauskäytännölle. Työn kokeellinen osa keskittyi Espoon piirilevytehtaan tuotannonohjausperiaatteen selvittämiseen. Espoon piirilevytehtaan nykyisessä tuotannonohjausperiaatteessa havaittujen ongelmakohtien ja liiketoimintaympäristön vaatimusten perusteella kehitettiin vaihtoehtoinen tuotannonohjaustapa. Vaihtoehtoisen tuotannonohjaustavan päämääränä oli läpimenoajan lyhentäminen sekä tuotannon parempi hallittavuus. Vaihtoehtoinen toimintamalli tavoitteiden saavuttamiseksi perustuu pullonkaulateoriaan, jossa keskeisin muutos nykyiseen toimintamalliin oli puolivalmisteiden varastointi toimitusajan lyhentämiseksi sekä tuotantovolyymin heilahdusten vaikutusten vähentämiseksi. Työn kokeellisessa osassa ilmeni, että kysynnän muutokset ja kapasiteetin suunnittelun puute aiheuttivat ongelmia piirilevytehtaan tuotannonohjauksessa.

Transfer pricing tool development process based on valve package production costs

Tämän diplomityön päätavoitteena oli parantaa kehitetyn kustannusperusteisen siirtohinnoittelutyökalun ominaisuuksia osastokohtaisen kustannusarviointiprosessin käyttöön. Työ on vaikeutunut lähimenneisyyden heikosta hintakyselyiden vastauskyvystä. Työn pääongelmana oli kerätä luotettavaa tuotannonohjausjärjestelmän kustannusaineistoa osittain vanhentuneista vakioventtiilien koneistus- ja materiaalitiedosta. Tutkimuksessa käytetyt tärkeimmät tutkimusmenetelmät voidaan jakaa siirtohinnoittelu- ja kustannusarvioprosessien kirjallisuustutkimukseen, kenttäanalyysiin ja nykyisen Microsoft Excel –siirtohinnoittelutyökalun kehittämiseen eri osastojen rajapinnassa. Siirtohinnoittelumenetelmät ovat yleisesti jaettu kustannus-, markkina- ja neuvotteluperusteisiin malleihin, jotka harvoin sellaisenaan kohtaavat siirtohinnoittelulle asetetut tavoitteet. Tämä ratkaisutapa voi johtaa tilanteisiin, jossa kaksi erillistä menetelmää sulautuvat yhteen. Lisäksi varsinaiseen siirtohinnoittelujärjestelmään yleensä vaikuttavat useat sisäiset ja ulkoiset tekijät. Lopullinen siirtohinnoittelumenetelmä tulisi ehdottomasti tukea myös yrityksen visiota ja muita liiketoiminnalle asetettuja strategioita. Työn tuloksena saatiin laajennettu Microsoft Excel –sovellus, joka vaatii sekä vuosittaista että kuukausittaista erikoisventtiilimateriaalien hinta- ja toimitusaikatietojen päivittämistä. Tämä ratkaisutapa ehdottomasti parantaa kustannusarviointiprosessia, koska myös alihankkijatietoja joudutaan tutkimaan systemaattisesti. Tämän jälkeen koko siirtohinnoitteluprosessia voidaan kehittää muuntamalla kokoonpano- ja testaustyövaiheiden kustannusrakennetta toimintoperustaisen kustannuslaskentamallin mukaiseksi.