Dynamic equilibrium through reinforcement learning

Reinforcement Learning is an area of Machine Learning that deals with how an agent should take actions in an environment such as to maximize the notion of accumulated reward. This type of learning is inspired by the way humans learn and has led to the creation of various algorithms for reinforcement learning. These algorithms focus on the way in which an agent’s behaviour can be improved, assuming independence as to their surroundings. The current work studies the application of reinforcement learning methods to solve the inverted pendulum problem. The importance of the variability of the environment (factors that are external to the agent) on the execution of reinforcement learning agents is studied by using a model that seeks to obtain equilibrium (stability) through dynamism – a Cart-Pole system or inverted pendulum. We sought to improve the behaviour of the autonomous agents by changing the information passed to them, while maintaining the agent’s internal parameters constant (learning rate, discount factors, decay rate, etc.), instead of the classical approach of tuning the agent’s internal parameters. The influence of changes on the state set and the action set on an agent’s capability to solve the Cart-pole problem was studied. We have studied typical behaviour of reinforcement learning agents applied to the classic BOXES model and a new form of characterizing the environment was proposed using the notion of convergence towards a reference value. We demonstrate the gain in performance of this new method applied to a Q-Learning agent.

Automação e controlo de um sistema de armazenamento de bicicletas

A crescente procura da bicicleta como meio de transporte alternativo torna relevante a criação e desenvolvimento de infra-estruturas de apoio, tais como ciclovias e parques para bicicletas. Os sistemas tradicionais de parqueamento de bicicletas com recurso a correntes e cadeados não fornecem segurança nem comodidade. No entanto, começam a surgir, em várias cidades do mundo, parque automáticos onde é possível guardar uma bicicleta em segurança, protegendo-a quer das intempéries quer de actos de vandalismo. Este trabalho apresenta uma proposta para um parque automático de armazenamento de bicicletas, com recurso a caixas individualizadas que garantem a sua segurança, e também de outros bens que podem ser guardados junto da mesma, como por exemplo um capacete ou uma mochila. O sistema proposto no âmbito deste trabalho é um complemento às alternativas existentes. As vantagens apresentadas pelo sistema proposto são: a sua construção modular e personalizada; e a possibilidade de instalação num terreno plano, sem recurso a obras de construção civil. O objectivo foi criar um projecto de automação e controlo de um protótipo, com base na proposta apresentada. O projecto de automação e controlo engloba a escolha dos sensores e dos actuadores. Para o dimensionamento dos motores foi necessário recorrer a um cálculo simplificado da estrutura do robô manipulador. Foi feita a escolha dos sensores, actuadores e do controlador com base nos requisitos funcionais. A programação foi desenvolvida numa linguagem normalizada. O modelo desenvolvido poderá servir de base para um projecto multidisciplinar entre vários departamentos do Instituto e dessa cooperação poderá surgir um novo projecto optimizado para produção e de menor custo.

Comparative evaluation of underlying causes of death processed by the Automated Classification of Medical Entities and the Underlying Cause of Death Selection Systems

INTRODUCTION: The correct identification of the underlying cause of death and its precise assignment to a code from the International Classification of Diseases are important issues to achieve accurate and universally comparable mortality statistics These factors, among other ones, led to the development of computer software programs in order to automatically identify the underlying cause of death. OBJECTIVE: This work was conceived to compare the underlying causes of death processed respectively by the Automated Classification of Medical Entities (ACME) and the "Sistema de Seleção de Causa Básica de Morte" (SCB) programs. MATERIAL AND METHOD: The comparative evaluation of the underlying causes of death processed respectively by ACME and SCB systems was performed using the input data file for the ACME system that included deaths which occurred in the State of S. Paulo from June to December 1993, totalling 129,104 records of the corresponding death certificates. The differences between underlying causes selected by ACME and SCB systems verified in the month of June, when considered as SCB errors, were used to correct and improve SCB processing logic and its decision tables. RESULTS: The processing of the underlying causes of death by the ACME and SCB systems resulted in 3,278 differences, that were analysed and ascribed to lack of answer to dialogue boxes during processing, to deaths due to human immunodeficiency virus [HIV] disease for which there was no specific provision in any of the systems, to coding and/or keying errors and to actual problems. The detailed analysis of these latter disclosed that the majority of the underlying causes of death processed by the SCB system were correct and that different interpretations were given to the mortality coding rules by each system, that some particular problems could not be explained with the available documentation and that a smaller proportion of problems were identified as SCB errors. CONCLUSION: These results, disclosing a very low and insignificant number of actual problems, guarantees the use of the version of the SCB system for the Ninth Revision of the International Classification of Diseases and assures the continuity of the work which is being undertaken for the Tenth Revision version.

Solução doméstica para televisão digital

Mestrado em Engenharia Electrotécnica e de Computadores

Convective patterns under the Indo-Atlantic << box >>

Using fluid mechanics, we reinterpret the mantle images obtained from global and regional tomography together with geochemical, geological and paleomagnetic observations, and attempt to unravel the pattern of convection in the Indo-Atlantic "box" and its temporal evolution over the last 260 Myr. The << box >> presently contains a) a broad slow seismic anomaly at the CMB which has a shape similar to Pangea 250 Myr ago, and which divides into several branches higher in the lower mantle, b) a "superswell, centered on the western edge of South Africa, c) at least 6 "primary hotspots" with long tracks related to traps, and d) numerous smaller hotspots. In the last 260 Myr, this mantle box has undergone 10 trap events, 7 of them related to continental breakup. Several of these past events are spatially correlated with present-day seismic anomalies and/or upwellings. Laboratory experiments show that superswells, long-lived hotspot tracks and traps may represent three evolutionary stages of the same phenomenon, i.e. episodic destabilization of a hot, chemically heterogeneous thermal boundary layer, close to the bottom of the mantle. When scaled to the Earth's mantle, its recurrence time is on the order of 100-200 Myr. At any given time, the Indo-Atlantic box should contain 3 to 9 of these instabilities at different stages of their development, in agreement with observations. The return flow of the downwelling slabs, although confined to two main << boxes >> (Indo-Atlantic and Pacific) by subduction zone geometry, may therefore not be passive, but rather take the form of active thermochemical instabilities. (c) 2005 Elsevier B.V. All rights reserved.

Pine processionary moth (Thaumetopoea pityocampa) and great tit (Parus major) in Portugal

In the present work, we studied a common outbreaking Lepidoptera species in Portuguese pine stands – Thaumetopoea pityocampa (Den. & Schiff.) - and one of its potential predators – Parus major (L.). The population dynamics of the immature stages of the Lepidoptera was studied in several types of Pinus pinaster (Aiton) plantations in three different areas: Setúbal Peninsula, Abrantes and National Pine Forest of Leiria. Location and plantation structure was the most important factors determining population density of T. pityocampa. Setubal and Abrantes was highly susceptible to attacks by the Lepidoptera, whereas Leiria had lower densities. Young and homogeneous pine stands was more susceptible to attacks than older and more heterogeneous pines stands. However, a desynchronized population of T. pityocampa, in which the larvae develops during summer instead of during winter, reached high densities also in Leiria. The impact of several mortality factors and climatic conditions on the immature stages of the insect (eggs and larvae), in normal and desynchronized populations are discussed, as well as possible evolutionary implications of the sudden appearance of the new version of T. pityocampa. The break of the pupa diapause and adult emergence times the annual life cycle of this insect. Adults from the desynchronized population emerged earlier than adults from the normal population, which in turn determined the change in the larvae development period. Different factors, potentially affecting the timing of adult emergence in both normal and abnormal populations are also discussed. To study P. Major, nest-boxes were placed in the areas of Setúbal and Leiria and they were monitored during three seasons. The nest-boxes increased the density of breeding and wintering birds in the studied pine plantations, indicating that a lack of natural holes are in fact a limiting factor for this populations. The earliest breeding start for this species was recorded in my study area, indicating that Portuguese coastal pines provide good breeding conditions earlier than in other areas of Europe and North Africa. This leads to an overlap between the end of the larvae stage of T. pityocampa and the beginning of the breeding season of P. major. Key-words: Thaumetopoea pityocampa, Parus major, Pinus pinaster, population dynamics, Portugal.

Relatório de estágio em reabilitação neurológica: intervenção da fisioterapia em crianças com necessidades especiais

Mestrado em Fisioterapia

Big Bang bifurcations and allee effect in Blumberg’s dynamics

This paper concerns dynamics and bifurcations properties of a class of continuous-defined one-dimensional maps, in a three-dimensional parameter space: Blumberg's functions. This family of functions naturally incorporates a major focus of ecological research: the Allee effect. We provide a necessary condition for the occurrence of this phenomenon, associated with the stability of a fixed point. A central point of our investigation is the study of bifurcations structure for this class of functions. We verified that under some sufficient conditions, Blumberg's functions have a particular bifurcations structure: the big bang bifurcations of the so-called "box-within-a-box" type, but for different kinds of boxes. Moreover, it is verified that these bifurcation cascades converge to different big bang bifurcation curves, where for the corresponding parameter values are associated distinct attractors. This work contributes to clarify the big bang bifurcation analysis for continuous maps. To support our results, we present fold and flip bifurcations curves and surfaces, and numerical simulations of several bifurcation diagrams.

Gás sulfídrico nas redes de águas residuais

Os colaboradores que realizam a sua atividade profissional, no setor das águas residuais, reconhecem que a sua atividade é perigosa. Esse conhecimento, foi adquirido pela experiência obtida ao longo dos anos, pelas várias formações e pelas notícias de acidentes, muitas vezes mortais, que têm acontecido na atividade. Muita das vezes, os colaboradores têm dificuldade em aceitar o risco, porque na realidade não existe um histórico conhecido do que se passa no interior das redes de drenagem de águas residuais e consideram que o risco é apenas teórico. Os portugueses são um Povo muito ligado à Cultura Popular,e são os ditados populares que melhor sintetiza as ideias expressas pelos trabalhador português “Ver para Crer, Como São Tomé” (SOUSA, Marcelo Rebelo de, Os Evangelhos de 2001, Lisboa, Bertrand Editora, 2001]. O objetivo principal com este trabalho, foi evidenciar que existem gases no interior das caixas de visita, mas principalmente, demostrar que seu aparecimento e respetiva concentração, é variável e que ocorre sem aviso prévio! Tornando assim, os trabalhos no interior das caixas de visita e das redes de drenagem de águas residuais domesticas, locais perigosos para quem opera no seu interior, sem a correta utilização dos equipamentos de proteção individual, (Epis e Epc). Sendo necessário, reforçar, a constante informação e formação para a utilização dos equipamentos de proteção individual e coletiva. Com o presente estágio, pretendeu-se iniciar um registo histórico de medições nas redes de drenagem de águas residuais domésticas. Com o resultado das medições que serão obtidas, pretende-se uma “Radiografia Fotográfica” ao interior das caixas de visitas das redes de drenagem de águas residuais. Com os resultados deste estudo, pretende-se contribuir de forma clara e objetiva, para os perigos iminentes que existem efetivamente, no interior das redes de saneamento, com a presença de altas concentrações de gás sulfídrico, nocivo para a vida humana!

Explosion birth and extinction: Double big bang bifurcations and Allee effect in Tsoularis-Wallace's growth models

This work concerns dynamics and bifurcations properties of a new class of continuous-defined one-dimensional maps: Tsoularis-Wallace's functions. This family of functions naturally incorporates a major focus of ecological research: the Allee effect. We provide a necessary condition for the occurrence of this phenomenon of extinction. To establish this result we introduce the notions of Allee's functions, Allee's effect region and Allee's bifurcation curve. Another central point of our investigation is the study of bifurcation structures for this class of functions, in a three-dimensional parameter space. We verified that under some sufficient conditions, Tsoularis-Wallace's functions have particular bifurcation structures: the big bang and the double big bang bifurcations of the so-called "box-within-a-box" type. The double big bang bifurcations are related to the existence of flip codimension-2 points. Moreover, it is verified that these bifurcation cascades converge to different big bang bifurcation curves, where for the corresponding parameter values are associated distinct kinds of boxes. This work contributes to clarify the big bang bifurcation analysis for continuous maps and understand their relationship with explosion birth and extinction phenomena.

Plataforma de gestão de guias de programação de TV digital

Com o consumismo de mais variedade e qualidade de informação, assim como, produtos interativos, surgiu a necessidade de apresentar mais conteúdos, para além da programação de televisão comum. Com os avanços tecnológicos ligados à indústria da televisão e sua distribuição nos lares portugueses pelos operadores de TV, a quantidade de oferta de canais deixou de ser um foco, passando a ser prioritário a melhoria da experiência do cliente. Com a introdução de novas funcionalidades nas caixas recetoras de sinais de transmissão de canais, como por exemplo, a capacidade de apresentar informações adicionais sobre os programas, desde da sua apresentação em modo trailer até ao elenco detalhado que o compõe, os clientes podem ter uma nova experiência de interação com os serviços de TV. A funcionalidade de gravação agendada de programas levou ao próximo ponto de melhoria de experiência do cliente. As gravações que resultavam em programas indevidamente cortados, quer no seu início quer no seu fim, foi um dos motivos que levou os operadores de TV a procurarem um melhor serviço de gestão de guias de programação digitais. A InfoPortugal, entidade detentora do seguinte projeto e EPG Provider de algumas operadoras de TV nacionais, viu-se obrigada a atualizar os seu sistemas de distribuição de conteúdos, para responder à evolução dos requisitos dos seus clientes.

Domestic vectors of Chagas' disease in three rural communities of Nicaragua

A triatomine survey was conducted in three rural settlements of Nicaragua (Santa Rosa, Quebrada Honda and Poneloya) where Chagas' disease is endemic, to determine rates of house infestation, evaluate the housing condition and to asess the performance of the María sensor box in detection of domestic vectors. A total of 184 households were selected and vectors were sought by the methods of timed manual capture and by sensor boxes. The sole vectors species found in this study was Triatoma dimidiata. Of the examined bugs 50, 60 and 33%, in the respective communities, were infected with T. cruzi. The rates of house infestation as determined by manual capture and sensor boxes were respectively, 48.3% and 54.2% in Santa Rosa, 29.8% and 51.2% in Quebrada Honda and in Poneloya 3.8 and 5.9% with significant difference between the methods in Quebrada Honda. When compared with the manual capture, the Maria sensor box detected vectors in 71.4% of positive houses in two of the communities but also was able to detect bugs in 39.3% and 41.1% of houses where manual capture had been negative. Housing condition was evaluated according to three structural parameters, in this way, in the first community 79.2% of houses were classified as bad, 20.8% as regular; in the second one 42.5% were bad and 57.5% regular, whereas in the third 62.5% of the houses were regular. Rates of infestation did not differ greatly between the different housing conditions. Our results show that the sensor box is as efficient as manual capture and could be implemented in our country.

Desenvolvimento de uma cola biodegradável

Este documento especifica o trabalho realizado no estágio, que decorreu na Colquímica, sediada em Valongo. Este trabalho teve como objetivo colaborar no desenvolvimento de uma cola com características biodegradáveis. O estágio foi orientado pela Engenheira Cristina Frutuoso, no que diz respeito à orientação por parte da Colquímica, e pelo Professor Dr. Gilberto Pinto e Alfredo Crispim, ISEP. O trabalho foi dividido em duas instâncias, sendo a primeira direcionada para a formulação de uma cola com características biodegradáveis e a segunda para a realização do teste de Sturm, um teste de biodegradabilidade capaz de quantificar a quantidade de CO2 produzido pelo processo de biodegradação. Inicialmente foram desenvolvidas e estudadas dez formulações através das variações de resinas e óleo plastificante e estudado o seu comportamento reológico relacionado à variação de viscosidade em função da temperatura. Com base nos resultados obtidos verificou-se que a melhor formulação seria a A8. Estudaram-se as propriedades mecânicas da formulação escolhida e decidiu-se que esta estaria apta a ser usada no fecho de caixas de cartão de produtos ultra congelados assim como no fecho de caixas de cartão de detergentes em pó com peso até 3 Kg. Depois de escolhida a formulação deu-se início ao teste de Sturm, teste meramente orientativo. Durante dezassete dias avaliou-se a quantidade de CO2 produzida durante o processo de biodegradação. Após avaliação dos resultados verificou-se que a taxa de biodegradação para a amostra foi de 0,055 g de CO2. Com a finalidade de confirmar os resultados obtidos no teste de Sturm foi efetuada uma réplica do ensaio, a qual se encontra a decorrer sendo os resultados facultados, posteriormente, à empresa.

Domestic Rhodnius ecuadoriensis (Hemiptera, Reduviidae) infestation in Northern Peru: a comparative trial of detection methods during a six-month follow-up

Two passive methods in the assessment of intradomiciliary infestation by Rhodnius ecuadoriensis were tested: (i) the Gomes Nuñez sensor box (GN), (ii) sheets of white typing paper and (iii) one active timed manual method. The study was carried out in the Alto Chicama River Valley, Province of Gran Chimú, Department of La Libertad. The study design consisted of an initial searching of triatomines inside of the domestic environment by the manual capture active procedure (man/hour) covering all the studied houses. Then, matched pairs of GN boxes and paper sheets were simultaneously installed in the bedrooms of 207 households distributed in 19 localities. A comparative prospective trial of these passive detection devices were monitored at 2, 4 and, finally 6 months follow-up. Parasitological Trypanosoma rangeli and/or T. cruzi infections were investigated in two houses with high level of infestation by R. ecuadoriensis. 16.9% of the 207 households investigated by an initial active manual method were infested with R. ecuadoriensis. The proportion of infested houses fluctuated from 6.2 to 55.5% amongst the 19 localities investigated. T. rangeli natural infection was detected in R. ecuadoriensis specimens collected in two households. Parasite rates in the bugs ranged from 16.6 to 21.7% respectively. The most striking fact was an average rate of salivary gland infection ranging from 7.4 to 8.3%. At the end of the sixth month period, a cumulative incidence of 31.4% of positive GN boxes against 15.9% for paper sheets was recorded. All three methods combined detected domestic infestation in 129 (62.3%) of the 207 houses studied in the 19 localities. The range of houses infested varies from 6.7% to 92.9%. In areas with low bug density infestation rates, the methodology experienced in our studies, seems to be the best choice for investigations on domestic R. ecuadoriensis populations.

Dynamic stability metrics for the container loading problem

The Container Loading Problem (CLP) literature has traditionally evaluated the dynamic stability of cargo by applying two metrics to box arrangements: the mean number of boxes supporting the items excluding those placed directly on the floor (M1) and the percentage of boxes with insufficient lateral support (M2). However, these metrics, that aim to be proxies for cargo stability during transportation, fail to translate real-world cargo conditions of dynamic stability. In this paper two new performance indicators are proposed to evaluate the dynamic stability of cargo arrangements: the number of fallen boxes (NFB) and the number of boxes within the Damage Boundary Curve fragility test (NB_DBC). Using 1500 solutions for well-known problem instances found in the literature, these new performance indicators are evaluated using a physics simulation tool (StableCargo), replacing the real-world transportation by a truck with a simulation of the dynamic behaviour of container loading arrangements. Two new dynamic stability metrics that can be integrated within any container loading algorithm are also proposed. The metrics are analytical models of the proposed stability performance indicators, computed by multiple linear regression. Pearson’s r correlation coefficient was used as an evaluation parameter for the performance of the models. The extensive computational results show that the proposed metrics are better proxies for dynamic stability in the CLP than the previous widely used metrics.