Unsupervised reinforcement learning via state entropy maximization

Reinforcement Learning (RL) provides a powerful framework to address sequential decision-making problems in which the transition dynamics is unknown or too complex to be represented. The RL approach is based on speculating what is the best decision to make given sample estimates obtained from previous interactions, a recipe that led to several breakthroughs in various domains, ranging from game playing to robotics. Despite their success, current RL methods hardly generalize from one task to another, and achieving the kind of generalization obtained through unsupervised pre-training in non-sequential problems seems unthinkable. Unsupervised RL has recently emerged as a way to improve generalization of RL methods. Just as its non-sequential counterpart, the unsupervised RL framework comprises two phases: An unsupervised pre-training phase, in which the agent interacts with the environment without external feedback, and a supervised fine-tuning phase, in which the agent aims to efficiently solve a task in the same environment by exploiting the knowledge acquired during pre-training. In this thesis, we study unsupervised RL via state entropy maximization, in which the agent makes use of the unsupervised interactions to pre-train a policy that maximizes the entropy of its induced state distribution. First, we provide a theoretical characterization of the learning problem by considering a convex RL formulation that subsumes state entropy maximization. Our analysis shows that maximizing the state entropy in finite trials is inherently harder than RL. Then, we study the state entropy maximization problem from an optimization perspective. Especially, we show that the primal formulation of the corresponding optimization problem can be (approximately) addressed through tractable linear programs. Finally, we provide the first practical methodologies for state entropy maximization in complex domains, both when the pre-training takes place in a single environment as well as multiple environments.

Novel methods of state estimation and parameter identification for high-performance motorcycles

This research activity aims at providing a reliable estimation of particular state variables or parameters concerning the dynamics and performance optimization of a MotoGP-class motorcycle, integrating the classical model-based approach with new methodologies involving artificial intelligence. The first topic of the research focuses on the estimation of the thermal behavior of the MotoGP carbon braking system. Numerical tools are developed to assess the instantaneous surface temperature distribution in the motorcycle's front brake discs. Within this application other important brake parameters are identified using Kalman filters, such as the disc convection coefficient and the power distribution in the disc-pads contact region. Subsequently, a physical model of the brake is built to estimate the instantaneous braking torque. However, the results obtained with this approach are highly limited by the knowledge of the friction coefficient (μ) between the disc rotor and the pads. Since the value of μ is a highly nonlinear function of many variables (namely temperature, pressure and angular velocity of the disc), an analytical model for the friction coefficient estimation appears impractical to establish. To overcome this challenge, an innovative hybrid solution is implemented, combining the benefit of artificial intelligence (AI) with classical model-based approach. Indeed, the disc temperature estimated through the thermal model previously implemented is processed by a machine learning algorithm that outputs the actual value of the friction coefficient thus improving the braking torque computation performed by the physical model of the brake. Finally, the last topic of this research activity regards the development of an AI algorithm to estimate the current sideslip angle of the motorcycle's front tire. While a single-track motorcycle kinematic model and IMU accelerometer signals theoretically enable sideslip calculation, the presence of accelerometer noise leads to a significant drift over time. To address this issue, a long short-term memory (LSTM) network is implemented.

Wavelets And Adaptive Grids For The Discontinuous Galerkin Method

In this paper, space adaptivity is introduced to control the error in the numerical solution of hyperbolic systems of conservation laws. The reference numerical scheme is a new version of the discontinuous Galerkin method, which uses an implicit diffusive term in the direction of the streamlines, for stability purposes. The decision whether to refine or to unrefine the grid in a certain location is taken according to the magnitude of wavelet coefficients, which are indicators of local smoothness of the numerical solution. Numerical solutions of the nonlinear Euler equations illustrate the efficiency of the method. © Springer 2005.

Dental caries profile in Monte Negro, Amazonian state of Rondônia, Brazil, in 2008

OBJECTIVE: This epidemiological survey assessed the dental caries profile in Monte Negro, a small town in the Amazonian state of Rondônia, Brazil, and its relationship with the northern region, national and global goals for oral health in the years 2000 and 2020. MATERIAL AND METHODS: The groups randomly examined were composed of individuals aged 5, 12, 15 to 19, 35 to 44, 65 to 74 years, living in both rural and urban areas. RESULTS: The means dft (standard deviation) and DMFT (standard deviation) for the groups were, respectively, 3.15 (3.12), 3.41 (2.69), 5.96 (4.19), 16.00 (7.30) and 25.96 (9.82). Caries-free individuals were 34.42%, 14.81% and 8.16% in the preschoolchildren, schoolchildren and adolescent groups, respectively. The Significant Caries Index percentages applied to the two younger groups were 6.65 and 6.70, and they increased to 32.00 in the individuals aged 65 to 74 years. Care Index percentages for adolescents, adults and elderly groups were, respectively, 29.40, 25.00 and 1.41. The dental caries profile in Monte Negro in 2008 shows that, 8 years after the year 2000, no FDI/WHO goal for any age settled in 1982 has been achieved. Dental caries increased with age and the main dental problem of adult and elderly groups was tooth loss. CONCLUSION: Oral health promotion and prevention of oral disease policies are urgent needs. Setting of oral health goals and targets to people living in Monte Negro or Amazonia to be pursuit and achieved in a near future is an important action to do because of the culture, sanitary conditions and socioeconomic aspects of this particular population.

First report of Eratyrus mucronatus, Stal, 1859, (Hemiptera, Reduviidae, Triatominae), in the State of Rondônia, Brazil

INTRODUCTION: This paper reports, for the first time, the presence of the Eratyrus mucronatus species in the State of Rondonia, Brazil. METHODS: These specimens were caught by chance in the forest and later they were collected using luminous traps. RESULTS: After finding these specimens, the number of the Triatominae genera in Rondonia rose to four, while its species rose to seven. CONCLUSIONS: Complimentary studies will be conducted in order to allow for clearer understanding the ecology of this arthropod, its possible role in transmitting Chagas' disease and its current geographical distribution.

Keratoconus prediction using a finite element model of the cornea with local biomechanical properties

PURPOSE: The ability to predict and understand which biomechanical properties of the cornea are responsible for the stability or progression of keratoconus may be an important clinical and surgical tool for the eye-care professional. We have developed a finite element model of the cornea, that tries to predicts keratoconus-like behavior and its evolution based on material properties of the corneal tissue. METHODS: Corneal material properties were modeled using bibliographic data and corneal topography was based on literature values from a schematic eye model. Commercial software was used to simulate mechanical and surface properties when the cornea was subject to different local parameters, such as elasticity. RESULTS: The simulation has shown that, depending on the corneal initial surface shape, changes in local material properties and also different intraocular pressures values induce a localized protuberance and increase in curvature when compared to the remaining portion of the cornea. CONCLUSIONS: This technique provides a quantitative and accurate approach to the problem of understanding the biomechanical nature of keratoconus. The implemented model has shown that changes in local material properties of the cornea and intraocular pressure are intrinsically related to keratoconus pathology and its shape/curvature.

Effects of flooding of the River Paraná on the temporal activity of Anopheles (Nyssorhynchus) darlingi Root (Diptera: Culicidae), at the border state of Mato Grosso do Sul and São Paulo, Brazil

INTRODUCTION: Study of the temporal activity of malaria vectors during the implantation of a hydroelectric power station on the River Paraná, intended to generate electrical energy. The river separates the States of São Paulo and Mato Grosso do Sul, in Brazil. The objective was to verify whether alterations occurred in the wealth and diversity indices of Anopheles, following two successive floods, extended to the temporal activity and nycthemeral rhythm followed over a five year period. METHODS: Mosquito capture was performed monthly using the Human Attraction Technique and Shannon Traps. The first, executed for 24h, provided the nycthemeral rhythm and the second, lasting 15h, permitted the tracking of Anopheles during the two floods. RESULTS: The bimodal pattern of Anopheles darlingi defined before these floods was modified throughout the environment interventions. The same effect had repercussions on the populations of An albitarsis s.l., An triannulatus and An galvaoi. Activity prior to twilight was less affected by the environment alterations. CONCLUSIONS: The dam construction provoked changes in Anopheles temporal activity patterns, permitting classification of the area as an ecologically steady and unstable situation. Differences observed in Anopheles behavior due to the capture methods revealed the influence of solo and multiple attractiveness inside the populations studied.

Sand fly captures with Disney traps in area of occurrence of Leishmania (Leishmania) amazonensis in the state of Mato Grosso do Sul, mid-western Brazil

INTRODUCTION: The work was conducted to study phlebotomine fauna (Diptera: Psychodidae) and aspects of American cutaneous leishmaniasis transmission in a forested area where Leishmania (Leishmania) amazonensis occurs, situated in the municipality of Bela Vista, State of Mato Grosso do Sul, Brazil. METHODS: The captures were conducted with modified Disney traps, using hamster (Mesocricetus auratus) as bait, from May 2004 to January 2006. RESULTS: Ten species of phlebotomine sandflies were captured: Brumptomyia avellari, Brumptomyia brumpti, Bichromomyia flaviscutellata, Evandromyia bourrouli, Evandromyia lenti, Lutzomyia longipalpis, Psathyromyia campograndensis, Psathyromyia punctigeniculata, Psathyromyia shannoni and Sciopemyia sordellii. The two predominant species were Ev bourrouli (57.3%) and Bi flaviscutellata (41.4%), present at all sampling sites. Two of the 36 hamsters used as bait presented natural infection with Leishmania. The parasite was identified as Leishmania (Leishmania) amazonensis. CONCLUSIONS: Analysis of the results revealed the efficiency of Disney traps for capturing Bichromomyia flaviscutellata and the simultaneous presence of both vector and the Leishmania species transmitted by the same can be considered a predictive factor of the occurrence of leishmaniasis outbreaks for the human population that occupies the location.

Study of infection by Rickettsiae of the spotted fever group in humans and ticks in an urban park located in the City of Londrina, State of Paraná, Brazil

INTRODUCTION: Spotted fevers are emerging zoonoses caused by Rickettsia species in the spotted fever group (SFG). Rickettsia rickettsii is the main etiologic agent of Brazilian spotted fever (BSF) and it is transmitted by Amblyomma spp. ticks. METHODS: The study aimed to investigate SFG rickettsiae in the Arthur Thomas Municipal Park in Londrina, PR, by collecting free-living ticks and ticks from capybaras and blood samples from personnel working in these areas. Samples from A. dubitatum and A. cajennense were submitted for PCR in pools to analyze the Rickettsia spp. gltA (citrate synthase gene). RESULTS: All the pools analyzed were negative. Human sera were tested by indirect immunofluorescence assay with R. rickettsii and R. parkeri as antigens. Among the 34 sera analyzed, seven (20.6%) were reactive for R. rickettsii: four of these had endpoint titers equal to 64, 2 titers were 128 and 1 titer was 256. None of the samples were reactive for R. parkeri. An epidemiological questionnaire was applied to the park staff, but no statistically significant associations were identified. CONCLUSIONS: The serological studies suggest the presence of Rickettsiae related to SFG that could be infecting the human population studied; however, analysis of the ticks collected was unable to determine which species may be involved in transmission to humans.

Impact of chronic disease on quality of life among the elderly in the state of São Paulo, Brazil: a population-based study

Determinar el impacto de las enfermedades crónicas y el número de enfermedades en los diversos aspectos de la calidad de vida relacionada con la salud (HRQOL) en adultos mayores de São Paulo, Brasil. MÉTODOS: Se empleó la encuesta de salud SF-36® para evaluar el impacto de las enfermedades crónicas de mayor prevalencia sobre la HRQOL. Se realizó un estudio poblacional transversal con un muestreo por conglomerados estratificado en dos etapas. Se obtuvieron los datos de una encuesta multicéntrica sobre la salud aplicada mediante entrevistas en hogares de varios municipios del estado de São Paulo. Se evaluaron siete enfermedades -artritis, dolor de espalda, depresión/ansiedad, diabetes, hipertensión arterial, osteoporosis y accidentes cerebrovasculares- y sus efectos sobre la calidad de vida. RESULTADOS: De los 1 958 adultos mayores de 60 años o más, 13,6% informaron no padecer ninguna de las enfermedades, mientras 45,7% presentaron tres enfermedades crónicas o más. La presencia de cualquiera de las siete enfermedades crónicas estudiadas influyó significativamente en la puntuación de casi todas las escalas de la SF-36®. La HRQOL alcanzó valores inferiores cuando la persona tenía depresión/ansiedad, osteoporosis o había sufrido un accidente cerebrovascular. A mayor número de enfermedades, mayores eran los efectos negativos en las dimensiones de la SF-36®. La presencia de tres enfermedades o más afectó significativamente la HRQOL en todas las áreas. Las escalas más afectadas por las enfermedades fueron dolor físico, salud general y vitalidad. CONCLUSIONES: Se encontró una alta prevalencia de enfermedades crónicas en la población de adultos mayores; la magnitud del efecto sobre la HRQOL dependió del tipo de enfermedad. Estos resultados destacan la importancia de prevenir y controlar las enfermedades crónicas para reducir la comorbilidad y disminuir su impacto sobre la HRQOL en los adultos mayores

Investigation of measles IgM-seropositive cases of febrile rash illnesses in the absence of documented measles virus transmission, State of São Paulo, Brazil, 2000-2004

Introdução: Revisar os casos de doenças febris exantemáticas com IgM reagente contra o sarampo, no estado de São Paulo, Brasil, durante os cinco anos seguidos a interrupção da transmissão do vírus do sarampo. Métodos: Nós revisamos 463 casos de doenças febris exantemáticas com IgM reagente contra o sarampo, no estado de São Paulo, Brasil, de 2000 a 2004. Indivíduos vacinados contra o sarampo 56 dias antes da coleta de amostra foram considerados expostos à vacina. Soros da fase aguda e de convalescença foram testados para a evidência de infecção de sarampo, rubéola, parvovírus B19 e herpes vírus 6. Na ausência de soroconversão para imunoglobulina G contra o sarampo, casos com IgM reagente contra o sarampo foram considerados falsos positivos em pessoas com evidência de outras infecções virais. Resultados: Entre as 463 pessoas com doenças febris exantemáticas que testaram positivo para anticorpos IgM contra o sarampo durante o período, 297 (64 por cento) pessoas foram classificadas como expostas à vacina. Entre os 166 casos não expostos à vacina, 109 (66 por cento) foram considerados falsos positivos baseado na ausência de soroconversão, dos quais 21 (13 por cento) tiveram evidência de infecção por vírus da rubéola, 49 (30 por cento) parvovírus B19 e 28 (17 por cento) infecção por herpes vírus humano 6. Conclusões: Após a interrupção da transmissão do vírus do sarampo é necessária exaustiva investigação dos casos com IgM reagente contra o sarampo, especialmente dos casos não expostos à vacina. Testes laboratoriais para etiologias das doenças febris exantemáticas ajudam na interpretação destes casos

Prevalence of inadequate nutrient intake in a group of adolescents using dietary reference intake- based methods

O objetivo do presente estudo foi avaliar a prevalência de ingestão inadequada de nutrientes em um grupo de adolescentes de São Bernardo do Campo-SP. Dados de consumo de energia e nutrientes foram obtidos por meio de recordatórios de 24 horas aplicados em 89 adolescentes. A prevalência de inadequação foi calculada utilizando o método EAR como ponto de corte, após ajuste pela variabilidade intrapessoal, utilizando o procedimento desenvolvido pela Iowa State University. As Referências de Ingestão Dietética (IDR) foram os valores de referência para ingestão. Para os nutrientes que não possuem EAR estabelecida, a distribuição do consumo foi comparada com a AI. As maiores prevalências de inadequação em ambos sexos foram observadas para o magnésio (99,3 por cento para o sexo masculino e 81,8 por cento para o feminino), zinco (44,0 por cento para o sexo masculino e 23,5 por cento para o feminino), vitamina C (57,2 por cento para o sexo masculino e 59,9 por cento para o feminino) e folato (34,8 por cento para o sexo feminino). A proporção de indivíduos com ingestão superior à AI foi insignificante (menor que 2,0 por cento) em ambos os sexos

An Extension of the Invariance Principle for a Class of Differential Equations with Finite Delay

An extension of the uniform invariance principle for ordinary differential equations with finite delay is developed. The uniform invariance principle allows the derivative of the auxiliary scalar function V to be positive in some bounded sets of the state space while the classical invariance principle assumes that. V <= 0. As a consequence, the uniform invariance principle can deal with a larger class of problems. The main difficulty to prove an invariance principle for functional differential equations is the fact that flows are defined on an infinite dimensional space and, in such spaces, bounded solutions may not be precompact. This difficulty is overcome by imposing the vector field taking bounded sets into bounded sets.

Bioassay guided purification of the antimicrobial fraction of a Brazilian propolis from Bahia state

Background: Brazilian propolis type 6 (Atlantic forest, Bahia) is distinct from the other types of propolis especially due to absence of flavonoids and presence of other non-polar, long chain compounds, but presenting good in vitro and in vivo antimicrobial activity. Several authors have suggested that fatty acids found in this propolis might be responsible for its antimicrobial activity; however, so far no evidence concerning this finding has been reported in the literature. The goals of this study were to evaluate the antibacterial activity of the main pure fatty acids in the ethanolic extract and fractions and elucidate the chemical nature of the bioactive compounds isolated from Brazilian propolis type 6. Methods: Brazilian propolis type 6 ethanolic extract (EEP), hexane fraction (H-Fr), major fatty acids, and isolated sub-fractions were analyzed using high performance liquid chromatography (HPLC), high resolution gas chromatography with flame ionization detection (HRGC-FID), and gas chromatography-mass spectrometry (GC-MS). Three sub-fractions of H-Fr were obtained through preparative HPLC. Antimicrobial activity of EEP, H-Fr, sub-fractions, and fatty acids were tested against Staphyloccus aureus ATCC 25923 and Streptococcus mutans Ingbritt 1600 using minimum inhibitory concentration (MIC) and minimum bactericidal concentration (MBC). Results: EEP and H-Fr inhibited the growth of the microorganisms tested; nevertheless, no antimicrobial activity was found for the major fatty acids. The three sub-fractions (1, 2, and 3) were isolated from H-Fr by preparative HPLC and only sub-fraction 1 showed antimicrobial activity. Conclusion: a) The major fatty acids tested were not responsible for the antimicrobial activity of propolis type 6; b) Sub-fraction 1, belonging to the benzophenone class, was responsible for the antimicrobial activity observed in the present study. The identification of the bioactive compound will improve the development of more efficient uses of this natural product.

Ionic and Histological Studies of Salivary Glands in Rats with Diabetes and Their Glycemic State After Laser Irradiation

Objectives: To evaluate the effect of laser irradiation (LI) on the glycemic state and the histological and ionic parameters of the parotid and submandibular glands in rats with diabetes. Methods: One hundred twenty female rats were divided into eight groups. Diabetes was induced by administration of streptozotocin and confirmed later according to results of glycemia testing. Twenty-nine days after the induction, the parotid and submandibular glands of the rats were irradiated with 5, 10, and 20 J/cm(2) using a laser diode (660nm/100mW) (without diabetes: C5, C10, and C20; with diabetes: D5, D10, and D20, respectively). On the following day, the rats were euthanized, and blood glucose determined. Histological and ionic analyses were performed. Results: Rats with diabetes without irradiation (D0) showed lipid droples accumulation in the parotid gland, but accumulation decreased after 5, 10, and 20 J/cm(2) of laser irradiation. A decrease in fasting glycemia level from 358.97 +/- 56.70 to 278.33 +/- 87.98mg/dL for D5 and from 409.50 +/- 124.41 to 231.80 +/- 120.18 mg/dL for D20 (p < 0.05) was also observed. Conclusion: LI should be explored as an auxiliary therapy for control of complications of diabetes because it can alter the carbohydrate and lipid metabolism of rats with diabetes.