Unsupervised reinforcement learning via state entropy maximization

Reinforcement Learning (RL) provides a powerful framework to address sequential decision-making problems in which the transition dynamics is unknown or too complex to be represented. The RL approach is based on speculating what is the best decision to make given sample estimates obtained from previous interactions, a recipe that led to several breakthroughs in various domains, ranging from game playing to robotics. Despite their success, current RL methods hardly generalize from one task to another, and achieving the kind of generalization obtained through unsupervised pre-training in non-sequential problems seems unthinkable. Unsupervised RL has recently emerged as a way to improve generalization of RL methods. Just as its non-sequential counterpart, the unsupervised RL framework comprises two phases: An unsupervised pre-training phase, in which the agent interacts with the environment without external feedback, and a supervised fine-tuning phase, in which the agent aims to efficiently solve a task in the same environment by exploiting the knowledge acquired during pre-training. In this thesis, we study unsupervised RL via state entropy maximization, in which the agent makes use of the unsupervised interactions to pre-train a policy that maximizes the entropy of its induced state distribution. First, we provide a theoretical characterization of the learning problem by considering a convex RL formulation that subsumes state entropy maximization. Our analysis shows that maximizing the state entropy in finite trials is inherently harder than RL. Then, we study the state entropy maximization problem from an optimization perspective. Especially, we show that the primal formulation of the corresponding optimization problem can be (approximately) addressed through tractable linear programs. Finally, we provide the first practical methodologies for state entropy maximization in complex domains, both when the pre-training takes place in a single environment as well as multiple environments.

Censored Linear Regression Models For Irregularly Observed Longitudinal Data Using The Multivariate-t Distribution.

In acquired immunodeficiency syndrome (AIDS) studies it is quite common to observe viral load measurements collected irregularly over time. Moreover, these measurements can be subjected to some upper and/or lower detection limits depending on the quantification assays. A complication arises when these continuous repeated measures have a heavy-tailed behavior. For such data structures, we propose a robust structure for a censored linear model based on the multivariate Student's t-distribution. To compensate for the autocorrelation existing among irregularly observed measures, a damped exponential correlation structure is employed. An efficient expectation maximization type algorithm is developed for computing the maximum likelihood estimates, obtaining as a by-product the standard errors of the fixed effects and the log-likelihood function. The proposed algorithm uses closed-form expressions at the E-step that rely on formulas for the mean and variance of a truncated multivariate Student's t-distribution. The methodology is illustrated through an application to an Human Immunodeficiency Virus-AIDS (HIV-AIDS) study and several simulation studies.

Reference Values For High-density Lipoprotein Particle Size And Volume By Dynamic Light Scattering In A Brazilian Population Sample And Their Relationships With Metabolic Parameters.

Current data indicate that the size of high-density lipoprotein (HDL) may be considered an important marker for cardiovascular disease risk. We established reference values of mean HDL size and volume in an asymptomatic representative Brazilian population sample (n=590) and their associations with metabolic parameters by gender. Size and volume were determined in HDL isolated from plasma by polyethyleneglycol precipitation of apoB-containing lipoproteins and measured using the dynamic light scattering (DLS) technique. Although the gender and age distributions agreed with other studies, the mean HDL size reference value was slightly lower than in some other populations. Both HDL size and volume were influenced by gender and varied according to age. HDL size was associated with age and HDL-C (total population); non- white ethnicity and CETP inversely (females); HDL-C and PLTP mass (males). On the other hand, HDL volume was determined only by HDL-C (total population and in both genders) and by PLTP mass (males). The reference values for mean HDL size and volume using the DLS technique were established in an asymptomatic and representative Brazilian population sample, as well as their related metabolic factors. HDL-C was a major determinant of HDL size and volume, which were differently modulated in females and in males.

Espectroscopia de Ressonância Magnética Nuclear de 13C no estudo de rotas biossintéticas de produtos naturais

During the last five decades, as a result of an interaction between natural product chemistry, synthetic organic chemistry, molecular biology and spectroscopy, scientists reached an extraordinary level of comprehension about the natural processes by which living organisms build up complex molecules. In this context, 13C nuclear magnetic resonance spectroscopy, allied with isotopic labeling, played a determinant role. Nowadays, the widespread use of modern NMR techniques allows an even more detailed picture of the biochemical steps by accurate manipulation of the atomic nuclei. This article focuses on the development of such techniques and their impact on biosynthetic studies.

Colheita de cana-de-açúcar com auxílio mecânico

Civil society and sugarcane farmer demands indicate that harvesting technology has enough limitations to jeopardize production in large sugar cane producing areas. This work analyses the current mechanical harvesting compared to a semi-mechanical harvesting proposal, on the bases of eleven characteristics considered determinant for a quick spreading of green cane harvesting on hilly areas, with lower impact on agricultural labor and farmers investment capacity.

Nossa notação m' na prova de Rorschach: ampliação e restrição do fator m de Klopfer

Klopfer's differentiation of movement scores, among which inanimate movement integrating the minor movements group, was no doubt opportune and needed. Their peculiar meaning has been clearly stressed and enriched by Piotrowski in his reformulation of Rorschach variables. It is our belief that Rorschach himself would take such step. Our criteria for scoring this determinant are somewhat different of both Klopfer's and Piotrowski's. On the one hand, masks, facial traits, emotional expressions, body parts in motion are not entered there. On the other, we score as such human or animal movement, provided this does not originate in the blot shape directly, but in the subjective reaction against the sensed muscular tension. Basic requirement for this scoring is the kinesthetic component, as for Mever since Rorschach's elaboration; and common trait distinctive for any response to be so scored ? be it an abstraction, an inanimate object, an animal or human being ? must be the subjective way of feeling the movement: (a) intention, blocking, struggle for achieving, for instance, or (b) activity of nature elements. Due to this subjective meaning we use the symbol m'intead of mfor this category. We already find these two kinds (a) and (b) of movement responses in Rorschach's text, respectively in the Examplesand in his posthumous Contribution.Either may point to a flight from emotional stress or to outstanding mental ability.

Procurando o lazer no Constituinte : sua inclusão como direito social na Constituição de 1988

Universidade Estadual de Campinas . Faculdade de Educação Física

Procedimentos pedagogicos para o ensino das lutas : contextos e possibilidades

Universidade Estadual de Campinas . Faculdade de Educação Física

Avaliação do desempenho anaeróbio de atletas de rugby em cadeira de rodas = : Anaerobic performance evaluation of wheelchair rugby athletes

Universidade Estadual de Campinas . Faculdade de Educação Física

Trabalho, tempo livre e emancipação humana : os determinantes ontologicos das politicas sociais de lazer

Universidade Estadual de Campinas . Faculdade de Educação Física

A pratica do basquetebol feminino no estado de São Paulo : conhecendo e analisando seu contexto

Universidade Estadual de Campinas . Faculdade de Educação Física

Prevalência de sobrepeso e obesidade e fatores associados em adolescentes na região centro-oeste do estado de São Paulo (SP, Brasil)

Objetivou-se avaliar a relação entre o Índice de Massa Corporal (IMC) e o índice CPOD em 207 adolescentes de 12 anos, de 8 escolas públicas e particulares da região centro-oeste do estado de São Paulo. A amostra foi constituída por 380 adolescentes aos 12 anos, de ambos os gêneros, sendo examinados 207. Utilizou-se o índice CPOD, IMC para peso, medida de estatura, e aplicou-se questionário sobre hábitos alimentares, características antropométricas e atividade física. Quanto ao peso corpóreo, 55,93% apresentaram normal (G4), 35,59% de baixo peso (G3), 8,47% de pré-obesos (G2), nas escolas particulares. Nas públicas, 52,03% apresentaram normal, baixo peso 41,22%, pré-obesos 4,73% e obesos (G1) 2,03%; não houve diferença significativa (p=0,45). Verificou-se que o CPOD nas escolas públicas foi 2,16 e nas particulares, 0,23 (p<0,05), sendo que 39,2% das crianças estavam livres de cárie nas municipais e nas particulares, 88,1%. Não houve correlação do maior IMC com o incremento de CPOD. Houve correlação negativa entre condições socioeconômicas e índice de cárie dentária. Concluiu-se que os grupos pré-obesos e obesos, embora houvesse maior frequência de ingestão de alimentos, não apresentaram correlação com o incremento de cárie dentária, mas as condições socioeconômicas foram determinantes para essa ocorrência.

Abrasion wear resistance of different artificial teeth opposed to metal and composite antagonists

One of the most important properties of artificial teeth is the abrasion wear resistance, which is determinant in the maintenance of the rehabilitation's occlusal pattern. OBJECTIVES: This in vitro study aims to evaluate the abrasion wear resistance of 7 brands of artificial teeth opposed to two types of antagonists. MATERIAL AND METHODS: Seven groups were prepared with 12 specimens each (BIOLUX & BL, TRILUX & TR, BLUE DENT & BD, BIOCLER & BC, POSTARIS & PO, ORTHOSIT & OR, GNATHOSTAR & GN), opposed to metallic (M & nickel-chromium alloy), and to composite antagonists (C & Solidex indirect composite). A mechanical loading device was used (240 cycles/min, 4 Hz speed, 10 mm antagonist course). Initial and final contours of each specimen were registered with aid of a profile projector (20x magnification). The linear difference between the two profiles was measured and the registered values were subjected to ANOVA and Tukey's test. RESULTS: Regarding the antagonists, only OR (M = 10.45 ± 1.42 µm and C = 2.77 ± 0.69 µm) and BC (M = 6.70 ± 1.37 µm and C = 4.48 ± 0.80 µm) presented statistically significant differences (p < 0.05). Best results were obtained with PO (C = 2.33 ± 0.91 µm and M = 1.78 ± 0.42 µm), followed by BL (C = 3.70 ± 1.32 µm and M = 3.70 ± 0.61 µm), statistically similar for both antagonists (p>0.05). Greater result variance was obtained with OR, which presented the worse results opposed to Ni-Cr (10.45 ± 1.42 µm), and results similar to the best ones against composite (2.77 ± 0.69 µm). CONCLUSIONS: Within the limitations of this study, it may be concluded that the antagonist material is a factor of major importance to be considered in the choice of the artificial teeth to be used in the prosthesis.

A comunicação dialógica como fator determinante para os processos de ensino-aprendizagem que ocorrem na capacitação rural: um estudo de caso em um órgão público de extensão localizado no interior do Estado de São Paulo

O objetivo deste artigo é discutir quais caminhos ou descaminhos estão sendo percorridos nos processos de capacitação rural de extensionistas e agricultores de algumas localidades do interior do Estado de São Paulo. Para tanto, a pesquisa aqui relatada teve como pressuposto analisar as práticas pedagógicas e os processos de comunicação que ocorrem durante as capacitações formativas entre extensionistas e agricultores, bem como entre formadores e extensionistas. Buscou-se, ainda, identificar as influências da comunicação nos processos de ensino e aprendizagem, a fim de se repensar práticas relativas à comunicação que devem ser utilizadas nos processos de ensino-aprendizagem. Para tanto, o método de coleta de dados foi o estudo de caso com viés fenomenológico. Os dados anunciam a necessidade da formação continuada com orientação de profissionais da pedagogia a fim de que mudanças efetivas possam ser realizadas.

Convecção linearmente organizada na área de Petrolina, semi-árido do Nordeste do Brasil: aspectos em meso e grande escala

É apresentado um estudo sobre sistemas convectivos linearmente organizados e observados por um radar meteorológico banda-C na região semi-árida do Nordeste do Brasil. São analisados três dias (27 a 29) de março de 1985, com ênfase na investigação do papel desempenhado por fatores locais e de grande escala no desenvolvimento dos sistemas. No cenário de grande escala, a área de cobertura do radar foi influenciada por um cavado de ar superior austral no dia 27 e por um vórtice ciclônico de altos níveis no dia 29. A convergência de umidade próxima à superfície favoreceu a atividade convectiva nos dias 27 e 29, enquanto que divergência de umidade próxima à superfície inibiu a atividade convectiva no dia 28. No cenário de mesoescala, foi observado que o aquecimento diurno é um fator importante para a formação de células convectivas, somando-se a ele o papel determinante da orografia na localização dos ecos. De maneira geral, as imagens de radar mostram os sistemas convectivos linearmente organizados em áreas elevadas e núcleos convectivos intensos envolvidos por uma área de precipitação estratiforme. Os resultados indicam que convergência do fluxo de umidade em grande escala e aquecimento radiativo, são fatores determinantes na evolução e desenvolvimento dos ecos na área de estudo.