Unsupervised reinforcement learning via state entropy maximization

Reinforcement Learning (RL) provides a powerful framework to address sequential decision-making problems in which the transition dynamics is unknown or too complex to be represented. The RL approach is based on speculating what is the best decision to make given sample estimates obtained from previous interactions, a recipe that led to several breakthroughs in various domains, ranging from game playing to robotics. Despite their success, current RL methods hardly generalize from one task to another, and achieving the kind of generalization obtained through unsupervised pre-training in non-sequential problems seems unthinkable. Unsupervised RL has recently emerged as a way to improve generalization of RL methods. Just as its non-sequential counterpart, the unsupervised RL framework comprises two phases: An unsupervised pre-training phase, in which the agent interacts with the environment without external feedback, and a supervised fine-tuning phase, in which the agent aims to efficiently solve a task in the same environment by exploiting the knowledge acquired during pre-training. In this thesis, we study unsupervised RL via state entropy maximization, in which the agent makes use of the unsupervised interactions to pre-train a policy that maximizes the entropy of its induced state distribution. First, we provide a theoretical characterization of the learning problem by considering a convex RL formulation that subsumes state entropy maximization. Our analysis shows that maximizing the state entropy in finite trials is inherently harder than RL. Then, we study the state entropy maximization problem from an optimization perspective. Especially, we show that the primal formulation of the corresponding optimization problem can be (approximately) addressed through tractable linear programs. Finally, we provide the first practical methodologies for state entropy maximization in complex domains, both when the pre-training takes place in a single environment as well as multiple environments.

The transformative potential of community engagement in higher education: an analysis of service-learning effects on students, communities, and institutions.

Service-learning in higher education is gaining attention as a reliable tool to support students’ learning and fulfil the mission of higher education institutions (HEIs). This dissertation addresses existing gaps in the literature by examining the effects and perspectives of service-learning in HEIs through three studies. The first study compares the effects of a voluntary semester-long service-learning course with traditional courses. A survey completed by 110 students before and after the lectures found no significant group differences in the psychosocial variables under inspection. Nevertheless, service-learning students showed higher scores concerning the quality of participation. Factors such as students’ perception of competence, duration of service-learning, and self-reported measures may have influenced the results. The second study explores the under-researched perspective of community partners in higher education and European settings. Twelve semi-structured interviews were conducted with community partners from various community organisations across Europe. The results highlight positive effects on community members and organisations, intrinsic motivations, organisational empowerment, different forms of reciprocity, the co-educational role of community partners, and the significant role of a sense of community and belonging. The third study focuses on faculty perspectives on service-learning in the European context. Twenty-two semi-structured interviews were conducted in 14 European countries. The findings confirm the transformative impact of service-learning on the community, students, teachers, and HEIs, emphasising the importance of motivation and institutionalisation processes in sustaining engaged scholarship. The study also identifies the relevance of the community experience, sense of community, and community responsibility with the service-learning experience; relatedness is proposed as the fifth pillar of service-learning. Overall, this dissertation provides new insights into the effects and perspectives of service-learning in higher education. It integrates the 4Rs model with the addition of relatedness, guiding the theoretical and practical implications of the findings. The dissertation also suggests limitations and areas for further research.

Machine Learning in Automatic Tabulation for Constraint Model Reformulation

Combinatorial decision and optimization problems belong to numerous applications, such as logistics and scheduling, and can be solved with various approaches. Boolean Satisfiability and Constraint Programming solvers are some of the most used ones and their performance is significantly influenced by the model chosen to represent a given problem. This has led to the study of model reformulation methods, one of which is tabulation, that consists in rewriting the expression of a constraint in terms of a table constraint. To apply it, one should identify which constraints can help and which can hinder the solving process. So far this has been performed by hand, for example in MiniZinc, or automatically with manually designed heuristics, in Savile Row. Though, it has been shown that the performances of these heuristics differ across problems and solvers, in some cases helping and in others hindering the solving procedure. However, recent works in the field of combinatorial optimization have shown that Machine Learning (ML) can be increasingly useful in the model reformulation steps. This thesis aims to design a ML approach to identify the instances for which Savile Row’s heuristics should be activated. Additionally, it is possible that the heuristics miss some good tabulation opportunities, so we perform an exploratory analysis for the creation of a ML classifier able to predict whether or not a constraint should be tabulated. The results reached towards the first goal show that a random forest classifier leads to an increase in the performances of 4 different solvers. The experimental results in the second task show that a ML approach could improve the performance of a solver for some problem classes.

Ecological Literacy And Beyond: Problem-based Learning For Future Professionals.

Ecological science contributes to solving a broad range of environmental problems. However, lack of ecological literacy in practice often limits application of this knowledge. In this paper, we highlight a critical but often overlooked demand on ecological literacy: to enable professionals of various careers to apply scientific knowledge when faced with environmental problems. Current university courses on ecology often fail to persuade students that ecological science provides important tools for environmental problem solving. We propose problem-based learning to improve the understanding of ecological science and its usefulness for real-world environmental issues that professionals in careers as diverse as engineering, public health, architecture, social sciences, or management will address. Courses should set clear learning objectives for cognitive skills they expect students to acquire. Thus, professionals in different fields will be enabled to improve environmental decision-making processes and to participate effectively in multidisciplinary work groups charged with tackling environmental issues.

Application Of A Semi-empirical Model For The Evaluation Of Transmission Properties Of Barite Mortar.

The aim of this study was to estimate barite mortar attenuation curves using X-ray spectra weighted by a workload distribution. A semi-empirical model was used for the evaluation of transmission properties of this material. Since ambient dose equivalent, H(⁎)(10), is the radiation quantity adopted by IAEA for dose assessment, the variation of the H(⁎)(10) as a function of barite mortar thickness was calculated using primary experimental spectra. A CdTe detector was used for the measurement of these spectra. The resulting spectra were adopted for estimating the optimized thickness of protective barrier needed for shielding an area in an X-ray imaging facility.

Evaluation Of Characteristic-to-total Spectrum Ratio: Comparison Between Experimental And A Semi-empirical Model.

Primary X-ray spectra were measured in the range of 80-150kV in order to validate a computer program based on a semiempirical model. The ratio between the characteristic and total air Kerma was considered to compare computed results and experimental data. Results show that the experimental spectra have higher first HVL and mean energy than the calculated ones. The ratios between the characteristic and total air Kerma for calculated spectra are in good agreement with experimental results for all filtrations used.

Semi-determinate Growth Habit Adjusts The Vegetative-to-reproductive Balance And Increases Productivity And Water-use Efficiency In Tomato (solanum Lycopersicum).

Tomato (Solanum lycopersicum) shows three growth habits: determinate, indeterminate and semi-determinate. These are controlled mainly by allelic variation in the SELF-PRUNING (SP) gene family, which also includes the florigen gene SINGLE FLOWER TRUSS (SFT). Determinate cultivars have synchronized flower and fruit production, which allows mechanical harvesting in the tomato processing industry, whereas indeterminate ones have more vegetative growth with continuous flower and fruit formation, being thus preferred for fresh market tomato production. The semi-determinate growth habit is poorly understood, although there are indications that it combines advantages of determinate and indeterminate growth. Here, we used near-isogenic lines (NILs) in the cultivar Micro-Tom (MT) with different growth habit to characterize semi-determinate growth and to determine its impact on developmental and productivity traits. We show that semi-determinate genotypes are equivalent to determinate ones with extended vegetative growth, which in turn impacts shoot height, number of leaves and either stem diameter or internode length. Semi-determinate plants also tend to increase the highly relevant agronomic parameter Brix×ripe yield (BRY). Water-use efficiency (WUE), evaluated either directly as dry mass produced per amount of water transpired or indirectly through C isotope discrimination, was higher in semi-determinate genotypes. We also provide evidence that the increases in BRY in semi-determinate genotypes are a consequence of an improved balance between vegetative and reproductive growth, a mechanism analogous to the conversion of the overly vegetative tall cereal varieties into well-balanced semi-dwarf ones used in the Green Revolution.

Resistência mecânica de hidrogéis termo-sensíveis constituídos de Alginato-Ca2+ / PNIPAAm, tipo Semi-IPN

Thermosensitive hydrogels were synthesized using alginate-Ca2+ in association with a thermosensitive polymer, such as PNIPAAm. The mechanical properties of the hydrogels were determined measuring the maximum tension of deformation. With the increase of the temperature by 25 to 40 ºC above the LCST the chains of PNIPAAm collapsed, dragging the alginate net and diminishing the size of the pores. The decrease in the size of the pores of the hydrogel was followed by an increase in the mechanicals resistance of the material.

Flicker perimetry in healthy subjects: influence of age and gender, learning effect and short-term fluctuation

PURPOSE: To determine the mean critical fusion frequency and the short-term fluctuation, to analyze the influence of age, gender, and the learning effect in healthy subjects undergoing flicker perimetry. METHODS: Study 1 - 95 healthy subjects underwent flicker perimetry once in one eye. Mean critical fusion frequency values were compared between genders, and the influence of age was evaluated using linear regression analysis. Study 2 - 20 healthy subjects underwent flicker perimetry 5 times in one eye. The first 3 sessions were separated by an interval of 1 to 30 days, whereas the last 3 sessions were performed within the same day. The first 3 sessions were used to investigate the presence of a learning effect, whereas the last 3 tests were used to calculate short-term fluctuation. RESULTS: Study 1 - Linear regression analysis demonstrated that mean global, foveal, central, and critical fusion frequency per quadrant significantly decreased with age (p<0.05).There were no statistically significant differences in mean critical fusion frequency values between males and females (p>0.05), with the exception of the central area and inferonasal quadrant (p=0.049 and p=0.011, respectively), where the values were lower in females. Study 2 - Mean global (p=0.014), central (p=0.008), and peripheral (p=0.03) critical fusion frequency were significantly lower in the first session compared to the second and third sessions. The mean global short-term fluctuation was 5.06±1.13 Hz, the mean interindividual and intraindividual variabilities were 11.2±2.8% and 6.4±1.5%, respectively. CONCLUSION: This study suggests that, in healthy subjects, critical fusion frequency decreases with age, that flicker perimetry is associated with a learning effect, and that a moderately high short-term fluctuation is expected.

Sensitivity and specificity of machine learning classifiers for glaucoma diagnosis using Spectral Domain OCT and standard automated perimetry

PURPOSE: To evaluate the sensitivity and specificity of machine learning classifiers (MLCs) for glaucoma diagnosis using Spectral Domain OCT (SD-OCT) and standard automated perimetry (SAP). METHODS: Observational cross-sectional study. Sixty two glaucoma patients and 48 healthy individuals were included. All patients underwent a complete ophthalmologic examination, achromatic standard automated perimetry (SAP) and retinal nerve fiber layer (RNFL) imaging with SD-OCT (Cirrus HD-OCT; Carl Zeiss Meditec Inc., Dublin, California). Receiver operating characteristic (ROC) curves were obtained for all SD-OCT parameters and global indices of SAP. Subsequently, the following MLCs were tested using parameters from the SD-OCT and SAP: Bagging (BAG), Naive-Bayes (NB), Multilayer Perceptron (MLP), Radial Basis Function (RBF), Random Forest (RAN), Ensemble Selection (ENS), Classification Tree (CTREE), Ada Boost M1(ADA),Support Vector Machine Linear (SVML) and Support Vector Machine Gaussian (SVMG). Areas under the receiver operating characteristic curves (aROC) obtained for isolated SAP and OCT parameters were compared with MLCs using OCT+SAP data. RESULTS: Combining OCT and SAP data, MLCs' aROCs varied from 0.777(CTREE) to 0.946 (RAN).The best OCT+SAP aROC obtained with RAN (0.946) was significantly larger the best single OCT parameter (p<0.05), but was not significantly different from the aROC obtained with the best single SAP parameter (p=0.19). CONCLUSION: Machine learning classifiers trained on OCT and SAP data can successfully discriminate between healthy and glaucomatous eyes. The combination of OCT and SAP measurements improved the diagnostic accuracy compared with OCT data alone.

Na "periferia" da quadra : educação fisica, cultura e sociabilidade na escola

Universidade Estadual de Campinas . Faculdade de Educação Física

Pedagogia do esporte : identificação, discussão e aplicação de procedimentos pedagogicos no processo de ensino-vivencia aprendizagem da modalidade basquetebol

Universidade Estadual de Campinas . Faculdade de Educação Física

Procedimentos pedagogicos para o ensino das lutas : contextos e possibilidades

Universidade Estadual de Campinas . Faculdade de Educação Física

A influencia dos agentes sociais nos interesses fisico-esportivos do lazer

Universidade Estadual de Campinas. Faculdade de Educação Física

A questão da licenciatura em Educação Fisica : a transição a pratica profissional

Universidade Estadual de Campinas. Faculdade de Educação Física