Dynamic equilibrium through reinforcement learning

Reinforcement Learning is an area of Machine Learning that deals with how an agent should take actions in an environment such as to maximize the notion of accumulated reward. This type of learning is inspired by the way humans learn and has led to the creation of various algorithms for reinforcement learning. These algorithms focus on the way in which an agent’s behaviour can be improved, assuming independence as to their surroundings. The current work studies the application of reinforcement learning methods to solve the inverted pendulum problem. The importance of the variability of the environment (factors that are external to the agent) on the execution of reinforcement learning agents is studied by using a model that seeks to obtain equilibrium (stability) through dynamism – a Cart-Pole system or inverted pendulum. We sought to improve the behaviour of the autonomous agents by changing the information passed to them, while maintaining the agent’s internal parameters constant (learning rate, discount factors, decay rate, etc.), instead of the classical approach of tuning the agent’s internal parameters. The influence of changes on the state set and the action set on an agent’s capability to solve the Cart-pole problem was studied. We have studied typical behaviour of reinforcement learning agents applied to the classic BOXES model and a new form of characterizing the environment was proposed using the notion of convergence towards a reference value. We demonstrate the gain in performance of this new method applied to a Q-Learning agent.

Brincar para aprender: as intenções do educador

Relatório da Prática Profissional Supervisionada Mestrado em Educação Pré-Escolar

An exploratory study of orthoptic student satisfaction in a blended learning environment

Purpose: To describe orthoptic student satisfaction in a blended learning environment. Methods: Blended learning and teaching approaches that include a mix of sessions with elearning are being used since 2011/2012 involving final year (4th year) students from an orthoptic program. This approach is used in the module of research in orthoptics during the 1 semester. Students experienced different teaching approaches, which include seminars, tutorial group discussions and e-learning activities using the moodle platform. The Constructivist OnLine Learning Environment Survey (COLLES ) was applied at the end of the semester with 24 questions grouped in 6 dimensions with 4 items each: Relevance to professional practice, Reflection, Interactivity, Tutor support, Peer support and Interpretation. A 5-point Likert scale was used to score each individual item of the questionnaire (1 - almost never to 5 – almost always). The sum of items in each dimension ranged between 4 (negative perception) and 20 (positive perception). Results: Twenty-four students replied to the questionnaire. Positive points were related with Relevance (16.13±2.63), Reflection (16.46±2.45), Tutor support (16.29±2.10) and Interpretation (15.38±2.16). The majority of the students (n=18; 75%) think that the on-line learning is relevant to students’ professional practice. Critical reflections about learning contents were frequent (n=19; 79.17%). The tutor was able to stimulate critical thinking (n=21; 87.50%), encouraged students to participate (n=18; 75%) and understood well the student’s contributions (n=15; 62.50%). Less positive points were related with Interactivity (14.13±2.77) and Peer support (13.29±2.60). Response from the colleagues to ideas (n=11; 45.83%) and valorization of individual contributions (n=10; 41.67%) scored lower than other items. Conclusions: The flow back and forth between face-to-face and online learning situations helps the students to make critical reflections. The majority of the students are satisfied with a blended e-learning system environment. However, more work needs to be done to improve interactivity and peer support.

An approach to absorptive capacity based on the Portuguese textile industry

The main purpose of this research is to identify the hidden knowledge and learning mechanisms in the organization in order to disclosure the tacit knowledge and transform it into explicit knowledge. Most firms usually tend to duplicate their efforts acquiring extra knowledge and new learning skills while forgetting to exploit the existing ones thus wasting one life time resources that could be applied to increase added value within the firm overall competitive advantage. This unique value in the shape of creation, acquisition, transformation and application of learning and knowledge is not disseminated throughout the individual, group and, ultimately, the company itself. This work is based on three variables that explain the behaviour of learning as the process of construction and acquisition of knowledge, namely internal social capital, technology and external social capital, which include the main attributes of learning and knowledge that help us to capture the essence of this symbiosis. Absorptive Capacity provides the right tool to explore this uncertainty within the firm it is possible to achieve the perfect match between learning skills and knowledge needed to support the overall strategy of the firm. This study has taken in to account a sample of the Portuguese textile industry and it is based on a multisectorial analysis that makes it possible a crossfunctional analysis to check on the validity of results in order to better understand and capture the dynamics of organizational behavior.

An approach to absorptive capacity based on the Portuguese textile industry(Final version)

The main purpose of this research is to identify the hidden knowledge and learning mechanisms in the organization in order to disclosure the tacit knowledge and transform it into explicit knowledge. Most firms usually tend to duplicate their efforts acquiring extra knowledge and new learning skills while forgetting to exploit the existing ones thus wasting one life time resources that could be applied to increase added value within the firm overall competitive advantage. This unique value in the shape of creation, acquisition, transformation and application of learning and knowledge is not disseminated throughout the individual, group and, ultimately, the company itself. This work is based on three variables that explain the behaviour of learning as the process of construction and acquisition of knowledge, namely internal social capital, technology and external social capital, which include the main attributes of learning and knowledge that help us to capture the essence of this symbiosis. Absorptive Capacity provides the right tool to explore this uncertainty within the firm it is possible to achieve the perfect match between learning skills and knowledge needed to support the overall strategy of the firm. This study has taken in to account a sample of the Portuguese textile industry and it is based on a multisectorial analysis that makes it possible a crossfunctional analysis to check on the validity of results in order to better understand and capture the dynamics of organizational behavior.

Learning and Knowledge in the Context of Absorptive Capacity

Develop a new model of Absorptive Capacity taking into account two variables namely Learning and knowledge to explain how companies transform information into knowledge

Learning dependent sources using mixtures of Dirichlet: applications on hyperspectral unmixing

This paper is an elaboration of the DECA algorithm [1] to blindly unmix hyperspectral data. The underlying mixing model is linear, meaning that each pixel is a linear mixture of the endmembers signatures weighted by the correspondent abundance fractions. The proposed method, as DECA, is tailored to highly mixed mixtures in which the geometric based approaches fail to identify the simplex of minimum volume enclosing the observed spectral vectors. We resort then to a statitistical framework, where the abundance fractions are modeled as mixtures of Dirichlet densities, thus enforcing the constraints on abundance fractions imposed by the acquisition process, namely non-negativity and constant sum. With respect to DECA, we introduce two improvements: 1) the number of Dirichlet modes are inferred based on the minimum description length (MDL) principle; 2) The generalized expectation maximization (GEM) algorithm we adopt to infer the model parameters is improved by using alternating minimization and augmented Lagrangian methods to compute the mixing matrix. The effectiveness of the proposed algorithm is illustrated with simulated and read data.

Statistical motion learning for improved transform domain Wyner-Ziv video coding

Wyner - Ziv (WZ) video coding is a particular case of distributed video coding (DVC), the recent video coding paradigm based on the Slepian - Wolf and Wyner - Ziv theorems which exploits the source temporal correlation at the decoder and not at the encoder as in predictive video coding. Although some progress has been made in the last years, WZ video coding is still far from the compression performance of predictive video coding, especially for high and complex motion contents. The WZ video codec adopted in this study is based on a transform domain WZ video coding architecture with feedback channel-driven rate control, whose modules have been improved with some recent coding tools. This study proposes a novel motion learning approach to successively improve the rate-distortion (RD) performance of the WZ video codec as the decoding proceeds, making use of the already decoded transform bands to improve the decoding process for the remaining transform bands. The results obtained reveal gains up to 2.3 dB in the RD curves against the performance for the same codec without the proposed motion learning approach for high motion sequences and long group of pictures (GOP) sizes.

Development of fire-resistant (B1 & B2) winter 1 KPU foams applied by gun or straw adapter and optimization of a gun applied winter 1 KPU foam

A realização do presente trabalho teve como principais objectivos o desenvolvimento de espumas de poliuretano de um componente com propriedades de resistência à chama superiores (B1 & B2), aplicadas por pistola ou por adaptador/tubo e a optimização de uma espuma de poliuretano de um componente de inverno aplicada por pistola. Todo o trabalho desenvolvido está dividido em dois projectos distintos: i. O primeiro projecto consistiu em desenvolver espumas de um componente com propriedades de resistência à chama (classificadas como B1 e B2 de acordo com a norma alemã DIN 4102), aplicadas por pistola (GWB1 e GWB2) ou por adaptador/tubo (AWB), utilizando polióis poliésteres aromáticos modificados e aditivos retardantes de chama halogenados. Estas espumas deveriam apresentar também propriedades aceitáveis a baixas temperaturas. Após realizar várias formulações foi possível desenvolver uma espuma AWB2 com apenas 3,3% de poliol poliéster no pré-polímero e com propriedades equivalentes às da melhor espuma comercial mesmo a 5/-10 (temperatura da lata/cura da espuma em °C) e também com uma altura de chama de apenas 11 cm. A partir de duas formulações (AWB2) que passaram o Teste B2, foram obtidas também, uma espuma GWB2 e outra GWB1 com propriedades equivalentes às da melhor espuma da concorrência a -10/-10 e a 23/5, respectivamente, embora não tenham sido submetidas ao teste B2 e B1 após as modificações efectuadas. ii. O segundo projecto consistiu em optimizar uma espuma de poliuretano de um componente de inverno aplicada por pistola (GWB3). A espuma inicial tinha problemas de glass bubbles quando esta era dispensada a partir de uma lata cheia, sendo necessário ultrapassar este problema. Este problema foi resolvido diminuindo a razão de GPL/DME através do aumento da percentagem em volume de DME no pré-polímero para 14% no entanto, a estabilidade dimensional piorou um pouco. O reagente FCA 400 foi removido da formulação anterior (6925) numa tentativa de diminuir o custo da espuma, obtendo-se uma espuma aceitável a 23/23 e a 5/5, com uma redução de 4% no custo da produção e com uma redução de 5,5% no custo por litro de espuma dispensada, quando comparada com a sua antecessora. Por último, foi avaliada a influência da concentração de diferentes surfactantes na formulação 6925, verificando-se o melhoramento da estrutura celular da espuma para concentrções mais elevadas de surfactante, sendo este efeito mais notório a temperaturas mais baixas (5/5). Dos surfactantes estudados, o B 8871 mostrou o melhor desempenho a 5/5 com a concentração mais baixa, sendo portanto o melhor surfactante, enquanto o Struksilon 8003 demonstrou ser o menos adequado para esta formulação específica, apresentando piores resultados globais. Pode-se ainda acrescentar que os surfactantes L-5351, L-5352 e B 8526 também não são adequados para esta formulação uma vez que as espumas resultantes apresentam cell collapse, especialmente a 5/5. No caso dos surfactantes L-5351 e L-5352, esta propriedade piora com concentrações mais elevadas. Em cada projecto foram também efectuados testes de benchmark em determinadas espumas comerciais com o principal objectivo de comparar todos os resultados das espumas desenvolvidas, em ambos os projectos, com espumas da concorrência.

Learning techniques for automatic email message tagging

A organização automática de mensagens de correio electrónico é um desafio actual na área da aprendizagem automática. O número excessivo de mensagens afecta cada vez mais utilizadores, especialmente os que usam o correio electrónico como ferramenta de comunicação e trabalho. Esta tese aborda o problema da organização automática de mensagens de correio electrónico propondo uma solução que tem como objectivo a etiquetagem automática de mensagens. A etiquetagem automática é feita com recurso às pastas de correio electrónico anteriormente criadas pelos utilizadores, tratando-as como etiquetas, e à sugestão de múltiplas etiquetas para cada mensagem (top-N). São estudadas várias técnicas de aprendizagem e os vários campos que compõe uma mensagem de correio electrónico são analisados de forma a determinar a sua adequação como elementos de classificação. O foco deste trabalho recai sobre os campos textuais (o assunto e o corpo das mensagens), estudando-se diferentes formas de representação, selecção de características e algoritmos de classificação. É ainda efectuada a avaliação dos campos de participantes através de algoritmos de classificação que os representam usando o modelo vectorial ou como um grafo. Os vários campos são combinados para classificação utilizando a técnica de combinação de classificadores Votação por Maioria. Os testes são efectuados com um subconjunto de mensagens de correio electrónico da Enron e um conjunto de dados privados disponibilizados pelo Institute for Systems and Technologies of Information, Control and Communication (INSTICC). Estes conjuntos são analisados de forma a perceber as características dos dados. A avaliação do sistema é realizada através da percentagem de acerto dos classificadores. Os resultados obtidos apresentam melhorias significativas em comparação com os trabalhos relacionados.

Influence of LPG composition on straw applied one componente PU winter foam

O principal objectivo desta tese é obter uma relação directa entre a composição dos gases liquefeitos de petróleo (GLP), propano, n-butano e isobutano, usados como aerossóis propulsores numa lata de poliuretano de um componente, com as propriedades das espumas produzidas por spray. As espumas obtidas, terão de ter como requisito principal, um bom desempenho a temperaturas baixas, -10ºC, sendo por isso designadas por espumas de Inverno. Uma espuma é considerada como tendo um bom desempenho se não apresentar a -10/-10ºC (temperatura lata/ spray) glass bubbles, base holes e cell collapse. As espumas deverão ainda ter densidades do spray no molde a +23/+23ºC abaixo dos 30 g/L, um rendimento superior a 30 L, boa estabilidade dimensional e um caudal de espuma a +5/+5ºC superior a 5 g/s. Os ensaios experimentais foram realizados a +23/+23ºC, +5/+5ºC e a -10/-10ºC. A cada temperatura, as espumas desenvolvidas, foram submetidas a testes que permitiram determinar a sua qualidade. Testes esses que incluem os designados por Quick Tests (QT): o spray no papel e no molde das espumas nas referidas temperaturas. As amostras do papel e no molde são especialmente analisadas, quanto, às glass bubbles, cell collapse, base holes, cell structur e, cutting shrinkage, para além de outras propriedades. Os QT também incluem a análise da densidade no molde (ODM) e o estudo do caudal de espumas. Além dos QT foram realizados os testes da estabilidade dimensional das espumas, testes físicos de compressão e adesão, testes de expansão das espumas após spray e do rendimento por lata de espuma. Em todos os ensaios foi utilizado um tubo adaptador colocado na válvula da lata como método de spray e ainda mantida constante a proporção das matérias-primas (excepto os gases, em estudo). As experiências iniciaram-se com o estudo de GLPs presentes no mercado de aerossóis. Estes resultaram que o GLP: propano/ n-butano/ isobutano: (30/ 0/ 70 w/w%), produz as melhores espumas de inverno a -10/-10ºC, reduzindo desta forma as glass bubbles, base holes e o cell collapse produzido pelos restantes GLP usados como aerossóis nas latas de poliuretano. Testes posteriores tiveram como objectivo estudar a influência directa de cada gás, propano, n-butano e isobutano nas espumas. Para tal, foram usadas duas referências do estudo com GLP comercializáveis, 7396 (30 /0 /70 w/w %) e 7442 (0/ 0/ 100 w/w %). Com estes resultados concluí-se que o n-butano produz más propriedades nas espumas a -10/- 10ºC, formando grandes quantidades de glass bubbles, base holes e cell collapse. Contudo, o uso de propano reduz essas glass bubbles, mas em contrapartida, forma cell collapse.Isobutano, porém diminui o cell collapse mas não as glass bubbles. Dos resultados experimentais podemos constatar que o caudal a +5/+5ºC e densidade das espumas a +23/+23ºC, são influenciados pela composição do GLP. O propano e n-butano aumentam o caudal de espuma das latas e a sua densidade, ao contrário com o que acontece com o isobutano. Todavia, pelos resultados obtidos, o isobutano proporciona os melhores rendimentos de espumas por lata. Podemos concluir que os GLPs que contivessem cerca de 30 w/w % de propano (bons caudais a +5/+5ºC e menos glass bubbles a -10/-10ºC), e cerca 70 w/w % de isobutano (bons rendimentos de espumas, bem como menos cell collapse a -10/-10ºC) produziam as melhores espumas. Também foram desenvolvidos testes sobre a influência da quantidade de gás GLP presente numa lata. A análise do volume de GLP usado, foi realizada com base na melhor espuma obtida nos estudos anteriores, 7396, com um GLP (30 / 0/ 70 w/w%), e foram feitas alterações ao seu volume gás GLP presente no pré-polímero. O estudo concluiu, que o aumento do volume pode diminuir a densidade das espumas, e o seu decréscimo, um aumento da densidade. Também indico u que um mau ajuste do volume poderá causar más propriedades nas espumas. A análise económica, concluiu que o custo das espumas com mais GLP nas suas formulações, reduz-se em cerca de 3%, a quando de um aumento do volume de GLP no pré-polímero de cerca de 8 %. Esta diminuição de custos deveu-se ao facto, de um aumento de volume de gás, implicar uma diminuição na quantidade das restantes matérias-primas, com custos superiores, já que o volume útil total da lata terá de ser sempre mantido nos 750 mL. Com o objectivo de melhorar a qualidade da espuma 7396 (30/0/70 w/w %) obtida nos ensaios anteriores adicionou-se à formulação 7396 o HFC-152a (1,1-di fluoroetano). Os resultados demonstram que se formam espumas com más propriedades, especialmente a -10/-10ºC, contudo proporcionou excelentes shaking rate da lata. Através de uma pequena análise de custos não é aconselhável o seu uso pelos resultados obtidos, não proporcionando um balanço custo/benefício favorável. As três melhores espumas obtidas de todos os estudos foram comparadas com uma espuma de inverno presente no mercado. 7396 e 7638 com um volume de 27 % no prépolímero e uma composição de GLP (30/ 0 / 70 w/w%) e (13,7/ 0/ 86,3 w/w%), respectivamente, e 7690, com 37 % de volume no pré-polímero e GLP (30/ 0 / 70 w/w%), apresentaram em geral melhores resultados, comparando com a espuma benchmark . Contudo, os seus shaking rate a -10/-10ºC, de cada espuma, apresentaram valores bastante inferiores à composição benchmarking.

Da leitura à escrita : texto expositivo

Projeto de intervênção apresentado à Escola Superior de Educação para a obtenção do grau de mestre em Didática da Língua Portuguesa em 1º e 2º Ciclos do Ensino Básico

Development of an in situ penetration test for the uptake of preservatives in applied wood

This paper concerns the study of biocides application in old timber structures of maritime pine (Pinus pinaster Ail.), previously impregnated with other products. A method was developed in laboratory to determine in situ the penetration depth of a product applied superficially. As initial treatment, three traditional products for sawn timber for buildings were used and, for new treatments, two newer, more environmentally benign products were used. Their ability to penetrate the pre-treated surfaces was evaluated after 1, 2 and 3 applications at 24 hours intervals and the results obtained are presented. Finally, the applicability of the developed test to the in-situ evaluation of timber structures is also discussed.

Extreme Value Theory versus traditional GARCH approaches applied to financial data: a comparative evaluation

Although stock prices fluctuate, the variations are relatively small and are frequently assumed to be normal distributed on a large time scale. But sometimes these fluctuations can become determinant, especially when unforeseen large drops in asset prices are observed that could result in huge losses or even in market crashes. The evidence shows that these events happen far more often than would be expected under the generalized assumption of normal distributed financial returns. Thus it is crucial to properly model the distribution tails so as to be able to predict the frequency and magnitude of extreme stock price returns. In this paper we follow the approach suggested by McNeil and Frey (2000) and combine the GARCH-type models with the Extreme Value Theory (EVT) to estimate the tails of three financial index returns DJI,FTSE 100 and NIKKEI 225 representing three important financial areas in the world. Our results indicate that EVT-based conditional quantile estimates are much more accurate than those from conventional AR-GARCH models assuming normal or Student’s t-distribution innovations when doing out-of-sample estimation (within the insample estimation, this is so for the right tail of the distribution of returns).

e-learning in the classroom?

As teachers, we are challenged everyday to solve pedagogical problems and we have to fight for our students’ attention in a media rich world. I will talk about how we use ICT in Initial Teacher Training and give you some insight on what we are doing. The most important benefit of using ICT in education is that it makes us reflect on our practice. There is no doubt that our classrooms need to be updated, but we need to be critical about every peace of hardware, software or service that we bring into them. It is not only because our budgets are short, but also because e‐learning is primarily about learning, not technology. Therefore, we need to have the knowledge and skills required to act in different situations, and choose the best tool for the job. Not all subjects are suitable for e‐learning, nor do all students have the skills to organize themselves their own study times. Also not all teachers want to spend time programming or learning about instructional design and metadata. The promised land of easy use of authoring tools (e.g. eXe and Reload) that will lead to all teachers become Learning Objects authors and share these LO in Repositories, all this failed, like previously HyperCard, Toolbook and others. We need to know a little bit of many different technologies so we can mobilize this knowledge when a situation requires it: integrate e‐learning technologies in the classroom, not a flipped classroom, just simple tools. Lecture capture, mobile phones and smartphones, pocket size camcorders, VoIP, VLE, live video broadcast, screen sharing, free services for collaborative work, save, share and sync your files. Do not feel stressed to use everything, every time. Just because we have a whiteboard does not mean we have to make it the centre of the classroom. Start from where you are, with your preferred subject and the tools you master. Them go slowly and try some new tool in a non‐formal situation and with just one or two students. And you don’t need to be alone: subscribe a mailing list and share your thoughts with other teachers in a dedicated forum, even better if both are part of a community of practice, and share resources. We did that for music teachers and it was a success, in two years arriving at 1.000 members. Just do it.