Reinforcement Learning by Policy Search

One objective of artificial intelligence is to model the behavior of an intelligent agent interacting with its environment. The environment's transformations can be modeled as a Markov chain, whose state is partially observable to the agent and affected by its actions; such processes are known as partially observable Markov decision processes (POMDPs). While the environment's dynamics are assumed to obey certain rules, the agent does not know them and must learn. In this dissertation we focus on the agent's adaptation as captured by the reinforcement learning framework. This means learning a policy---a mapping of observations into actions---based on feedback from the environment. The learning can be viewed as browsing a set of policies while evaluating them by trial through interaction with the environment. The set of policies is constrained by the architecture of the agent's controller. POMDPs require a controller to have a memory. We investigate controllers with memory, including controllers with external memory, finite state controllers and distributed controllers for multi-agent systems. For these various controllers we work out the details of the algorithms which learn by ascending the gradient of expected cumulative reinforcement. Building on statistical learning theory and experiment design theory, a policy evaluation algorithm is developed for the case of experience re-use. We address the question of sufficient experience for uniform convergence of policy evaluation and obtain sample complexity bounds for various estimators. Finally, we demonstrate the performance of the proposed algorithms on several domains, the most complex of which is simulated adaptive packet routing in a telecommunication network.

Fast Learning by Bounding Likelihoods in Sigmoid Type Belief Networks

Sigmoid type belief networks, a class of probabilistic neural networks, provide a natural framework for compactly representing probabilistic information in a variety of unsupervised and supervised learning problems. Often the parameters used in these networks need to be learned from examples. Unfortunately, estimating the parameters via exact probabilistic calculations (i.e, the EM-algorithm) is intractable even for networks with fairly small numbers of hidden units. We propose to avoid the infeasibility of the E step by bounding likelihoods instead of computing them exactly. We introduce extended and complementary representations for these networks and show that the estimation of the network parameters can be made fast (reduced to quadratic optimization) by performing the estimation in either of the alternative domains. The complementary networks can be used for continuous density estimation as well.

Cambio climático y mundo en desarrollo: las insidias del learning by doing… (Otros Temas)

No hay duda, el mundo parece estar de acuerdo en que la reunión de Copenhague, el pasado mes de diciembre, fue un fracaso rotundo. Hay quienes no rescatan nada de la cumbre, sencillamente todo acabó en una operación de fachada de última hora, en la que un puñado de países, en nombre de todos los demás y para sacar la cara frente al mundo, presentaba el famoso Acuerdo de Copenhague.

Rapid word learning by 15-month-olds under tightly-controlled conditions

Infants (12 to 17 months) were taught 2 novel words for 2 images of novel objects, by pairing isolated auditory labels with to-be-associated images. Comprehension was tested using a preferential looking task in which the infant was presented with both images together with an isolated auditory label. The auditory label usually, but not always, matched one of the images. Infants looked preferentially at images that matched the auditory stimulus. The experiment controlled within-subjects for both side bias and preference for previously named items. Infants showed learning after 12 presentations of the new words. Evidence is presented that, in certain circumstances, the duration of longest look at a target may be a more robust measure of target preference than overall looking time. The experiment provides support for previous demonstrations of rapid word learning by pre-vocabulary spurt children, and offers some methodological improvements to the preferential looking task.

History of hypertension and eplerenone in patients with acute myocardial infarction complicated by heart failure

In the Eplerenone Post-Acute Myocardial Infarction Heart Failure Efficacy and Survival Study ( n = 6632), eplerenone- associated reduction in all- cause mortality was significantly greater in those with a history of hypertension ( Hx- HTN). There were 4007 patients with Hx- HTN ( eplerenone: n = 1983) and 2625 patients without Hx- HTN ( eplerenone: n = 1336). Propensity scores for eplerenone use, separately calculated for patients with and without Hx- HTN, were used to assemble matched cohorts of 1838 and 1176 pairs of patients. In patients with Hx- HTN, all- cause mortality occurred in 18% of patients treated with placebo ( rate, 1430/ 10 000 person- years) and 14% of patients treated with eplerenone ( rate, 1058/ 10 000 person- years) during 2350 and 2457 years of follow- up, respectively ( hazard ratio [ HR]: 0.71; 95% CI: 0.59 to 0.85; P < 0.0001). Composite end point of cardiovascular hospitalization or cardiovascular mortality occurred in 33% of placebo-treated patients ( 3029/ 10 000 person- years) and 28% of eplerenone- treated patients (2438/10 000 person- years) with Hx- HTN ( HR: 0.82; 95% CI: 0.72 to 0.94; P = 0.003). In patients without Hx- HTN, eplerenone reduced heart failure hospitalization ( HR: 73; 95% CI: 0.55 to 0.97; P = 0.028) but had no effect on mortality ( HR: 0.91; 95% CI: 0.72 to 1.15; P = 0.435) or on the composite end point ( HR: 0.91; 95% CI: 0.76 to 1.10; P = 0.331). Eplerenone should, therefore, be prescribed to all of the post - acute myocardial infarction patients with reduced left ventricular ejection fraction and heart failure regardless of Hx- HTN.

Learning by Reading : A literature study on the use of authentic texts in the EFL upper elementary classroom

The English language is widely used throughout the world and has become a core subject in many countries, especially for students in the upper elementary classroom. While textbooks have been the preferred EFL teaching method for a long time, this belief has seemingly changed within the last few years. Therefore, this study looks at what prior research says about the use of authentic texts in the EFL upper elementary classroom with an aim to answer research questions on how teachers can work with authentic texts, what the potential benefits of using authentic texts are and what teachers and students say about the use of authentic texts in the EFL classroom. While this thesis is written from a Swedish perspective, it is recognized that many countries teach EFL. Therefore, international results have also been taken into consideration and seven previous research studies have been analyzed in order to gain a better understanding of the use of authentic texts in the EFL classroom. Results indicate that the use of authentic texts is beneficial in teaching EFL. However, many teachers are still reluctant to use these, mainly because of time constraints and the belief that such texts are too difficult for their students. Since these findings are mainly focused on areas outside of Sweden, additional research is needed before conclusions can be drawn on the use of authentic texts in the Swedish upper elementary EFL classroom.

Student learning: "the heart of quality" in education and training

This paper surveys the development of various approaches to quality that are essentially learning-centred:

•In the Schools sector: a brief overview of the Victorian Quality in Schools project;

•In Higher Education: work in progress at two Australian universities (Victoria University of Technology and Swinburne Universities of Technology in Melbourne); and

•In Vocational Education and Training: work in progress in re-orienting the policy approach to Quality towards a more flexible and learning-centred model.

This paper will argue that when looked at from the perspective of the individual learner, there is a strong case for student learning to be placed at the very heart of Quality Systems in all sectors of education, and also therefore in related sectoral Quality Assurance programs and processes.

Plagiarism – can preventative measures and a commitment to the value of learning by academics and students reduce plagiarism in classrooms?

Plagiarism is an interruptive act in the teaching and learning value chain. This paper analyses the impact of both plagiarism and effective assessment in the learning value chain. Effective assessment is the positive outcome. Plagiarism negates or breaks the chain. Anecdotal evidence suggests the incidences of plagiarism are increasing in faculties across the Queensland University of Technology (QUT), and where every subject/unit outline includes a statement and warning to students of the penalties of this activity, deterrence tools do not overcome the harmful impact of plagiarism on the teacher/student learning relationship. A working party established in 2000 in the Faculty of Business at QUT, examined the literature, university policies, teaching and learning practices, and examples of plagiarism and penalties in order to identify the wider learning and assessment issues surrounding plagiarism and options for action and policy. A three-semester study of acts of plagiarism and wide consultation with staff presented the working party with an opportunity to develop a set of preventative measures, and also exposed the legal, cultural and accountability issues of diverse attitudes and values.

The paper presents the findings and recommendations from the investigation, and also proposes an interpretation of marketing theory of the impact of plagiarism on the teaching and learning value chain.