Reinforcement learning for parameter estimation in statistical spoken dialogue systems

Reinforcement techniques have been successfully used to maximise the expected cumulative reward of statistical dialogue systems. Typically, reinforcement learning is used to estimate the parameters of a dialogue policy which selects the system's responses based on the inferred dialogue state. However, the inference of the dialogue state itself depends on a dialogue model which describes the expected behaviour of a user when interacting with the system. Ideally the parameters of this dialogue model should be also optimised to maximise the expected cumulative reward. This article presents two novel reinforcement algorithms for learning the parameters of a dialogue model. First, the Natural Belief Critic algorithm is designed to optimise the model parameters while the policy is kept fixed. This algorithm is suitable, for example, in systems using a handcrafted policy, perhaps prescribed by other design considerations. Second, the Natural Actor and Belief Critic algorithm jointly optimises both the model and the policy parameters. The algorithms are evaluated on a statistical dialogue system modelled as a Partially Observable Markov Decision Process in a tourist information domain. The evaluation is performed with a user simulator and with real users. The experiments indicate that model parameters estimated to maximise the expected reward function provide improved performance compared to the baseline handcrafted parameters. © 2011 Elsevier Ltd. All rights reserved.

Climate resilience of schools: a case study

Social and political concerns are frequently reflected in the design of school buildings, often in turn leading to the development of technical innovations. One example is a recurrent concern about the physical health of the nation, which has at several points over the last century prompted new design approaches to natural light and ventilation. The most critical concern of the current era is the global, rather than the indoor, environment. The resultant political focus on mitigating climate change has resulted in new regulations, and in turn considerable technical changes in building design and construction. The vanguard of this movement has again been in school buildings, set the highest targets for reducing operational carbon by the previous Government. The current austerity measures have moved the focus to the refurbishment and retrofit of existing buildings, in order to bring them up to the exacting new standards. Meanwhile there is little doubt that climate change is happening already, and that the impacts will be considerable. Climate scientists have increasing confidence in their predictions for the future; if today’s buildings are to be resilient to these changes, building designers will need to understand and design for the predicted climates in order to continue to provide comfortable and healthy spaces through the lifetimes of the buildings. This paper describes the decision processes, and the planned design measures, for adapting an existing school for future climates. The project is at St Faith’s School in Cambridge, and focuses on three separate buildings: a large Victorian block built as a substantial domestic dwelling in 1885, a smaller single storey 1970s block with a new extension, and an as-yet unbuilt single storey block designed to passivhaus principles and using environmentally friendly materials. The implications of climate change have been considered for the three particular issues of comfort, construction, and water, as set out in the report on Design for Future Climate: opportunities for adaptation in the built environment (Gething, 2010). The adaptation designs aim to ensure each of the three very different buildings remains fit for purpose throughout the 21st century, continuing to provide a healthy environment for the children. A forth issue, the reduction of carbon and the mitigation of other negative environmental impacts of the construction work, is also a fundamental aim for the school and the project team. Detailed modelling of both the operational and embodied energy and carbon of the design options is therefore being carried out, in order that the whole life carbon costs of the adaptation design options may be minimised. The project has been funded by the Technology Strategy Board as part of the Design for Future Climates programme; the interdisciplinary team includes the designers working on the current school building projects and the school bursar, supported by researchers from the University of Cambridge Centre for Sustainable Development. It is hoped that lessons from the design process, as well as the solutions themselves, will be transferable to other buildings in similar climatic regions.

Task-Irrelevant Perceptual Learning Specific to the Contrast Polarity of Motion Stimuli

Studies of perceptual learning have focused on aspects of learning that are related to early stages of sensory processing. However, conclusions that perceptual learning results in low-level sensory plasticity are of great controversy, largely because such learning can often be attributed to plasticity in later stages of sensory processing or in the decision processes. To address this controversy, we developed a novel random dot motion (RDM) stimulus to target motion cells selective to contrast polarity, by ensuring the motion direction information arises only from signal dot onsets and not their offsets, and used these stimuli in conjunction with the paradigm of task-irrelevant perceptual learning (TIPL). In TIPL, learning is achieved in response to a stimulus by subliminally pairing that stimulus with the targets of an unrelated training task. In this manner, we are able to probe learning for an aspect of motion processing thought to be a function of directional V1 simple cells with a learning procedure that dissociates the learned stimulus from the decision processes relevant to the training task. Our results show learning for the exposed contrast polarity and that this learning does not transfer to the unexposed contrast polarity. These results suggest that TIPL for motion stimuli may occur at the stage of directional V1 simple cells.

Applications of Feller-Reuter-Riley transition functions

A Feller–Reuter–Riley function is a Markov transition function whose corresponding semigroup maps the set of the real-valued continuous functions vanishing at infinity into itself. The aim of this paper is to investigate applications of such functions in the dual problem, Markov branching processes, and the Williams-matrix. The remarkable property of a Feller–Reuter–Riley function is that it is a Feller minimal transition function with a stable q-matrix. By using this property we are able to prove that, in the theory of branching processes, the branching property is equivalent to the requirement that the corresponding transition function satisfies the Kolmogorov forward equations associated with a stable q-matrix. It follows that the probabilistic definition and the analytic definition for Markov branching processes are actually equivalent. Also, by using this property, together with the Resolvent Decomposition Theorem, a simple analytical proof of the Williams' existence theorem with respect to the Williams-matrix is obtained. The close link between the dual problem and the Feller–Reuter–Riley transition functions is revealed. It enables us to prove that a dual transition function must satisfy the Kolmogorov forward equations. A necessary and sufficient condition for a dual transition function satisfying the Kolmogorov backward equations is also provided.

The influence of gender in the direction of Spanish cooperative societies on profitability and indebtedness: an empirical analysis

In this paper the claim for the market for a new business management to ensure the presence of women in decision -making to respond to new social needs addressed. Thus, this paper analyzes the influence of gender diversity of the directors on the profitability and the level of debt for a sample of 5,199 Spanish cooperatives. Unlike capitalist societies, these organizations have a number of peculiarities in their government, and that the partners are themselves major time, agents and customers. The study focuses on the Spanish context, where there is an open debate on the importance of women's business management, as in other countries, driven by the proliferation of legislation on gender equality, being, in addition, Spain, the pioneer in having specific legislation on Social Economy. The results show that cooperatives with greater female representation in theirs Boards have higher profitability. On the other hand, those Boards with a higher percentage of women show a lower level of indebtedness.

¿Barreras invisibles? Actitudes de los estudiantes universitarios ante sus compañeros con discapacidad

El presente trabajo analiza los conocimientos, opiniones y actitudes de los estudiantes universitarios sobre sus compañeros con discapacidad. La integración educativa y social de estos estudiantes es objeto de preocupación y estudio en todo el sistema educativo español, incluida la enseñanza superior en la que 21.942 universitarios con discapacidad se encontraban matriculados en el curso 2013-2014. El estudio se realiza en el marco de dos Proyectos de Innovación Educativa[1] de la Universidad de Málaga en los que se examinaron las actitudes del personal docente e investigador, el alumnado en general y el alumnado con discapacidad de esta institución. En este artículo se recogen las perspectivas de los estudiantes sin discapacidad, necesarias para medir el grado de aceptación dentro de las aulas universitarias. Se empleó una encuesta suministrada mediante correo electrónico a la totalidad de la población de estudiantes de la Universidad de Málaga (N= 32.464). El tamaño de la muestra fue de 620 estudiantes, por lo que trabajamos con un margen de error del 3,9% para la estimación de los porcentajes con un nivel de confianza del 95%. Los datos, analizados mediante análisis estadístico, visibilizan la importancia de un elevado grado de información y conocimiento así como de una experiencia directa con la discapacidad. Ambas circunstancias se traducen en actitudes más positivas y favorables. Tan solo el 20% de los alumnos manifestó haber recibido en alguna ocasión cualquier tipo de información sobre este asunto por parte de la Universidad; no obstante, la mayoría valora positivamente la presencia de alumnos con discapacidad en la Universidad, la posibilidad de contar ellos como compañeros de clase y el enriquecimiento que supone esta convivencia dentro del aula. Se discuten posibles acciones para mejorar las actitudes hacia los estudiantes con discapacidad. Además, se plantea incluir una perspectiva que promueva el empoderamiento de estos estudiantes. [1] PIE 10-135 y PIE 13-103.

Dynamic pricing models for used products in remanufacturing with lost-sales and uncertain quality

In this paper, we investigate the remanufacturing problem of pricing single-class used products (cores) in the face of random price-dependent returns and random demand. Specifically, we propose a dynamic pricing policy for the cores and then model the problem as a continuous-time Markov decision process. Our models are designed to address three objectives: finite horizon total cost minimization, infinite horizon discounted cost, and average cost minimization. Besides proving optimal policy uniqueness and establishing monotonicity results for the infinite horizon problem, we also characterize the structures of the optimal policies, which can greatly simplify the computational procedure. Finally, we use computational examples to assess the impacts of specific parameters on optimal price and reveal the benefits of a dynamic pricing policy. © 2013 Elsevier B.V. All rights reserved.

Dynamic pricing for used products in remanufacturing with back orders(ABS 3* Journal, Accepted with Revision on 2nd December 2011).

In remanufacturing, the supply of used products and the demand for remanufactured products are usually mismatched because of the great uncertainties on both sides. In this paper, we propose a dynamic pricing policy to balance this uncertain supply and demand. Specifically, we study a remanufacturer’s problem of pricing a single class of cores with random price-dependent returns and random demand for the remanufactured products with backlogs. We model this pricing task as a continuous-time Markov decision process, which addresses both the finite and infinite horizon problems, and provide managerial insights by analyzing the structural properties of the optimal policy. We then use several computational examples to illustrate the impacts of particular system parameters on pricing policy.

Improving appropriate polypharmacy for older people in primary care: selecting components of an evidence-based intervention to target prescribing and dispensing.

Background
The use of multiple medicines (polypharmacy) is increasingly common in older people. Ensuring that patients receive the most appropriate combinations of medications (appropriate polypharmacy) is a significant challenge. The quality of evidence to support the effectiveness of interventions to improve appropriate polypharmacy is low. Systematic identification of mediators of behaviour change, using the Theoretical Domains Framework (TDF), provides a theoretically robust evidence base to inform intervention design. This study aimed to (1) identify key theoretical domains that were perceived to influence the prescribing and dispensing of appropriate polypharmacy to older patients by general practitioners (GPs) and community pharmacists, and (2) map domains to associated behaviour change techniques (BCTs) to include as components of an intervention to improve appropriate polypharmacy in older people in primary care.

Methods
Semi-structured interviews were conducted with members of each healthcare professional (HCP) group using tailored topic guides based on TDF version 1 (12 domains). Questions covering each domain explored HCPs’ perceptions of barriers and facilitators to ensuring the prescribing and dispensing of appropriate polypharmacy to older people. Interviews were audio-recorded and transcribed verbatim. Data analysis involved the framework method and content analysis. Key domains were identified and mapped to BCTs based on established methods and discussion within the research team.

Results
Thirty HCPs were interviewed (15 GPs, 15 pharmacists). Eight key domains were identified, perceived to influence prescribing and dispensing of appropriate polypharmacy: ‘Skills’, ‘Beliefs about capabilities’, ‘Beliefs about consequences’, ‘Environmental context and resources’, ‘Memory, attention and decision processes’, ‘Social/professional role and identity’, ‘Social influences’ and ‘Behavioural regulation’. Following mapping, four BCTs were selected for inclusion in an intervention for GPs or pharmacists: ‘Action planning’, ‘Prompts/cues’, ‘Modelling or demonstrating of behaviour’ and ‘Salience of consequences’. An additional BCT (‘Social support or encouragement’) was selected for inclusion in a community pharmacy-based intervention in order to address barriers relating to interprofessional working that were encountered by pharmacists.

Conclusions
Selected BCTs will be operationalised in a theory-based intervention to improve appropriate polypharmacy for older people, to be delivered in GP practice and community pharmacy settings. Future research will involve development and feasibility testing of this intervention.

A influência do terrorismo nos comportamentos de viagem internacionais

A partir de uma amostra de 600 turistas internacionais que circulam em Portugal, Espanha e Itália, este estudo identifica as principais os conceitos chave relacionados com o terrorismo, a percepção de risco, envolvimento e motivação para a segurança dos turistas internacionais. Diferentes níveis de preocupação relativamente à segurança pode influenciar as decisões dos turistas. No seu processo de decisão, os turistas avaliam vários factores, nomeadamente, o nível de risco ou de segurança que consideram nos destinos (Sonmez, 1998). Os turistas adoptam uma atitude protectora alterando os seus comportamentos durante os processos de decisão, substituindo os destinos que consideram inseguros por outros associados a uma maior segurança (Gu & Martin, 1992; Mansfeld, 1996). O terrorismo exacerbado pelos media tem efeitos graves nas receitas dos destinos turísticos (Taylor, 2006). Através da publicidade negativa, um destino turístico que experiencia um incidente terrorista pode ver a sua reputação danificada e a actividade turística severamente comprometida (Sonmez, 1998). Inclusivamente, a imageme negativa de um destino pode ser generalizada e pode também afectar outros países ou regiões por períodos de tempo indeterminados (Taylor, 2006). Um modelo de equações estruturais revela que os turistas são motivados para adquirir informação sobre o terrorismo nos media, nomeadamente mostram atenção e interesse sobre essas notícias e esse facto influencia directamente o seu risco percebido. A percepção de risco influencia directamente o envolvimento dos turistas no planeamento da viagem, especificamente a procura de informação antes da viagem e quando estão no destino. A percepção de risco e o envolvimento dos turistas influencia a percepção da importância da segurança.A discussão foca as implicações deste modelo para a teoria e para as instituições e organizações turísticas. São igualmente apresentadas recomendações para os gestores e promotores dos destinos e para os gestores das organizações turísticas. Direcções futuras de investigação são igualmente apresentadas.

U-TRACER® - the use of communication technology in higher education - an information visualization tool for the context of Portuguese public higher education

Information Visualization is gradually emerging to assist the representation and comprehension of large datasets about Higher Education Institutions, making the data more easily understood. The importance of gaining insights and knowledge regarding higher education institutions is little disputed. Within this knowledge, the emerging and urging area in need of a systematic understanding is the use of communication technologies, area that is having a transformative impact on educational practices worldwide. This study focused on the need to visually represent a dataset about how Portuguese Public Higher Education Institutions are using Communication Technologies as a support to teaching and learning processes. Project TRACER identified this need, regarding the Portuguese public higher education context, and carried out a national data collection. This study was developed within project TRACER, and worked with the dataset collected in order to conceptualize an information visualization tool U-TRACER®. The main goals of this study related to: conceptualization of the information visualization tool U-TRACER®, to represent the data collected by project TRACER; understand higher education decision makers perception of usefulness regarding the tool. The goals allowed us to contextualize the phenomenon of information visualization tools regarding higher education data, realizing the existing trends. The research undertaken was of qualitative nature, and followed the method of case study with four moments of data collection.The first moment regarded the conceptualization of the U-TRACER®, with two focus group sessions with Higher Education professionals, with the aim of defining the interaction features the U-TRACER® should offer. The second data collection moment involved the proposal of the graphical displays that would represent the dataset, which reading effectiveness was tested by end-users. The third moment involved the development of a usability test to the UTRACER ® performed by higher education professionals and which resulted in the proposal of improvements to the final prototype of the tool. The fourth moment of data collection involved conducting exploratory, semi-structured interviews, to the institutional decision makers regarding their perceived usefulness of the U-TRACER®. We consider that the results of this study contribute towards two moments of reflection. The challenges of involving end-users in the conceptualization of an information visualization tool; the relevance of effective visual displays for an effective communication of the data and information. The second relates to the reflection about how the higher education decision makers, stakeholders of the U-TRACER® tool, perceive usefulness of the tool, both for communicating their institutions data and for benchmarking exercises, as well as a support for decision processes. Also to reflect on the main concerns about opening up data about higher education institutions in a global market.

Les processus additifs markoviens et leurs applications en finance mathématique

Cette thèse porte sur les questions d'évaluation et de couverture des options dans un modèle exponentiel-Lévy avec changements de régime. Un tel modèle est construit sur un processus additif markovien un peu comme le modèle de Black- Scholes est basé sur un mouvement Brownien. Du fait de l'existence de plusieurs sources d'aléa, nous sommes en présence d'un marché incomplet et ce fait rend inopérant les développements théoriques initiés par Black et Scholes et Merton dans le cadre d'un marché complet. Nous montrons dans cette thèse que l'utilisation de certains résultats de la théorie des processus additifs markoviens permet d'apporter des solutions aux problèmes d'évaluation et de couverture des options. Notamment, nous arrivons à caracté- riser la mesure martingale qui minimise l'entropie relative à la mesure de probabilit é historique ; aussi nous dérivons explicitement sous certaines conditions, le portefeuille optimal qui permet à un agent de minimiser localement le risque quadratique associé. Par ailleurs, dans une perspective plus pratique nous caract érisons le prix d'une option Européenne comme l'unique solution de viscosité d'un système d'équations intégro-di érentielles non-linéaires. Il s'agit là d'un premier pas pour la construction des schémas numériques pour approcher ledit prix.

Le projet organisant : vers une ontologie du projet d'aménagement

Codirection: Dr. Gonzalo Lizarralde

Zur denkmalpolitischen Praxis der Gegenwart - institutionelle Handlungsspielräume und diskursive Prozesse

Die Arbeit behandelt im Rahmen eines induktiven Ansatzes die Problematik aktueller kommunalpolitischer Zielkonflikte im Umgang mit Baudenkmalen in Deutschland. Dabei wird das Politikfeld Denkmalschutz in seiner kulturell-politischen Mehrdimensionalität unter der Ausgangsfrage untersucht, wie Entscheidungsprozesse verlaufen, bei denen entwicklungsbezogene Interessen und Belange des Denkmalschutzes eine besondere Rolle spielen. Vier Beispielfälle bilden den empirischen Kern der Untersuchung: Ein ortsbildprägendes und architektonisch qualitätsvolles Industriedenkmal wandelt sich mittels staatlicher Förderung zu einer Brachfläche; der Umgebungsschutz eines Gartendenkmals von Weltrang muss den Bedürfnissen des kommerzialisierten Fußballsports den Vortritt lassen; ein historisches Lichtspieltheater wird trotz Massenprotesten von Bürgern zu einem Buchladen umgebaut; eine freistehende Gründerzeitvilla wird unter der Maßgabe maximaler Verkaufsflächengröße durch ein Einkaufszentrum eingehaust. Aufbauend auf einer Analyse der jeweiligen Entscheidungsprozesse werden die Spezifika politischer Auseinandersetzungen um Denkmale fallübergreifend herausgearbeitet. Das Untersuchungsprinzip entspricht einem explorativen Verfahren, wobei der argumentative Austausch als empirischer Schlüssel zu sprachlich materialisierten Deutungsangeboten von Akteuren einen Schwerpunkt der Untersuchung bildet. In der Gegenüberstellung diskursiver Prozesse wird untersucht, wie Deutungsangebote im politischen Prozess entstehen, sich verändern und diskursiv vermittelt werden. Im Mittelpunkt steht der Einblick in das Zusammenspiel empirisch bestimmter Einflussgrößen. Dabei kristallisieren sich mehrere Thesen heraus, die das kulturelle Verständnis, die Rolle des institutionellen Kontextes und die politische Aushandlung als Prozess betreffen. Es wird aufgezeigt, weshalb die Kluft zwischen dem elitären Erhaltungsinteresse der Fachwelt und dem Denkmalverständnis des „Durchschnittsbürgers" als notwendige Triebfeder der denkmalpflegerischen Vermittlungsarbeit und für eine kreative Auseinandersetzung mit dem Denkmal ebenso wie der hoheitliche Denkmalschutz unverzichtbar bleibt.

Análisis del estado de las instituciones del Mercosur y las asimetrías en la toma de decisión entre los Estados Miembros, periodo 1999 – 2006

Tras 18 años de fundado el Mercosur se pretende analizar la forma como sus Instituciones, a partir de 1999, han evolucionado en el Mercado Común a la luz del Institucionalismo Neoliberal y al mismo tiempo han fomentado las asimetrías de poder entre socios mayores y menores en procesos de decisión.