On-line policy optimisation of Bayesian spoken dialogue systems via human interaction

A partially observable Markov decision process has been proposed as a dialogue model that enables robustness to speech recognition errors and automatic policy optimisation using reinforcement learning (RL). However, conventional RL algorithms require a very large number of dialogues, necessitating a user simulator. Recently, Gaussian processes have been shown to substantially speed up the optimisation, making it possible to learn directly from interaction with human users. However, early studies have been limited to very low dimensional spaces and the learning has exhibited convergence problems. Here we investigate learning from human interaction using the Bayesian Update of Dialogue State system. This dynamic Bayesian network based system has an optimisation space covering more than one hundred features, allowing a wide range of behaviours to be learned. Using an improved policy model and a more robust reward function, we show that stable learning can be achieved that significantly outperforms a simulator trained policy. © 2013 IEEE.

Principles of Safe Policy Routing Dynamics

We introduce the Dynamic Policy Routing (DPR) model that captures the propagation of route updates under arbitrary changes in topology or path preferences. DPR introduces the notion of causation chains where the route flap at one node causes a flap at the next node along the chain. Using DPR, we model the Gao-Rexford (economic) guidelines that guarantee the safety (i.e., convergence) of policy routing. We establish three principles of safe policy routing dynamics. The non-interference principle provides insight into which ASes can directly induce route changes in one another. The single cycle principle and the multi-tiered cycle principle provide insight into how cycles of routing updates can manifest in any network. We develop INTERFERENCEBEAT, a distributed algorithm that propagates a small token along causation chains to check adherence to these principles. To enhance the diagnosis power of INTERFERENCEBEAT, we model four violations of the Gao-Rexford guidelines (e.g., transiting between peers) and characterize the resulting dynamics.

An Adaptive Management Approach to Resolving Policy Conflicts

The Border Gateway Protocol (BGP) is the current inter-domain routing protocol used to exchange reachability information between Autonomous Systems (ASes) in the Internet. BGP supports policy-based routing which allows each AS to independently define a set of local policies on which routes it accepts and advertises from/to other networks, as well as on which route it prefers when more than one route becomes available. However, independently chosen local policies may cause global conflicts, which result in protocol divergence. In this paper, we propose a new algorithm, called Adaptive Policy Management Scheme(APMS), to resolve policy conflicts in a distributed manner. Akin to distributed feedback control systems, each AS independently classifies the state of the network as either conflict-free or potentially conflicting by observing its local history only (namely, route flaps). Based on the degree of measured conflicts, each AS dynamically adjusts its own path preferences---increasing its preference for observably stable paths over flapping paths. APMS also includes a mechanism to distinguish route flaps due to topology changes, so as not to confuse them with those due to policy conflicts. A correctness and convergence analysis of APMS based on the sub-stability property of chosen paths is presented. Implementation in the SSF network simulator is performed, and simulation results for different performance metrics are presented. The metrics capture the dynamic performance (in terms of instantaneous throughput, delay, etc.) of APMS and other competing solutions, thus exposing the often neglected aspects of performance.

Pretty good governance: balancing policy drivers and stakeholder interests in developing fisheries ecosystem plans

The central research question that this thesis addresses is whether there is a significant gap between fishery stakeholder values and the principles and policy goals implicit in an Ecosystem Approach to Fisheries Management (EAFM). The implications of such a gap for fisheries governance are explored. Furthermore an assessment is made of what may be practically achievable in the implementation of an EAFM in fisheries in general and in a case study fishery in particular. The research was mainly focused on a particular case study, the Celtic Sea Herring fishery and its management committee, the Celtic Sea Herring Management Advisory Committee (CSHMAC). The Celtic Sea Herring fishery exhibits many aspects of an EAFM and the fish stock has successfully recovered to healthy levels in the past 5 years. However there are increasing levels of governance related conflict within the fishery which threaten the future sustainability of the stock. Previous research on EAFM governance has tended to focus either on higher levels of EAFM governance or on individual behaviour but very little research has attempted to link the two spheres or explore the relationship between them. Two main themes within this study aimed to address this gap. The first was what role governance could play in facilitating EAFM implementation. The second theme concerned the degree of convergence between high-level EAFM goals and stakeholder values. The first method applied was governance benchmarking to analyse systemic risks to EAFM implementation. This found that there are no real EU or national level policies which provide stakeholders or managers with clear targets for EAFM implementation. The second method applied was the use of cognitive mapping to explore stakeholders understandings of the main ecological, economic and institutional driving forces in the Celtic Sea Herring fishery. The main finding from this was that a long-term outlook can and has been incentivised through a combination of policy drivers and participatory management. However the fundamental principle of EAFM, accounting for ecosystem linkages rather than target stocks was not reflected in stakeholders cognitive maps. This was confirmed in a prioritisation of stakeholders management priorities using Analytic Hierarchy Process which found that the overriding concern is for protection of target stock status but that wider ecosystem health was not a priority for most management participants. The conclusion reached is that moving to sustainable fisheries may be a more complex process than envisioned in much of the literature and may consist of two phases. The first phase is a transition to a long-term but still target stock focused approach. This achievable transition is mainly a strategic change, which can be incentivised by policies and supported by stakeholders. In the Celtic Sea Herring fishery, and an increasing number of global and European fisheries, such transitions have contributed to successful stock recoveries. The second phase however, implementation of an ecosystem approach, may present a greater challenge in terms of governability, as this research highlights some fundamental conflicts between stakeholder perceptions and values and those inherent in an EAFM. This phase may involve the setting aside of fish for non-valued ecosystem elements and will require either a pronounced mind-set and value change or some strong top-down policy incentives in order to succeed. Fisheries governance frameworks will need to carefully explore the most effective balance between such endogenous and exogenous solutions. This finding of low prioritisation of wider ecosystem elements has implications for rights based management within an ecosystem approach, regardless of whether those rights are individual or collective.

Practitioners’ perception of the impact of the vision of policy-makers on practice: The example of the recommendations of the Moser Commission

This paper presents the perception of practitioners of the impact of the Moser Committee recommendations and the Skills for Life agenda it generated. The paper further explores areas of convergence and divergence between practitioners’ perceptions and the underpinning values of the Moser Committee recommendations. The study utilised a range of research tools including an online questionnaire, documentary analysis and elements of discourse analysis in the collection and analysis of data. It found that there is substantial divergence between the perception of practitioners and the values underpinning policy. It concludes by suggesting that a varying perception of what constitutes sustainable education and the lack of input from practitioners into policy might be responsible for this significant divergence of opinion and also raised a question on the perceived role of practitioners in the policy‐making process.

Where there is smoke, there is (the potential for) fire: soft indicators of research and policy impact

There is a growing literature examining the impact of research on informing policy, and of research and policy on practice. Research and policy do not have the same types of impact on practice but can be evaluated using similar approaches. Sometimes the literature provides a platform for methodological debate but mostly it is concerned with how research can link to improvements in the process and outcomes of education, how it can promote innovative policies and practice, and how it may be successfully disseminated. Whether research-informed or research-based, policy and its implementation is often assessed on such 'hard' indicators of impact as changes in the number of students gaining five or more A to C grades in national examinations or a percentage fall in the number of exclusions in inner city schools. Such measures are necessarily crude, with large samples smoothing out errors and disguising instances of significant success or failure. Even when 'measurable' in such a fashion, however, the impact of any educational change or intervention may require a period of years to become observable. This paper considers circumstances in which short-term change may be implausible or difficult to observe. It explores how impact is currently theorized and researched and promotes the concept of 'soft' indicators of impact in circumstances in which the pursuit of conventional quantitative and qualitative evidence is rendered impractical within a reasonable cost and timeframe. Such indicators are characterized by their avowedly subjective, anecdotal and impressionistic provenance and have particular importance in the context of complex community education issues where the assessment of any impact often faces considerable problems of access. These indicators include the testimonies of those on whom the research intervention or policy focuses (for example, students, adult learners), the formative effects that are often reported (for example, by head teachers, community leaders) and media coverage. The collation and convergence of a wide variety of soft indicators (Where there is smoke …) is argued to offer a credible means of identifying subtle processes that are often neglected as evidence of potential and actual impact (… there is fire).

La convergence de la sécurité informatique et de la protection des renseignements personnels : vers une nouvelle approche juridique

"Mémoire présenté à la Faculté des études supérieures en vue de l'obtention du grade de maîtrise en droit (LL.M.) option Nouvelles technologies de l'information"

La convergence de la sécurité informatique et la protection des données à caractère personnel : vers une nouvelle approche juridique

Le développement exponentiel des réseaux informatiques a largement contribué à augmenter le volume des renseignements personnels disponibles et à remplacer les méthodes désuètes de collecte des renseignements par des méthodes plus rapides et plus efficaces. La vie privée et le contrôle sur les informations personnelles, tels que nous les connaissions il y a quelques décennies, sont des notions difficilement compatibles avec la société ouverte et commerciale comme la nôtre. Face à cette nouvelle réalité menaçante pour les droits et libertés de l’homme, il est essentiel de donner un cadre technique et légal stable qui garantisse une protection adéquate de ces données personnelles. Pour rester dans le marché ou bénéficier de la confiance des individus, les entreprises et les gouvernements doivent posséder une infrastructure de sécurité efficace. Cette nouvelle donne a tendance à devenir plus qu’une règle de compétitivité, elle se transforme en une authentique obligation légale de protéger les données à caractère personnel par des mesures de sécurité adéquates et suffisantes. Ce mémoire aborde justement ces deux points, soit l’étude du développement d’une obligation légale de sécurité et l’encadrement juridique de la mise en place d’un programme de sécurisation des données personnelles par des mesures de sécurités qui respectent les standards minimaux imposés par les textes législatifs nationaux et internationaux.

Reinforcement Learning by Policy Search

One objective of artificial intelligence is to model the behavior of an intelligent agent interacting with its environment. The environment's transformations can be modeled as a Markov chain, whose state is partially observable to the agent and affected by its actions; such processes are known as partially observable Markov decision processes (POMDPs). While the environment's dynamics are assumed to obey certain rules, the agent does not know them and must learn. In this dissertation we focus on the agent's adaptation as captured by the reinforcement learning framework. This means learning a policy---a mapping of observations into actions---based on feedback from the environment. The learning can be viewed as browsing a set of policies while evaluating them by trial through interaction with the environment. The set of policies is constrained by the architecture of the agent's controller. POMDPs require a controller to have a memory. We investigate controllers with memory, including controllers with external memory, finite state controllers and distributed controllers for multi-agent systems. For these various controllers we work out the details of the algorithms which learn by ascending the gradient of expected cumulative reinforcement. Building on statistical learning theory and experiment design theory, a policy evaluation algorithm is developed for the case of experience re-use. We address the question of sufficient experience for uniform convergence of policy evaluation and obtain sample complexity bounds for various estimators. Finally, we demonstrate the performance of the proposed algorithms on several domains, the most complex of which is simulated adaptive packet routing in a telecommunication network.

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

The growth effects of EU cohesion policy: a meta-analysis. Bruegel Working Paper 2012/14, October 2012

We run a standard income convergence analysis for the last decade and confirm an already established finding in the growth economics literature. EU countries are converging. Regions in Europe are also converging. But, within countries, regional disparities are on the rise. At the same time, there is probably no reason for EU Cohesion Policy to be concerned with what happens inside countries. Ultimately, our data shows that national governments redistribute well across regions, whether they are fiscally centralised or decentralised. It is difficult to establish if Structural and Cohesion Funds play any role in recent growth convergence patterns in Europe. Generally, macroeconomic simulations produce better results than empirical tests. It is thus possible that Structural Funds do not fully realise their potential either because they are not efficiently allocated or are badly managed or are used for the wrong investments, or a combination of all three. The approach to assess the effectiveness of EU funds should be consistent with the rationale behind the post-1988 EU Cohesion Policy. Standard income convergence analysis is certainly not sufficient and should be accompanied by an assessment of the changes in the efficiency of the capital stock in the recipient countries or regions as well as by a more qualitative assessment. EU funds for competitiveness and employment should be allocated by looking at each region’s capital efficiency to maximise growth generating effects or on a pure competitive.

Continental shift? An analysis of convergence trends in European real estate equities

European economic and political integration have been recognised as having implications for patterns of performance in national real estate and capital markets and have generated a wide body of research and commentary. In 1999, progress towards monetary integration within the European Union culminated in the introduction of a common currency and monetary policy. This paper investigates the effects of this ‘event’ on the behaviour of stock returns in European real estate companies. A range of statistical tests is applied to the performance of European property companies to test for changes in segmentation, co-movement and causality. The results suggest that, relative to the wider equity markets, the dispersion of performance is higher, correlations are lower, a common contemporaneous factor has much lower explanatory power whilst lead-lag relationships are stronger. Consequently, the evidence of transmission of monetary integration to real estate securities is less noticeable than to general securities. Less and slower integration is attributed to the relatively small size of the real estate securities market and the local and national nature of the majority of the companies’ portfolios.

Output convergence in Mercosur: multivariate time series evidence

The aim of this paper is to provide evidence on output convergence among the Mercosur countries and associates, using multivariate time-series tests. The methodology is based on a combination of tests and estimation procedures, both univariate and multivariate, applied to the differences in per capita real income. We use the definitions of time-series convergence proposed by Bernard & Durlauf and apply unit root and tests proposed by Abuaf & Jorion and Taylor & Sarno. In this same multivariate context, the Flôres, Preumont & Szafarz and Breuer, MbNown & Wallace tests, which allow for the existence of correlations across the series without imposing a common speed of mean reversion, identify the countries that convergence. Concerning the empirical results, there is evidence of long-run convergence or, at least, catching up, for the smaller countries, Bolivia, Paraguay, Peru and Uruguay, towards Brazil and, to some extent, Argentina. In contrast, the evidence on convergence for the larger countries is weaker, as they have followed different (or rather opposing) macroeconomic policy strategies. Thus the future of the whole area will critically depend on the ability of Brazil, Argentina and Chile to find some scope for more cooperative policy actions.

Brazilian regional development distribution, evidences for socio-economic policy

As to many Latin american countries, the impacts of the recent economic globalization on the Brazilian economy have revealed a diversified tendency in spatial development when regional economic indicators are observed. This is due to the specificities or each region, as regard their sector structure, the availability of human resources and the degree of technological innovation undertaken by local enterprises. From a situation of regional inequalities observed in lhe socio-economic levels of development at the beginning of the eighties the dynamics of the Brazilian regional evolution has presented different speeds and intensities in the several spaees. This paper aims to evaluate the dynamics of Brazilian regional development during the 1985-95 period and the impacts over the working population and regional disparities in order to offer some elements to assist social and economic policy. For this purpose Dispersion Quotients and Dispersion lntensity Coefficients were calculated based on two variables, the Regional Gross Domestic Product anel the Working Population. The results of the analysis confirm the existence of considerable regional disparities and it was observed that thc sector and regional redistribution of the GDP indicate that in a general way, no remarkable changes occurred in the regional development in the period. The results show that although the economic policies did stimulate a global convergence process of the per capita product among regions, those policies did not attenuate economic dynamism concentration to the desired extent.

Renewable energy sources in Latin America and the Caribbean: situation and policy proposals

Includes bibliography