Code-specific policy gradient rules for spiking neurons

Although it is widely believed that reinforcement learning is a suitable tool for describing behavioral learning, the mechanisms by which it can be implemented in networks of spiking neurons are not fully understood. Here, we show that different learning rules emerge from a policy gradient approach depending on which features of the spike trains are assumed to influence the reward signals, i.e., depending on which neural code is in effect. We use the framework of Williams (1992) to derive learning rules for arbitrary neural codes. For illustration, we present policy-gradient rules for three different example codes - a spike count code, a spike timing code and the most general "full spike train" code - and test them on simple model problems. In addition to classical synaptic learning, we derive learning rules for intrinsic parameters that control the excitability of the neuron. The spike count learning rule has structural similarities with established Bienenstock-Cooper-Munro rules. If the distribution of the relevant spike train features belongs to the natural exponential family, the learning rules have a characteristic shape that raises interesting prediction problems.

On-line policy optimisation of Bayesian spoken dialogue systems via human interaction

A partially observable Markov decision process has been proposed as a dialogue model that enables robustness to speech recognition errors and automatic policy optimisation using reinforcement learning (RL). However, conventional RL algorithms require a very large number of dialogues, necessitating a user simulator. Recently, Gaussian processes have been shown to substantially speed up the optimisation, making it possible to learn directly from interaction with human users. However, early studies have been limited to very low dimensional spaces and the learning has exhibited convergence problems. Here we investigate learning from human interaction using the Bayesian Update of Dialogue State system. This dynamic Bayesian network based system has an optimisation space covering more than one hundred features, allowing a wide range of behaviours to be learned. Using an improved policy model and a more robust reward function, we show that stable learning can be achieved that significantly outperforms a simulator trained policy. © 2013 IEEE.

City versus national energy use: implications for urban energy policy and strategies

The global trend towards urbanization means that over half of the world's population now lives in cities. Cities use energy in different proportions to national energy use averages, typically corresponding to whether a country is industrialized or developing. Cities in industrialized countries tend to use less energy per capita than the national average while cities in developing countries use more. This paper looks at existing World Bank data in respect to urban energy consumption, the emissions inventory work done by New York City, and discusses how this data highlights the need for a focus on: energy policy for buildings in industrialized cities; masterplanning and new construction standards in developing cities; and how urban energy policy can become more effective in reducing urban greenhouse gas emissions.

The temporal and spatial evaluation on China's agricultural policy output since 1978

Since reform and opening up, how much contribution has China's implementation of new agricultural policy made to agricultural output? This paper is trying to establish an agricultural policy output econometric model for doing a quantitative analysis of China's new agricultural policy. The results show that China's agricultural policies on agricultural output have an average contribution rate of about 7% since 1978, which is consistent with the OECD's basic forecast. There are obvious temporal and spatial differences. Generally speaking, we can divide the contribution of agricultural policy into three periods, which are the start-up phase from 1978 to 1991 (14 years), the stationary phase from 1992 to 2002 (11 years) and the rising phase from 2003 to 2008 (6 years). In space, the contribution of agricultural policy underwent a process from the all-low in the start-up phase, the gradual increase in the stationary phase to the all-high in the rising phase. Northern and western regions are more sensitive to policies. There are three major factors that can affect the contribution of regional agricultural policies, which are the process of national industrialization strategy, terrain and the level of local finance.

How smokers may react to cigarette taxes and price increases in Brazil: data from a national survey

Background: Despite being the third largest tobacco producer in the world, Brazil has developed a comprehensive tobacco control policy that includes a broad restriction on both advertising and smoking in indoor public places, compulsory pictorial warning labels, and a menthol cigarette ban. However, tax and pricing policies have been developed slowly and only very recently were stronger measures implemented. This study investigated the expected responses of smokers to hypothetical price increases in Brazil.Methods: We analyzed smokers' responses to hypothetical future price increases according to sociodemographic characteristics and smoking conditions in a multistage sample of Brazilian current cigarette smokers aged >= 14 years (n = 500). Logistic regression analysis was used to examine the relationship between possible responses and different predictors.Results: in most subgroups investigated, smokers most frequently said they would react to a hypothetical price increase by taking up alternatives that might have a positive impact on health, i.e., they would try to stop smoking (52.3%) or smoke fewer cigarettes (46.8%). However, a considerable percentage responded that they would use alternatives that would reduce the effect of price increases, such as the same brand with lower cost (48.1%). After controlling for sex age group (14-19, 20-39, 40-59, and >= 60 years), schooling level (>= 9 versus <= 9 years), number of cigarettes per day (>20 versus <= 20), and stage of change for smoking cessation (precontemplation, contemplation, and preparation), lower levels of dependence were positively associated with the response I would try to stop smoking (odds ratio [OR], 2.19). Young age was associated with I would decrease the number of cigarettes (OR, 3.44). A low schooling level was strongly associated with all responses.Conclusions: Taxes and prices increases have great potential to stimulate cessation or reduction of cigarette consumption further among two important vulnerable populations of smokers in Brazil: young smokers and those of low educational level. the results from the present study also suggest that seeking illegal products may reduce the impact of increased taxes, but does not eliminate it.

Maritime policy in Scotland.

This paper begins by defining the meaning of the term `maritime policy'. Since devolution in the UK, the opening of the new Scottish parliament in 1998 and the creation of the Scottish Executive, little effort has been made to establish a distinct maritime policy for Scotland. As was evident prior to devolution, the primary emphasis from any maritime policy perspective has continued to be a focus on lifeline island ferry services. This ignores significant developments in several other key maritime transport sectors, and this paper provides examples of areas that require some form of policy response, including intra-European short sea shipping, UK coastal shipping, urban/river transport and global container shipping. Long-standing institutional bias against maritime transport coupled with subsidy devoted almost entirely to land transport systems has resulted in a quite distorted marketplace. This suggests that a maritime policy is now imperative if maritime transport is to play a more significant role in the overall transport system. Further discussion centres on the need to consider, from a Scottish policy perspective, the role of various state-sponsored maritime service providers and how these bodies might fit better within evolving policy. The conclusion is that formulation of a maritime policy by the Scottish Executive is overdue and that a degree of restructuring of transport responsibilities within the Executive, combined with adequate resource allocation towards the maritime industry, will be necessary in order that market distortions can be overcome, so enabling Scotland to fully exploit the competitive and environmental advantages that maritime transport can provide.

An integrated firewall policy validation tool.

Security policies are increasingly being implemented by organisations. Policies are mapped to device configurations to enforce the policies. This is typically performed manually by network administrators. The development and management of these enforcement policies is a difficult and error prone task. This thesis describes the development and evaluation of an off-line firewall policy parser and validation tool. This provides the system administrator with a textual interface and the vendor specific low level languages they trust and are familiar with, but the support of an off-line compiler tool. The tool was created using the Microsoft C#.NET language, and the Microsoft Visual Studio Integrated Development Environment (IDE). This provided an object environment to create a flexible and extensible system, as well as simple Web and Windows prototyping facilities to create GUI front-end applications for testing and evaluation. A CLI was provided with the tool, for more experienced users, but it was also designed to be easily integrated into GUI based applications for non-expert users. The evaluation of the system was performed from a custom built GUI application, which can create test firewall rule sets containing synthetic rules, to supply a variety of experimental conditions, as well as record various performance metrics. The validation tool was created, based around a pragmatic outlook, with regard to the needs of the network administrator. The modularity of the design was important, due to the fast changing nature of the network device languages being processed. An object oriented approach was taken, for maximum changeability and extensibility, and a flexible tool was developed, due to the possible needs of different types users. System administrators desire, low level, CLI-based tools that they can trust, and use easily from scripting languages. Inexperienced users may prefer a more abstract, high level, GUI or Wizard that has an easier to learn process. Built around these ideas, the tool was implemented, and proved to be a usable, and complimentary addition to the many network policy-based systems currently available. The tool has a flexible design and contains comprehensive functionality. As opposed to some of the other tools which perform across multiple vendor languages, but do not implement a deep range of options for any of the languages. It compliments existing systems, such as policy compliance tools, and abstract policy analysis systems. Its validation algorithms were evaluated for both completeness, and performance. The tool was found to correctly process large firewall policies in just a few seconds. A framework for a policy-based management system, with which the tool would integrate, is also proposed. This is based around a vendor independent XML-based repository of device configurations, which could be used to bring together existing policy management and analysis systems.

The total margin of exposure of ethanol and acetaldehyde for heavy drinkers consuming cider or vodka

Heavy drinkers in Scotland may consume 1600 g ethanol per week. Due to its low price, cider may be preferred over other beverages. Anecdotal evidence has linked cider to specific health hazards beyond other alcoholic beverages. To examine this hypothesis, nine apple and pear cider samples were chemically analysed for constituents and contaminants. None of the products exceeded regulatory or toxicological thresholds, but the regular occurrence of acetaldehyde in cider was detected. To provide a quantitative risk assessment, two collectives of exclusive drinkers of cider and vodka were compared and the intake of acetaldehyde was estimated using probabilistic MonteeCarlo type analysis. The cider consumers were found to ingest more than 200-times the amount of acetaldehyde consumed by vodka consumers. The margins of exposure (MOE) of acetaldehyde were 224 for the cider and over 220,000 for vodka consumers. However, if the effects of ethanol were considered in a cumulative assessment of the combined MOE, the effect of acetaldehyde was minor and the combined MOE for both groups was 0.3. We suggest that alcohol policy priority should be given on reducing ethanol intake by measures such as minimum pricing, rather than to focus on acetaldehyde.

MUES (Mid Wales - Users - Ethnic Services) Ethnic services provision 2007-08. Policy guidance for Libraries for Life: Delivering the entitlement agenda for library users in Wales 2007-09

Urquhart, C., Thomas, R., Crossan, S. & Hines, B. (2008). MUES (Mid Wales - Users - Ethnic Services) Ethnic services provision 2007-08. Policy guidance for Libraries for Life: Delivering the entitlement agenda for library users in Wales 2007-09. Aberystwyth: Department of Information Studies, Aberystwyth University. Relates to report of same title - http://hdl.handle.net/2160/609 Sponsorship: CyMAL