Code-specific policy gradient rules for spiking neurons

Although it is widely believed that reinforcement learning is a suitable tool for describing behavioral learning, the mechanisms by which it can be implemented in networks of spiking neurons are not fully understood. Here, we show that different learning rules emerge from a policy gradient approach depending on which features of the spike trains are assumed to influence the reward signals, i.e., depending on which neural code is in effect. We use the framework of Williams (1992) to derive learning rules for arbitrary neural codes. For illustration, we present policy-gradient rules for three different example codes - a spike count code, a spike timing code and the most general "full spike train" code - and test them on simple model problems. In addition to classical synaptic learning, we derive learning rules for intrinsic parameters that control the excitability of the neuron. The spike count learning rule has structural similarities with established Bienenstock-Cooper-Munro rules. If the distribution of the relevant spike train features belongs to the natural exponential family, the learning rules have a characteristic shape that raises interesting prediction problems.

Discrete multicomponent model for biodiesel spray combustion simulation

An important first step in spray combustion simulation is an accurate determination of the fuel properties which affects the modelling of spray formation and reaction. In a practical combustion simulation, the implementation of a multicomponent model is important in capturing the relative volatility of different fuel components. A Discrete Multicomponent (DM) model is deemed to be an appropriate candidate to model a composite fuel like biodiesel which consists of four components of fatty acid methyl esters (FAME). In this paper, the DM model is compared with the traditional Continuous Thermodynamics (CTM) model for both diesel and biodiesel. The CTM model is formulated based on mixing rules that incorporate the physical and thermophysical properties of pure components into a single continuous surrogate for the composite fuel. The models are implemented within the open-source CFD code OpenFOAM, and a semi-quantitative comparison is made between the predicted spray-combustion characteristics and optical measurements of a swirl-stabilised flame of diesel and biodiesel. The DM model performs better than the CTM model in predicting a higher magnitude of heat release rate in the top flame brush region of the biodiesel flame compared to that of the diesel flame. Using both the DM and CTM models, the simulation successfully reproduces the droplet size, volume flux, and droplet density profiles of diesel and biodiesel. The DM model predicts a longer spray penetration length for biodiesel compared to that of diesel, as seen in the experimental data. Also, the DM model reproduces a segregated biodiesel fuel vapour field and spray in which the most abundant FAME component has the longest vapour penetration. In the biodiesel flame, the relative abundance of each fuel component is found to dominate over the relative volatility in terms of the vapour species distribution and vice versa in the liquid species distribution. © 2014 Elsevier Ltd. All rights reserved.

Cell types, network homeostasis, and pathological compensation from a biologically plausible ion channel expression model.

How do neurons develop, control, and maintain their electrical signaling properties in spite of ongoing protein turnover and perturbations to activity? From generic assumptions about the molecular biology underlying channel expression, we derive a simple model and show how it encodes an "activity set point" in single neurons. The model generates diverse self-regulating cell types and relates correlations in conductance expression observed in vivo to underlying channel expression rates. Synaptic as well as intrinsic conductances can be regulated to make a self-assembling central pattern generator network; thus, network-level homeostasis can emerge from cell-autonomous regulation rules. Finally, we demonstrate that the outcome of homeostatic regulation depends on the complement of ion channels expressed in cells: in some cases, loss of specific ion channels can be compensated; in others, the homeostatic mechanism itself causes pathological loss of function.

a flexible negotiation model for an agent-based software process modelling

Our Agent-based Software Process Modelling (ASPM) approach describes a software process as a set of cooperative agents. Negotiation is the way in which the agents construct their cooperative relations, and thus the software process. Currently, most negotiation models use a fixed negotiation protocol and fixed strategies. In order to achieve the flexibility that the negotiation of the agents in ASPM requires, we propose a negotiation model NM-PA. NM-PA mainly includes a generic negotiation protocol and some rules, which possibly change in different negotiation processes. By changing the rules, the model can support multi-protocols and multi-decision-making strategies at a lower cost.

A Computational Model for the Acquisition and Use of Phonological Knowledge

Does knowledge of language consist of symbolic rules? How do children learn and use their linguistic knowledge? To elucidate these questions, we present a computational model that acquires phonological knowledge from a corpus of common English nouns and verbs. In our model the phonological knowledge is encapsulated as boolean constraints operating on classical linguistic representations of speech sounds in term of distinctive features. The learning algorithm compiles a corpus of words into increasingly sophisticated constraints. The algorithm is incremental, greedy, and fast. It yields one-shot learning of phonological constraints from a few examples. Our system exhibits behavior similar to that of young children learning phonological knowledge. As a bonus the constraints can be interpreted as classical linguistic rules. The computational model can be implemented by a surprisingly simple hardware mechanism. Our mechanism also sheds light on a fundamental AI question: How are signals related to symbols?

AMAR: A Computational Model of Autosegmental Phonology

This report describes a computational system with which phonologists may describe a natural language in terms of autosegmental phonology, currently the most advanced theory pertaining to the sound systems of human languages. This system allows linguists to easily test autosegmental hypotheses against a large corpus of data. The system was designed primarily with tonal systems in mind, but also provides support for tree or feature matrix representation of phonemes (as in The Sound Pattern of English), as well as syllable structures and other aspects of phonological theory. Underspecification is allowed, and trees may be specified before, during, and after rule application. The association convention is automatically applied, and other principles such as the conjunctivity condition are supported. The method of representation was designed such that rules are designated in as close a fashion as possible to the existing conventions of autosegmental theory while adhering to a textual constraint for maximum portability.

A novel Information sharing model using binary decision diagrams for redundancy, shadowing, generalisation and correlation.

This paper outlines a novel information sharing method using Binary Decision Diagrams (BBDs). It is inspired by the work of Al-Shaer and Hamed, who applied BDDs into the modelling of network firewalls. This is applied into an information sharing policy system which optimizes the search of redundancy, shadowing, generalisation and correlation within information sharing rules.

Evaluation of binary decision diagrams for redundancy, shadowing, generalisation and correlation in an Information sharing model.

This paper defines a structured methodology which is based on the foundational work of Al-Shaer et al. in [1] and that of Hamed and Al-Shaer in [2]. It defines a methodology for the declaration of policy field elements, through to the syntax, ontology and functional verification stages. In their works of [1] and [2] the authors concentrated on developing formal definitions of possible anomalies between rules in a network firewall rule set. Their work is considered as the foundation for further works on anomaly detection, including those of Fitzgerald et al. [3], Chen et al. [4], Hu et al. [5], among others. This paper extends this work by applying the methods to information sharing policies, and outlines the evaluation related to these.

A Topological Analysis of the Magnetic Breakout Model for an Eruptive Solar Flare

Maclean, R., Beveridge, C., Longcope, D.W., Brown, D.S. and Priest, E.R., 2005, A topological analysis of the magnetic breakout model for an eruptive solar flare, Proc. Roy. Soc., 461, 2099-2120. Sponsorship: PPARC/STFC

Political participation of non-national residents in Portugal

Em Portugal, novas medidas de participação política têm vindo a ser introduzidas tanto por força da transição para uma sociedade multicultural como pela sua participação na integração política europeia que se direcciona para um novo nível de cidadania. O presente artigo pretende mostrar até que ponto os residentes não-nacionais, de países terceiros e da União Europeia, efectivamente usam os seus novos direitos políticos e participam nos actos eleitorais em Portugal, conferindo-lhes a oportunidade de uma maior integração política no seu Estado de residência. In Portugal, new rules of political participation have been imposed by the transition to a multicultural society, in addition to the European political integration that is currently developing towards a new level of citizenship. This paper intends to show to what extent non-national residents, both from EU and non-EU countries, effectively use their new political rights and participate in the Portuguese electoral acts that give them the opportunity for a wider political integration in their state of residence.

Friendly Virtual Machine: Leveraging a Feedback-Control Model for Application Adaptation

With the increased use of "Virtual Machines" (VMs) as vehicles that isolate applications running on the same host, it is necessary to devise techniques that enable multiple VMs to share underlying resources both fairly and efficiently. To that end, one common approach is to deploy complex resource management techniques in the hosting infrastructure. Alternately, in this paper, we advocate the use of self-adaptation in the VMs themselves based on feedback about resource usage and availability. Consequently, we define a "Friendly" VM (FVM) to be a virtual machine that adjusts its demand for system resources, so that they are both efficiently and fairly allocated to competing FVMs. Such properties are ensured using one of many provably convergent control rules, such as AIMD. By adopting this distributed application-based approach to resource management, it is not necessary to make assumptions about the underlying resources nor about the requirements of FVMs competing for these resources. To demonstrate the elegance and simplicity of our approach, we present a prototype implementation of our FVM framework in User-Mode Linux (UML)-an implementation that consists of less than 500 lines of code changes to UML. We present an analytic, control-theoretic model of FVM adaptation, which establishes convergence and fairness properties. These properties are also backed up with experimental results using our prototype FVM implementation.

Lack of cross-scale linkages reduces robustness of community-based fisheries management.

Community-based management and the establishment of marine reserves have been advocated worldwide as means to overcome overexploitation of fisheries. Yet, researchers and managers are divided regarding the effectiveness of these measures. The "tragedy of the commons" model is often accepted as a universal paradigm, which assumes that unless managed by the State or privatized, common-pool resources are inevitably overexploited due to conflicts between the self-interest of individuals and the goals of a group as a whole. Under this paradigm, the emergence and maintenance of effective community-based efforts that include cooperative risky decisions as the establishment of marine reserves could not occur. In this paper, we question these assumptions and show that outcomes of commons dilemmas can be complex and scale-dependent. We studied the evolution and effectiveness of a community-based management effort to establish, monitor, and enforce a marine reserve network in the Gulf of California, Mexico. Our findings build on social and ecological research before (1997-2001), during (2002) and after (2003-2004) the establishment of marine reserves, which included participant observation in >100 fishing trips and meetings, interviews, as well as fishery dependent and independent monitoring. We found that locally crafted and enforced harvesting rules led to a rapid increase in resource abundance. Nevertheless, news about this increase spread quickly at a regional scale, resulting in poaching from outsiders and a subsequent rapid cascading effect on fishing resources and locally-designed rule compliance. We show that cooperation for management of common-pool fisheries, in which marine reserves form a core component of the system, can emerge, evolve rapidly, and be effective at a local scale even in recently organized fisheries. Stakeholder participation in monitoring, where there is a rapid feedback of the systems response, can play a key role in reinforcing cooperation. However, without cross-scale linkages with higher levels of governance, increase of local fishery stocks may attract outsiders who, if not restricted, will overharvest and threaten local governance. Fishers and fishing communities require incentives to maintain their management efforts. Rewarding local effective management with formal cross-scale governance recognition and support can generate these incentives.

CMIP5 climate model analyses: Climate extremes in the United States

Given the increases in spatial resolution and other improvements in climate modeling capabilities over the last decade since the CMIP3 simulations were completed, CMIP5 provides a unique opportunity to assess scientific understanding of climate variability and change over a range of historical and future conditions. With participation from over 20 modeling groups and more than 40 global models, CMIP5 represents the latest and most ambitious coordinated international climate model intercomparison exercise to date. Observations dating back to 1900 show that the temperatures in the twenty-first century have the largest spatial extent of record breaking and much above normal mean monthly maximum and minimum temperatures. The 20-yr return value of the annual maximum or minimum daily temperature is one measure of changes in rare temperature extremes.

Safety-critical medical device development using the UPP2SF model translation tool

Software-based control of life-critical embedded systems has become increasingly complex, and to a large extent has come to determine the safety of the human being. For example, implantable cardiac pacemakers have over 80,000 lines of code which are responsible for maintaining the heart within safe operating limits. As firmware-related recalls accounted for over 41% of the 600,000 devices recalled in the last decade, there is a need for rigorous model-driven design tools to generate verified code from verified software models. To this effect, we have developed the UPP2SF model-translation tool, which facilitates automatic conversion of verified models (in UPPAAL) to models that may be simulated and tested (in Simulink/Stateflow). We describe the translation rules that ensure correct model conversion, applicable to a large class of models. We demonstrate how UPP2SF is used in themodel-driven design of a pacemaker whosemodel is (a) designed and verified in UPPAAL (using timed automata), (b) automatically translated to Stateflow for simulation-based testing, and then (c) automatically generated into modular code for hardware-level integration testing of timing-related errors. In addition, we show how UPP2SF may be used for worst-case execution time estimation early in the design stage. Using UPP2SF, we demonstrate the value of integrated end-to-end modeling, verification, code-generation and testing process for complex software-controlled embedded systems. © 2014 ACM.

Vulnerability of coastal ecosystems to changes in harmful algal bloom distribution in response to climate change: projections based on model analysis

Harmful algal blooms (HABs), those proliferations of algae that can cause fish kills, contaminate seafood with toxins, form unsightly scums, or detrimentally alter ecosystem function have been increasing in frequency, magnitude, and duration worldwide. Here, using a global modeling approach, we show, for three regions of the globe, the potential effects of nutrient loading and climate change for two HAB genera, pelagic Prorocentrum and Karenia, each with differing physiological characteristics for growth. The projections (end of century, 2090-2100) are based on climate change resulting from the A1B scenario of the Intergovernmental Panel on Climate Change Institut Pierre Simon Laplace Climate Model (IPCC, IPSL-CM4), applied in a coupled oceanographic-biogeochemical model, combined with a suite of assumed physiological 'rules' for genera-specific bloom development. Based on these models, an expansion in area and/or number of months annually conducive to development of these HABs along the NW European Shelf-Baltic Sea system and NE Asia was projected for both HAB genera, but no expansion (Prorocentrum spp.), or actual contraction in area and months conducive for blooms (Karenia spp.), was projected in the SE Asian domain. The implications of these projections, especially for Northern Europe, are shifts in vulnerability of coastal systems to HAB events, increased regional HAB impacts to aquaculture, increased risks to human health and ecosystems, and economic consequences of these events due to losses to fisheries and ecosystem services.