Code-specific policy gradient rules for spiking neurons

Although it is widely believed that reinforcement learning is a suitable tool for describing behavioral learning, the mechanisms by which it can be implemented in networks of spiking neurons are not fully understood. Here, we show that different learning rules emerge from a policy gradient approach depending on which features of the spike trains are assumed to influence the reward signals, i.e., depending on which neural code is in effect. We use the framework of Williams (1992) to derive learning rules for arbitrary neural codes. For illustration, we present policy-gradient rules for three different example codes - a spike count code, a spike timing code and the most general "full spike train" code - and test them on simple model problems. In addition to classical synaptic learning, we derive learning rules for intrinsic parameters that control the excitability of the neuron. The spike count learning rule has structural similarities with established Bienenstock-Cooper-Munro rules. If the distribution of the relevant spike train features belongs to the natural exponential family, the learning rules have a characteristic shape that raises interesting prediction problems.

Contribuições para o estudo do código neural

Os recentes avanços técnicos das duas últimas décadas para o registro de sinais neuroeletrofisiológicos foram essenciais para que se testassem hipóteses há muito propostas acerca de como células nervosas processam e armazenam informação. No entanto, ao permitir maior detalhamento dos dados coletados, as novas tecnologias levam inevitavelmente ao aumento de sua complexidade estatística e, consequentemente, à necessidade de novas ferramentas matemático-computacionais para sua análise. Nesta tese, apresentamos novos métodos para a análise de dois componentes fundamentais nas atuais teorias da codificação neural: (1) assembleias celulares, definidas pela co-ativação de subgrupos neuronais; e (2) o padrão temporal de atividade de neurônios individuais. Em relação a (1), desenvolvemos um método baseado em análise de componentes independentes para identificar e rastrear padrões de co-ativação significativos com alta resolução temporal. Superamos limitações de métodos anteriores, ao efetivamente isolar assembleias e abrir a possibilidade de analisar simultaneamente grandes populações neuronais. Em relação a (2), apresentamos uma nova técnica para a extração de padrões de atividade em trens de disparo baseada na decomposição wavelet. Demonstramos, por meio de simulações e de aplicação a dados reais, que nossa ferramenta supera as mais utilizadas atualmente para decodificar respostas de neurônios e estimar a informação de Shannon entre trens de disparos e estímulos externos.

Stimulus Integration and Parsing in the Primate Auditory Midbrain

Integrating information from multiple sources is a crucial function of the brain. Examples of such integration include multiple stimuli of different modalties, such as visual and auditory, multiple stimuli of the same modality, such as auditory and auditory, and integrating stimuli from the sensory organs (i.e. ears) with stimuli delivered from brain-machine interfaces.

The overall aim of this body of work is to empirically examine stimulus integration in these three domains to inform our broader understanding of how and when the brain combines information from multiple sources.

First, I examine visually-guided auditory, a problem with implications for the general problem in learning of how the brain determines what lesson to learn (and what lessons not to learn). For example, sound localization is a behavior that is partially learned with the aid of vision. This process requires correctly matching a visual location to that of a sound. This is an intrinsically circular problem when sound location is itself uncertain and the visual scene is rife with possible visual matches. Here, we develop a simple paradigm using visual guidance of sound localization to gain insight into how the brain confronts this type of circularity. We tested two competing hypotheses. 1: The brain guides sound location learning based on the synchrony or simultaneity of auditory-visual stimuli, potentially involving a Hebbian associative mechanism. 2: The brain uses a ‘guess and check’ heuristic in which visual feedback that is obtained after an eye movement to a sound alters future performance, perhaps by recruiting the brain’s reward-related circuitry. We assessed the effects of exposure to visual stimuli spatially mismatched from sounds on performance of an interleaved auditory-only saccade task. We found that when humans and monkeys were provided the visual stimulus asynchronously with the sound but as feedback to an auditory-guided saccade, they shifted their subsequent auditory-only performance toward the direction of the visual cue by 1.3-1.7 degrees, or 22-28% of the original 6 degree visual-auditory mismatch. In contrast when the visual stimulus was presented synchronously with the sound but extinguished too quickly to provide this feedback, there was little change in subsequent auditory-only performance. Our results suggest that the outcome of our own actions is vital to localizing sounds correctly. Contrary to previous expectations, visual calibration of auditory space does not appear to require visual-auditory associations based on synchrony/simultaneity.

My next line of research examines how electrical stimulation of the inferior colliculus influences perception of sounds in a nonhuman primate. The central nucleus of the inferior colliculus is the major ascending relay of auditory information before it reaches the forebrain, and thus an ideal target for understanding low-level information processing prior to the forebrain, as almost all auditory signals pass through the central nucleus of the inferior colliculus before reaching the forebrain. Thus, the inferior colliculus is the ideal structure to examine to understand the format of the inputs into the forebrain and, by extension, the processing of auditory scenes that occurs in the brainstem. Therefore, the inferior colliculus was an attractive target for understanding stimulus integration in the ascending auditory pathway.

Moreover, understanding the relationship between the auditory selectivity of neurons and their contribution to perception is critical to the design of effective auditory brain prosthetics. These prosthetics seek to mimic natural activity patterns to achieve desired perceptual outcomes. We measured the contribution of inferior colliculus (IC) sites to perception using combined recording and electrical stimulation. Monkeys performed a frequency-based discrimination task, reporting whether a probe sound was higher or lower in frequency than a reference sound. Stimulation pulses were paired with the probe sound on 50% of trials (0.5-80 µA, 100-300 Hz, n=172 IC locations in 3 rhesus monkeys). Electrical stimulation tended to bias the animals’ judgments in a fashion that was coarsely but significantly correlated with the best frequency of the stimulation site in comparison to the reference frequency employed in the task. Although there was considerable variability in the effects of stimulation (including impairments in performance and shifts in performance away from the direction predicted based on the site’s response properties), the results indicate that stimulation of the IC can evoke percepts correlated with the frequency tuning properties of the IC. Consistent with the implications of recent human studies, the main avenue for improvement for the auditory midbrain implant suggested by our findings is to increase the number and spatial extent of electrodes, to increase the size of the region that can be electrically activated and provide a greater range of evoked percepts.

My next line of research employs a frequency-tagging approach to examine the extent to which multiple sound sources are combined (or segregated) in the nonhuman primate inferior colliculus. In the single-sound case, most inferior colliculus neurons respond and entrain to sounds in a very broad region of space, and many are entirely spatially insensitive, so it is unknown how the neurons will respond to a situation with more than one sound. I use multiple AM stimuli of different frequencies, which the inferior colliculus represents using a spike timing code. This allows me to measure spike timing in the inferior colliculus to determine which sound source is responsible for neural activity in an auditory scene containing multiple sounds. Using this approach, I find that the same neurons that are tuned to broad regions of space in the single sound condition become dramatically more selective in the dual sound condition, preferentially entraining spikes to stimuli from a smaller region of space. I will examine the possibility that there may be a conceptual linkage between this finding and the finding of receptive field shifts in the visual system.

In chapter 5, I will comment on these findings more generally, compare them to existing theoretical models, and discuss what these results tell us about processing in the central nervous system in a multi-stimulus situation. My results suggest that the brain is flexible in its processing and can adapt its integration schema to fit the available cues and the demands of the task.

Timing analysis of assembler code control-flow paths

Timinganalysis of assembler code is essential to achieve the strongest possible guarantee of correctness for safety-critical, real-time software. Previous work has shown how timingconstrain ts on controlflow paths through high-level language programs can be formalised using the semantics of the statements comprisingthe path. We extend these results to assembler-level code where it becomes possible to not only determine timingconstrain ts, but also to verify them against the known execution times for each instruction. A minimal formal model is developed with both a weakest liberal precondition and a strongest postcondition semantics. However, despite the formalism’s simplicity, it is shown that complex timingb ehaviour associated with instruction pipeliningand iterative code can be modelled accurately.

Transit timing variations in WASP-10b induced by stellar activity

The hot-JupiterWASP-10bwas reported by Maciejewski et al. to showtransit timing variations (TTVs) with an amplitude of ~3.5 min. These authors proposed that the observed TTVs were caused by a 0.1M_Jup perturbing companion with an orbital period of ~5.23 d, and hence, close to the outer 5:3 mean-motion resonance with WASP-10b. To test this scenario, we present eight new transit light curves of WASP-10b obtained with the Faulkes Telescope North and the Liverpool Telescope. The new light curves, together with 22 previously published ones, were modelled with a Markov Chain Monte Carlo transit fitting code. Transit depth differences reported forWASP-10b are thought to be due to starspot-induced brightness modulation of the host star. Assuming the star is brighter at the activity minimum, we favour a small planetary radius. We find Rp = 1.039^+0.043 _-0.049R_Jup in agreement with Johnson et al. and Maciejewski et al. Recent studies find no evidence for a significant eccentricity in this system. We present consistent system parameters for a circular orbit and refine the orbital ephemeris ofWASP-10b. Our homogeneously derived transit times do not support the previous claimed TTV signal, which was strongly dependent on two previously published transits that have been incorrectly normalized. Nevertheless, a linear ephemeris is not a statistically good fit to the transit times of WASP-10b. We show that the observed transit time variations are due to spot occultation features or systematics. We discuss and exemplify the effects of occultation spot features in the measured transit times and show that despite spot occultation during egress and ingress being difficult to distinguish in the transit light curves, they have a significant effect in the measured transit times. We conclude that if we account for spot features, the transit times of WASP-10b are consistent with a linear ephemeris with the exception of one transit (epoch 143) which is a partial transit. Therefore, there is currently no evidence for the existence of a companion to WASP-10b. Our results support the lack of TTVs of hot-Jupiters reported for the Kepler sample. 

Exploiting dynamic timing margins in microprocessors for frequency-over-scaling with instruction-based clock adjustment

Static timing analysis provides the basis for setting the clock period of a microprocessor core, based on its worst-case critical path. However, depending on the design, this critical path is not always excited and therefore dynamic timing margins exist that can theoretically be exploited for the benefit of better speed or lower power consumption (through voltage scaling). This paper introduces predictive instruction-based dynamic clock adjustment as a technique to trim dynamic timing margins in pipelined microprocessors. To this end, we exploit the different timing requirements for individual instructions during the dynamically varying program execution flow without the need for complex circuit-level measures to detect and correct timing violations. We provide a design flow to extract the dynamic timing information for the design using post-layout dynamic timing analysis and we integrate the results into a custom cycle-accurate simulator. This simulator allows annotation of individual instructions with their impact on timing (in each pipeline stage) and rapidly derives the overall code execution time for complex benchmarks. The design methodology is illustrated at the microarchitecture level, demonstrating the performance and power gains possible on a 6-stage OpenRISC in-order general purpose processor core in a 28nm CMOS technology. We show that employing instruction-dependent dynamic clock adjustment leads on average to an increase in operating speed by 38% or to a reduction in power consumption by 24%, compared to traditional synchronous clocking, which at all times has to respect the worst-case timing identified through static timing analysis.

Chemical and Sr isotopic compositions and timing of carbonate mineral precipitation, ODP Leg 115

The strontium isotope ratios of authigenic carbonates from Indian Ocean sea-floor basalts have been used to determine the timing of carbonate mineral precipitation and fluid flow. The samples include calcites from 57.2 Ma crust from Ocean Drilling Project (ODP) Site 715, and calcites, aragonites, and siderites from 63.7 Ma crust from ODP Site 707. At Site 715, calcite precipitation may have begun at any time after the basalts cooled, and it continued until approximately 31 Ma, or 26 m.y. after basalt eruption. At Site 707, aragonite and siderite did not begin to precipitate until about 36 Ma, almost 30 m.y. after basalt eruption, and continued to precipitate until at least 30 and 28 Ma, respectively. Calcite precipitation began at approximately 32 Ma and continued until 22 Ma. These ages suggest that vein mineral deposition and low-temperature fluid circulation in the ocean crust may continue for much longer periods of time than previously observed.

Adding Run-Time Monitoring to UML-RT by Modifying the Papyrus-RT Code Generator

In Model-Driven Engineering (MDE), the developer creates a model using a language such as Unified Modeling Language (UML) or UML for Real-Time (UML-RT) and uses tools such as Papyrus or Papyrus-RT that generate code for them based on the model they create. Tracing allows developers to get insights such as which events occur and timing information into their own application as it runs. We try to add monitoring capabilities using Linux Trace Toolkit: next generation (LTTng) to models created in UML-RT using Papyrus-RT. The implementation requires changing the code generator to add tracing statements for the events that the user wants to monitor to the generated code. We also change the makefile to automate the build process and we create an Extensible Markup Language (XML) file that allows developers to view their traces visually using Trace Compass, an Eclipse-based trace viewing tool. Finally, we validate our results using three models we create and trace.

Parallelisation of a 3-D electromagnetic scattering code

A high-frequency time domain finite element scattering code using a combination of edge and piecewise constant elements on unstructured tetrahedral meshes is described. A comparison of computation with theory is given for scattering from a sphere. A parallel implementation making use of the bulk synchronous parallel (BSP) programming model is described in detail; a BSP performance model of the parallelized field calculation is derived and compared to timing measurements on up to 128 processors on a Cray T3D.