Gradient estimation in dendritic reinforcement learning

We study synaptic plasticity in a complex neuronal cell model where NMDA-spikes can arise in certain dendritic zones. In the context of reinforcement learning, two kinds of plasticity rules are derived, zone reinforcement (ZR) and cell reinforcement (CR), which both optimize the expected reward by stochastic gradient ascent. For ZR, the synaptic plasticity response to the external reward signal is modulated exclusively by quantities which are local to the NMDA-spike initiation zone in which the synapse is situated. CR, in addition, uses nonlocal feedback from the soma of the cell, provided by mechanisms such as the backpropagating action potential. Simulation results show that, compared to ZR, the use of nonlocal feedback in CR can drastically enhance learning performance. We suggest that the availability of nonlocal feedback for learning is a key advantage of complex neurons over networks of simple point neurons, which have previously been found to be largely equivalent with regard to computational capability.

Hydrodynamic Characterization of Bile Salt Host-Guest Complexes by Pulsed Field Gradient NMR Spectroscopy

The work described herein is aimed at understanding primary and secondary aggregation of bile salt micelles and how micelles can perform chiral recognition of binapthyl analytes. Previous work with cholate and deoxycholate using micellar electrokinetic chromatography (MEKC) and nuclear magnetic resonance (NMR) has provided insightinto cholate and deoxycholate micelle formation, especially with respect to the critical micelle concentration (CMC). Chiral separations of the model analyte, 1,1â??-binaphthyl-2,2â??-diyl hydrogen phosphate (BNDHP), via cholate (C) and deoxycholate (DC) mediated MEKC separataions previously have shown the DC CMC to be 7-10 mM andthe cholate CMC at 14 mM at ph 12. A second model analyte,1,1â??-binaphthol (BN), was also previously investigated to probe micellar structure, but the MEKC data for this analyte implied a higher CMC, which may be interpreted as secondary aggregation. Thiswork extends the investigation of bile salts to include pulsed field gradient spin echo (PFGSE) NMR experiments being used to gain information about the size and degree of polydispersity of cholate and deoxycholate micelles. Concentrations of cholate below 10mM show a large variation in effective radius likely due to the existence of transient preliminary aggregates. The onset of the primary micelle shows a dramatic increase in effective radius of the micelle in cholate and deoxycholate. In the region of expectedsecondary aggregation a gradual increase of effective radius was observed with cholate; deoxycholate showed a persistent aggregate size in the secondary micelle region that is modulated by the presence of an analyte molecule. Effective radii of cholate anddeoxycholate (individually) were compared with and without R- and S-BNDHP in order to observe the effective radius difference of micelles with and without analyte present. The presence of S-BNDHP consistently resulted in a larger effective aggregate radius incholate and deoxycholate, confirming previous data of the S-BNDHP interacting more with the micelle than R-BNDHP. In total, various NMR techniques, like diffusion NMR can be used to gain a greater understanding of the bile salt micellization process and chiral resolution.

Using Micro-Gravity Techniques to Map Alluvium Thickness and Pleistocene Location of the West Branch of the Susquehanna River Near Muncy, Pennsylvania

Laurentide glaciation during the early Pleistocene (~970 ka) dammed the southeast-flowing West Branch of the Susquehanna River (WBSR), scouring bedrock and creating 100-km-long glacial Lake Lesley near the Great Bend at Muncy, Pennsylvania (Ramage et al., 1998). Local drill logs and well data indicate that subsequent paleo-outwash floods and modern fluvial processes have deposited as much as 30 meters of alluvium in this area, but little is known about the valley fill architecture and the bedrock-alluvium interface. By gaining a greater understanding of the bedrock-alluvium interface the project will not only supplement existing depth to bedrock information, but also provide information pertinent to the evolution of the Muncy Valley landscape. This project determined if variations in the thickness of the valley fill were detectable using micro-gravity techniques to map the bedrock-alluvium interface. The gravity method was deemed appropriate due to scale of the study area (~30 km2), ease of operation by a single person, and the available geophysical equipment. A LaCoste and Romberg Gravitron unit was used to collect gravitational field readings at 49 locations over 5 transects across the Muncy Creek and Susquehanna River valleys (approximately 30 km2), with at least two gravity base stations per transect. Precise latitude, longitude and ground surface elevation at each location were measured using an OPUS corrected Trimble RTK-GPS unit. Base stations were chosen based on ease of access due to the necessity of repeat measurements. Gravity measurement locations were selected and marked to provide easy access and repeat measurements. The gravimeter was returned to a base station within every two hours and a looping procedure was used to determine drift and maximize confidence in the gravity measurements. A two-minute calibration reading at each station was used to minimize any tares in the data. The Gravitron digitally recorded finite impulse response filtered gravity measurements every 20 seconds at each station. A measurement period of 15 minutes was used for each base station occupation and a minimum of 5 minutes at all other locations. Longer or multiple measurements were utilized at some sites if drift or other externalities (i.e. train or truck traffic) were effecting readings. Average, median, standard deviation and 95% confidence interval were calculated for each station. Tidal, drift, latitude, free-air, Bouguer and terrain corrections were then applied. The results show that the gravitational field decreases as alluvium thickness increases across the axes of the Susquehanna River and Muncy Creek valleys. However, the location of the gravity low does not correspond with the present-day location of the West Branch of the Susquehanna River (WBSR), suggesting that the WBSR may have been constrained along Bald Eagle Mountain by a glacial lobe originating from the Muncy Creek Valley to the northeast. Using a 3-D inversion model, the topography of the bedrock-alluvium interface was determined over the extent of the study area using a density contrast of -0.8 g/cm3. Our results are consistent with the bedrock geometry of the area, and provide a low-cost, non-invasive and efficient method for exploring the subsurface and for supplementing existing well data.