Spike-based decision learning of nash equilibria in two-player games

Humans and animals face decision tasks in an uncertain multi-agent environment where an agent's strategy may change in time due to the co-adaptation of others strategies. The neuronal substrate and the computational algorithms underlying such adaptive decision making, however, is largely unknown. We propose a population coding model of spiking neurons with a policy gradient procedure that successfully acquires optimal strategies for classical game-theoretical tasks. The suggested population reinforcement learning reproduces data from human behavioral experiments for the blackjack and the inspector game. It performs optimally according to a pure (deterministic) and mixed (stochastic) Nash equilibrium, respectively. In contrast, temporal-difference(TD)-learning, covariance-learning, and basic reinforcement learning fail to perform optimally for the stochastic strategy. Spike-based population reinforcement learning, shown to follow the stochastic reward gradient, is therefore a viable candidate to explain automated decision learning of a Nash equilibrium in two-player games.

Two Stable Equilibria of the Atlantic Subpolar Gyre

The cyclonic circulation of the Atlantic subpolar gyre is a key mechanism for North Atlantic climate variability on a wide range of time scales. It is generally accepted that it is driven by both cyclonic winds and buoyancy forcing, yet the individual importance and dynamical interactions of the two contributions remain unclear. The authors propose a simplified four-box model representing the convective basin of the Labrador Sea and its shallow and deep boundary current system, the western subpolar gyre. Convective heat loss drives a baroclinic flow of relatively light water around the dense center. Eddy salt flux from the boundary current to the center increases with a stronger circulation, favors the formation of dense waters, and thereby sustains a strong baroclinic flow, approximately 10%–25% of the total. In contrast, when the baroclinic flow is not active, surface waters may be too fresh to convect, and a buoyancy-driven circulation cannot develop. This situation corresponds to a second stable circulation mode. A hysteresis is found for variations in surface freshwater flux and the salinity of the near-surface boundary current. An analytical solution is presented and analyzed.

Identifiability of diffusion and sorption parameters from in situ diffusion experiments by using simultaneously tracer dilution and claystone data

In situ diffusion experiments are performed in geological formations at underground research laboratories to overcome the limitations of laboratory diffusion experiments and investigate scale effects. Tracer concentrations are monitored at the injection interval during the experiment (dilution data) and measured from host rock samples around the injection interval at the end of the experiment (overcoring data). Diffusion and sorption parameters are derived from the inverse numerical modeling of the measured tracer data. The identifiability and the uncertainties of tritium and Na-22(+) diffusion and sorption parameters are studied here by synthetic experiments having the same characteristics as the in situ diffusion and retention (DR) experiment performed on Opalinus Clay. Contrary to previous identifiability analyses of in situ diffusion experiments, which used either dilution or overcoring data at approximate locations, our analysis of the parameter identifiability relies simultaneously on dilution and overcoring data, accounts for the actual position of the overcoring samples in the claystone, uses realistic values of the standard deviation of the measurement errors, relies on model identification criteria to select the most appropriate hypothesis about the existence of a borehole disturbed zone and addresses the effect of errors in the location of the sampling profiles. The simultaneous use of dilution and overcoring data provides accurate parameter estimates in the presence of measurement errors, allows the identification of the right hypothesis about the borehole disturbed zone and diminishes other model uncertainties such as those caused by errors in the volume of the circulation system and the effective diffusion coefficient of the filter. The proper interpretation of the experiment requires the right hypothesis about the borehole disturbed zone. A wrong assumption leads to large estimation errors. The use of model identification criteria helps in the selection of the best model. Small errors in the depth of the overcoring samples lead to large parameter estimation errors. Therefore, attention should be paid to minimize the errors in positioning the depth of the samples. The results of the identifiability analysis do not depend on the particular realization of random numbers. (C) 2012 Elsevier B.V. All rights reserved.