Learning in neural networks and stochastic approximation methods with averaging

The problem of adjusting the weights (learning) in multilayer feedforward neural networks (NN) is known to be of a high importance when utilizing NN techniques in various practical applications. The learning procedure is to be performed as fast as possible and in a simple computational fashion, the two requirements which are usually not satisfied practically by the methods developed so far. Moreover, the presence of random inaccuracies are usually not taken into account. In view of these three issues, an alternative stochastic approximation approach discussed in the paper, seems to be very promising.

Electron spin resonance study of puff-resolved free radical formation in mainstream cigarette smoke

Puff-by-puff resolved gas phase free radicals were measured in mainstream smoke from Kentucky 2R4F reference cigarettes using ESR spectroscopy. Three spin-trapping reagents were evaluated: PBN, DMPO and DEPMPO. Two procedures were used to collect gas phase smoke on a puff-resolved basis: i) the accumulative mode, in which all the gas phase smoke up to a particular puff was bubbled into the trap (i.e., the 5th puff corresponded to the total smoke from the 1st to 5th puffs). In this case, after a specified puff, an aliquot of the spin trap was taken and analysed; or, ii) the individual mode, in which the spin trap was analysed and then replaced after each puff. Spin concentrations were determined by double-integration of the first derivative of the ESR signal. This was compared with the integrals of known standards using the TEMPO free radical. The radicals trapped with PBN were mainly carbon-centred, whilst the oxygen-centred radicals were identified with DMPO and DEPMPO. With each spin trap, the puff-resolved radical concentrations showed a characteristic pattern as a function of the puff number. Based on the spin concentrations, the DMPO and DEPMPO spin traps showed better trapping efficiencies than PBN. The implication for gas phase free radical analysis is that a range of different spin traps should be used to probe complex free radical reactions in cigarette smoke.

Electron paramagnetic resonance investigation of the free radicals in the gas and particulate phases of cigarette smoke using spin-trapping techniques

Free radicals in cigarette smoke have been studied using spin trapping EPR techniques. 2R4F reference cigarettes were smoked using 35 ml puff volumes of 2 seconds duration, once every 60 seconds. The particulate phase of the smoke was separated from the gas phase by passing the smoke through a Cambridge filter pad. For both phases, free radicals were measured and identified. A range of spin-traps was employed: PBN, DMPO, DEPMPO, and DPPH-PBN. In the gas-phase, short-lived carbon- and oxygen- centered radicals were identified; the ratios between them changed during the smoking runs. For the first puffs, C-centered radicals predominated while for the later puffs, O-centered radicals were mainly observed. The particulate phase and the ‘tar’ were studied as well.

A hydrogen-1 nuclear magnetic resonance, mass spectral and extended Hückel study of some olefin complexes of rhodium(I) and iridium(I) and the crystal and molecular structure of η4-2,4-dimethylpenta-1,4-diene(η5-formylcyclopentadienyl)rhodium(I)

A 1H NMR study of monosubstituted η-cyclopentadienyl-rhodium(I) complexes of type LLRh(C5H4X) and -iridium(I) complexes of type L2Ir(C5H4X) (L = ethene, LL = 1,3- or 1,5-diolefin; X = C(C6H5)3, CHO, or COOCH3) has been carried out. For complexes of both metals in which the neutral ligand is ethene or a non-conjugated diolefin the NMR spectra of the cyclopentadienyl protons are unusual in that H(2), H(5) resonate to high field either at room temperature or below. The corresponding NMR spectra for the cyclopentadienyl ring protons of complexes where the neutral ligand is a conjugated diene are, with one exception, normal. A single crystal X-ray structural analysis of (η4-2,4-dimethylpenta-1,4-diene)(η5-formylcyclopentadienyl)rhodium(I) (which exhibits an abnormal 1H NMR spectrum) reveals substantial localisation of electron density in the C(3)C(4) Cp ring bond (1.283(33) Å) which may be consistent with a contribution from an ‘allyl-ene’ rotamer to the ring—metal bonding scheme. An extended Hückel calculation with self consistent charge iteration was performed on this complex. The results predict a greater Mulliken overlap population for the C(3)C(4) bond in the cyclopentadienyl ring and show that the localisation is dependent on both the Cp ring substituent and the nature of the diolefin. The mass spectral fragmentation patterns of some representative diene complexes of iridium(I) and rhodium(I) are presented.

Large-scale length and time scales for use with stochastic convective parameterization

Many numerical models for weather prediction and climate studies are run at resolutions that are too coarse to resolve convection explicitly, but too fine to justify the local equilibrium assumed by conventional convective parameterizations. The Plant-Craig (PC) stochastic convective parameterization scheme, developed in this paper, solves this problem by removing the assumption that a given grid-scale situation must always produce the same sub-grid-scale convective response. Instead, for each timestep and gridpoint, one of the many possible convective responses consistent with the large-scale situation is randomly selected. The scheme requires as input the large-scale state as opposed to the instantaneous grid-scale state, but must nonetheless be able to account for genuine variations in the largescale situation. Here we investigate the behaviour of the PC scheme in three-dimensional simulations of radiative-convective equilibrium, demonstrating in particular that the necessary space-time averaging required to produce a good representation of the input large-scale state is not in conflict with the requirement to capture large-scale variations. The resulting equilibrium profiles agree well with those obtained from established deterministic schemes, and with corresponding cloud-resolving model simulations. Unlike the conventional schemes the statistics for mass flux and rainfall variability from the PC scheme also agree well with relevant theory and vary appropriately with spatial scale. The scheme is further shown to adapt automatically to changes in grid length and in forcing strength.

Integrating spatial dependence into stochastic frontier analysis

An approach to incorporate spatial dependence into stochastic frontier analysis is developed and applied to a sample of 215 dairy farms in England and Wales. A number of alternative specifications for the spatial weight matrix are used to analyse the effect of these on the estimation of spatial dependence. Estimation is conducted using a Bayesian approach and results indicate that spatial dependence is present when explaining technical inefficiency.

Working Group 1: Future developments in deterministic and stochastic parametrizations

Our group considered the desirability of including representations of uncertainty in the development of parameterizations. (By ‘uncertainty’ here we mean the deviation of sub-grid scale fluxes or tendencies in any given model grid box from truth.) We unanimously agreed that the ECWMF should attempt to provide a more physical basis for uncertainty estimates than the very effective but ad hoc methods being used at present. Our discussions identified several issues that will arise.

Stochastic parameterization: uncertainties from convection

In 2005, the ECMWF held a workshop on stochastic parameterisation, at which the convection was seen as being a key issue. That much is clear from the working group reports and particularly the statement from working group 1 that “it is clear that a stochastic convection scheme is desirable”. The present note aims to consider our current status in comparison with some of the issues raised and hopes expressed in that working group report.

Stochastic perturbations to dynamical systems: a response theory approach

Using the formalism of the Ruelle response theory, we study how the invariant measure of an Axiom A dynamical system changes as a result of adding noise, and describe how the stochastic perturbation can be used to explore the properties of the underlying deterministic dynamics. We first find the expression for the change in the expectation value of a general observable when a white noise forcing is introduced in the system, both in the additive and in the multiplicative case. We also show that the difference between the expectation value of the power spectrum of an observable in the stochastically perturbed case and of the same observable in the unperturbed case is equal to the variance of the noise times the square of the modulus of the linear susceptibility describing the frequency-dependent response of the system to perturbations with the same spatial patterns as the considered stochastic forcing. This provides a conceptual bridge between the change in the fluctuation properties of the system due to the presence of noise and the response of the unperturbed system to deterministic forcings. Using Kramers-Kronig theory, it is then possible to derive the real and imaginary part of the susceptibility and thus deduce the Green function of the system for any desired observable. We then extend our results to rather general patterns of random forcing, from the case of several white noise forcings, to noise terms with memory, up to the case of a space-time random field. Explicit formulas are provided for each relevant case analysed. As a general result, we find, using an argument of positive-definiteness, that the power spectrum of the stochastically perturbed system is larger at all frequencies than the power spectrum of the unperturbed system. We provide an example of application of our results by considering the spatially extended chaotic Lorenz 96 model. These results clarify the property of stochastic stability of SRB measures in Axiom A flows, provide tools for analysing stochastic parameterisations and related closure ansatz to be implemented in modelling studies, and introduce new ways to study the response of a system to external perturbations. Taking into account the chaotic hypothesis, we expect that our results have practical relevance for a more general class of system than those belonging to Axiom A.

Attention through self-synchronisation in the spiking neuron stochastic diffusion network

The paper discusses ensemble behaviour in the Spiking Neuron Stochastic Diffusion Network, SNSDN, a novel network exploring biologically plausible information processing based on higher order temporal coding. SNSDN was proposed as an alternative solution to the binding problem [1]. SNSDN operation resembles Stochastic Diffusin on Search, SDS, a non-deterministic search algorithm able to rapidly locate the best instantiation of a target pattern within a noisy search space ([3], [5]). In SNSDN, relevant information is encoded in the length of interspike intervals. Although every neuron operates in its own time, ‘attention’ to a pattern in the search space results in self-synchronised activity of a large population of neurons. When multiple patterns are present in the search space, ‘switching of at- tention’ results in a change of the synchronous activity. The qualitative effect of attention on the synchronicity of spiking behaviour in both time and frequency domain will be discussed.