Finite-size effects in on-line learning of multilayer neural networks

We complement recent advances in thermodynamic limit analyses of mean on-line gradient descent learning dynamics in multi-layer networks by calculating fluctuations possessed by finite dimensional systems. Fluctuations from the mean dynamics are largest at the onset of specialisation as student hidden unit weight vectors begin to imitate specific teacher vectors, increasing with the degree of symmetry of the initial conditions. In light of this, we include a term to stimulate asymmetry in the learning process, which typically also leads to a significant decrease in training time.

Finite-size effects and error-free communication in Gaussian channels

The efficacy of a specially constructed Gallager-type error-correcting code to communication in a Gaussian channel is examined. The construction is based on the introduction of complex matrices, used in both encoding and decoding, which comprise sub-matrices of cascading connection values. The finite-size effects are estimated for comparing the results with the bounds set by Shannon. The critical noise level achieved for certain code rates and infinitely large systems nearly saturates the bounds set by Shannon even when the connectivity used is low.

Finite-size effects in low-density parity-check codes applied to cryptography

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT

A mechanistic description of combined hardening and size effects

Mechanistic models based on geometrically necessary dislocations are re-examined in light of recent experiments exhibiting the indentation size effect. A simple method is developed to combine work hardening, solid solution hardening, radiation hardening and size effects. The model is verified by experiments in ionic salt crystals. © 2002 Acta Materialia Inc. Published by Elsevier Science Ltd. All rights reserved.

Unfolding grain size effects in barium titanate ferroelectric ceramics

Grain size effects on the physical properties of polycrystalline ferroelectrics have been extensively studied for decades; however there are still major controversies regarding the dependence of the piezoelectric and ferroelectric properties on the grain size. Dense BaTiO3 ceramics with different grain sizes were fabricated by either conventional sintering or spark plasma sintering using micro- and nano-sized powders. The results show that the grain size effect on the dielectric permittivity is nearly independent of the sintering method and starting powder used. A peak in the permittivity is observed in all the ceramics with a grain size near 1μm and can be attributed to a maximum domain wall density and mobility. The piezoelectric coefficient d33 and remnant polarization Pr show diverse grain size effects depending on the particle size of the starting powder and sintering temperature. This suggests that besides domain wall density, other factors such as back fields and point defects, which influence the domain wall mobility, could be responsible for the different grain size dependence observed in the dielectric and piezoelectric/ferroelectric properties. In cases where point defects are not the dominant contributor, the piezoelectric constant d33 and the remnant polarization Pr increase with increasing grain size.

An international cohort comparison of size effects on job growth

The contribution of different-sized businesses to job creation continues to attract policymakers’ attention; however, it has recently been recognised that conclusions about size were confounded with the effect of age. We probe the role of size, controlling for age, by comparing the cohorts of firms born in 1998 over their first decade of life, using variation across half a dozen northern European countries Austria, Finland, Germany, Norway, Sweden and the UK to pin down size effects. We find that a very small proportion of the smallest firms play a crucial role in accounting for cross-country differences in job growth. A closer analysis reveals that the initial size distribution and survival rates do not seem to explain job growth differences between countries, rather it is a small number of rapidly growing firms that are driving this result.

Magnetic resonance imaging of a magnetic field-dependent chemical wave

The magnetic field dependence of the travelling wave formed during the reaction of (ethylenediaminetetraacetato)cobalt (II) (Co(II)EDTA2- ) and hydrogen peroxide was studied using magnetic resonance imaging (MRI). The reaction was investigated in a vertical tube, in which the wave was initiated from above. The wave propagated downwards, initially with a flat wavefront before forming a finger. Magnetic field effects were observed only once the finger had formed. The wave propagation was accelerated by a magnetic field with a negative gradient (i.e., when the field was stronger at the top of the tube than at the bottom) and slightly decelerated by positive field gradients.

The correlation of indentation size effect experiments with pyramidal and spherical indenters

Experiments were conducted in annealed iridium using pyramidal and spherical indenters over a wide range of load. For a Berkovich pyramidal indenter, the hardness increased with decreasing depth of penetration. However, for spherical indenters, hardness increased with decreasing sphere radius. Based on the number of geometrically necessary dislocations generated during indentation, a theory that takes into account the work hardening differences between pyramidal and spherical indenters is developed to correlate the indentation size effects measured with the two indenters. The experimental results verify the theoretical correlation.

Statistical physics of low density parity check error correcting codes

We study the performance of Low Density Parity Check (LDPC) error-correcting codes using the methods of statistical physics. LDPC codes are based on the generation of codewords using Boolean sums of the original message bits by employing two randomly-constructed sparse matrices. These codes can be mapped onto Ising spin models and studied using common methods of statistical physics. We examine various regular constructions and obtain insight into their theoretical and practical limitations. We also briefly report on results obtained for irregular code constructions, for codes with non-binary alphabet, and on how a finite system size effects the error probability.

Efficient Bayesian inference for learning in the Ising linear perceptron and signal detection in CDMA

Efficient new Bayesian inference technique is employed for studying critical properties of the Ising linear perceptron and for signal detection in code division multiple access (CDMA). The approach is based on a recently introduced message passing technique for densely connected systems. Here we study both critical and non-critical regimes. Results obtained in the non-critical regime give rise to a highly efficient signal detection algorithm in the context of CDMA; while in the critical regime one observes a first-order transition line that ends in a continuous phase transition point. Finite size effects are also studied. © 2006 Elsevier B.V. All rights reserved.

Genetics and competing strategies in a threshold model for mail processing

Multi-agent algorithms inspired by the division of labour in social insects are applied to a problem of distributed mail retrieval in which agents must visit mail producing cities and choose between mail types under certain constraints.The efficiency (i.e. the average amount of mail retrieved per time step), and the flexibility (i.e. the capability of the agents to react to changes in the environment) are investigated both in static and dynamic environments. New rules for mail selection and specialisation are introduced and are shown to exhibit improved efficiency and flexibility compared to existing ones. We employ a genetic algorithm which allows the various rules to evolve and compete. Apart from obtaining optimised parameters for the various rules for any environment, we also observe extinction and speciation. From a more theoretical point of view, in order to avoid finite size effects, most results are obtained for large population sizes. However, we do analyse the influence of population size on the performance. Furthermore, we critically analyse the causes of efficiency loss, derive the exact dynamics of the model in the large system limit under certain conditions, derive theoretical upper bounds for the efficiency, and compare these with the experimental results.

Inference by replication in densely connected systems

An efficient Bayesian inference method for problems that can be mapped onto dense graphs is presented. The approach is based on message passing where messages are averaged over a large number of replicated variable systems exposed to the same evidential nodes. An assumption about the symmetry of the solutions is required for carrying out the averages; here we extend the previous derivation based on a replica-symmetric- (RS)-like structure to include a more complex one-step replica-symmetry-breaking-like (1RSB-like) ansatz. To demonstrate the potential of the approach it is employed for studying critical properties of the Ising linear perceptron and for multiuser detection in code division multiple access (CDMA) under different noise models. Results obtained under the RS assumption in the noncritical regime give rise to a highly efficient signal detection algorithm in the context of CDMA; while in the critical regime one observes a first-order transition line that ends in a continuous phase transition point. Finite size effects are also observed. While the 1RSB ansatz is not required for the original problems, it was applied to the CDMA signal detection problem with a more complex noise model that exhibits RSB behavior, resulting in an improvement in performance. © 2007 The American Physical Society.

The effect of load on agent-based algorithms for distributed task allocation

Multi-agent algorithms inspired by the division of labour in social insects and by markets, are applied to a constrained problem of distributed task allocation. The efficiency (average number of tasks performed), the flexibility (ability to react to changes in the environment), and the sensitivity to load (ability to cope with differing demands) are investigated in both static and dynamic environments. A hybrid algorithm combining both approaches, is shown to exhibit improved efficiency and robustness. We employ nature inspired particle swarm optimisation to obtain optimised parameters for all algorithms in a range of representative environments. Although results are obtained for large population sizes to avoid finite size effects, the influence of population size on the performance is also analysed. From a theoretical point of view, we analyse the causes of efficiency loss, derive theoretical upper bounds for the efficiency, and compare these with the experimental results.

Influence of check and field size on the visual evoked magnetic response to a pattern shift stimulus

A decrease in the check size of a pattern shift stimulus increases the latency and amplitude of the visual evoked potential (VEP) P100. In addition, for a given check size, decreasing the size of the stimulus field increases the latency and amplitude of the P100. These results imply that the central regions of the retina make a significant contribution to the generation of the electrical P100. However, the corresponding magnetic P100m may have a different origin. We have studied the effects of check and field size on the P100m in five normal subjects using a DC-Squid, second-order gradiometer. Magnetic responses were recorded at the positive maximum of the P100m over the occipital scalp to six check sizes (10-100') presented in a large (13 degrees 34') and small (5 degrees 14') field and to a large check (100') presented in seven field sizes (1 degree 45' - 15 degrees 10'). No responses were recorded to any check size with a small field. Decreasing the check size presented in a large field increased latency of the P100m by approx. 30 ms while the amplitude of the response decreased with the largest reduction occurring between 70' and 12' checks. Using a large check, latency increased and amplitude decreased as the field size was reduced. The latency changes in response to check and field size were similar to those described for the VEP although the magnitudes of the magnetic changes were greater. Unlike the VEP, amplitude responses were maximal when large checks were presented in a large stimulus field. This suggests that regions outside the central retina make a more significant contribution to the visual evoked magnetic response than they do to the VEP, and that the P100m may be useful clinically in the study of diseases that affect the more peripheral regions of the retina.