Finite-size effects in on-line learning of multilayer neural networks

We complement recent advances in thermodynamic limit analyses of mean on-line gradient descent learning dynamics in multi-layer networks by calculating fluctuations possessed by finite dimensional systems. Fluctuations from the mean dynamics are largest at the onset of specialisation as student hidden unit weight vectors begin to imitate specific teacher vectors, increasing with the degree of symmetry of the initial conditions. In light of this, we include a term to stimulate asymmetry in the learning process, which typically also leads to a significant decrease in training time.

Finite-size effects and error-free communication in Gaussian channels

The efficacy of a specially constructed Gallager-type error-correcting code to communication in a Gaussian channel is examined. The construction is based on the introduction of complex matrices, used in both encoding and decoding, which comprise sub-matrices of cascading connection values. The finite-size effects are estimated for comparing the results with the bounds set by Shannon. The critical noise level achieved for certain code rates and infinitely large systems nearly saturates the bounds set by Shannon even when the connectivity used is low.

Finite-size effects in low-density parity-check codes applied to cryptography

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT

A mechanistic description of combined hardening and size effects

Mechanistic models based on geometrically necessary dislocations are re-examined in light of recent experiments exhibiting the indentation size effect. A simple method is developed to combine work hardening, solid solution hardening, radiation hardening and size effects. The model is verified by experiments in ionic salt crystals. © 2002 Acta Materialia Inc. Published by Elsevier Science Ltd. All rights reserved.

Unfolding grain size effects in barium titanate ferroelectric ceramics

Grain size effects on the physical properties of polycrystalline ferroelectrics have been extensively studied for decades; however there are still major controversies regarding the dependence of the piezoelectric and ferroelectric properties on the grain size. Dense BaTiO3 ceramics with different grain sizes were fabricated by either conventional sintering or spark plasma sintering using micro- and nano-sized powders. The results show that the grain size effect on the dielectric permittivity is nearly independent of the sintering method and starting powder used. A peak in the permittivity is observed in all the ceramics with a grain size near 1μm and can be attributed to a maximum domain wall density and mobility. The piezoelectric coefficient d33 and remnant polarization Pr show diverse grain size effects depending on the particle size of the starting powder and sintering temperature. This suggests that besides domain wall density, other factors such as back fields and point defects, which influence the domain wall mobility, could be responsible for the different grain size dependence observed in the dielectric and piezoelectric/ferroelectric properties. In cases where point defects are not the dominant contributor, the piezoelectric constant d33 and the remnant polarization Pr increase with increasing grain size.

An international cohort comparison of size effects on job growth

The contribution of different-sized businesses to job creation continues to attract policymakers’ attention; however, it has recently been recognised that conclusions about size were confounded with the effect of age. We probe the role of size, controlling for age, by comparing the cohorts of firms born in 1998 over their first decade of life, using variation across half a dozen northern European countries Austria, Finland, Germany, Norway, Sweden and the UK to pin down size effects. We find that a very small proportion of the smallest firms play a crucial role in accounting for cross-country differences in job growth. A closer analysis reveals that the initial size distribution and survival rates do not seem to explain job growth differences between countries, rather it is a small number of rapidly growing firms that are driving this result.

The correlation of indentation size effect experiments with pyramidal and spherical indenters

Experiments were conducted in annealed iridium using pyramidal and spherical indenters over a wide range of load. For a Berkovich pyramidal indenter, the hardness increased with decreasing depth of penetration. However, for spherical indenters, hardness increased with decreasing sphere radius. Based on the number of geometrically necessary dislocations generated during indentation, a theory that takes into account the work hardening differences between pyramidal and spherical indenters is developed to correlate the indentation size effects measured with the two indenters. The experimental results verify the theoretical correlation.

Statistical physics of low density parity check error correcting codes

We study the performance of Low Density Parity Check (LDPC) error-correcting codes using the methods of statistical physics. LDPC codes are based on the generation of codewords using Boolean sums of the original message bits by employing two randomly-constructed sparse matrices. These codes can be mapped onto Ising spin models and studied using common methods of statistical physics. We examine various regular constructions and obtain insight into their theoretical and practical limitations. We also briefly report on results obtained for irregular code constructions, for codes with non-binary alphabet, and on how a finite system size effects the error probability.

Efficient Bayesian inference for learning in the Ising linear perceptron and signal detection in CDMA

Efficient new Bayesian inference technique is employed for studying critical properties of the Ising linear perceptron and for signal detection in code division multiple access (CDMA). The approach is based on a recently introduced message passing technique for densely connected systems. Here we study both critical and non-critical regimes. Results obtained in the non-critical regime give rise to a highly efficient signal detection algorithm in the context of CDMA; while in the critical regime one observes a first-order transition line that ends in a continuous phase transition point. Finite size effects are also studied. © 2006 Elsevier B.V. All rights reserved.

Genetics and competing strategies in a threshold model for mail processing

Multi-agent algorithms inspired by the division of labour in social insects are applied to a problem of distributed mail retrieval in which agents must visit mail producing cities and choose between mail types under certain constraints.The efficiency (i.e. the average amount of mail retrieved per time step), and the flexibility (i.e. the capability of the agents to react to changes in the environment) are investigated both in static and dynamic environments. New rules for mail selection and specialisation are introduced and are shown to exhibit improved efficiency and flexibility compared to existing ones. We employ a genetic algorithm which allows the various rules to evolve and compete. Apart from obtaining optimised parameters for the various rules for any environment, we also observe extinction and speciation. From a more theoretical point of view, in order to avoid finite size effects, most results are obtained for large population sizes. However, we do analyse the influence of population size on the performance. Furthermore, we critically analyse the causes of efficiency loss, derive the exact dynamics of the model in the large system limit under certain conditions, derive theoretical upper bounds for the efficiency, and compare these with the experimental results.

Inference by replication in densely connected systems

An efficient Bayesian inference method for problems that can be mapped onto dense graphs is presented. The approach is based on message passing where messages are averaged over a large number of replicated variable systems exposed to the same evidential nodes. An assumption about the symmetry of the solutions is required for carrying out the averages; here we extend the previous derivation based on a replica-symmetric- (RS)-like structure to include a more complex one-step replica-symmetry-breaking-like (1RSB-like) ansatz. To demonstrate the potential of the approach it is employed for studying critical properties of the Ising linear perceptron and for multiuser detection in code division multiple access (CDMA) under different noise models. Results obtained under the RS assumption in the noncritical regime give rise to a highly efficient signal detection algorithm in the context of CDMA; while in the critical regime one observes a first-order transition line that ends in a continuous phase transition point. Finite size effects are also observed. While the 1RSB ansatz is not required for the original problems, it was applied to the CDMA signal detection problem with a more complex noise model that exhibits RSB behavior, resulting in an improvement in performance. © 2007 The American Physical Society.

The effect of load on agent-based algorithms for distributed task allocation

Multi-agent algorithms inspired by the division of labour in social insects and by markets, are applied to a constrained problem of distributed task allocation. The efficiency (average number of tasks performed), the flexibility (ability to react to changes in the environment), and the sensitivity to load (ability to cope with differing demands) are investigated in both static and dynamic environments. A hybrid algorithm combining both approaches, is shown to exhibit improved efficiency and robustness. We employ nature inspired particle swarm optimisation to obtain optimised parameters for all algorithms in a range of representative environments. Although results are obtained for large population sizes to avoid finite size effects, the influence of population size on the performance is also analysed. From a theoretical point of view, we analyse the causes of efficiency loss, derive theoretical upper bounds for the efficiency, and compare these with the experimental results.

Perception while watching movies:effects of physical screen size and scene type

Over the last decade, television screens and display monitors have increased in size considerably, but has this improved our televisual experience? Our working hypothesis was that the audiences adopt a general strategy that “bigger is better.” However, as our visual perceptions do not tap directly into basic retinal image properties such as retinal image size (C. A. Burbeck, 1987), we wondered whether object size itself might be an important factor. To test this, we needed a task that would tap into the subjective experiences of participants watching a movie on different-sized displays with the same retinal subtense. Our participants used a line bisection task to self-report their level of “presence” (i.e., their involvement with the movie) at several target locations that were probed in a 45-min section of the movie “The Good, The Bad, and The Ugly.” Measures of pupil dilation and reaction time to the probes were also obtained. In Experiment 1, we found that subjective ratings of presence increased with physical screen size, supporting our hypothesis. Face scenes also produced higher presence scores than landscape scenes for both screen sizes. In Experiment 2, reaction time and pupil dilation results showed the same trends as the presence ratings and pupil dilation correlated with presence ratings, providing some validation of the method. Overall, the results suggest that real-time measures of subjective presence might be a valuable tool for measuring audience experience for different types of (i) display and (ii) audiovisual material.

Effects of grain size and microstructure on threshold values and near threshold crack growth in powder-formed Ni-base superalloy

Threshold stress intensity values, ranging from ∼6 to 16 MN m −3/2 can be obtained in powder-formed Nimonic AP1 by changing the microstructure. The threshold and low crack growth rate behaviour at room temperature of a number of widely differing API microstructures, with both ‘necklace’ and fully recrystallized grain structures of various sizes and uniform and bimodal γ′-distributions, have been investigated. The results indicate that grain size is an important microstructural parameter which can control threshold behaviour, with the value of threshold stress intensity increasing with increasing grain size, but that the γ′-distribution is also important. In this Ni-base alloy, as in many others, near threshold fatigue crack growth occurs in a crystallographic manner along {111} planes. This is due to the development of a dislocation structure involving persistent slip bands on {111} planes in the plastic zone, caused by the presence of ordered shearable precipitates in the microstructure. However, as the stress intensity range is increased, a striated growth mode takes over. The results presented show that this transition from faceted to striated growth is associated with a sudden increase in crack propagation rate and occurs when the size of the reverse plastic zone at the crack tip becomes equal to the grain size, independent of any other microstructural variables.