The theory of on-line learning: a statistical physics approach

In this paper we review recent theoretical approaches for analysing the dynamics of on-line learning in multilayer neural networks using methods adopted from statistical physics. The analysis is based on monitoring a set of macroscopic variables from which the generalisation error can be calculated. A closed set of dynamical equations for the macroscopic variables is derived analytically and solved numerically. The theoretical framework is then employed for defining optimal learning parameters and for analysing the incorporation of second order information into the learning process using natural gradient descent and matrix-momentum based methods. We will also briefly explain an extension of the original framework for analysing the case where training examples are sampled with repetition.

Statistical physics of low density parity check error correcting codes

We study the performance of Low Density Parity Check (LDPC) error-correcting codes using the methods of statistical physics. LDPC codes are based on the generation of codewords using Boolean sums of the original message bits by employing two randomly-constructed sparse matrices. These codes can be mapped onto Ising spin models and studied using common methods of statistical physics. We examine various regular constructions and obtain insight into their theoretical and practical limitations. We also briefly report on results obtained for irregular code constructions, for codes with non-binary alphabet, and on how a finite system size effects the error probability.

Low density parity check codes: a statistical physics perspective

The modem digital communication systems are made transmission reliable by employing error correction technique for the redundancies. Codes in the low-density parity-check work along the principles of Hamming code, and the parity-check matrix is very sparse, and multiple errors can be corrected. The sparseness of the matrix allows for the decoding process to be carried out by probability propagation methods similar to those employed in Turbo codes. The relation between spin systems in statistical physics and digital error correcting codes is based on the existence of a simple isomorphism between the additive Boolean group and the multiplicative binary group. Shannon proved general results on the natural limits of compression and error-correction by setting up the framework known as information theory. Error-correction codes are based on mapping the original space of words onto a higher dimensional space in such a way that the typical distance between encoded words increases.

Magnetization enumerator for LDPC codes - a statistical physics approach

We propose a method based on the magnetization enumerator to determine the critical noise level for Gallager type low density parity check error correcting codes (LDPC). Our method provides an appealingly simple interpretation to the relation between different decoding schemes, and provides more optimistic critical noise levels than those reported in the information theory literature.

Noisy random Boolean formulae:a statistical physics perspective

Properties of computing Boolean circuits composed of noisy logical gates are studied using the statistical physics methodology. A formula-growth model that gives rise to random Boolean functions is mapped onto a spin system, which facilitates the study of their typical behavior in the presence of noise. Bounds on their performance, derived in the information theory literature for specific gates, are straightforwardly retrieved, generalized and identified as the corresponding macroscopic phase transitions. The framework is employed for deriving results on error-rates at various function-depths and function sensitivity, and their dependence on the gate-type and noise model used. These are difficult to obtain via the traditional methods used in this field.

Statistical physics of error-correcting codes

In this thesis we use statistical physics techniques to study the typical performance of four families of error-correcting codes based on very sparse linear transformations: Sourlas codes, Gallager codes, MacKay-Neal codes and Kanter-Saad codes. We map the decoding problem onto an Ising spin system with many-spins interactions. We then employ the replica method to calculate averages over the quenched disorder represented by the code constructions, the arbitrary messages and the random noise vectors. We find, as the noise level increases, a phase transition between successful decoding and failure phases. This phase transition coincides with upper bounds derived in the information theory literature in most of the cases. We connect the practical decoding algorithm known as probability propagation with the task of finding local minima of the related Bethe free-energy. We show that the practical decoding thresholds correspond to noise levels where suboptimal minima of the free-energy emerge. Simulations of practical decoding scenarios using probability propagation agree with theoretical predictions of the replica symmetric theory. The typical performance predicted by the thermodynamic phase transitions is shown to be attainable in computation times that grow exponentially with the system size. We use the insights obtained to design a method to calculate the performance and optimise parameters of the high performance codes proposed by Kanter and Saad.

Network optimisation - A statistical physics perspective

Inference and optimisation of real-value edge variables in sparse graphs are studied using the tree based Bethe approximation optimisation algorithms. Equilibrium states of general energy functions involving a large set of real edge-variables that interact at the network nodes are obtained for networks in various cases. These include different cost functions, connectivity values, constraints on the edge bandwidth and the case of multiclass optimisation.

Networking - a statistical physics perspective

Networking encompasses a variety of tasks related to the communication of information on networks; it has a substantial economic and societal impact on a broad range of areas including transportation systems, wired and wireless communications and a range of Internet applications. As transportation and communication networks become increasingly more complex, the ever increasing demand for congestion control, higher traffic capacity, quality of service, robustness and reduced energy consumption requires new tools and methods to meet these conflicting requirements. The new methodology should serve for gaining better understanding of the properties of networking systems at the macroscopic level, as well as for the development of new principled optimization and management algorithms at the microscopic level. Methods of statistical physics seem best placed to provide new approaches as they have been developed specifically to deal with nonlinear large-scale systems. This review aims at presenting an overview of tools and methods that have been developed within the statistical physics community and that can be readily applied to address the emerging problems in networking. These include diffusion processes, methods from disordered systems and polymer physics, probabilistic inference, which have direct relevance to network routing, file and frequency distribution, the exploration of network structures and vulnerability, and various other practical networking applications. © 2013 IOP Publishing Ltd.

The influence of external forces on a monopolistic market:a statistical physics perspective

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT

A statistical physics perspective on alignment-independent protein sequence comparison.

Motivation: Within bioinformatics, the textual alignment of amino acid sequences has long dominated the determination of similarity between proteins, with all that implies for shared structure, function, and evolutionary descent. Despite the relative success of modern-day sequence alignment algorithms, so-called alignment-free approaches offer a complementary means of determining and expressing similarity, with potential benefits in certain key applications, such as regression analysis of protein structure-function studies, where alignment-base similarity has performed poorly. Results: Here, we offer a fresh, statistical physics-based perspective focusing on the question of alignment-free comparison, in the process adapting results from “first passage probability distribution” to summarize statistics of ensemble averaged amino acid propensity values. In this paper, we introduce and elaborate this approach.

Approach to chaos in ultracold atomic and molecular physics: Statistics of near-threshold bound states for Li+CaH and Li+CaF

We calculate near-threshold bound states and Feshbach resonance positions for atom–rigid-rotor models of the highly anisotropic systems Li+CaH and Li+CaF. We perform statistical analysis on the resonance positions to compare with the predictions of random matrix theory. For Li+CaH with total angular momentum J=0 we find fully chaotic behavior in both the nearest-neighbor spacing distribution and the level number variance. However, for J>0 we find different behavior due to the presence of a nearly conserved quantum number. Li+CaF (J=0) also shows apparently reduced levels of chaotic behavior despite its stronger effective coupling. This may indicate the development of another good quantum number relating to a bending motion of the complex. However, continuously varying the rotational constant over a wide range shows unexpected structure in the degree of chaotic behavior, including a dramatic reduction around the rotational constant of CaF. This demonstrates the complexity of the relationship between coupling and chaotic behavior.

Thermodynamics and Heat Transport of doped Spin-Ice Materials

In this thesis the low-temperature magnetism of the spin-ice systems Dy2Ti2O7 and Ho2Ti2O7 is investigated. In general, a clear experimental evidence for a sizable magnetic contribution kappa_{mag} to the low-temperature, zero-field heat transport of both spin-ice materials is observed. This kappa_{mag} can be attributed to the magnetic monopole excitations, which are highly mobile in zero field and are suppressed by a rather small external field resulting in a drop of kappa(H). Towards higher magnetic fields, significant field dependencies of the phononic heat conductivities kappa_{ph}(H) of Ho2Ti2O7 and Dy2Ti2O7 are found, which are, however, of opposite signs, as it is also found for the highly dilute reference materials (Ho0.5Y0.5)2Ti2O7 and (Dy0.5Y0.5)2Ti2O7. The dominant effect in the Ho-based materials is the scattering of phonons by spin flips which appears to be significantly stronger than in the Dy-based materials. Here, the thermal conductivity is suppressed due to enhanced lattice distortions observed in the magnetostriction. Furthermore, the thermal conductivity of Dy2Ti2O7 has been investigated concerning strong hysteresis effects and slow-relaxation processes towards equilibrium states in the low-temperature and low-field regime. The thermal conductivity in the hysteretic regions slowly relaxes towards larger values suggesting that there is an additional suppression of the heat transport by disorder in the non-equilibrium states. The equilibration can even be governed by the heat current for particular configurations. A special focus was put on the dilution series Dy2Ti2O7x. From specific heat measurements, it was found that the ultra-slow thermal equilibration in pure spin ice Dy2Ti2O7 is rapidly suppressed upon dilution with non-magnetic yttrium and vanishes completely for x>=0.2 down to the lowest accessible temperatures. In general, the low-temperature entropy of (Dy1-xYx)2Ti2O7, considerably decreases with increasing x, whereas its temperature-dependence drastically increases. Thus, it could be clarified that there is no experimental evidence for a finite zero-temperature entropy in (Dy1-xYx)2Ti2O7 above x>=0.2, in clear contrast to the finite residual entropy S_{P}(x) expected from a generalized Pauling approximation. A similar discrepancy is also present between S_{P}(x) and the low-temperature entropy obtained by Monte Carlo simulations, which reproduce the experimental data from 25 K down to 0.7 K, whereas the data at 0.4 K are overestimated. A straightforward description of the field-dependence kappa(H) of the dilution series with qualitative models justifies the extraction of kappa_{mag}. It was observed that kappa_{mag} systematically scales with the degree of dilution and its low-field decrease is related to the monopole excitation energy. The diffusion coefficient D_{mag} for the monopole excitations was calculated by means of c_{mag} and kappa_{mag}. It exhibits a broad maximum around 1.6 K and is suppressed for T<=0.5 K, indicating a non-degenerate ground state in the long-time limit, and in the high-temperature range for T>=4 K where spin-ice physics is eliminated. A mean-free path of 0.3 mum is obtained for Dy2Ti2O7 at about 1 K within the kinetic gas theory.

Political theory and theoretical physics

This paper explains, somewhat along a Simmelian line, that political theory may produce practical and universal theories like those developed in theoretical physics. The reasoning behind this paper is to show that the Element of Democracy Theory may be true by way of comparing it to Einstein’s Special Relativity – specifically concerning the parameters of symmetry, unification, simplicity, and utility. These parameters are what make a theory in physics as meeting them not only fits with current knowledge, but also produces paths towards testing (application). As the Element of Democracy Theory meets these same parameters, it could settle the debate concerning the definition of democracy. This will be shown firstly by discussing why no one has yet achieved a universal definition of democracy; secondly by explaining the parameters chosen (as in why these and not others confirm or scuttle theories); and thirdly by comparing how Special Relativity and the Element of Democracy match the parameters.

Political theory and theoretical physics [Revised]

This paper explains, somewhat along a Simmelian line, that political theory may produce practical and universal theories like those developed in theoretical physics. The reasoning behind this paper is to show that the Element of Democracy Theory may be true by way of comparing it to Einstein’s Special Relativity – specifically concerning the parameters of symmetry, unification, simplicity, and utility. These parameters are what make a theory in physics as meeting them not only fits with current knowledge, but also produces paths towards testing (application). As the Element of Democracy Theory meets these same parameters, it could settle the debate concerning the definition of democracy. This will be shown firstly by discussing why no one has yet achieved a universal definition of democracy; secondly by explaining the parameters chosen (as in why these and not others confirm or scuttle theories); and thirdly by comparing how Special Relativity and the Element of Democracy match the parameters.

GIS-based mapping and statistical analysis of air pollution and mortality in Brisbane, Australia

In this thesis, the relationship between air pollution and human health has been investigated utilising Geographic Information System (GIS) as an analysis tool. The research focused on how vehicular air pollution affects human health. The main objective of this study was to analyse the spatial variability of pollutants, taking Brisbane City in Australia as a case study, by the identification of the areas of high concentration of air pollutants and their relationship with the numbers of death caused by air pollutants. A correlation test was performed to establish the relationship between air pollution, number of deaths from respiratory disease, and total distance travelled by road vehicles in Brisbane. GIS was utilized to investigate the spatial distribution of the air pollutants. The main finding of this research is the comparison between spatial and non-spatial analysis approaches, which indicated that correlation analysis and simple buffer analysis of GIS using the average levels of air pollutants from a single monitoring station or by group of few monitoring stations is a relatively simple method for assessing the health effects of air pollution. There was a significant positive correlation between variable under consideration, and the research shows a decreasing trend of concentration of nitrogen dioxide at the Eagle Farm and Springwood sites and an increasing trend at CBD site. Statistical analysis shows that there exists a positive relationship between the level of emission and number of deaths, though the impact is not uniform as certain sections of the population are more vulnerable to exposure. Further statistical tests found that the elderly people of over 75 years age and children between 0-15 years of age are the more vulnerable people exposed to air pollution. A non-spatial approach alone may be insufficient for an appropriate evaluation of the impact of air pollutant variables and their inter-relationships. It is important to evaluate the spatial features of air pollutants before modeling the air pollution-health relationships.