The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

Sample complexity results from computational learning theory, when applied to neural network learning for pattern classification problems, suggest that for good generalization performance the number of training examples should grow at least linearly with the number of adjustable parameters in the network. Results in this paper show that if a large neural network is used for a pattern classification problem and the learning algorithm finds a network with small weights that has small squared error on the training patterns, then the generalization performance depends on the size of the weights rather than the number of weights. For example, consider a two-layer feedforward network of sigmoid units, in which the sum of the magnitudes of the weights associated with each unit is bounded by A and the input dimension is n. We show that the misclassification probability is no more than a certain error estimate (that is related to squared error on the training set) plus A3 √((log n)/m) (ignoring log A and log m factors), where m is the number of training patterns. This may explain the generalization performance of neural networks, particularly when the number of training examples is considerably smaller than the number of weights. It also supports heuristics (such as weight decay and early stopping) that attempt to keep the weights small during training. The proof techniques appear to be useful for the analysis of other pattern classifiers: when the input domain is a totally bounded metric space, we use the same approach to give upper bounds on misclassification probability for classifiers with decision boundaries that are far from the training examples.

Ceremony and cybrary : Digital libraries and the dialectic of place and space

This article examines social, cultural and technological change in the systems and economies of educational information management. Since the Sumerians first collected, organized and supervised administrative and religious records some six millennia ago, libraries have been key physical depositories and cultural signifiers in the production and mediation of social capital and power through education. To date, the textual, archival and discursive practices perpetuating libraries have remained exempt from inquiry. My aim here is to remedy this hiatus by making the library itself the terrain and object of critical analysis and investigation. The paper argues that in the three dominant communications eras—namely, oral, print and digital cultures—society’s centres of knowledge and learning have resided in the ceremony, the library and the cybrary respectively. In a broad-brush historical grid, each of these key educational institutions—the ceremony in oral culture, the library in print culture and the cybrary in digital culture—are mapped against social, cultural and technological orders pertaining to their era. Following a description of these shifts in society’s collective cultural memory, the paper then examines the question of what the development of global information systems and economies mean for schools and libraries of today, and for teachers and learners as knowledge consumers and producers?

Group maintenance scheduling : a case study for a pipeline network

This paper presents a group maintenance scheduling case study for a water distributed network. This water pipeline network presents the challenge of maintaining aging pipelines with the associated increases in annual maintenance costs. The case study focuses on developing an effective maintenance plan for the water utility. Current replacement planning is difficult as it needs to balance the replacement needs under limited budgets. A Maintenance Grouping Optimization (MGO) model based on a modified genetic algorithm was utilized to develop an optimum group maintenance schedule over a 20-year cycle. The adjacent geographical distribution of pipelines was used as a grouping criterion to control the searching space of the MGO model through a Judgment Matrix. Based on the optimum group maintenance schedule, the total cost was effectively reduced compared with the schedules without grouping maintenance jobs. This optimum result can be used as a guidance to optimize the current maintenance plan for the water utility.

A review of natural convection and heat transfer in attic-shaped space

This paper presents a review of studies on natural convection heat transfer in the triangular enclosure namely, in attic-shaped space. Much research activity has been devoted to this topic over the last three decades with a view to providing thermal comfort to the occupants in attic-shaped buildings and to minimising the energy costs associated with heating and air-conditioning. Two basic thermal boundary conditions of attic are considered to represent hot and cold climates or day and night time. This paper also reports on a significant number of studies which have been performed recently on other topics related to the attic space, for example, attics subject to localized heating and attics filled with porous media.

Evaluation of texture and geometry for dimensional facial expression recognition

Facial expression recognition (FER) algorithms mainly focus on classification into a small discrete set of emotions or representation of emotions using facial action units (AUs). Dimensional representation of emotions as continuous values in an arousal-valence space is relatively less investigated. It is not fully known whether fusion of geometric and texture features will result in better dimensional representation of spontaneous emotions. Moreover, the performance of many previously proposed approaches to dimensional representation has not been evaluated thoroughly on publicly available databases. To address these limitations, this paper presents an evaluation framework for dimensional representation of spontaneous facial expressions using texture and geometric features. SIFT, Gabor and LBP features are extracted around facial fiducial points and fused with FAP distance features. The CFS algorithm is adopted for discriminative texture feature selection. Experimental results evaluated on the publicly accessible NVIE database demonstrate that fusion of texture and geometry does not lead to a much better performance than using texture alone, but does result in a significant performance improvement over geometry alone. LBP features perform the best when fused with geometric features. Distributions of arousal and valence for different emotions obtained via the feature extraction process are compared with those obtained from subjective ground truth values assigned by viewers. Predicted valence is found to have a more similar distribution to ground truth than arousal in terms of covariance or Bhattacharya distance, but it shows a greater distance between the means.

Natural convection adjacent to an inclined flat plate and in an attic space under various thermal forcing conditions

The effect of viscous dissipation on natural convection from a vertical plate placed in a thermally stratified environment has been investigated numerically. The reduced equations are integrated by employing the implicit finite difference scheme or Ke1ler-box method and obtained the effect of heat due to viscous dissipation on the local skin-friction and loca1 Nusselt number at various stratification levels, for fluids having Prandtl number equals 10, 50, and 100. Solutions are also obtained using the perturbation technique for small values of viscous dissipation parameters and compared with the Finite Difference solutions. Effect of the heat transfer due to viscous dissipation and the temperature stratification are also shown on the velocity and temperature distributions in the boundary layer region. A numerical study of laminar doubly diffusive free convection flows adjacent to a vertical surface in a stable thermally stratified medium is also considered for this study. Solutions are obtained using the implicit Finite Difference method and compared with the local non-similarity method. The velocity and temperature distributions for different values of stratification parameter are shown graphically. The results show many interesting aspects of complex interaction of the two buoyant mechanisms.

Auditory spatial acuity approximates the resolving power of space-specific neurons

The relationship between neuronal acuity and behavioral performance was assessed in the barn owl (Tyto alba), a nocturnal raptor renowned for its ability to localize sounds and for the topographic representation of auditory space found in the midbrain. We measured discrimination of sound-source separation using a newly developed procedure involving the habituation and recovery of the pupillary dilation response. The smallest discriminable change of source location was found to be about two times finer in azimuth than in elevation. Recordings from neurons in its midbrain space map revealed that their spatial tuning, like the spatial discrimination behavior, was also better in azimuth than in elevation by a factor of about two. Because the PDR behavioral assay is mediated by the same circuitry whether discrimination is assessed in azimuth or in elevation, this difference in vertical and horizontal acuity is likely to reflect a true difference in sensory resolution, without additional confounding effects of differences in motor performance in the two dimensions. Our results, therefore, are consistent with the hypothesis that the acuity of the midbrain space map determines auditory spatial discrimination.