The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

Sample complexity results from computational learning theory, when applied to neural network learning for pattern classification problems, suggest that for good generalization performance the number of training examples should grow at least linearly with the number of adjustable parameters in the network. Results in this paper show that if a large neural network is used for a pattern classification problem and the learning algorithm finds a network with small weights that has small squared error on the training patterns, then the generalization performance depends on the size of the weights rather than the number of weights. For example, consider a two-layer feedforward network of sigmoid units, in which the sum of the magnitudes of the weights associated with each unit is bounded by A and the input dimension is n. We show that the misclassification probability is no more than a certain error estimate (that is related to squared error on the training set) plus A3 √((log n)/m) (ignoring log A and log m factors), where m is the number of training patterns. This may explain the generalization performance of neural networks, particularly when the number of training examples is considerably smaller than the number of weights. It also supports heuristics (such as weight decay and early stopping) that attempt to keep the weights small during training. The proof techniques appear to be useful for the analysis of other pattern classifiers: when the input domain is a totally bounded metric space, we use the same approach to give upper bounds on misclassification probability for classifiers with decision boundaries that are far from the training examples.

Overcoming the complexity of diagnostic problems due to sensor network architecture

In fault detection and diagnostics, limitations coming from the sensor network architecture are one of the main challenges in evaluating a system’s health status. Usually the design of the sensor network architecture is not solely based on diagnostic purposes, other factors like controls, financial constraints, and practical limitations are also involved. As a result, it quite common to have one sensor (or one set of sensors) monitoring the behaviour of two or more components. This can significantly extend the complexity of diagnostic problems. In this paper a systematic approach is presented to deal with such complexities. It is shown how the problem can be formulated as a Bayesian network based diagnostic mechanism with latent variables. The developed approach is also applied to the problem of fault diagnosis in HVAC systems, an application area with considerable modeling and measurement constraints.

The rademacher complexity of coregularized kernel classes

In the multi-view approach to semisupervised learning, we choose one predictor from each of multiple hypothesis classes, and we co-regularize our choices by penalizing disagreement among the predictors on the unlabeled data. We examine the co-regularization method used in the co-regularized least squares (CoRLS) algorithm, in which the views are reproducing kernel Hilbert spaces (RKHS's), and the disagreement penalty is the average squared difference in predictions. The final predictor is the pointwise average of the predictors from each view. We call the set of predictors that can result from this procedure the co-regularized hypothesis class. Our main result is a tight bound on the Rademacher complexity of the co-regularized hypothesis class in terms of the kernel matrices of each RKHS. We find that the co-regularization reduces the Rademacher complexity by an amount that depends on the distance between the two views, as measured by a data dependent metric. We then use standard techniques to bound the gap between training error and test error for the CoRLS algorithm. Experimentally, we find that the amount of reduction in complexity introduced by co regularization correlates with the amount of improvement that co-regularization gives in the CoRLS algorithm.

Narratives of complexity : an ethnographic exploration of resilience and wellbeing among single refugee women in Brisbane

This study aimed to explore resilience and wellbeing among a group of eight refugee women originating from several countries (mainly African) and living in Brisbane, most of whom were single mothers. To challenge mostly quantitative and gender-blind explorations of mental health concepts among refugee groups, the project sought an emic and contextual understanding of resilience and wellbeing. Established perspectives, while useful, tend to overlook the complexities of refugee mental health experiences and can neglect the dense nature of individual stories. The purpose of my study was to contest relatively simplistic narratives of mental health constructs that tend to dominate migrant and refugee studies and influence practice paradigms in the human services field. In this ethnographic exploration of mental health constructs conducted in 2008 and 2009, the use of in-depth interviews, participant observations, and visual ethnographic elements provided an opportunity for refugee women to tell their own stories. The participants’ unique narratives of pre- and post-migration experiences, shaped by specific gender, age, social, cultural and political aspects prevailing in their lives, yielded ‘thick’ ethnographic description (Geertz, 1973) of their social worlds. The findings explored in this study, namely language issues, the impact of community dynamics, and the single status of refugee women, clearly demonstrate that mental health constructs are fluid, multifaceted and complex in reality. In fact, language, community dynamics, and being a single mother, represented both opportunities and barriers in the lives of participants. In some contexts, these factors were conducive to resilience and wellbeing, while in other circumstances, these three elements acted as a hindrance to positive mental health outcomes. There are multiple dimensions to the findings, signifying that the social worlds of refugee women cannot be simplified using set definitions and neat notions of resilience and wellbeing. Instead, the intricacies and complexities embedded in the mundane of the everyday highlight novel conceptualisations of resilience and wellbeing. Based on the particular circumstances of single refugee mothers, whose experiences differ from that of married women, this thesis presents novel articulations of mental health constructs, as an alternative view to existing trends in the literature on refugee issues. Rich and multi-dimensional meanings associated with the socio-cultural determinants of mental health emerged in the process. This thesis’ findings highlight a significant gap in diasporic studies as well as simplistic assumptions about refugee women’s resettlement experiences. Single refugee women’s distinct issues are so complex and dense, that a contextual approach is critical to yield accurate depictions of their circumstances. It is therefore essential to understand refugee lived experiences within broader socio-political contexts to truly appreciate the depth of these narratives. In this manner, critical aspects salient to refugee journeys can inform different understandings of resilience, wellbeing and mental health, and shape contemporary policy and human service practice paradigms.

Complexity, evolution, and the structure of demand

This chapter argues that evolutionary economics should be founded upon complex systems theory rather than neo-Darwinian analogies concerning natural selection, which focus on supply side considerations and competition amongst firms and technologies. It suggests that conceptions such as production and consumption functions should be replaced by network representations, in which the preferences or, more correctly, the aspirations of consumers are fundamental and, as such, the primary drivers of economic growth. Technological innovation is viewed as a process that is intermediate between these aspirational networks, and the organizational networks in which goods and services are produced. Consumer knowledge becomes at least as important as producer knowledge in determining how economic value is generated. It becomes clear that the stability afforded by connective systems of rules is essential for economic flexibility to exist, but that too many rules result in inert and structurally unstable states. In contrast, too few rules result in a more stable state, but at a low level of ordered complexity. Economic evolution from this perspective is explored using random and scale free network representations of complex systems.

Measuring complexity for building projects : a Delphi study

Purpose---The aim of this study is to identify complexity measures for building projects in the People’s Republic of China (PRC). Design/Methodology/Approach---A three-round of Delphi questionnaire survey was conducted to identify the key parameters that measure the degree of project complexity. A complexity index (CI) was developed based on the identified measures and their relative importance. Findings---Six key measures of project complexity have been identified, which include, namely (1) building structure & function; (2) construction method; (3) the urgency of the project schedule; (4) project size/scale; (5) geological condition; and (6) neighboring environment. Practical implications---These complexity measures help stakeholders assess degrees of project complexity and better manage the potential risks that might be induced to different levels of project complexity. Originality/Value---The findings provide insightful perspectives to define and understand project complexity. For stakeholders, understanding and addressing the complexity help to improve project planning and implementation.

Risk, uncertainty and complexity in science education

Increasingly societies and their governments are facing important social issues that have science and technology as key features. A number of these socio-scientific issues have two features that distinguish them from the restricted contexts in which school science has traditionally been presented. Some of their science is uncertain and scientific knowledge is not the only knowledge involved. As a result, the concepts of uncertainty, risk and complexity become essential aspects of the science underlying these issues. In this chapter we discuss the nature and role of these concepts in the public understanding of science and consider their links with school science. We argue that these same concepts and their role in contemporary scientific knowledge need to be addressed in school science curricula. The new features for content, pedagogy and assessment of this urgent challenge for science educators are outlined. These will be essential if the goal of science education for citizenship is to be achieved with our students, who will increasingly be required to make personal and collective decisions on issues involving science and technology.

Measuring reef complexity and rugosity from monocular video bathymetric reconstruction

Topographic structural complexity of a reef is highly correlated to coral growth rates, coral cover and overall levels of biodiversity, and is therefore integral in determining ecological processes. Modeling these processes commonly includes measures of rugosity obtained from a wide range of different survey techniques that often fail to capture rugosity at different spatial scales. Here we show that accurate estimates of rugosity can be obtained from video footage captured using underwater video cameras (i.e., monocular video). To demonstrate the accuracy of our method, we compared the results to in situ measurements of a 2m x 20m area of forereef from Glovers Reef atoll in Belize. Sequential pairs of images were used to compute fine scale bathymetric reconstructions of the reef substrate from which precise measurements of rugosity and reef topographic structural complexity can be derived across multiple spatial scales. To achieve accurate bathymetric reconstructions from uncalibrated monocular video, the position of the camera for each image in the video sequence and the intrinsic parameters (e.g., focal length) must be computed simultaneously. We show that these parameters can be often determined when the data exhibits parallax-type motion, and that rugosity and reef complexity can be accurately computed from existing video sequences taken from any type of underwater camera from any reef habitat or location. This technique provides an infinite array of possibilities for future coral reef research by providing a cost-effective and automated method of determining structural complexity and rugosity in both new and historical video surveys of coral reefs.

Cross-cultural language learning and web design complexity

Accepting the fact that culture and language are interrelated in second language learning (SLL), the web sites should be designed to integrate with the cultural aspects. Yet many SLL web sites fail to integrate with the cultural aspects and/or focus on language acquisition only. This study identified three issues: (1) anthropologists’ cultural models mostly adopted in cross-cultural web user interface have been superficially used; (2) web designers deal with culture as a fixed one which needs to be modeled into interface design elements, so (3) there is a need for a communication framework between educators and design practitioners, which can be utilized in web design processes. This paper discusses what anthropology can contribute to language learning, mediated through web design processes and suggests a cultural user experience framework for web-based SLL by presenting an exemplary matrix. To evaluate the effectiveness of the framework, the key stakeholders (learners, teachers, and designers) participated in a case scenario-based evaluation. The result shows a high possibility that the framework can enhance the effective communication and collaboration for the cultural integration.

Engaging complexity through collaborative brand design

In this research we used inductive reasoning through design to understand how stakeholders in the Waterfall Way (New South Wales, Australia) perceive the relationships between themselves and the place they live in. This paper describes a collaborative design methodology used to release information about local identities, which guided the regional brand exercise. The methodology is explicit about the uncertainties and complexities of the design process and of its reception system. As such, it aims to engage with local stakeholders and experts in order to help elicit tacit knowledge and identify system patterns and trends that would possibly not be visible if a top-down expert-based process was used. Through collective design, local people were drawn together in search for a symbol to represent the meaning attached to their places/region in relation to sustainable tourism activity.