The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

Sample complexity results from computational learning theory, when applied to neural network learning for pattern classification problems, suggest that for good generalization performance the number of training examples should grow at least linearly with the number of adjustable parameters in the network. Results in this paper show that if a large neural network is used for a pattern classification problem and the learning algorithm finds a network with small weights that has small squared error on the training patterns, then the generalization performance depends on the size of the weights rather than the number of weights. For example, consider a two-layer feedforward network of sigmoid units, in which the sum of the magnitudes of the weights associated with each unit is bounded by A and the input dimension is n. We show that the misclassification probability is no more than a certain error estimate (that is related to squared error on the training set) plus A3 √((log n)/m) (ignoring log A and log m factors), where m is the number of training patterns. This may explain the generalization performance of neural networks, particularly when the number of training examples is considerably smaller than the number of weights. It also supports heuristics (such as weight decay and early stopping) that attempt to keep the weights small during training. The proof techniques appear to be useful for the analysis of other pattern classifiers: when the input domain is a totally bounded metric space, we use the same approach to give upper bounds on misclassification probability for classifiers with decision boundaries that are far from the training examples.

Multitask learning with expert advice

We consider the problem of prediction with expert advice in the setting where a forecaster is presented with several online prediction tasks. Instead of competing against the best expert separately on each task, we assume the tasks are related, and thus we expect that a few experts will perform well on the entire set of tasks. That is, our forecaster would like, on each task, to compete against the best expert chosen from a small set of experts. While we describe the “ideal” algorithm and its performance bound, we show that the computation required for this algorithm is as hard as computation of a matrix permanent. We present an efficient algorithm based on mixing priors, and prove a bound that is nearly as good for the sequential task presentation case. We also consider a harder case where the task may change arbitrarily from round to round, and we develop an efficient approximate randomized algorithm based on Markov chain Monte Carlo techniques.

Co-constructing self-efficacy narratives : a study of four mature age university students

This thesis reports on a study in which research participants, four mature aged females starting an undergraduate degree at a regional Australian university, collaborated with the researcher in co-constructing a self-efficacy narrative. For the purpose of the study, self-efficacy was conceptualized as a means by which an individual initiates action to engage in a task or set of tasks, applies effort to perform the task or set of tasks, and persists in the face of obstacles encountered in order to achieve successful completion of the task or set of tasks. Qualitative interviews were conducted with the participants, initially investigating their respective life histories for an understanding of how they made the decision to embark on their respective academic program. Additional data were generated from a written exercise, prompting participants to furnish specific examples of self-efficacy. These data were incorporated into the individual's self-efficacy narrative, produced as the outcome of the "narrative analysis". Another aspect of the study entailed "analysis of narrative" in which analytic procedures were used to identify themes common to the self-efficacy narratives. Five main themes were identified: (a) participants' experience of schooling . for several participants their formative experience of school was not always positive, and yet their narratives demonstrated their agency in persevering and taking on university-level studies as mature aged persons; (b) recognition of family as an early influence . these influences were described as being both positive, in the sense of being supportive and encouraging, as well as posing obstacles that participants had to overcome in order to pursue their goals; (c) availability of supportive persons – the support of particular persons was acknowledged as a factor that enabled participants to persist in their respective endeavours; (d) luck or chance factors were recognised as placing participants at the right place at the right time, from which circumstances they applied considerable effort in order to convert the opportunity into a successful outcome; and (e) self-efficacy was identified as a major theme found in the narratives. The study included an evaluation of the research process by participants. A number of themes were identified in respect of the manner in which the research process was experienced as a helpful process. Participants commented that: (a) the research process was helpful in clarifying their respective career goals; (b) they appreciated opportunities provided by the research process to view their life from a different perspective and to better understand what motivated them, and what their preferred learning styles were; (c) their past successes in a range of different spheres were made more evident to them as they were guided in self-reflection, and their self-efficacious behaviour was affirmed; and (d) the opportunities provided by their participation in the research process to identify strengths of which they had not been consciously aware, to find confirmation of strengths they knew they possessed, and in some instances to rectify misconceptions they had held about aspects of their personality. The study made three important contributions to knowledge. Firstly, it provided a detailed explication of a qualitative narrative method in exploring self-efficacy, with the potential for application to other issues in educational, counselling and psychotherapy research. Secondly, it consolidated and illustrated social cognitive theory by proposing a dynamic model of self-efficacy, drawing on constructivist and interpretivist paradigms and extending extant theory and models. Finally, the study made a contribution to the debate concerning the nexus of qualitative research and counselling by providing guidelines for ethical practice in both endeavours for the practitioner-researcher.

On the data consumption benefits of accepting increased uncertainty

In the context of learning paradigms of identification in the limit, we address the question: why is uncertainty sometimes desirable? We use mind change bounds on the output hypotheses as a measure of uncertainty and interpret ‘desirable’ as reduction in data memorization, also defined in terms of mind change bounds. The resulting model is closely related to iterative learning with bounded mind change complexity, but the dual use of mind change bounds — for hypotheses and for data — is a key distinctive feature of our approach. We show that situations exist where the more mind changes the learner is willing to accept, the less the amount of data it needs to remember in order to converge to the correct hypothesis. We also investigate relationships between our model and learning from good examples, set-driven, monotonic and strong-monotonic learners, as well as class-comprising versus class-preserving learnability.

Assessment for learning - Why the theory needs the practice

Assessment for Learning is a pedagogical practice with anticipated gains of increased student motivation, mastery and autonomy as learners develop their capacity to monitor and plan their own learning progress. Assessment for Learning (AfL) differs from Assessment of learning in its timing, occurring within the regular flow of learning rather than end point, in its purpose of improving student learning rather than summative grading and in the ownership of the learning where the student voice is heard in judging quality. Since Black and Wiliam (1998) highlighted the achievement gains that AfL practices seem to bring to all learners in classrooms, it has become part of current educational policy discourse in Australia, yet teacher adoption of the practices is not a straightforward implementation of techniques within an existing classroom repertoire. As can be seen from the following meta-analysis, recent research highlights a more complex interrelationship between teacher and student beliefs about learning and assessment, and the social and cultural interactions in and contexts of the classroom. More research is needed from a sociocultural perspective that allows meaning to emerge from practice. Before another policy push, we need to understand better the many factors within the assessment relationship. We need to hear from teachers and students through long-term AfL case studies both to inform AfL theory and to shed light on the complexities of pedagogical change for enhancing learner autonomy.

Predicting residue-wise contact orders in proteins by support vector regression

Background The residue-wise contact order (RWCO) describes the sequence separations between the residues of interest and its contacting residues in a protein sequence. It is a new kind of one-dimensional protein structure that represents the extent of long-range contacts and is considered as a generalization of contact order. Together with secondary structure, accessible surface area, the B factor, and contact number, RWCO provides comprehensive and indispensable important information to reconstructing the protein three-dimensional structure from a set of one-dimensional structural properties. Accurately predicting RWCO values could have many important applications in protein three-dimensional structure prediction and protein folding rate prediction, and give deep insights into protein sequence-structure relationships. Results We developed a novel approach to predict residue-wise contact order values in proteins based on support vector regression (SVR), starting from primary amino acid sequences. We explored seven different sequence encoding schemes to examine their effects on the prediction performance, including local sequence in the form of PSI-BLAST profiles, local sequence plus amino acid composition, local sequence plus molecular weight, local sequence plus secondary structure predicted by PSIPRED, local sequence plus molecular weight and amino acid composition, local sequence plus molecular weight and predicted secondary structure, and local sequence plus molecular weight, amino acid composition and predicted secondary structure. When using local sequences with multiple sequence alignments in the form of PSI-BLAST profiles, we could predict the RWCO distribution with a Pearson correlation coefficient (CC) between the predicted and observed RWCO values of 0.55, and root mean square error (RMSE) of 0.82, based on a well-defined dataset with 680 protein sequences. Moreover, by incorporating global features such as molecular weight and amino acid composition we could further improve the prediction performance with the CC to 0.57 and an RMSE of 0.79. In addition, combining the predicted secondary structure by PSIPRED was found to significantly improve the prediction performance and could yield the best prediction accuracy with a CC of 0.60 and RMSE of 0.78, which provided at least comparable performance compared with the other existing methods. Conclusion The SVR method shows a prediction performance competitive with or at least comparable to the previously developed linear regression-based methods for predicting RWCO values. In contrast to support vector classification (SVC), SVR is very good at estimating the raw value profiles of the samples. The successful application of the SVR approach in this study reinforces the fact that support vector regression is a powerful tool in extracting the protein sequence-structure relationship and in estimating the protein structural profiles from amino acid sequences.

Crisis communication under transformative change : the emergent context and roles of primary and secondary organisations

In an era of complex challenges that draw sustained media attention and entangle multiple organisational actors, this thesis addresses the gap between current trends in society and business, and existing scholarship in public relations and crisis communication. By responding to calls from crisis communication researchers to develop theory (Coombs, 2006a), to examine the interdependencies of crises (Seeger, Sellnow, & Ulmer, 1998), and to consider variation in crisis response (Seeger, 2002), this thesis contributes to theory development in crisis communication and public relations. Through transformative change, this thesis extends existing scholarship built on a preservation or conservation logic where public relations is used to maintain stability by incrementally responding to changes in an organisation‘s environment (Cutlip, Center, & Broom, 2006; Everett, 2001; Grunig, 2000; Spicer, 1997). Based on the opportunity to contribute to ongoing theoretical development in the literature, the overall research problem guiding this thesis asks: How does transformative change during crisis influence corporate actors’ communication? This thesis adopts punctuated equilibrium theory, which describes change as alternating between long periods of stability and short periods of revolutionary or transformative change (Gersick, 1991; Romanelli & Tushman, 1994; Siggelkow, 2002; Tushman, Newman, & Romanelli, 1986; Tushman & Romanelli, 1985). As a theory for change, punctuated equilibrium provides an opportunity to examine public relations and transformative change, building on scholarship that is based primarily on incremental change. Further, existing scholarship in public relations and crisis communication focuses on the actions of single organisations in situational or short-term crisis events. Punctuated equilibrium theory enables the study of multiple crises and multiple organisational responses during transformative change. In doing so, punctuated equilibrium theory provides a framework to explain both the context for transformative change and actions or strategies enacted by organisations during transformative change (Tushman, Newman, & Romanelli, 1986; Tushman & Romanelli, 1985; Tushman, Virany, & Romanelli, 1986). The connections between context and action inform the research questions that guide this thesis: RQ1: What symbolic and substantive strategies persist and change as crises develop from situational events to transformative and multiple linked events? RQ2: What features of the crisis context influence changes in symbolic and substantive strategies? To shed light on these research questions, the thesis adopts a qualitative approach guided by process theory and methods to explicate the events, sequences and activities that were essential to change (Pettigrew, 1992; Van de Ven, 1992). Specifically, the thesis draws on an alternative template strategy (Langley, 1999) that provides several alternative interpretations of the same events (Allison, 1971; Allison & Zelikow, 1999). Following Allison (1971) and Allison and Zelikow (1999), this thesis uses three alternative templates of crisis or strategic response typologies to construct three narratives using media articles and organisational documents. The narratives are compared to identify and draw out different patterns of crisis communication strategies that operate within different crisis contexts. The thesis is based on the crisis events that affected three organisations within the pharmaceutical industry for four years. The primary organisation is Merck, as its product recall crisis triggered transformative change affecting, in different ways, the secondary organisations of Pfizer and Novartis. Three narratives are presented based on the crisis or strategic response typologies of Coombs (2006b), Allen and Caillouet (1994), and Oliver (1991). The findings of this thesis reveal different stories about crisis communication under transformative change. By zooming in to a micro perspective (Nicolini, 2009) to focus on the crisis communication and actions of a single organisation and zooming out to a macro perspective (Nicolini, 2009) to consider multiple organisations, new insights about crisis communication, change and the relationships among multiple organisations are revealed at context and action levels. At the context level, each subsequent narrative demonstrates greater connections among multiple corporate actors. By zooming out from Coombs‘ (2006b) focus on single organisations to consider Allen and Caillouet‘s (1994) integration of the web of corporate actors, the thesis demonstrates how corporate actors add accountability pressures to the primary organisation. Next, by zooming further out to the macro perspective by considering Oliver‘s (1991) strategic responses to institutional processes, the thesis reveals a greater range of corporate actors that are caught up in the process of transformative change and accounts for their varying levels of agency over their environment. By zooming in to a micro perspective and out to a macro perspective (Nicolini, 2009) across alternative templates, the thesis sheds light on sequences, events, and actions of primary and secondary organisations. Although the primary organisation remains the focus of sustained media attention across the four-year time frame, the secondary organisations, even when one faced a similar starting situation to the primary organisation, were buffered by the process of transformative change. This understanding of crisis contexts in transforming environments builds on existing knowledge in crisis communication. At the action level, the thesis also reveals different interpretations from each alternative template. Coombs‘ (2006b) narrative shows persistence in the primary organisation‘s crisis or strategic responses over the four-year time frame of the thesis. That is, the primary organisation consistently applies a diminish crisis response. At times, the primary organisation drew on denial responses when corporate actors questioned its legitimacy or actions. To close the crisis, the primary organisation uses a rebuild crisis posture (Coombs, 2006). These finding are replicated in Allen and Caillouet‘s (1994) narrative, noting this template‘s limitation to communication messages only. Oliver‘s (1991) narrative is consistent with Coombs‘ (2006b) but also demonstrated a shift from a strategic response that signals conformity to the environment to one that signals more active resistance to the environment over time. Specifically, the primary organisation‘s initial response demonstrates conformity but these same messages were used some three years later to set new expectations in the environment in order to shape criteria and build acceptance for future organisational decisions. In summary, the findings demonstrate the power of crisis or strategic responses when considered over time and in the context of transformative change. The conclusions of this research contribute to scholarship in the public relations and management literatures. Based on the significance of organisational theory, the primary contribution of the theory relates to the role of interorganisational linkages or legitimacy buffers that form during the punctuation of equilibrium. The network of linkages among the corporate actors are significant also to the crisis communication literature as they form part of the process model of crisis communication under punctuated equilibrium. This model extends existing research that focuses on crisis communication of single organisations to consider the emergent context that incorporates secondary organisations as well as the localised contests of legitimacy and buffers from regulatory authorities. The thesis also provides an empirical base for punctuated equilibrium in public relations and crisis communication, extending Murphy‘s (2000) introduction of the theory to the public relations literature. In doing this, punctuated equilibrium theory reinvigorates theoretical development in crisis communication by extending existing scholarship around incrementalist approaches and demonstrating how public relations works in the context of transformative change. Further research in this area could consider using alternative templates to study transformative change caused by a range of crisis types from natural disasters to product tampering, and to add further insight into the dynamics between primary and secondary organisations. This thesis contributes to practice by providing guidelines for crisis response strategy selection and indicators related to the emergent context for crises under transformative change that will help primary and secondary organisations‘ responses to crises.

Towards learner autonomy : an assessment for learning approach

Using Assessment for Learning (AfL) may develop learner autonomy however, very often AfL is reduced to a set of strategies that do not always achieve the desired outcome. This research adopted a different approach that examined AfL as a cultural practice, situated within influential social relationships that shape learner identity. The study addressed the question “What are the qualities of the teacher-student relationship that support student learning autonomy in an AfL context?” Three case studies of the interactions of Queensland middle school teachers and their classes of Year 7, 8 and 9 were developed over one year. Data were collected from field notes and video recordings of classroom interactions and individual and focus group interviews with teachers and students. The analysis began with a close look at the field data. Interpretations that emerged from a sociocultural theoretical understanding were helpful in informing the process of analysis. Themes and patterns of interrelationships were identified through thematic coding using a constant comparative approach. Validation was achieved through methodological triangulation. Four findings that inform an understanding of AfL and the development of learner autonomy emerged. Firstly, autonomy is theorised as a context-specific identity mediated through the teacher-student relationship. Secondly, it was observed that learners negotiated their identities as knowers through AfL practices in various tacit, explicit, group and individual ways in a ‘generative dance’ of knowing in action (Cook & Brown, 2005). Thirdly, teachers and learners negotiated their participation by drawing from identities in multiple communities of practice. Finally it is proposed that a new participative identity or narrative for assessment is needed. This study contributes to understandings about teacher AfL practices that can help build teacher assessment capacity. Importantly, autonomy is understood as an identity that is available to all learners. This study is also significant as it affirms the importance of teacher assessment to support learners in developing autonomy, a focus that challenges the singular assessment policy focus on measuring performance. Finally this study contributes to a sociocultural theoretical understanding of AfL.

A theory of waste behaviour in the construction industry

Levels of waste within the construction industry need to be reduced for environmental and economic reasons. Changing people's wasteful behaviour can make a significant contribution. This paper describes a research project that used Ajzen's 'theory of planned behaviour' to investigate the attitudinal forces that shape behaviour at the operative level. It concludes that operatives see waste as an inevitable by-product of construction activity. Attitudes towards waste management are not negative, although they are pragmatic and impeded by perceptions of a lack of managerial commitment. Waste management is perceived as a low project priority, and there is an absence of appropriate resources and incentives to support it. A theory of waste behaviour is proposed for the construction industry, and recommendations are made to help managers improve operatives' attitudes towards waste.