Margin-adaptive model selection in statistical learning

A classical condition for fast learning rates is the margin condition, first introduced by Mammen and Tsybakov. We tackle in this paper the problem of adaptivity to this condition in the context of model selection, in a general learning framework. Actually, we consider a weaker version of this condition that allows one to take into account that learning within a small model can be much easier than within a large one. Requiring this “strong margin adaptivity” makes the model selection problem more challenging. We first prove, in a general framework, that some penalization procedures (including local Rademacher complexities) exhibit this adaptivity when the models are nested. Contrary to previous results, this holds with penalties that only depend on the data. Our second main result is that strong margin adaptivity is not always possible when the models are not nested: for every model selection procedure (even a randomized one), there is a problem for which it does not demonstrate strong margin adaptivity.

Reconceptualizing statistical learning in the early years

This chapter argues for the need to restructure children’s statistical experiences from the beginning years of formal schooling. The ability to understand and apply statistical reasoning is paramount across all walks of life, as seen in the variety of graphs, tables, diagrams, and other data representations requiring interpretation. Young children are immersed in our data-driven society, with early access to computer technology and daily exposure to the mass media. With the rate of data proliferation have come increased calls for advancing children’s statistical reasoning abilities, commencing with the earliest years of schooling (e.g., Langrall et al. 2008; Lehrer and Schauble 2005; Shaughnessy 2010; Whitin and Whitin 2011). Several articles (e.g., Franklin and Garfield 2006; Langrall et al. 2008) and policy documents (e.g., National Council of Teachers ofMathematics 2006) have highlighted the need for a renewed focus on this component of early mathematics learning, with children working mathematically and scientifically in dealing with realworld data. One approach to this component in the beginning school years is through data modelling (English 2010; Lehrer and Romberg 1996; Lehrer and Schauble 2000, 2007)...

A statistical framework for natural feature representation

This paper presents a robust stochastic framework for the incorporation of visual observations into conventional estimation, data fusion, navigation and control algorithms. The representation combines Isomap, a non-linear dimensionality reduction algorithm, with expectation maximization, a statistical learning scheme. The joint probability distribution of this representation is computed offline based on existing training data. The training phase of the algorithm results in a nonlinear and non-Gaussian likelihood model of natural features conditioned on the underlying visual states. This generative model can be used online to instantiate likelihoods corresponding to observed visual features in real-time. The instantiated likelihoods are expressed as a Gaussian mixture model and are conveniently integrated within existing non-linear filtering algorithms. Example applications based on real visual data from heterogenous, unstructured environments demonstrate the versatility of the generative models.

Leveraging Web 2.0 data for scalable semi-supervised learning of domain-specific sentiment lexicons

Since manually constructing domain-specific sentiment lexicons is extremely time consuming and it may not even be feasible for domains where linguistic expertise is not available. Research on the automatic construction of domain-specific sentiment lexicons has become a hot topic in recent years. The main contribution of this paper is the illustration of a novel semi-supervised learning method which exploits both term-to-term and document-to-term relations hidden in a corpus for the construction of domain specific sentiment lexicons. More specifically, the proposed two-pass pseudo labeling method combines shallow linguistic parsing and corpusbase statistical learning to make domain-specific sentiment extraction scalable with respect to the sheer volume of opinionated documents archived on the Internet these days. Another novelty of the proposed method is that it can utilize the readily available user-contributed labels of opinionated documents (e.g., the user ratings of product reviews) to bootstrap the performance of sentiment lexicon construction. Our experiments show that the proposed method can generate high quality domain-specific sentiment lexicons as directly assessed by human experts. Moreover, the system generated domain-specific sentiment lexicons can improve polarity prediction tasks at the document level by 2:18% when compared to other well-known baseline methods. Our research opens the door to the development of practical and scalable methods for domain-specific sentiment analysis.

Learning domain-specific sentiment lexicons for predicting product sales

Generic sentiment lexicons have been widely used for sentiment analysis these days. However, manually constructing sentiment lexicons is very time-consuming and it may not be feasible for certain application domains where annotation expertise is not available. One contribution of this paper is the development of a statistical learning based computational method for the automatic construction of domain-specific sentiment lexicons to enhance cross-domain sentiment analysis. Our initial experiments show that the proposed methodology can automatically generate domain-specific sentiment lexicons which contribute to improve the effectiveness of opinion retrieval at the document level. Another contribution of our work is that we show the feasibility of applying the sentiment metric derived based on the automatically constructed sentiment lexicons to predict product sales of certain product categories. Our research contributes to the development of more effective sentiment analysis system to extract business intelligence from numerous opinionated expressions posted to the Web

Young children's statistical reasoning : a tale of two contexts

This thesis explored the knowledge and reasoning of young children in solving novel statistical problems, and the influence of problem context and design on their solutions. It found that young children's statistical competencies are underestimated, and that problem design and context facilitated children's application of a wide range of knowledge and reasoning skills, none of which had been taught. A qualitative design-based research method, informed by the Models and Modeling perspective (Lesh & Doerr, 2003) underpinned the study. Data modelling activities incorporating picture story books were used to contextualise the problems. Children applied real-world understanding to problem solving, including attribute identification, categorisation and classification skills. Intuitive and metarepresentational knowledge together with inductive and probabilistic reasoning was used to make sense of data, and beginning awareness of statistical variation and informal inference was visible.

A stochastic model for natural feature representation

This paper presents a robust stochastic model for the incorporation of natural features within data fusion algorithms. The representation combines Isomap, a non-linear manifold learning algorithm, with Expectation Maximization, a statistical learning scheme. The representation is computed offline and results in a non-linear, non-Gaussian likelihood model relating visual observations such as color and texture to the underlying visual states. The likelihood model can be used online to instantiate likelihoods corresponding to observed visual features in real-time. The likelihoods are expressed as a Gaussian Mixture Model so as to permit convenient integration within existing nonlinear filtering algorithms. The resulting compactness of the representation is especially suitable to decentralized sensor networks. Real visual data consisting of natural imagery acquired from an Unmanned Aerial Vehicle is used to demonstrate the versatility of the feature representation.

The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

Sample complexity results from computational learning theory, when applied to neural network learning for pattern classification problems, suggest that for good generalization performance the number of training examples should grow at least linearly with the number of adjustable parameters in the network. Results in this paper show that if a large neural network is used for a pattern classification problem and the learning algorithm finds a network with small weights that has small squared error on the training patterns, then the generalization performance depends on the size of the weights rather than the number of weights. For example, consider a two-layer feedforward network of sigmoid units, in which the sum of the magnitudes of the weights associated with each unit is bounded by A and the input dimension is n. We show that the misclassification probability is no more than a certain error estimate (that is related to squared error on the training set) plus A3 √((log n)/m) (ignoring log A and log m factors), where m is the number of training patterns. This may explain the generalization performance of neural networks, particularly when the number of training examples is considerably smaller than the number of weights. It also supports heuristics (such as weight decay and early stopping) that attempt to keep the weights small during training. The proof techniques appear to be useful for the analysis of other pattern classifiers: when the input domain is a totally bounded metric space, we use the same approach to give upper bounds on misclassification probability for classifiers with decision boundaries that are far from the training examples.

Surviving an avalanche of data

Open the sports or business section of your daily newspaper, and you are immediately bombarded with an array of graphs, tables, diagrams, and statistical reports that require interpretation. Across all walks of life, the need to understand statistics is fundamental. Given that our youngsters’ future world will be increasingly data laden, scaffolding their statistical understanding and reasoning is imperative, from the early grades on. The National Council of Teachers of Mathematics (NCTM) continues to emphasize the importance of early statistical learning; data analysis and probability was the Council’s professional development “Focus of the Year” for 2007–2008. We need such a focus, especially given the results of the statistics items from the 2003 NAEP. As Shaughnessy (2007) noted, students’ performance was weak on more complex items involving interpretation or application of items of information in graphs and tables. Furthermore, little or no gains were made between the 2000 NAEP and the 2003 NAEP studies. One approach I have taken to promote young children’s statistical reasoning is through data modeling. Having implemented in grades 3 –9 a number of model-eliciting activities involving working with data (e.g., English 2010), I observed how competently children could create their own mathematical ideas and representations—before being instructed how to do so. I thus wished to introduce data-modeling activities to younger children, confi dent that they would likewise generate their own mathematics. I recently implemented data-modeling activities in a cohort of three first-grade classrooms of six year- olds. I report on some of the children’s responses and discuss the components of data modeling the children engaged in.

On the relations and differences between Popper dimension, exclusion dimension and VC-Dimension

A high-level relationPopper dimension—( Exclusion dimension—( VC dimension—( between Karl Popper’s ideas on “falsifiability of scientific theories” and the notion of “overfitting”Overfitting in statistical learning theory can be easily traced. However, it was pointed out that at the level of technical details the two concepts are significantly different. One possible explanation that we suggest is that the process of falsification is an active process, whereas statistical learning theory is mainly concerned with supervised learningSupervised learning, which is a passive process of learning from examples arriving from a stationary distribution. We show that concepts that are closer (although still distant) to Karl Popper’s definitions of falsifiability can be found in the domain of learning using membership queries, and derive relations between Popper’s dimension, exclusion dimension, and the VC-dimensionVC dimension.

A comparison of methods for classifying clinical samples based on proteomics data : a case study for statistical and machine learning approaches

The discovery of protein variation is an important strategy in disease diagnosis within the biological sciences. The current benchmark for elucidating information from multiple biological variables is the so called “omics” disciplines of the biological sciences. Such variability is uncovered by implementation of multivariable data mining techniques which come under two primary categories, machine learning strategies and statistical based approaches. Typically proteomic studies can produce hundreds or thousands of variables, p, per observation, n, depending on the analytical platform or method employed to generate the data. Many classification methods are limited by an n≪p constraint, and as such, require pre-treatment to reduce the dimensionality prior to classification. Recently machine learning techniques have gained popularity in the field for their ability to successfully classify unknown samples. One limitation of such methods is the lack of a functional model allowing meaningful interpretation of results in terms of the features used for classification. This is a problem that might be solved using a statistical model-based approach where not only is the importance of the individual protein explicit, they are combined into a readily interpretable classification rule without relying on a black box approach. Here we incorporate statistical dimension reduction techniques Partial Least Squares (PLS) and Principal Components Analysis (PCA) followed by both statistical and machine learning classification methods, and compared them to a popular machine learning technique, Support Vector Machines (SVM). Both PLS and SVM demonstrate strong utility for proteomic classification problems.

Components of learning and assessment in linear algebra

Linear algebra provides theory and technology that are the cornerstones of a range of cutting edge mathematical applications, from designing computer games to complex industrial problems, as well as more traditional applications in statistics and mathematical modelling. Once past introductions to matrices and vectors, the challenges of balancing theory, applications and computational work across mathematical and statistical topics and problems are considerable, particularly given the diversity of abilities and interests in typical cohorts. This paper considers two such cohorts in a second level linear algebra course in different years. The course objectives and materials were almost the same, but some changes were made in the assessment package. In addition to considering effects of these changes, the links with achievement in first year courses are analysed, together with achievement in a following computational mathematics course. Some results that may initially appear surprising provide insight into the components of student learning in linear algebra.

Beyond statistical procedures for predictive modelling: Data mining algorithms and support for university research at QUT

In a seminal data mining article, Leo Breiman [1] argued that to develop effective predictive classification and regression models, we need to move away from the sole dependency on statistical algorithms and embrace a wider toolkit of modeling algorithms that include data mining procedures. Nevertheless, many researchers still rely solely on statistical procedures when undertaking data modeling tasks; the sole reliance on these procedures has lead to the development of irrelevant theory and questionable research conclusions ([1], p.199). We will outline initiatives that the HPC & Research Support group is undertaking to engage researchers with data mining tools and techniques; including a new range of seminars, workshops, and one-on-one consultations covering data mining algorithms, the relationship between data mining and the research cycle, and limitations and problems with these new algorithms. Organisational limitations and restrictions to these initiatives are also discussed.

Innovation in clinical learning for the acute hospital environment : nursing grand rounds

The literature reports that workload factors affect nurses' ability to fully engage in continuing professional development. Hence the work environment in acute care calls for innovative approaches to achieve continuous development of nursing practice and work satisfaction. This study employs a one group pre-test post-test design to test the effectiveness of nursing grand rounds on nursing worklife satisfaction and work environment in an acute surgical ward. The effect of nursing grand rounds was measured using the Nursing Worklife Satisfaction Scale and the Practice Environment Scale. There was no change between pre- and post-test on these measures but trends were evident in some component scores. Statistical results were inconclusive but observational data indicated that nursing grand rounds was found to be feasible, well attended with tested processes for implementation in an acute care environment.

Wellness education : an integrated theoretical framework for fostering transformative learning

Optimum Wellness involves the development, refinement and practice of lifestyle choices which resonate with personally meaningful frames of reference. Personal transformations are the means by which our frames of reference are refined across the lifespan. It is through critical reflection, supportive relationships and meaning making of our experiences that we construct and reconstruct our life paths. When individuals are able to be what they are destined to be or reach their higher purpose, then they are able to contribute to the world in positive and meaningful ways. Transformative education facilitates the changes in perspective that enable one to contemplate and travel a path in life that leads to self-actualisation. This thesis argues for an integrated theoretical framework for optimum Wellness Education. It establishes a learner centred approach to Wellness education in the form of an integrated instructional design framework derived from both Wellness and Transformative education constructs. Students’ approaches to learning and their study strategies in a Wellness education context serve to highlight convergences in the manner in which students can experience perspective transformation. As they learn to critically reflect, pursue relationships and adapt their frames of reference to sustain their pursuit of both learning and Wellness goals, strengthening the nexus between instrumental and transformative learning is a strategically important goal for educators. The aim of this exploratory research study was to examine those facets that serve to optimise the learning experiences of students in a Wellness course. This was accomplished through three research issues: 1) What are the relationships between Wellness, approaches to learning and academic success? 2) How are students approaching learning in an undergraduate Wellness subject? Why are students approaching their learning in the ways they do? 3) What sorts of transformations are students experiencing in their Wellness? How can transformative education be formulated in the context of an undergraduate Wellness subject? Subsequent to a thorough review of the literature pertaining to Wellness education, a mixed method embedded case study design was formulated to explore the research issues. This thesis examines the interrelationships between student, content and context in a one semester university undergraduate unit (a coherent set of learning activities which is assigned a unit code and a credit point value). The experiences of a cohort of 285 undergraduate students in a Wellness course formed the unit of study and seven individual students from a total of sixteen volunteers whose profiles could be constructed from complete data sets were selected for analysis as embedded cases. The introductory level course required participants to engage in a personal project involving a behaviour modification plan for a self-selected, single dimension of Wellness. Students were given access to the Standard Edition Testwell Survey to assess and report their Wellness as a part of their personal projects. To identify relationships among the constructs of Self-Regulated Learning (SRL), Wellness and Student Approaches to Learning (SAL) a blend of quantitative and qualitative methods to collect and analyse data was formulated. Surveys were the primary instruments for acquiring quantitative data. Sources included the Wellness data from Testwell surveys, SAL data from R-SPQ surveys, SRL data from MSLQ surveys and student self-evaluation data from an end of semester survey. Students’ final grades and GPA scores were used as indicators of academic performance. The sources of qualitative data included subject documentation, structured interview transcripts and open-ended responses to survey items. Subsequent to a pilot study in which survey reliability and validity were tested in context, amendments to processes for and instruments of data collection were made. Students who adopted meaning oriented (deep/achieving) approaches tended to assess their Wellness at a higher level, seek effective learning strategies and perform better in formal study. Posttest data in the main study revealed that there were significant positive statistical relationships between academic performance and total wellness scores (rs=.297, n=205, p<.01). Deep (rs=.343, n=137, p<.01) and achieving (rs=.286, n=123, p<.01) approaches to learning also significantly correlated with Wellness whilst surface approaches had negative correlations that were not significant. SRL strategies including metacognitive selfregulation, effort, help-seeking and critical thinking were increasingly correlated with Wellness. Qualitative findings suggest that while all students adopt similar patterns of day to day activities for example attending classes, taking notes, working on assignments the level of care with which these activities is undertaken varies considerably. The dominant motivational trigger for students in this cohort was the personal relevance and associated benefits of the material being learned and practiced. Students were inclined to set goals that had a positive impact on affect and used “sense of happiness” to evaluate their achievement status. Students who had a higher drive to succeed and/or understand tended to have or seek a wider range of strategies. Their goal orientations were generally learning rather than performance based and barriers presented a challenge which could be overcome as opposed to a blockage which prevented progress. Findings from an empirical analysis of the Testwell data suggest that a single third order Wellness construct exists. A revision of the instrument is necessary in order to juxtapose it with the chosen six dimensional Wellness model that forms the foundation construct in the course. Further, redevelopment should be sensitive to the Australian context and culture including choice of language, examples and scenarios used in item construction. This study concludes with an heuristic for use in Wellness education. Guided by principles of Transformative education theory and behaviour change theory, and informed by this representative case study the “CARING” heuristic is proposed as an instructional design tool for Wellness educators seeking to foster transformative learning. Based upon this study, recommendations were made for university educators to provide authentic and personal experiences in Wellness curricula. Emphasis must focus on involving students and teachers in a partnership for implementing Wellness programs both in the curriculum and co-curricularly. The implications of this research for practice are predicated on the willingness of academics to embrace transformative learning at a personal level and a professional one. To explore students’ profiles in detail is not practical however teaching students how to guide us in supporting them through the “pain” of learning is a skill which would benefit them and optimise the learning and teaching process. At a theoretical level, this research contributes to an ecological theory of Wellness education as transformational change. By signposting the wider contexts in which learning takes place, it seeks to encourage changing paradigms to ones which harness the energy of each successive contextual layer in which students live. Future research which amplifies the qualities of individuals and groups who are “Well” and seeks the refinement and development of instruments to measure Wellness constructs would be desirable for both theoretical and applied knowledge bases. Mixed method Wellness research derived and conducted by teams that incorporate expertise from multiple disciplines such as psychology, anthropology, education, and medicine would enable creative and multi-perspective programs of investigation to be designed and implemented. Congruences and inconsistencies in health promotion and education would provide valuable material for strengthening the nexus between transformational learning and behaviour change theories. Future development of and research on the effectiveness of the CARING heuristic would be valuable in advancing the understanding of pedagogies which advance rather than impede learning as a transformative process. Exploring pedagogical models that marry with transformative education may render solutions to the vexing challenge of teaching and learning in diverse contexts.