Statistical and Information-Theoretic Methods for Data-Analysis

In this Thesis, we develop theory and methods for computational data analysis. The problems in data analysis are approached from three perspectives: statistical learning theory, the Bayesian framework, and the information-theoretic minimum description length (MDL) principle. Contributions in statistical learning theory address the possibility of generalization to unseen cases, and regression analysis with partially observed data with an application to mobile device positioning. In the second part of the Thesis, we discuss so called Bayesian network classifiers, and show that they are closely related to logistic regression models. In the final part, we apply the MDL principle to tracing the history of old manuscripts, and to noise reduction in digital signals.

Learning Nonlinear Visual Processing from Natural Images

The paradigm of computational vision hypothesizes that any visual function -- such as the recognition of your grandparent -- can be replicated by computational processing of the visual input. What are these computations that the brain performs? What should or could they be? Working on the latter question, this dissertation takes the statistical approach, where the suitable computations are attempted to be learned from the natural visual data itself. In particular, we empirically study the computational processing that emerges from the statistical properties of the visual world and the constraints and objectives specified for the learning process. This thesis consists of an introduction and 7 peer-reviewed publications, where the purpose of the introduction is to illustrate the area of study to a reader who is not familiar with computational vision research. In the scope of the introduction, we will briefly overview the primary challenges to visual processing, as well as recall some of the current opinions on visual processing in the early visual systems of animals. Next, we describe the methodology we have used in our research, and discuss the presented results. We have included some additional remarks, speculations and conclusions to this discussion that were not featured in the original publications. We present the following results in the publications of this thesis. First, we empirically demonstrate that luminance and contrast are strongly dependent in natural images, contradicting previous theories suggesting that luminance and contrast were processed separately in natural systems due to their independence in the visual data. Second, we show that simple cell -like receptive fields of the primary visual cortex can be learned in the nonlinear contrast domain by maximization of independence. Further, we provide first-time reports of the emergence of conjunctive (corner-detecting) and subtractive (opponent orientation) processing due to nonlinear projection pursuit with simple objective functions related to sparseness and response energy optimization. Then, we show that attempting to extract independent components of nonlinear histogram statistics of a biologically plausible representation leads to projection directions that appear to differentiate between visual contexts. Such processing might be applicable for priming, \ie the selection and tuning of later visual processing. We continue by showing that a different kind of thresholded low-frequency priming can be learned and used to make object detection faster with little loss in accuracy. Finally, we show that in a computational object detection setting, nonlinearly gain-controlled visual features of medium complexity can be acquired sequentially as images are encountered and discarded. We present two online algorithms to perform this feature selection, and propose the idea that for artificial systems, some processing mechanisms could be selectable from the environment without optimizing the mechanisms themselves. In summary, this thesis explores learning visual processing on several levels. The learning can be understood as interplay of input data, model structures, learning objectives, and estimation algorithms. The presented work adds to the growing body of evidence showing that statistical methods can be used to acquire intuitively meaningful visual processing mechanisms. The work also presents some predictions and ideas regarding biological visual processing.

Language learning in infancy

Although immensely complex, speech is also a very efficient means of communication between humans. Understanding how we acquire the skills necessary for perceiving and producing speech remains an intriguing goal for research. However, while learning is likely to begin as soon as we start hearing speech, the tools for studying the language acquisition strategies in the earliest stages of development remain scarce. One prospective strategy is statistical learning. In order to investigate its role in language development, we designed a new research method. The method was tested in adults using magnetoencephalography (MEG) as a measure of cortical activity. Neonatal brain activity was measured with electroencephalography (EEG). Additionally, we developed a method for assessing the integration of seen and heard syllables in the developing brain as well as a method for assessing the role of visual speech when learning phoneme categories. The MEG study showed that adults learn statistical properties of speech during passive listening of syllables. The amplitude of the N400m component of the event-related magnetic fields (ERFs) reflected the location of syllables within pseudowords. The amplitude was also enhanced for syllables in a statistically unexpected position. The results suggest a role for the N400m component in statistical learning studies in adults. Using the same research design with sleeping newborn infants, the auditory event-related potentials (ERPs) measured with EEG reflected the location of syllables within pseudowords. The results were successfully replicated in another group of infants. The results show that even newborn infants have a powerful mechanism for automatic extraction of statistical characteristics from speech. We also found that 5-month-old infants integrate some auditory and visual syllables into a fused percept, whereas other syllable combinations are not fully integrated. Auditory syllables were paired with visual syllables possessing a different phonetic identity, and the ERPs for these artificial syllable combinations were compared with the ERPs for normal syllables. For congruent auditory-visual syllable combinations, the ERPs did not differ from those for normal syllables. However, for incongruent auditory-visual syllable combinations, we observed a mismatch response in the ERPs. The results show an early ability to perceive speech cross-modally. Finally, we exposed two groups of 6-month-old infants to artificially created auditory syllables located between two stereotypical English syllables in the formant space. The auditory syllables followed, equally for both groups, a unimodal statistical distribution, suggestive of a single phoneme category. The visual syllables combined with the auditory syllables, however, were different for the two groups, one group receiving visual stimuli suggestive of two separate phoneme categories, the other receiving visual stimuli suggestive of only one phoneme category. After a short exposure, we observed different learning outcomes for the two groups of infants. The results thus show that visual speech can influence learning of phoneme categories. Altogether, the results demonstrate that complex language learning skills exist from birth. They also suggest a role for the visual component of speech in the learning of phoneme categories.

Data fusion and matching by maximizing statistical dependencies

The core aim of machine learning is to make a computer program learn from the experience. Learning from data is usually defined as a task of learning regularities or patterns in data in order to extract useful information, or to learn the underlying concept. An important sub-field of machine learning is called multi-view learning where the task is to learn from multiple data sets or views describing the same underlying concept. A typical example of such scenario would be to study a biological concept using several biological measurements like gene expression, protein expression and metabolic profiles, or to classify web pages based on their content and the contents of their hyperlinks. In this thesis, novel problem formulations and methods for multi-view learning are presented. The contributions include a linear data fusion approach during exploratory data analysis, a new measure to evaluate different kinds of representations for textual data, and an extension of multi-view learning for novel scenarios where the correspondence of samples in the different views or data sets is not known in advance. In order to infer the one-to-one correspondence of samples between two views, a novel concept of multi-view matching is proposed. The matching algorithm is completely data-driven and is demonstrated in several applications such as matching of metabolites between humans and mice, and matching of sentences between documents in two languages.

Moral Professionalism in Interaction : Educators' Relational Moral Voices in Urban Schools

The aim of this study was to investigate educators relational moral voices in urban schools and to listen to what they told about moral professionalism and moral practices in challenging urban schools. Their relational moral voices were investigated through the following three questions: 1. What are the educators moral voices in relation to themselves and other people? 2. What are the educators moral voices in relation to their work and society? 3. What kind of interaction process lies between the educators moral voices and the urban school context? The research data of this study were gathered in four urban schools in Jyväskylä and Helsinki. The research schools were chosen for this study according to the criteria of the international Socrates Comenius project called Leading Schools Successfully in Challenging Urban Context: Strategies for Improvement. This study formed part of this project, which investigated successful urban schools as challenging learning environments in nine European countries and explored the principals success in leadership in particular. The data, which included 37 narratively constructed interviews with four principals and key informants selected by the principals, were gathered in interviews conducted in 2006. In other words, the data comprised three interviews with each of four principals, and interviews with two teachers, two parents, and two pupils from each school. In addition, the school deacon from one school was also interviewed. Furthermore, part of the data from one of the research schools included a medium report of the school deacon s work. This study combined the case study method, the narrative approach and the critical incident technique as the methodological framework. In addition, all of these methods served as practical tools for both analyzing and reporting the data. The educators' narrations and the results of the study appear in the original articles (Hanhimäki & Tirri 2008; Hanhimäki 2008b; Hanhimäki & Tirri 2009; Hanhimäki 2008a). The educators moral voices in relation to themselves and other people emerged through the main themes of moral leadership, the development and evaluation process, moral sensitivity, gender, values, and student well-being. The educators moral voices in relation to their work and society emerged through the main themes of multiprofessional cooperation, families and parental involvement, and moral school culture. The idea of moral interaction connected moral professionalism and the methodological combination of this study, which together emphasized social interaction and the creation of understanding and meaning in this interaction. The main point of this study was to state that the educators moral voices emerged in the interaction between the educators themselves and the urban school context. In this interaction, the educators moral professionalism was constructed and shaped in relation to themselves, other people, their work and society. The loudest relational moral voices heard through the main themes were those of caring, cooperation, respect, commitment, and professionalism. When the results were compared to the codes of ethics which guided these educators moral professional work, the ethical principles and values of the codes were clearly visible in their moral practices. The loudest message from the educators narration could be summarized in the words caring, respect and cooperation: at its best, there is just a human being and a human being with caring, respect and cooperation between them. The results of this study emphasize the need for practical approaches such as case studies and the narrative approach in teacher education to encourage educators to become moral professionals capable of meeting the needs of people of varied backgrounds. In addition, opportunities for moral, religious and spiritual education should be noticed and utilized in the plural interaction of urban schools when nurturing pupils and creating a moral school culture. Furthermore, multiprofessional cooperation and parents as the school s primary cooperation partner are needed to carry out the shared duty of moral education in urban schools. Keywords: moral professionalism, educator, relational moral voice, interaction, urban school