Learning Nonlinear Visual Processing from Natural Images

The paradigm of computational vision hypothesizes that any visual function -- such as the recognition of your grandparent -- can be replicated by computational processing of the visual input. What are these computations that the brain performs? What should or could they be? Working on the latter question, this dissertation takes the statistical approach, where the suitable computations are attempted to be learned from the natural visual data itself. In particular, we empirically study the computational processing that emerges from the statistical properties of the visual world and the constraints and objectives specified for the learning process. This thesis consists of an introduction and 7 peer-reviewed publications, where the purpose of the introduction is to illustrate the area of study to a reader who is not familiar with computational vision research. In the scope of the introduction, we will briefly overview the primary challenges to visual processing, as well as recall some of the current opinions on visual processing in the early visual systems of animals. Next, we describe the methodology we have used in our research, and discuss the presented results. We have included some additional remarks, speculations and conclusions to this discussion that were not featured in the original publications. We present the following results in the publications of this thesis. First, we empirically demonstrate that luminance and contrast are strongly dependent in natural images, contradicting previous theories suggesting that luminance and contrast were processed separately in natural systems due to their independence in the visual data. Second, we show that simple cell -like receptive fields of the primary visual cortex can be learned in the nonlinear contrast domain by maximization of independence. Further, we provide first-time reports of the emergence of conjunctive (corner-detecting) and subtractive (opponent orientation) processing due to nonlinear projection pursuit with simple objective functions related to sparseness and response energy optimization. Then, we show that attempting to extract independent components of nonlinear histogram statistics of a biologically plausible representation leads to projection directions that appear to differentiate between visual contexts. Such processing might be applicable for priming, \ie the selection and tuning of later visual processing. We continue by showing that a different kind of thresholded low-frequency priming can be learned and used to make object detection faster with little loss in accuracy. Finally, we show that in a computational object detection setting, nonlinearly gain-controlled visual features of medium complexity can be acquired sequentially as images are encountered and discarded. We present two online algorithms to perform this feature selection, and propose the idea that for artificial systems, some processing mechanisms could be selectable from the environment without optimizing the mechanisms themselves. In summary, this thesis explores learning visual processing on several levels. The learning can be understood as interplay of input data, model structures, learning objectives, and estimation algorithms. The presented work adds to the growing body of evidence showing that statistical methods can be used to acquire intuitively meaningful visual processing mechanisms. The work also presents some predictions and ideas regarding biological visual processing.

A Floor Control Server in a Distributed Conference Service

The conferencing systems in IP Multimedia (IM) networks are going through restructuring, accomplished in the near future. One of the changes introduced is the concept of floors and floor control in its current form with matching entity roles. The Binary Floor Control Protocol (BFCP) is a novelty to be exploited in distributed tightly coupled conferencing services. The protocol defines the floor control server (FCS), which implements floor control giving access to shared resources. As the newest tendency is to distribute the conferencing services, the locations of different functionality units play an important role in developing the standards. The floor control server location is not yet single-mindedly fixed in different standardization bodies, and the debate goes on where to place it within the media server, providing the conferencing service. The thesis main objective is to evaluate two distinctive alternatives in respect the Mp interface protocol between the respective nodes, as the interface in relation to floor control is under standardization work at the moment. The thesis gives a straightforward preamble in IMS network, nodes of interest including floor control server and conferencing. Knowledge on several protocols – BFCP, SDP, SIP and H.248 provides an important background for understanding the functionality changes introduced in the Mp interface and therefore introductions on those protocols and how they are connected to the full picture is given. The actual analysis on the impact of the floor control server into the Mp reference point is concluded in relation to the locations, giving basic flows, requirements analysis including a limited implementation proposal on supporting protocol parameters. The overall conclusion of the thesis is that even if both choices are seemingly useful, not one of the locations is clearly the most suitable in the light of this work. The thesis suggests a solution having both possibilities available to be chosen from in separate circumstances, realized with consistent standardization. It is evident, that if the preliminary assumption for the analysis is kept regarding to only one right place for the floor control server, more work is to be done in connected areas to discover the one most appropriate location.

Mechanisms for alphavirus nonstructural polyprotein processing

Replication and transcription of the RNA genome of alphaviruses relies on a set of virus-encoded nonstructural proteins. They are synthesized as a long polyprotein precursor, P1234, which is cleaved at three processing sites to yield nonstructural proteins nsP1, nsP2, nsP3 and nsP4. All the four proteins function as constitutive components of the membrane-associated viral replicase. Proteolytic processing of P1234 polyprotein is precisely orchestrated and coordinates the replicase assembly and maturation. The specificity of the replicase is also controlled by proteolytic cleavages. The early replicase is composed of P123 polyprotein intermediate and nsP4. It copies the positive sense RNA genome to complementary minus-strand. Production of new plus-strands requires complete processing of the replicase. The papain-like protease residing in nsP2 is responsible for all three cleavages in P1234. This study addressed the mechanisms of proteolytic processing of the replicase polyprotein in two alphaviruses Semliki Forest virus (SFV) and Sindbis virus (SIN) representing different branches of the genus. The survey highlighted the functional relation of the alphavirus nsP2 protease to the papain-like enzymes. A new structural motif the Cys-His catalytic dyad accompanied with an aromatic residue following the catalytic His was described for nsP2 and a subset of other thiol proteases. Such an architecture of the catalytic center was named the glycine specificity motif since it was implicated in recognition of a specific Gly residue in the substrate. In particular, the presence of the motif in nsP2 makes the appearance of this amino acid at the second position upstream of the scissile bond a necessary condition for the cleavage. On top of that, there were four distinct mechanisms identified, which provide affinity for the protease and specifically direct the enzyme to different sites in the P1234 polyprotein. Three factors RNA, the central domain of nsP3 and the N-terminus of nsP2 were demonstrated to be external modulators of the nsP2 protease. Here I suggest that the basal nsP2 protease specificity is inherited from the ancestral papain-like enzyme and employs the recognition of the upstream amino acid signature in the immediate vicinity of the scissile bond. This mechanism is responsible for the efficient processing of the SFV nsP3/nsP4 junction. I propose that the same mechanism is involved in the cleavage of the nsP1/nsP2 junction of both viruses as well. However, in this case it rather serves to position the substrate, whereas the efficiency of the processing is ensured by the capability of nsP2 to cut its own N-terminus in cis. Both types of cleavages are demonstrated here to be inhibited by RNA, which is interpreted as impairing the basal papain-like recognition of the substrate. In contrast, processing of the SIN nsP3/nsP4 junction was found to be activated by RNA and additionally potentiated by the presence of the central region of nsP3 in the protease. The processing of the nsP2/nsP3 junction in both viruses occurred via another mechanism, requiring the exactly processed N-terminus of nsP2 in the protease and insensitive to RNA addition. Therefore, the three processing events in the replicase polyprotein maturation are performed via three distinct mechanisms in each of two studied alphaviruses. Distinct sets of conditions required for each cleavage ensure sequential maturation of P1234 polyprotein: nsP4 is released first, then the nsP1/nsP2 site is cut in cis, and liberation of the nsP2 N-terminus activates the cleavage of the nsP2/nsP3 junction at last. The first processing event occurs differently in SFV and SIN, whereas the subsequent cleavages are found to be similar in the two viruses and therefore, their mechanisms are suggested to be conserved in the genus. The RNA modulation of the alphavirus nonstructural protease activity, discovered here, implies bidirectional functional interplay between the alphavirus RNA metabolism and protease regulation. The nsP2 protease emerges as a signal transmitting moiety, which senses the replication stage and responds with proteolytic cleavages. A detailed hypothetical model of the alphavirus replicase core was inferred from the data obtained in the study. Similar principles in replicase organization and protease functioning are expected to be employed by other RNA viruses.

Tactile processing in human somatosensory and auditory cortices

Tactile sensation plays an important role in everyday life. While the somatosensory system has been studied extensively, the majority of information has come from studies using animal models. Recent development of high-resolution anatomical and functional imaging techniques has enabled the non-invasive study of human somatosensory cortex and thalamus. This thesis provides new insights into the functional organization of the human brain areas involved in tactile processing using magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI). The thesis also demonstrates certain optimizations of MEG and fMRI methods. Tactile digit stimulation elicited stimulus-specific responses in a number of brain areas. Contralateral activation was observed in somatosensory thalamus (Study II), primary somatosensory cortex (SI; I, III, IV), and post-auditory belt area (III). Bilateral activation was observed in secondary somatosensory cortex (SII; II, III, IV). Ipsilateral activation was found in the post-central gyrus (area 2 of SI cortex; IV). In addition, phasic deactivation was observed within ipsilateral SI cortex and bilateral primary motor cortex (IV). Detailed investigation of the tactile responses demonstrated that the arrangement of distal-proximal finger representations in area 3b of SI in humans is similar to that found in monkeys (I). An optimized MEG approach was sufficient to resolve such fine detail in functional organization. The SII region appeared to contain double representations for fingers and toes (II). The detection of activations in the SII region and thalamus improved at the individual and group levels when cardiac-gated fMRI was used (II). Better detection of body part representations at the individual level is an important improvement, because identification of individual representations is crucial for studying brain plasticity in somatosensory areas. The posterior auditory belt area demonstrated responses to both auditory and tactile stimuli (III), implicating this area as a physiological substrate for the auditory-tactile interaction observed in earlier psychophysical studies. Comparison of different smoothing parameters (III) demonstrated that proper evaluation of co-activation should be based on individual subject analysis with minimal or no smoothing. Tactile input consistently influenced area 3b of the human ipsilateral SI cortex (IV). The observed phasic negative fMRI response is proposed to result from interhemispheric inhibition via trans-callosal connections. This thesis contributes to a growing body of human data suggesting that processing of tactile stimuli involves multiple brain areas, with different spatial patterns of cortical activation for different stimuli.

Brain imaging of chronic pain

Acute pain has substantial survival value because of its protective function in the everyday environment. Instead, chronic pain lacks survival and adaptive function, causes great amount of individual suffering, and consumes the resources of the society due to the treatment costs and loss of production. The treatment of chronic pain has remained challenging because of inadequate understanding of mechanisms working at different levels of the nervous system in the development, modulation, and maintenance of chronic pain. Especially in unclear chronic pain conditions the treatment may be suboptimal because it can not be targeted to the underlying mechanisms. Noninvasive neuroimaging techniques have greatly contributed to our understanding of brain activity associated with pain in healthy individuals. Many previous studies, focusing on brain activations to acute experimental pain in healthy individuals, have consistently demonstrated a widely-distributed network of brain regions that participate in the processing of acute pain. The aim of the present thesis was to employ non-invasive brain imaging to better understand the brain mechanisms in patients suffering from chronic pain. In Study I, we used magnetoencephalography (MEG) to measure cortical responses to painful laser stimulation in healthy individuals for optimization of the stimulus parameters for patient studies. In Studies II and III, we monitored with MEG the cortical processing of touch and acute pain in patients with complex regional pain syndrome (CRPS). We found persisting plastic changes in the hand representation area of the primary somatosensory (SI) cortex, suggesting that chronic pain causes cortical reorganization. Responses in the posterior parietal cortex to both tactile and painful laser stimulation were attenuated, which could be associated with neglect-like symptoms of the patients. The primary motor cortex reactivity to acute pain was reduced in patients who had stronger spontaneous pain and weaker grip strength in the painful hand. The tight coupling between spontaneous pain and motor dysfunction supports the idea that motor rehabilitation is important in CRPS. In Studies IV and V we used MEG and functional magnetic resonance imaging (fMRI) to investigate the central processing of touch and acute pain in patients who suffered from recurrent herpes simplex virus infections and from chronic widespread pain in one side of the body. With MEG, we found plastic changes in the SI cortex, suggesting that many different types of chronic pain may be associated with similar cortical reorganization. With fMRI, we found functional and morphological changes in the central pain circuitry, as an indication of central contribution for the pain. These results show that chronic pain is associated with morphological and functional changes in the brain, and that such changes can be measured with functional imaging.

Collecting data from distributed FOSS projects

A key trait of Free and Open Source Software (FOSS) development is its distributed nature. Nevertheless, two project-level operations, the fork and the merge of program code, are among the least well understood events in the lifespan of a FOSS project. Some projects have explicitly adopted these operations as the primary means of concurrent development. In this study, we examine the effect of highly distributed software development, is found in the Linux kernel project, on collection and modelling of software development data. We find that distributed development calls for sophisticated temporal modelling techniques where several versions of the source code tree can exist at once. Attention must be turned towards the methods of quality assurance and peer review that projects employ to manage these parallel source trees. Our analysis indicates that two new metrics, fork rate and merge rate, could be useful for determining the role of distributed version control systems in FOSS projects. The study presents a preliminary data set consisting of version control and mailing list data.

Collecting data from distributed FOSS projects

A key trait of Free and Open Source Software (FOSS) development is its distributed nature. Nevertheless, two project-level operations, the fork and the merge of program code, are among the least well understood events in the lifespan of a FOSS project. Some projects have explicitly adopted these operations as the primary means of concurrent development. In this study, we examine the effect of highly distributed software development, is found in the Linux kernel project, on collection and modelling of software development data. We find that distributed development calls for sophisticated temporal modelling techniques where several versions of the source code tree can exist at once. Attention must be turned towards the methods of quality assurance and peer review that projects employ to manage these parallel source trees. Our analysis indicates that two new metrics, fork rate and merge rate, could be useful for determining the role of distributed version control systems in FOSS projects. The study presents a preliminary data set consisting of version control and mailing list data.

Music in the recovering brain

Listening to music involves a widely distributed bilateral network of brain regions that controls many auditory perceptual, cognitive, emotional, and motor functions. Exposure to music can also temporarily improve mood, reduce stress, and enhance cognitive performance as well as promote neural plasticity. However, very little is currently known about the relationship between music perception and auditory and cognitive processes or about the potential therapeutic effects of listening to music after neural damage. This thesis explores the interplay of auditory, cognitive, and emotional factors related to music processing after a middle cerebral artery (MCA) stroke. In the acute recovery phase, 60 MCA stroke patients were randomly assigned to a music listening group, an audio book listening group, or a control group. All patients underwent neuropsychological assessments, magnetoencephalography (MEG) measurements, and magnetic resonance imaging (MRI) scans repeatedly during a six-month post-stroke period. The results revealed that amusia, a deficit of music perception, is a common and persistent deficit after a stroke, especially if the stroke affects the frontal and temporal brain areas in the right hemisphere. Amusia is clearly associated with deficits in both auditory encoding, as indicated by the magnetic mismatch negativity (MMNm) response, and domain-general cognitive processes, such as attention, working memory, and executive functions. Furthermore, both music and audio book listening increased the MMNm, whereas only music listening improved the recovery of verbal memory and focused attention as well as prevented a depressed and confused mood during the first post-stroke months. These findings indicate a close link between musical, auditory, and cognitive processes in the brain. Importantly, they also encourage the use of listening to music as a rehabilitative leisure activity after a stroke and suggest that the auditory environment can induce long-term plastic changes in the recovering brain.