Dynamics of on-line learning in radial basis function networks

On-line learning is examined for the radial basis function network, an important and practical type of neural network. The evolution of generalization error is calculated within a framework which allows the phenomena of the learning process, such as the specialization of the hidden units, to be analyzed. The distinct stages of training are elucidated, and the role of the learning rate described. The three most important stages of training, the symmetric phase, the symmetry-breaking phase, and the convergence phase, are analyzed in detail; the convergence phase analysis allows derivation of maximal and optimal learning rates. As well as finding the evolution of the mean system parameters, the variances of these parameters are derived and shown to be typically small. Finally, the analytic results are strongly confirmed by simulations.

On-line learning with adaptive back-propagation in two-layer networks

An adaptive back-propagation algorithm parameterized by an inverse temperature 1/T is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, we analyse these learning algorithms in both the symmetric and the convergence phase for finite learning rates in the case of uncorrelated teachers of similar but arbitrary length T. These analyses show that adaptive back-propagation results generally in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent.

The attenuation surface for contrast sensitivity has the form of a witch’s hat within the central visual field

Over the full visual field, contrast sensitivity is fairly well described by a linear decline in log sensitivity as a function of eccentricity (expressed in grating cycles). However, many psychophysical studies of spatial visual function concentrate on the central ±4.5 deg (or so) of the visual field. As the details of the variation in sensitivity have not been well documented in this region we did so for small patches of target contrast at several spatial frequencies (0.7–4 c/deg), meridians (horizontal, vertical, and oblique), orientations (horizontal, vertical, and oblique), and eccentricities (0–18 cycles). To reduce the potential effects of stimulus uncertainty, circular markers surrounded the targets. Our analysis shows that the decline in binocular log sensitivity within the central visual field is bilinear: The initial decline is steep, whereas the later decline is shallow and much closer to the classical results. The bilinear decline was approximately symmetrical in the horizontal meridian and declined most steeply in the superior visual field. Further analyses showed our results to be scale-invariant and that this property could not be predicted from cone densities. We used the results from the cardinal meridians to radially interpolate an attenuation surface with the shape of a witch's hat that provided good predictions for the results from the oblique meridians. The witch's hat provides a convenient starting point from which to build models of contrast sensitivity, including those designed to investigate signal summation and neuronal convergence of the image contrast signal. Finally, we provide Matlab code for constructing the witch's hat.

Competition for shortest paths on sparse graphs

Optimal paths connecting randomly selected network nodes and fixed routers are studied analytically in the presence of a nonlinear overlap cost that penalizes congestion. Routing becomes more difficult as the number of selected nodes increases and exhibits ergodicity breaking in the case of multiple routers. The ground state of such systems reveals nonmonotonic complex behaviors in average path length and algorithmic convergence, depending on the network topology, and densities of communicating nodes and routers. A distributed linearly scalable routing algorithm is also devised. © 2012 American Physical Society.

Seizure detection in subdural recordings using nonlinear decision functions

This dissertation proposed a new approach to seizure detection in intracranial EEG recordings using nonlinear decision functions. It implemented well-established features that were designed to deal with complex signals such as brain recordings, and proposed a 2-D domain of analysis. Since the features considered assume both the time and frequency domains, the analysis was carried out both temporally and as a function of different frequency ranges in order to ascertain those measures that were most suitable for seizure detection. In retrospect, this study established a generalized approach to seizure detection that works across several features and across patients. ^ Clinical experiments involved 8 patients with intractable seizures that were evaluated for potential surgical interventions. A total of 35 iEEG data files collected were used in a training phase to ascertain the reliability of the formulated features. The remaining 69 iEEG data files were then used in the testing phase. ^ The testing phase revealed that the correlation sum is the feature that performed best across all patients with a sensitivity of 92% and an accuracy of 99%. The second best feature was the gamma power with a sensitivity of 92% and an accuracy of 96%. In the frequency domain, all of the 5 other spectral bands considered, revealed mixed results in terms of low sensitivity in some frequency bands and low accuracy in other frequency bands, which is expected given that the dominant frequencies in iEEG are those of the gamma band. In the time domain, other features which included mobility, complexity, and activity, all performed very well with an average a sensitivity of 80.3% and an accuracy of 95%. ^ The computational requirement needed for these nonlinear decision functions to be generated in the training phase was extremely long. It was determined that when the duration dimension was rescaled, the results improved and the convergence rates of the nonlinear decision functions were reduced dramatically by more than a 100 fold. Through this rescaling, the sensitivity of the correlation sum improved to 100% and the sensitivity of the gamma power to 97%, which meant that there were even less false negatives and false positives detected. ^

Turkish-American relations in the post-Cold War era, 1990-2005

This study examines the contours of Turkish-American foreign relations in the post-Cold War era from 1990 to 2005. While providing an interpretive analysis, the study highlights elements of continuity and change and of convergence and divergence in the relationship between Ankara and Washington. Turkey’s encounter with its Kurdish problem at home intertwined with the emergence of an autonomous Kurdish authority in northern Iraq after the Gulf War that left a political vacuum in the region. The main argument of this dissertation is that the Kurdish question has been the central element in shaping and redefining the nature and scope of Turkish-American relations since 1991. This study finds that systemic factors primarily prevail in the early years of the post-Cold War Turkish-American relations, as had been the case during the Cold War era. However, the Turkish parliament’s rejection of the deployment of the U.S. troops in Turkey for the invasion of Iraq in 2003 could not be explained by the primacy of distribution of capabilities in the system. Instead, the role of identity, ideology, norms, and the socialization of agency through interaction and language must be considered. The Justice and Development Party’s ascension to power in 2002 magnified a wider transformation in domestic and foreign politics and reflected changes in Turkey’s own self-perception and the definition of its core interests towards the United States.

Aplicação do exame da proporcionalidade diante da implantação de empreendimentos eólicos em áreas de preservação permanente

The setting up of wind power enterprises at Permanent Preservation Areas reflects the obvious conflict and necessary convergence between free market and energy security on the one hand, and the promotion of environmental quality on the other. From the perspective of energy sustainability, and in order to achieve development (in its complex meaning, which converges economic, social, environmental and cultural aspects), the harmonization between free market and an ecologically sustainable environment is required. This work aims to identify the link between the protection system of the Permanent Preservation Areas and the current constitutional order, by analyzing the implementation of wind power enterprises in these protected zones focusing on the proportionality aspects. A legal and purposeful research was developed, from a theoretical method, followed by collecting and analyzing both primary and secondary data. From these data, the law, the legal literature and judicial decisions were cross-examined, under the light of the Constitution and guided by the theory of proportionality and related development imperatives. In this context, the present study identified the link between the principles of the economic order, environment and energy law, finding their basis under the Federal Constitution and development. By reproducing this interrelationship and by means of post-crisis institutional reforms, the guiding objectives of the Brazilian electric sector began to corroborate the precepts of development, although issues regarding its sustainability still persist. The appraisal of proportionality indicates that the Permanent Preservation Areas protection system is insufficient to materialize the right to a healthy quality of life upon the implementation of wind projects at Permanent Preservation Areas, albeit seeking the harmonization between free market and environmental protection.

Aplicação do exame da proporcionalidade diante da implantação de empreendimentos eólicos em áreas de preservação permanente

The setting up of wind power enterprises at Permanent Preservation Areas reflects the obvious conflict and necessary convergence between free market and energy security on the one hand, and the promotion of environmental quality on the other. From the perspective of energy sustainability, and in order to achieve development (in its complex meaning, which converges economic, social, environmental and cultural aspects), the harmonization between free market and an ecologically sustainable environment is required. This work aims to identify the link between the protection system of the Permanent Preservation Areas and the current constitutional order, by analyzing the implementation of wind power enterprises in these protected zones focusing on the proportionality aspects. A legal and purposeful research was developed, from a theoretical method, followed by collecting and analyzing both primary and secondary data. From these data, the law, the legal literature and judicial decisions were cross-examined, under the light of the Constitution and guided by the theory of proportionality and related development imperatives. In this context, the present study identified the link between the principles of the economic order, environment and energy law, finding their basis under the Federal Constitution and development. By reproducing this interrelationship and by means of post-crisis institutional reforms, the guiding objectives of the Brazilian electric sector began to corroborate the precepts of development, although issues regarding its sustainability still persist. The appraisal of proportionality indicates that the Permanent Preservation Areas protection system is insufficient to materialize the right to a healthy quality of life upon the implementation of wind projects at Permanent Preservation Areas, albeit seeking the harmonization between free market and environmental protection.

Bayesian Inference in Large-scale Problems

Many modern applications fall into the category of "large-scale" statistical problems, in which both the number of observations n and the number of features or parameters p may be large. Many existing methods focus on point estimation, despite the continued relevance of uncertainty quantification in the sciences, where the number of parameters to estimate often exceeds the sample size, despite huge increases in the value of n typically seen in many fields. Thus, the tendency in some areas of industry to dispense with traditional statistical analysis on the basis that "n=all" is of little relevance outside of certain narrow applications. The main result of the Big Data revolution in most fields has instead been to make computation much harder without reducing the importance of uncertainty quantification. Bayesian methods excel at uncertainty quantification, but often scale poorly relative to alternatives. This conflict between the statistical advantages of Bayesian procedures and their substantial computational disadvantages is perhaps the greatest challenge facing modern Bayesian statistics, and is the primary motivation for the work presented here.

Two general strategies for scaling Bayesian inference are considered. The first is the development of methods that lend themselves to faster computation, and the second is design and characterization of computational algorithms that scale better in n or p. In the first instance, the focus is on joint inference outside of the standard problem of multivariate continuous data that has been a major focus of previous theoretical work in this area. In the second area, we pursue strategies for improving the speed of Markov chain Monte Carlo algorithms, and characterizing their performance in large-scale settings. Throughout, the focus is on rigorous theoretical evaluation combined with empirical demonstrations of performance and concordance with the theory.

One topic we consider is modeling the joint distribution of multivariate categorical data, often summarized in a contingency table. Contingency table analysis routinely relies on log-linear models, with latent structure analysis providing a common alternative. Latent structure models lead to a reduced rank tensor factorization of the probability mass function for multivariate categorical data, while log-linear models achieve dimensionality reduction through sparsity. Little is known about the relationship between these notions of dimensionality reduction in the two paradigms. In Chapter 2, we derive several results relating the support of a log-linear model to nonnegative ranks of the associated probability tensor. Motivated by these findings, we propose a new collapsed Tucker class of tensor decompositions, which bridge existing PARAFAC and Tucker decompositions, providing a more flexible framework for parsimoniously characterizing multivariate categorical data. Taking a Bayesian approach to inference, we illustrate empirical advantages of the new decompositions.

Latent class models for the joint distribution of multivariate categorical, such as the PARAFAC decomposition, data play an important role in the analysis of population structure. In this context, the number of latent classes is interpreted as the number of genetically distinct subpopulations of an organism, an important factor in the analysis of evolutionary processes and conservation status. Existing methods focus on point estimates of the number of subpopulations, and lack robust uncertainty quantification. Moreover, whether the number of latent classes in these models is even an identified parameter is an open question. In Chapter 3, we show that when the model is properly specified, the correct number of subpopulations can be recovered almost surely. We then propose an alternative method for estimating the number of latent subpopulations that provides good quantification of uncertainty, and provide a simple procedure for verifying that the proposed method is consistent for the number of subpopulations. The performance of the model in estimating the number of subpopulations and other common population structure inference problems is assessed in simulations and a real data application.

In contingency table analysis, sparse data is frequently encountered for even modest numbers of variables, resulting in non-existence of maximum likelihood estimates. A common solution is to obtain regularized estimates of the parameters of a log-linear model. Bayesian methods provide a coherent approach to regularization, but are often computationally intensive. Conjugate priors ease computational demands, but the conjugate Diaconis--Ylvisaker priors for the parameters of log-linear models do not give rise to closed form credible regions, complicating posterior inference. In Chapter 4 we derive the optimal Gaussian approximation to the posterior for log-linear models with Diaconis--Ylvisaker priors, and provide convergence rate and finite-sample bounds for the Kullback-Leibler divergence between the exact posterior and the optimal Gaussian approximation. We demonstrate empirically in simulations and a real data application that the approximation is highly accurate, even in relatively small samples. The proposed approximation provides a computationally scalable and principled approach to regularized estimation and approximate Bayesian inference for log-linear models.

Another challenging and somewhat non-standard joint modeling problem is inference on tail dependence in stochastic processes. In applications where extreme dependence is of interest, data are almost always time-indexed. Existing methods for inference and modeling in this setting often cluster extreme events or choose window sizes with the goal of preserving temporal information. In Chapter 5, we propose an alternative paradigm for inference on tail dependence in stochastic processes with arbitrary temporal dependence structure in the extremes, based on the idea that the information on strength of tail dependence and the temporal structure in this dependence are both encoded in waiting times between exceedances of high thresholds. We construct a class of time-indexed stochastic processes with tail dependence obtained by endowing the support points in de Haan's spectral representation of max-stable processes with velocities and lifetimes. We extend Smith's model to these max-stable velocity processes and obtain the distribution of waiting times between extreme events at multiple locations. Motivated by this result, a new definition of tail dependence is proposed that is a function of the distribution of waiting times between threshold exceedances, and an inferential framework is constructed for estimating the strength of extremal dependence and quantifying uncertainty in this paradigm. The method is applied to climatological, financial, and electrophysiology data.

The remainder of this thesis focuses on posterior computation by Markov chain Monte Carlo. The Markov Chain Monte Carlo method is the dominant paradigm for posterior computation in Bayesian analysis. It has long been common to control computation time by making approximations to the Markov transition kernel. Comparatively little attention has been paid to convergence and estimation error in these approximating Markov Chains. In Chapter 6, we propose a framework for assessing when to use approximations in MCMC algorithms, and how much error in the transition kernel should be tolerated to obtain optimal estimation performance with respect to a specified loss function and computational budget. The results require only ergodicity of the exact kernel and control of the kernel approximation accuracy. The theoretical framework is applied to approximations based on random subsets of data, low-rank approximations of Gaussian processes, and a novel approximating Markov chain for discrete mixture models.

Data augmentation Gibbs samplers are arguably the most popular class of algorithm for approximately sampling from the posterior distribution for the parameters of generalized linear models. The truncated Normal and Polya-Gamma data augmentation samplers are standard examples for probit and logit links, respectively. Motivated by an important problem in quantitative advertising, in Chapter 7 we consider the application of these algorithms to modeling rare events. We show that when the sample size is large but the observed number of successes is small, these data augmentation samplers mix very slowly, with a spectral gap that converges to zero at a rate at least proportional to the reciprocal of the square root of the sample size up to a log factor. In simulation studies, moderate sample sizes result in high autocorrelations and small effective sample sizes. Similar empirical results are observed for related data augmentation samplers for multinomial logit and probit models. When applied to a real quantitative advertising dataset, the data augmentation samplers mix very poorly. Conversely, Hamiltonian Monte Carlo and a type of independence chain Metropolis algorithm show good mixing on the same dataset.

A Computer Model of Drafting Effects on Collective Behavior in Elite 10,000 m Runners

Purpose Drafting in cycling influences collective behaviour of pelotons. Whilst evidence for collective behaviour in competitive running events exists, it is not clear if this results from energetic savings conferred by drafting. This study modelled the effects of drafting on behavior in elite 10,000 m runners. Methods Using performance data from a men’s elite 10,000 m track running event, computer simulations were constructed using Netlogo 5.1 to test the effects of three different drafting quantities on collective behaviour: no drafting, drafting to 3m behind with up to ~8% energy savings (a realistic running draft); and drafting up to 3m behind with up to 38% energy savings (a realistic cycling draft). Three measures of collective behaviour were analysed in each condition; mean speed, mean group stretch (distance between first and last placed runner), and Runner Convergence Ratio (RCR) which represents the degree of drafting benefit obtained by the follower in a pair of coupled runners. Results Mean speeds were 6.32±0.28m.s-1, 5.57±0.18 m.s-1, and 5.51±0.13 m.s-1 in the cycling draft, runner draft, and no draft conditions respectively (all P<0.001). RCR was lower in the cycling draft condition, but did not differ between the other two. Mean stretch did not differ between conditions. Conclusions Collective behaviours observed in running events cannot be fully explained through energetic savings conferred by realistic drafting benefits. They may therefore result from other, possibly psychological, processes. The benefits or otherwise of engaging in such behavior are, as yet, unclear.

Transmedia storytelling e a construção de um universo participativo: estudo de caso do projeto “collider”

Dissertação apresentada à Escola Superior de Comunicação Social como parte dos requisitos para obtenção de grau de mestre em Audiovisual e Multimédia.

Asymptotic Change-Point Analysis of the Dependencies in Time Series

In recent papers, Wied and his coauthors have introduced change-point procedures to detect and estimate structural breaks in the correlation between time series. To prove the asymptotic distribution of the test statistic and stopping time as well as the change-point estimation rate, they use an extended functional Delta method and assume nearly constant expectations and variances of the time series. In this thesis, we allow asymptotically infinitely many structural breaks in the means and variances of the time series. For this setting, we present test statistics and stopping times which are used to determine whether or not the correlation between two time series is and stays constant, respectively. Additionally, we consider estimates for change-points in the correlations. The employed nonparametric statistics depend on the means and variances. These (nuisance) parameters are replaced by estimates in the course of this thesis. We avoid assuming a fixed form of these estimates but rather we use "blackbox" estimates, i.e. we derive results under assumptions that these estimates fulfill. These results are supplement with examples. This thesis is organized in seven sections. In Section 1, we motivate the issue and present the mathematical model. In Section 2, we consider a posteriori and sequential testing procedures, and investigate convergence rates for change-point estimation, always assuming that the means and the variances of the time series are known. In the following sections, the assumptions of known means and variances are relaxed. In Section 3, we present the assumptions for the mean and variance estimates that we will use for the mean in Section 4, for the variance in Section 5, and for both parameters in Section 6. Finally, in Section 7, a simulation study illustrates the finite sample behaviors of some testing procedures and estimates.

Load balancing for aspect ratio in adaptive finite element simulations

In parallel adaptive finite element simulations the work load on the individual processors may change frequently. To (re)distribute the load evenly over the processors a load balancing heuristic is needed. Common strategies try to minimise subdomain dependencies by optimising the cutsize of the partitioning. However for certain solvers cutsize only plays a minor role, and their convergence is highly dependent on the subdomain shapes. Degenerated subdomain shapes cause them to need significantly more iterations to converge. In this work a new parallel load balancing strategy is introduced which directly addresses the problem of generating and conserving reasonably good subdomain shapes in a dynamically changing Finite Element Simulation. Geometric data is used to formulate several cost functions to rate elements in terms of their suitability to be migrated. The well known diffusive method which calculates the necessary load flow is enhanced by weighting the subdomain edges with the help of these cost functions. The proposed methods have been tested and results are presented.

Quichua-Spanish language contact in Salcedo, Ecuador: revisiting media lengua syncretic language practices

The purpose of the current thesis is to develop a better understanding of the interaction between Spanish and Quichua in the Salcedo region and provide more information for the processes that might have given rise to Media Lengua, a ‘mixed’ language comprised of a Quichua grammar and Spanish lexicon. Muysken attributes the formation of Media Lengua to relexification, ruling out any influence from other bilingual phenomena. I argue that the only characteristic that distinguishes Media Lengua from other language contact varieties in central Ecuador is the quantity of the overall Spanish borrowings and not the type of processes that might have been employed by Quichua speakers during the genesis of Media Lengua. The results from the Salcedo data that I have collected show how processes such as adlexification, code-mixing, and structural convergence produce Media Lengua-type sentences, evidence that supports an alternative analysis to Muysken’s relexification hypothesis. Overall, this dissertation is developed around four main objectives: (1) to describe the variation of Spanish loanwords within a bilingual community in Salcedo; (2) to analyze some of the prominent and recent structural changes in Quichua and Spanish; (3) to determine whether Spanish loanword use can be explained by the relationship consultants have with particular social categories; and (4) to analyze the consultants’ language ideologies toward syncretic uses of Spanish and Quichua. Overall, 58% of the content words, 39% of the basic vocabulary, and 50% of the subject pronouns in the Salcedo corpus were derived from Spanish. When compared to Muysken’s description of highlander Quichua in the 1970’s, Spanish loanwords have more than doubled in each category. The overall level of Spanish loanwords in Salcedo Quichua has grown to a level between highlander Quichua in the 1970’s and Media Lengua. Similar to Spanish’s lexical influence in Media Lengua, the increase of Spanish borrowings in today’s rural Quichua can be seen in non-basic and basic vocabularies as well as the subject pronoun system. Significantly, most of the growth has occurred through forms of adlexification i.e., doublets, well-established borrowings, and cultural borrowings, suggesting that ‘ordinary’ lexical borrowing is also capable of producing Media Lengua-type sentences. I approach the second objective by investigating two separate phenomena related to structural convergence. The first examines the complex verbal constructions that have developed in Quichua through Spanish loan translations while the second describes the type of Quichua particles that are attached to Spanish lexemes while speaking Spanish. The calquing of the complex verbal constructions from Spanish were employed when speaking standard Quichua. Since this standard form is typically used by language purists, I argue that their use of calques is a strategy of exploiting the full range of expression from Spanish without incorporating any of the Spanish lexemes which would give the appearance of ‘contamination’. The use of Quichua particles in local varieties of Spanish is a defining characteristic of Quichuacized Spanish, spoken most frequently by women and young children in the community. Although the use of Quichua particles was probably not the main catalyst engendering Media Lengua, I argue that its contribution as a source language to other ‘mixed’ varieties, such as Media Lengua, needs to be accounted for in descriptions of BML genesis. Contrary to Muysken’s representation of relatively ‘unmixed’ Spanish and Quichua as the two source languages of Media Lengua, I propose that local varieties of Spanish might have already been ‘mixed’ to a large degree before Media Lengua was created. The third objective attempts to draw a relationship between particular social variables and the use of Spanish loanwords. Whisker Boxplots and ANOVAs were used to determine which social group, if any, have been introducing new Spanish borrowings into the bilingual communities in Salcedo. Specifically, I controlled for age, education, native language, urban migration, and gender. The results indicate that none of the groups in each of the five social variables indicate higher or lower loanword use. The implication of these results are twofold: (a) when lexical borrowing occurs, it is immediately adopted as the community-wide norm and spoken by members from different backgrounds and generations, or (b) this level of Spanish borrowing (58%) is not a recent phenomenon. The fourth and final objective draws on my ethnographic research that addresses the attitudes of syncretic language use. I observed that Quichuacized Spanish and Hispanicized Quichua are highly stigmatized varieties spoken by the country’s most marginalized populations and families, yet within the community, syncretic ways of speaking are in fact the norm. It was shown that there exists a range of different linguistic definitions for ‘Chaupi Lengua’ and other syncretic language practices as well as many contrasting connotations, most of which were negative. One theme that emerged from the interviews was that speaking syncretic varieties of Quichua weakened the consultant’s claim to an indigenous identity. The linguistic and social data presented in this dissertation supports an alternative view to Muysken’s relexification hypothesis, one that has the advantage of operating with well-precedented linguistic processes and which is actually observable in the present-day Salcedo area. The results from the study on lexical borrowing are significant because they demonstrate how a dynamic bilingual speech community has gradually diversified their Quichua lexicon under intense pressure to shift toward Spanish. They also show that Hispanicized Quichua (Quichua with heavy lexical borrowing) clearly arose from adlexification and prolonged lexical borrowing, and is one of at least six identifiable speech styles found in Salcedo. These results challenge particular interpretations of language contact outcomes, such as, ones that depict sources languages as discrete and ‘unmixed.’ The bilingual continuum presented in this thesis shows on the one hand, the range of speech styles that are accessible to different speakers, and on the other hand, the overlapping, syncretic features that are shared among the different registers and language varieties. It was observed that syncretic speech styles in Salcedo are employed by different consultants in varied interactional contexts, and in turn, produce different evaluations by other fellow community members. In the current dissertation, I challenge the claim that relexification and Media Lengua-type sentences develop in isolation and without the influence of other bilingual phenomena. Based on Muysken's Media Lengua example sentences and the speech styles from the Salcedo corpus, I argue that Media Lengua may have arisen as an institutionalized variant of the highly mixed "middle ground" within the range of the Salcedo bilingual continuum discussed above. Such syncretic forms of Spanish and Quichua strongly resemble Media Lengua sentences in Muysken’s research, and therefore demonstrate how its development could have occurred through several different language contact processes and not only through relexification.

O Eu neuropsíquico : Ensaio sobre a consciência

Tese de Doutoramento em Psicologia Clínica apresentada ao ISPA - Instituto Universitário