The Informational Complexity of Learning from Examples

This thesis attempts to quantify the amount of information needed to learn certain tasks. The tasks chosen vary from learning functions in a Sobolev space using radial basis function networks to learning grammars in the principles and parameters framework of modern linguistic theory. These problems are analyzed from the perspective of computational learning theory and certain unifying perspectives emerge.

Optimization of apodization functions in terahertz transient spectrometry

A quadratic programming optimization procedure for designing asymmetric apodization windows tailored to the shape of time-domain sample waveforms recorded using a terahertz transient spectrometer is proposed. By artificially degrading the waveforms, the performance of the designed window in both the time and the frequency domains is compared with that of conventional rectangular, triangular (Mertz), and Hamming windows. Examples of window optimization assuming Gaussian functions as the building elements of the apodization window are provided. The formulation is sufficiently general to accommodate other basis functions. (C) 2007 Optical Society of America

Neurofuzzy network model construction using Bézier-Bernstein polynomial functions

Neurofuzzy modelling systems combine fuzzy logic with quantitative artificial neural networks via a concept of fuzzification by using a fuzzy membership function usually based on B-splines and algebraic operators for inference, etc. The paper introduces a neurofuzzy model construction algorithm using Bezier-Bernstein polynomial functions as basis functions. The new network maintains most of the properties of the B-spline expansion based neurofuzzy system, such as the non-negativity of the basis functions, and unity of support but with the additional advantages of structural parsimony and Delaunay input space partitioning, avoiding the inherent computational problems of lattice networks. This new modelling network is based on the idea that an input vector can be mapped into barycentric co-ordinates with respect to a set of predetermined knots as vertices of a polygon (a set of tiled Delaunay triangles) over the input space. The network is expressed as the Bezier-Bernstein polynomial function of barycentric co-ordinates of the input vector. An inverse de Casteljau procedure using backpropagation is developed to obtain the input vector's barycentric co-ordinates that form the basis functions. Extension of the Bezier-Bernstein neurofuzzy algorithm to n-dimensional inputs is discussed followed by numerical examples to demonstrate the effectiveness of this new data based modelling approach.

Generalized neurofuzzy network modeling algorithms using Bezier-Bernstein polynomial functions and additive decomposition

This paper introduces a new neurofuzzy model construction algorithm for nonlinear dynamic systems based upon basis functions that are Bezier-Bernstein polynomial functions. This paper is generalized in that it copes with n-dimensional inputs by utilising an additive decomposition construction to overcome the curse of dimensionality associated with high n. This new construction algorithm also introduces univariate Bezier-Bernstein polynomial functions for the completeness of the generalized procedure. Like the B-spline expansion based neurofuzzy systems, Bezier-Bernstein polynomial function based neurofuzzy networks hold desirable properties such as nonnegativity of the basis functions, unity of support, and interpretability of basis function as fuzzy membership functions, moreover with the additional advantages of structural parsimony and Delaunay input space partition, essentially overcoming the curse of dimensionality associated with conventional fuzzy and RBF networks. This new modeling network is based on additive decomposition approach together with two separate basis function formation approaches for both univariate and bivariate Bezier-Bernstein polynomial functions used in model construction. The overall network weights are then learnt using conventional least squares methods. Numerical examples are included to demonstrate the effectiveness of this new data based modeling approach.

Neural network basis function center selection using cluster analysis

This paper deals with the selection of centres for radial basis function (RBF) networks. A novel mean-tracking clustering algorithm is described as a way in which centers can be chosen based on a batch of collected data. A direct comparison is made between the mean-tracking algorithm and k-means clustering and it is shown how mean-tracking clustering is significantly better in terms of achieving an RBF network which performs accurate function modelling.

Online modeling with tunable RBF network

In this paper, we propose a novel online modeling algorithm for nonlinear and nonstationary systems using a radial basis function (RBF) neural network with a fixed number of hidden nodes. Each of the RBF basis functions has a tunable center vector and an adjustable diagonal covariance matrix. A multi-innovation recursive least square (MRLS) algorithm is applied to update the weights of RBF online, while the modeling performance is monitored. When the modeling residual of the RBF network becomes large in spite of the weight adaptation, a node identified as insignificant is replaced with a new node, for which the tunable center vector and diagonal covariance matrix are optimized using the quantum particle swarm optimization (QPSO) algorithm. The major contribution is to combine the MRLS weight adaptation and QPSO node structure optimization in an innovative way so that it can track well the local characteristic in the nonstationary system with a very sparse model. Simulation results show that the proposed algorithm has significantly better performance than existing approaches.

Complex-valued b-spline neural networks for modelling and inverse of wiener systems

Communication signal processing applications often involve complex-valued (CV) functional representations for signals and systems. CV artificial neural networks have been studied theoretically and applied widely in nonlinear signal and data processing [1–11]. Note that most artificial neural networks cannot be automatically extended from the real-valued (RV) domain to the CV domain because the resulting model would in general violate Cauchy-Riemann conditions, and this means that the training algorithms become unusable. A number of analytic functions were introduced for the fully CV multilayer perceptrons (MLP) [4]. A fully CV radial basis function (RBF) nework was introduced in [8] for regression and classification applications. Alternatively, the problem can be avoided by using two RV artificial neural networks, one processing the real part and the other processing the imaginary part of the CV signal/system. A even more challenging problem is the inverse of a CV

Exact Search Directions for Optimization of Linear and Nonlinear Models Based on Generalized Orthonormal Functions

A novel technique for selecting the poles of orthonormal basis functions (OBF) in Volterra models of any order is presented. It is well-known that the usual large number of parameters required to describe the Volterra kernels can be significantly reduced by representing each kernel using an appropriate basis of orthonormal functions. Such a representation results in the so-called OBF Volterra model, which has a Wiener structure consisting of a linear dynamic generated by the orthonormal basis followed by a nonlinear static mapping given by the Volterra polynomial series. Aiming at optimizing the poles that fully parameterize the orthonormal bases, the exact gradients of the outputs of the orthonormal filters with respect to their poles are computed analytically by using a back-propagation-through-time technique. The expressions relative to the Kautz basis and to generalized orthonormal bases of functions (GOBF) are addressed; the ones related to the Laguerre basis follow straightforwardly as a particular case. The main innovation here is that the dynamic nature of the OBF filters is fully considered in the gradient computations. These gradients provide exact search directions for optimizing the poles of a given orthonormal basis. Such search directions can, in turn, be used as part of an optimization procedure to locate the minimum of a cost-function that takes into account the error of estimation of the system output. The Levenberg-Marquardt algorithm is adopted here as the optimization procedure. Unlike previous related work, the proposed approach relies solely on input-output data measured from the system to be modeled, i.e., no information about the Volterra kernels is required. Examples are presented to illustrate the application of this approach to the modeling of dynamic systems, including a real magnetic levitation system with nonlinear oscillatory behavior.

Optimization Of Groundwater Remediation With New Efficient Parallel Algorithm For Global Optimization

Application of optimization algorithm to PDE modeling groundwater remediation can greatly reduce remediation cost. However, groundwater remediation analysis requires a computational expensive simulation, therefore, effective parallel optimization could potentially greatly reduce computational expense. The optimization algorithm used in this research is Parallel Stochastic radial basis function. This is designed for global optimization of computationally expensive functions with multiple local optima and it does not require derivatives. In each iteration of the algorithm, an RBF is updated based on all the evaluated points in order to approximate expensive function. Then the new RBF surface is used to generate the next set of points, which will be distributed to multiple processors for evaluation. The criteria of selection of next function evaluation points are estimated function value and distance from all the points known. Algorithms created for serial computing are not necessarily efficient in parallel so Parallel Stochastic RBF is different algorithm from its serial ancestor. The application for two Groundwater Superfund Remediation sites, Umatilla Chemical Depot, and Former Blaine Naval Ammunition Depot. In the study, the formulation adopted treats pumping rates as decision variables in order to remove plume of contaminated groundwater. Groundwater flow and contamination transport is simulated with MODFLOW-MT3DMS. For both problems, computation takes a large amount of CPU time, especially for Blaine problem, which requires nearly fifty minutes for a simulation for a single set of decision variables. Thus, efficient algorithm and powerful computing resource are essential in both cases. The results are discussed in terms of parallel computing metrics i.e. speedup and efficiency. We find that with use of up to 24 parallel processors, the results of the parallel Stochastic RBF algorithm are excellent with speed up efficiencies close to or exceeding 100%.

Comparative study between RBF and radial-PPS neural networks

The study of function approximation is motivated by the human limitation and inability to register and manipulate with exact precision the behavior variations of the physical nature of a phenomenon. These variations are referred to as signals or signal functions. Many real world problem can be formulated as function approximation problems and from the viewpoint of artificial neural networks these can be seen as the problem of searching for a mapping that establishes a relationship from an input space to an output space through a process of network learning. Several paradigms of artificial neural networks (ANN) exist. Here we will be investigated a comparative of the ANN study of RBF with radial Polynomial Power of Sigmoids (PPS) in function approximation problems. Radial PPS are functions generated by linear combination of powers of sigmoids functions. The main objective of this paper is to show the advantages of the use of the radial PPS functions in relationship traditional RBF, through adaptive training and ridge regression techniques.

Aprendizado não-supervisionado em redes neurais pulsadas de base radial: um estudo da capacidade de agrupamento para a classificação de pixels

Redes neurais pulsadas - redes que utilizam uma codificação temporal da informação - têm despontado como uma promissora abordagem dentro do paradigma conexionista, emergente da ciência cognitiva. Um desses novos modelos é a rede neural pulsada com função de base radial, que é capaz de armazenar informação nos tempos de atraso axonais dos neurônios. Um algoritmo de aprendizado foi aplicado com sucesso nesta rede pulsada, que se mostrou capaz de mapear uma seqüência de pulsos de entrada em uma seqüência de pulsos de saída. Mais recentemente, um método baseado no uso de campos receptivos gaussianos foi proposto para codificar dados constantes em uma seqüência de pulsos temporais. Este método tornou possível a essa rede lidar com dados computacionais. O processo de aprendizado desta nova rede não se encontra plenamente compreendido e investigações mais profundas são necessárias para situar este modelo dentro do contexto do aprendizado de máquinas e também para estabelecer as habilidades e limitações desta rede. Este trabalho apresenta uma investigação desse novo classificador e um estudo de sua capacidade de agrupar dados em três dimensões, particularmente procurando estabelecer seus domínios de aplicação e horizontes no campo da visão computacional.

Analytical functions for the calculation of hyperspherical potential curves of atomic systems

We present angular basis functions for the Schrödinger equation of two-electron systems in hyperspherical coordinates. By using the hyperspherical adiabatic approach, the wave functions of two-electron systems are expanded in analytical functions, which generalizes the Jacobi polynomials. We show that these functions, obtained by selecting the diagonal terms of the angular equation, allow efficient diagonalization of the Hamiltonian for all values of the hyperspherical radius. The method is applied to the determination of the 1S e energy levels of the Li + and we show that the precision can be improved in a systematic and controllable way. ©2000 The American Physical Society.

Treinamento de uma rede neural de base radial usando computação evolutiva: implementação e aplicações

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Desenvolvimento de metodologias automáticas para obtenção do parâmetro de forma ótimo para o método RPIM

Neste trabalho, são propostas metodologias para otimização do parâmetro de forma local c do método RPIM (Radial Point Interpolation Method). Com as técnicas apresentadas, é possível reduzir problemas com inversão de matrizes comuns em métodos sem malha e, também, garantir um maior grau de liberdade e precisão para a utilização da técnica, já que se torna possível uma definição semi-automática dos fatores de forma mais adequados para cada domínio de suporte. Além disso, é apresentado um algoritmo baseado no Line Sweep para a geração eficiente dos domínios de suporte.

Estudo de solvatocromismo em líquidos moleculares orgânicos via método seqüencial Monte Carlo/Mecânica Quântica

Utilizou-se o método seqüencial Monte Carlo / Mecânica Quântica para obterem-se os desvios de solvatocromismo e os momentos de dipolo dos sistemas de moléculas orgânicas: Uracil em meio aquoso, -Caroteno em Ácido Oléico, Ácido Ricinoléico em metanol e em Etanol e Ácido Oléico em metanol e em Etanol. As otimizações das geometrias e as distribuições de cargas foram obtidas através da Teoria do Funcional Densidade com o funcional B3LYP e os conjuntos de funções de base 6-31G(d) para todas as moléculas exceto para a água e Uracil, as quais, foram utilizadas o conjunto de funções de base 6-311++G(d,p). No tratamento clássico, Monte Carlo, aplicou-se o algoritmo Metropólis através do programa DICE. A separação de configurações estatisticamente relevantes para os cálculos das propriedades médias foi implementada com a utilização da função de auto-correlação calculada para cada sistema. A função de distribuição radial dos líquidos moleculares foi utilizada para a separação da primeira camada de solvatação, a qual, estabelece a principal interação entre soluto-solvente. As configurações relevantes da primeira camada de solvatação de cada sistema foram submetidas a cálculos quânticos a nível semi-empírico com o método ZINDO/S-CI. Os espectros de absorção foram obtidos para os solutos em fase gasosa e para os sistemas de líquidos moleculares comentados. Os momentos de dipolo elétrico dos mesmos também foram obtidos. Todas as bandas dos espectros de absorção dos sistemas tiveram um desvio para o azul, exceto a segunda banda do sistema de Beta-Caroteno em Ácido Oléico que apresentou um desvio para o vermelho. Os resultados encontrados apresentam-se em excelente concordância com os valores experimentais encontrados na literatura. Todos os sistemas tiveram aumento no momento de dipolo elétrico devido às moléculas dos solventes serem moléculas polares. Os sistemas de ácidos graxos em álcoois apresentaram resultados muito semelhantes, ou seja, os ácidos graxos mencionados possuem comportamentos espectroscópicos semelhantes submetidos aos mesmos solventes. As simulações através do método seqüencial Monte Carlo / Mecânica Quântica estudadas demonstraram que a metodologia é eficaz para a obtenção das propriedades espectroscópicas dos líquidos moleculares analisados.