Continuous speech recognition based on ICA and geometrical learning

We investigate the use of independent component analysis (ICA) for speech feature extraction in digits speech recognition systems. We observe that this may be true for recognition tasks based on Geometrical Learning with little training data. In contrast to image processing, phase information is not essential for digits speech recognition. We therefore propose a new scheme that shows how the phase sensitivity can be removed by using an analytical description of the ICA-adapted basis functions. Furthermore, since the basis functions are not shift invariant, we extend the method to include a frequency-based ICA stage that removes redundant time shift information. The digits speech recognition results show promising accuracy. Experiments show that the method based on ICA and Geometrical Learning outperforms HMM in a different number of training samples.

Continuous speech recognition based on ICA and geometrical learning

We investigate the use of independent component analysis (ICA) for speech feature extraction in digits speech recognition systems. We observe that this may be true for recognition tasks based on Geometrical Learning with little training data. In contrast to image processing, phase information is not essential for digits speech recognition. We therefore propose a new scheme that shows how the phase sensitivity can be removed by using an analytical description of the ICA-adapted basis functions. Furthermore, since the basis functions are not shift invariant, we extend the method to include a frequency-based ICA stage that removes redundant time shift information. The digits speech recognition results show promising accuracy. Experiments show that the method based on ICA and Geometrical Learning outperforms HMM in a different number of training samples.

Spatial distribution of soil heavy metal pollution estimated by different interpolation methods: Accuracy and uncertainty analysis

Mapping the spatial distribution of contaminants in soils is the basis of pollution evaluation and risk control. Interpolation methods are extensively applied in the mapping processes to estimate the heavy metal concentrations at unsampled sites. The performances of interpolation methods (inverse distance weighting, local polynomial, ordinary kriging and radial basis functions) were assessed and compared using the root mean square error for cross validation. The results indicated that all interpolation methods provided a high prediction accuracy of the mean concentration of soil heavy metals. However, the classic method based on percentages of polluted samples, gave a pollution area 23.54-41.92% larger than that estimated by interpolation methods. The difference in contaminated area estimation among the four methods reached 6.14%. According to the interpolation results, the spatial uncertainty of polluted areas was mainly located in three types of region: (a) the local maxima concentration region surrounded by low concentration (clean) sites, (b) the local minima concentration region surrounded with highly polluted samples; and (c) the boundaries of the contaminated areas. (C) 2010 Elsevier Ltd. All rights reserved.

A Theory of Networks for Appxoimation and Learning

Learning an input-output mapping from a set of examples, of the type that many neural networks have been constructed to perform, can be regarded as synthesizing an approximation of a multi-dimensional function, that is solving the problem of hypersurface reconstruction. From this point of view, this form of learning is closely related to classical approximation techniques, such as generalized splines and regularization theory. This paper considers the problems of an exact representation and, in more detail, of the approximation of linear and nolinear mappings in terms of simpler functions of fewer variables. Kolmogorov's theorem concerning the representation of functions of several variables in terms of functions of one variable turns out to be almost irrelevant in the context of networks for learning. We develop a theoretical framework for approximation based on regularization techniques that leads to a class of three-layer networks that we call Generalized Radial Basis Functions (GRBF), since they are mathematically related to the well-known Radial Basis Functions, mainly used for strict interpolation tasks. GRBF networks are not only equivalent to generalized splines, but are also closely related to pattern recognition methods such as Parzen windows and potential functions and to several neural network algorithms, such as Kanerva's associative memory, backpropagation and Kohonen's topology preserving map. They also have an interesting interpretation in terms of prototypes that are synthesized and optimally combined during the learning stage. The paper introduces several extensions and applications of the technique and discusses intriguing analogies with neurobiological data.

A Connection Between GRBF and MLP

Both multilayer perceptrons (MLP) and Generalized Radial Basis Functions (GRBF) have good approximation properties, theoretically and experimentally. Are they related? The main point of this paper is to show that for normalized inputs, multilayer perceptron networks are radial function networks (albeit with a non-standard radial function). This provides an interpretation of the weights w as centers t of the radial function network, and therefore as equivalent to templates. This insight may be useful for practical applications, including better initialization procedures for MLP. In the remainder of the paper, we discuss the relation between the radial functions that correspond to the sigmoid for normalized inputs and well-behaved radial basis functions, such as the Gaussian. In particular, we observe that the radial function associated with the sigmoid is an activation function that is good approximation to Gaussian basis functions for a range of values of the bias parameter. The implication is that a MLP network can always simulate a Gaussian GRBF network (with the same number of units but less parameters); the converse is true only for certain values of the bias parameter. Numerical experiments indicate that this constraint is not always satisfied in practice by MLP networks trained with backpropagation. Multiscale GRBF networks, on the other hand, can approximate MLP networks with a similar number of parameters.

Accurate localized resolution of identity approach for linear-scaling hybrid density functionals and for many-body perturbation theory

© 2015 IOP Publishing Ltd and Deutsche Physikalische Gesellschaft.A key component in calculations of exchange and correlation energies is the Coulomb operator, which requires the evaluation of two-electron integrals. For localized basis sets, these four-center integrals are most efficiently evaluated with the resolution of identity (RI) technique, which expands basis-function products in an auxiliary basis. In this work we show the practical applicability of a localized RI-variant ('RI-LVL'), which expands products of basis functions only in the subset of those auxiliary basis functions which are located at the same atoms as the basis functions. We demonstrate the accuracy of RI-LVL for Hartree-Fock calculations, for the PBE0 hybrid density functional, as well as for RPA and MP2 perturbation theory. Molecular test sets used include the S22 set of weakly interacting molecules, the G3 test set, as well as the G2-1 and BH76 test sets, and heavy elements including titanium dioxide, copper and gold clusters. Our RI-LVL implementation paves the way for linear-scaling RI-based hybrid functional calculations for large systems and for all-electron many-body perturbation theory with significantly reduced computational and memory cost.

Full-Wave Analysis of Layered Aperture Arrays

The new rigorous numerical-analytical technique based upon Galerkin method with the entire domain basis functions has been developed and applied to the study of the periodic aperture arrays containing multiple dissimilar apertures of complex shapes in stratified medium. The rapid uniform convergence of the solutions has enabled a comprehensive parametric study of complex array arrangements. The developed theory has revealed new effects of the aperture shape and layout on the array performance. The physical mechanisms underlying the TM wave resonances and Luebbers' anomaly have been explained for the first time.

A two-stage algorithm for identification of nonlinear dynamic systems

This paper investigates the two-stage stepwise identification for a class of nonlinear dynamic systems that can be described by linear-in-the-parameters models, and the model has to be built from a very large pool of basis functions or model terms. The main objective is to improve the compactness of the model that is obtained by the forward stepwise methods, while retaining the computational efficiency. The proposed algorithm first generates an initial model using a forward stepwise procedure. The significance of each selected term is then reviewed at the second stage and all insignificant ones are replaced, resulting in an optimised compact model with significantly improved performance. The main contribution of this paper is that these two stages are performed within a well-defined regression context, leading to significantly reduced computational complexity. The efficiency of the algorithm is confirmed by the computational complexity analysis, and its effectiveness is demonstrated by the simulation results.

Wavelets for the Analysis and Compression of Power System Disturbances

Wavelets introduce new classes of basis functions for time-frequency signal analysis and have properties particularly suited to the transient components and discontinuities evident in power system disturbances. Wavelet analysis involves representing signals in terms of simpler, fixed building blocks at different scales and positions. This paper examines the analysis and subsequent compression properties of the discrete wavelet and wavelet packet transforms and evaluates both transforms using an actual power system disturbance from a digital fault recorder. The paper presents comparative compression results using the wavelet and discrete cosine transforms and examines the application of wavelet compression in power monitoring to mitigate against data communications overheads.

Decay spectrum and decay subspace of normal operators

Let A be a self-adjoint operator on a Hilbert space. It is well known that A admits a unique decomposition into a direct sum of three self-adjoint operators A(p), A(ac) and A(sc) such that there exists an orthonormal basis of eigenvectors for the operator A(p) the operator A(ac) has purely absolutely continuous spectrum and the operator A(sc) has purely singular continuous spectrum. We show the existence of a natural further decomposition of the singular continuous component A c into a direct sum of two self-adjoint operators A(sc)(D) and A(sc)(ND). The corresponding subspaces and spectra are called decaying and purely non-decaying singular subspaces and spectra. Similar decompositions are also shown for unitary operators and for general normal operators.

On velocity-based local model networks for nonlinear identification

This paper exposes the strengths and weaknesses of the recently proposed velocity-based local model (LM) network. The global dynamics of the velocity-based blended representation are directly related to the dynamics of the underlying local models, an important property in the design of local controller networks. Furthermore, the sub-models are continuous-time and linear providing continuity with established linear theory and methods. This is not true for the conventional LM framework, where the global dynamics are only weakly related to the affine sub-models. In this paper, a velocity-based multiple model network is identified for a highly nonlinear dynamical system. The results show excellent dynamical modelling performances, highlighting the value of the velocity-based approach for the design and analysis of LM based control. Three important practical issues are also addressed. These relate to the blending of the velocity-based local models, the use of normalised Gaussian basis functions and the requirement of an input derivative.

Efficient modeling of novel uniplanar left-handed metamaterials

This paper presents an efficient. modeling technique for the derivation of the dispersion characteristics of novel uniplanar metallodielectric periodic structures. The analysis is based on the method of moments and an interpolation scheme, which significantly accelerates the computations. Triangular basis functions are used that allow for modeling of arbitrary shaped metallic elements. Based on this method, novel uniplanar left-handed (LH) metamaterials are proposed. Variations of the split rectangular-loop element printed on grounded dielectric substrate are demonstrated to possess LH propagation properties. Full-wave dispersion curves are presented. Based on the dual transmission-line concept, we study the distribution of the modal fields And the variation of series capacitance and shunt inductance for all the proposed elements. A verification of the left-handedness is presented by means of full-wave simulation of finite uniplanar arrays using commercial software (HFSS). The cell dimensions are a small fraction of the wavelength (approximately lambda/24) so that the structures can he considered as a homogeneous effective medium. The structures are simple, readily scalable to higher frequencies, and compatible with low-cost fabrication techniques.

Integrated structure selection and parameter optimisation for eng-genes neural models

The eng-genes concept involves the use of fundamental known system functions as activation functions in a neural model to create a 'grey-box' neural network. One of the main issues in eng-genes modelling is to produce a parsimonious model given a model construction criterion. The challenges are that (1) the eng-genes model in most cases is a heterogenous network consisting of more than one type of nonlinear basis functions, and each basis function may have different set of parameters to be optimised; (2) the number of hidden nodes has to be chosen based on a model selection criterion. This is a mixed integer hard problem and this paper investigates the use of a forward selection algorithm to optimise both the network structure and the parameters of the system-derived activation functions. Results are included from case studies performed on a simulated continuously stirred tank reactor process, and using actual data from a pH neutralisation plant. The resulting eng-genes networks demonstrate superior simulation performance and transparency over a range of network sizes when compared to conventional neural models. (c) 2007 Elsevier B.V. All rights reserved.

Electromagnetic wave diffraction from an infinite array of complex-shaped microstrip reflectors

A numerical-analytical method is developed for solving surface integral equations (IEs) describing electromagnetic wave diffraction from arrays of complex-shaped planar reflectors. Solutions to these equations are regularized via analytical transformation of the separated singular part of the matrix kernel. Basis functions satisfying the metal-edge condition are determined on the entire surface of the complex region. The amplitude and phase responses of arrays consisting of polygonal reflectors are numerically investigated.

Protection Assessment using Reduced Power System Fault Data

Wavelet transforms provide basis functions for time-frequency analysis and have properties that are particularly useful for the compression of analogue point on wave transient and disturbance power system signals. This paper evaluates the compression properties of the discrete wavelet transform using actual power system data. The results presented in the paper indicate that reduction ratios up to 10:1 with acceptable distortion are achievable. The paper discusses the application of the reduction method for expedient fault analysis and protection assessment.