The n-tuple network: coping with sub-optimality

The subject of this thesis is the n-tuple net.work (RAMnet). The major advantage of RAMnets is their speed and the simplicity with which they can be implemented in parallel hardware. On the other hand, this method is not a universal approximator and the training procedure does not involve the minimisation of a cost function. Hence RAMnets are potentially sub-optimal. It is important to understand the source of this sub-optimality and to develop the analytical tools that allow us to quantify the generalisation cost of using this model for any given data. We view RAMnets as classifiers and function approximators and try to determine how critical their lack of' universality and optimality is. In order to understand better the inherent. restrictions of the model, we review RAMnets showing their relationship to a number of well established general models such as: Associative Memories, Kamerva's Sparse Distributed Memory, Radial Basis Functions, General Regression Networks and Bayesian Classifiers. We then benchmark binary RAMnet. model against 23 other algorithms using real-world data from the StatLog Project. This large scale experimental study indicates that RAMnets are often capable of delivering results which are competitive with those obtained by more sophisticated, computationally expensive rnodels. The Frequency Weighted version is also benchmarked and shown to perform worse than the binary RAMnet for large values of the tuple size n. We demonstrate that the main issues in the Frequency Weighted RAMnets is adequate probability estimation and propose Good-Turing estimates in place of the more commonly used :Maximum Likelihood estimates. Having established the viability of the method numerically, we focus on providillg an analytical framework that allows us to quantify the generalisation cost of RAMnets for a given datasetL. For the classification network we provide a semi-quantitative argument which is based on the notion of Tuple distance. It gives a good indication of whether the network will fail for the given data. A rigorous Bayesian framework with Gaussian process prior assumptions is given for the regression n-tuple net. We show how to calculate the generalisation cost of this net and verify the results numerically for one dimensional noisy interpolation problems. We conclude that the n-tuple method of classification based on memorisation of random features can be a powerful alternative to slower cost driven models. The speed of the method is at the expense of its optimality. RAMnets will fail for certain datasets but the cases when they do so are relatively easy to determine with the analytical tools we provide.

Interactive semantics

Much research pursues machine intelligence through better representation of semantics. What is semantics? People in different areas view semantics from different facets although it accompanies interaction through civilization. Some researchers believe that humans have some innate structure in mind for processing semantics. Then, what the structure is like? Some argue that humans evolve a structure for processing semantics through constant learning. Then, how the process is like? Humans have invented various symbol systems to represent semantics. Can semantics be accurately represented? Turing machines are good at processing symbols according to algorithms designed by humans, but they are limited in ability to process semantics and to do active interaction. Super computers and high-speed networks do not help solve this issue as they do not have any semantic worldview and cannot reflect themselves. Can future cyber-society have some semantic images that enable machines and individuals (humans and agents) to reflect themselves and interact with each other with knowing social situation through time? This paper concerns these issues in the context of studying an interactive semantics for the future cyber-society. It firstly distinguishes social semantics from natural semantics, and then explores the interactive semantics in the category of social semantics. Interactive semantics consists of an interactive system and its semantic image, which co-evolve and influence each other. The semantic worldview and interactive semantic base are proposed as the semantic basis of interaction. The process of building and explaining semantic image can be based on an evolving structure incorporating adaptive multi-dimensional classification space and self-organized semantic link network. A semantic lens is proposed to enhance the potential of the structure and help individuals build and retrieve semantic images from different facets, abstraction levels and scales through time.

A Hierarchical Architecture with Parallel Comunication for Implementing P Systems

Membrane systems are computational equivalent to Turing machines. However, its distributed and massively parallel nature obtain polynomial solutions opposite to traditional non-polynomial ones. Nowadays, developed investigation for implementing membrane systems has not yet reached the massively parallel character of this computational model. Better published approaches have achieved a distributed architecture denominated “partially parallel evolution with partially parallel communication” where several membranes are allocated at each processor, proxys are used to communicate with membranes allocated at different processors and a policy of access control to the communications is mandatory. With these approaches, it is obtained processors parallelism in the application of evolution rules and in the internal communication among membranes allocated inside each processor. Even though, external communications share a common communication line, needed for the communication among membranes arranged in different processors, are sequential. In this work, we present a new hierarchical architecture that reaches external communication parallelism among processors and substantially increases parallelization in the application of evolution rules and internal communications. Consequently, necessary time for each evolution step is reduced. With all of that, this new distributed hierarchical architecture is near to the massively parallel character required by the model.

Parametric pattern selection in a reaction-diffusion model

We compare spot patterns generated by Turing mechanisms with those generated by replication cascades, in a model one-dimensional reaction-diffusion system. We determine the stability region of spot solutions in parameter space as a function of a natural control parameter (feed-rate) where degenerate patterns with different numbers of spots coexist for a fixed feed-rate. While it is possible to generate identical patterns via both mechanisms, we show that replication cascades lead to a wider choice of pattern profiles that can be selected through a tuning of the feed-rate, exploiting hysteresis and directionality effects of the different pattern pathways.

Mode-locking via dissipative Faraday instability

Emergence of coherent structures and patterns at the nonlinear stage of modulation instability of a uniform state is an inherent feature of many biological, physical and engineering systems. There are several well-studied classical modulation instabilities, such as Benjamin-Feir, Turing and Faraday instability, which play a critical role in the self-organization of energy and matter in non-equilibrium physical, chemical and biological systems. Here we experimentally demonstrate the dissipative Faraday instability induced by spatially periodic zig-zag modulation of a dissipative parameter of the system - spectrally dependent losses - achieving generation of temporal patterns and high-harmonic mode-locking in a fibre laser. We demonstrate features of this instability that distinguish it from both the Benjamin-Feir and the purely dispersive Faraday instability. Our results open the possibilities for new designs of mode-locked lasers and can be extended to other fields of physics and engineering.

Dynamic instabilities in scalar neural field equations with space-dependent delays

In this paper we consider a class of scalar integral equations with a form of space-dependent delay. These non-local models arise naturally when modelling neural tissue with active axons and passive dendrites. Such systems are known to support a dynamic (oscillatory) Turing instability of the homogeneous steady state. In this paper we develop a weakly nonlinear analysis of the travelling and standing waves that form beyond the point of instability. The appropriate amplitude equations are found to be the coupled mean-field Ginzburg-Landau equations describing a Turing-Hopf bifurcation with modulation group velocity of O(1). Importantly we are able to obtain the coefficients of terms in the amplitude equations in terms of integral transforms of the spatio-temporal kernels defining the neural field equation of interest. Indeed our results cover not only models with axonal or dendritic delays but those which are described by a more general distribution of delayed spatio-temporal interactions. We illustrate the predictive power of this form of analysis with comparison against direct numerical simulations, paying particular attention to the competition between standing and travelling waves and the onset of Benjamin-Feir instabilities.

Modeling electrocortical activity through improved local approximations of integral neural field equations

Neural field models of firing rate activity typically take the form of integral equations with space-dependent axonal delays. Under natural assumptions on the synaptic connectivity we show how one can derive an equivalent partial differential equation (PDE) model that properly treats the axonal delay terms of the integral formulation. Our analysis avoids the so-called long-wavelength approximation that has previously been used to formulate PDE models for neural activity in two spatial dimensions. Direct numerical simulations of this PDE model show instabilities of the homogeneous steady state that are in full agreement with a Turing instability analysis of the original integral model. We discuss the benefits of such a local model and its usefulness in modeling electrocortical activity. In particular we are able to treat "patchy'" connections, whereby a homogeneous and isotropic system is modulated in a spatially periodic fashion. In this case the emergence of a "lattice-directed" traveling wave predicted by a linear instability analysis is confirmed by the numerical simulation of an appropriate set of coupled PDEs. Article published and (c) American Physical Society 2007

Damage and fracture algorithm using the screened Poisson equation and local remeshing

We propose a crack propagation algorithm which is independent of particular constitutive laws and specific element technology. It consists of a localization limiter in the form of the screened Poisson equation with local mesh refinement. This combination allows the cap- turing of strain localization with good resolution, even in the absence of a sufficiently fine initial mesh. In addition, crack paths are implicitly defined from the localized region, cir- cumventing the need for a specific direction criterion. Observed phenomena such as mul- tiple crack growth and shielding emerge naturally from the algorithm. In contrast with alternative regularization algorithms, curved cracks are correctly represented. A staggered scheme for standard equilibrium and screened equations is used. Element subdivision is based on edge split operations using a given constitutive quantity (either damage or void fraction). To assess the robustness and accuracy of this algorithm, we use both quasi-brittle benchmarks and ductile tests.