Implementation of two-stage Hopfield model and its application in nonlinear systems

This paper presents an efficient neural network for solving constrained nonlinear optimization problems. More specifically, a two-stage neural network architecture is developed and its internal parameters are computed using the valid-subspace technique. The main advantage of the developed network is that it treats optimization and constraint terms in different stages with no interference with each other. Moreover, the proposed approach does not require specification of penalty or weighting parameters for its initialization.

The size of excavators within a polymorphic termite species governs tunnel topology

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

A fast electric load forecasting using adaptive neural networks

This work presents a procedure for electric load forecasting based on adaptive multilayer feedforward neural networks trained by the Backpropagation algorithm. The neural network architecture is formulated by two parameters, the scaling and translation of the postsynaptic functions at each node, and the use of the gradient-descendent method for the adjustment in an iterative way. Besides, the neural network also uses an adaptive process based on fuzzy logic to adjust the network training rate. This methodology provides an efficient modification of the neural network that results in faster convergence and more precise results, in comparison to the conventional formulation Backpropagation algorithm. The adapting of the training rate is effectuated using the information of the global error and global error variation. After finishing the training, the neural network is capable to forecast the electric load of 24 hours ahead. To illustrate the proposed methodology it is used data from a Brazilian Electric Company. © 2003 IEEE.

Using remote lab networks to provide support to public secondary school education level

The advantages of networking are widely known in many areas (from business to personal ones). One particular area where networks have also proved their benefits is education. Taking the secondary school education level into account, some successful cases can be found in literature. In this paper we describe a particular remote lab network supporting physical experiments accessible to students of institutions geographically separated. The network architecture and application examples of using some of the available remote experiments are illustrated in detail. ©2008 IEEE.

Argentina: recent dynamics in the software and IT services industry

Includes bibliography.

Previsão de carga multinodal usando a rede neural ART-ARTMAP fuzzy

Pós-graduação em Engenharia Elétrica - FEIS

Detecção de refletores sísmicos por rede neural discreta

As redes neurais artificiais têm provado serem uma poderosa técnica na resolução de uma grande variedade de problemas de otimização. Nesta dissertação é desenvolvida uma nova rede neural, tipo recorrente, sem realimentação (self-feedback loops) e sem neurônios ocultos, para o processamento do sinal sísmico, para fornecer a posição temporal, a polaridade e as amplitudes estimadas dos refletores sísmicos, representadas pelos seus coeficientes de reflexão. A principal característica dessa nova rede neural consiste no tipo de função de ativação utilizada, a qual permite três possíveis estados para o neurônio. Busca-se estimar a posição dos refletores sísmicos e reproduzir as verdadeiras polaridades desses refletores. A idéia básica desse novo tipo de rede, aqui denominada rede neural discreta (RND), é relacionar uma função objeto, que descreve o problema geofísico, com a função de Liapunov, que descreve a dinâmica da rede neural. Deste modo, a dinâmica da rede leva a uma minimização local da sua função de Liapunov e consequentemente leva a uma minimização da função objeto. Assim, com uma codificação conveniente do sinal de saída da rede tem-se uma solução do problema geofísico. A avaliação operacional da arquitetura desta rede neural artificial é realizada em dados sintéticos gerados através do modelo convolucional simples e da teoria do raio. A razão é para explicar o comportamento da rede com dados contaminados por ruído, e diante de pulsos fonte de fases mínima, máxima e misturada.

Identificação automática das primeiras quebras em traços sísmicos por meio de uma rede neural direta

Apesar do avanço tecnológico ocorrido na prospecção sísmica, com a rotina dos levantamentos 2D e 3D, e o significativo aumento na quantidade de dados, a identificação dos tempos de chegada da onda sísmica direta (primeira quebra), que se propaga diretamente do ponto de tiro até a posição dos arranjos de geofones, permanece ainda dependente da avaliação visual do intérprete sísmico. O objetivo desta dissertação, insere-se no processamento sísmico com o intuito de buscar um método eficiente, tal que possibilite a simulação computacional do comportamento visual do intérprete sísmico, através da automação dos processos de tomada de decisão envolvidos na identificação das primeiras quebras em um traço sísmico. Visando, em última análise, preservar o conhecimento intuitivo do intérprete para os casos complexos, nos quais o seu conhecimento será, efetivamente, melhor aproveitado. Recentes descobertas na tecnologia neurocomputacional produziram técnicas que possibilitam a simulação dos aspectos qualitativos envolvidos nos processos visuais de identificação ou interpretação sísmica, com qualidade e aceitabilidade dos resultados. As redes neurais artificiais são uma implementação da tecnologia neurocomputacional e foram, inicialmente, desenvolvidas por neurobiologistas como modelos computacionais do sistema nervoso humano. Elas diferem das técnicas computacionais convencionais pela sua habilidade em adaptar-se ou aprender através de uma repetitiva exposição a exemplos, pela sua tolerância à falta de alguns dos componentes dos dados e pela sua robustez no tratamento com dados contaminados por ruído. O método aqui apresentado baseia-se na aplicação da técnica das redes neurais artificiais para a identificação das primeiras quebras nos traços sísmicos, a partir do estabelecimento de uma conveniente arquitetura para a rede neural artificial do tipo direta, treinada com o algoritmo da retro-propagação do erro. A rede neural artificial é entendida aqui como uma simulação computacional do processo intuitivo de tomada de decisão realizado pelo intérprete sísmico para a identificação das primeiras quebras nos traços sísmicos. A aplicabilidade, eficiência e limitações desta abordagem serão avaliadas em dados sintéticos obtidos a partir da teoria do raio.

A control and automation system for wave basins

This paper presents the new active absorption wave basin, named Hydrodynamic Calibrator (HC), constructed at the University of São Paulo (USP), in the Laboratory facilities of the Numerical Offshore Tank (TPN). The square (14 m 14 m) tank is able to generate and absorb waves from 0.5 Hz to 2.0 Hz, by means of 148 active hinged flap wave makers. An independent mechanical system drives each flap by means of a 1HP servo-motor and a ball-screw based transmission system. A customized ultrasonic wave probe is installed in each flap, and is responsible for measuring wave elevation in the flap. A complex automation architecture was implemented, with three Programmable Logic Computers (PLCs), and a low-level software is responsible for all the interlocks and maintenance functions of the tank. Furthermore, all the control algorithms for the generation and absorption are implemented using higher level software (MATLAB /Simulink block diagrams). These algorithms calculate the motions of the wave makers both to generate and absorb the required wave field by taking into account the layout of the flaps and the limits of wave generation. The experimental transfer function that relates the flap amplitude to the wave elevation amplitude is used for the calculation of the motion of each flap. This paper describes the main features of the tank, followed by a detailed presentation of the whole automation system. It includes the measuring devices, signal conditioning, PLC and network architecture, real-time and synchronizing software and motor control loop. Finally, a validation of the whole automation system is presented, by means of the experimental analysis of the transfer function of the waves generated and the calculation of all the delays introduced by the automation system.

Synaptic connectivity in micropatterned networks of neuronal cells

The presented thesis describes the formation of functional neuronal networks on an underlying micropattern. Small circuits of interconnected neurons defined by the geometry of the patterned substrate could be observed and were utilised as a model system of reduced complexity for the behaviour of neuronal network formation and activity. The first set of experiments was conducted to investigate aspects of the substrate preparation. Micropatterned substrates were created by microcontact printing of physiological proteins onto polystyrene culture dishes. The substrates displayed a high contrast between the repellant background and the cell attracting pattern, such that neurons seeded onto these surfaces aligned with the stamped structure. Both the patterning process and the cell culture were optimised, yielding highly compliant low-density networks of living neuronal cells. In the second step, cellular physiology of the cells grown on these substrates was investigated by patch-clamp measurements and compared to cells cultivated under control conditions. It could be shown that the growth on a patterned substrate did not result in an impairment of cellular integrity nor that it had an impact on synapse formation or synaptic efficacy. Due to the extremely low-density cell culture that was applied, cellular connectivity through chemical synapses could be observed at the single cell level. Having established that single cells were not negatively affected by the growth on patterned substrates, aspects of network formation were investigated. The formation of physical contact between two cells was analysed through microinjection studies and related to the rate at which functional synaptic contacts formed between two neighbouring cells. Surprisingly, the rate of synapse formation between physically contacting cells was shown to be unaltered in spite of the drastic reduction of potential interaction partners on the micropattern. Additional features of network formation were investigated and found consistent with results reported by other groups: A different rate of synapse formation by excitatory and inhibitory neurons could be reproduced as well as a different rate of frequency-dependent depression at excitatory and inhibitory synapses. Furthermore, regarding simple feedback loops, a significant enrichment of reciprocal connectivity between mixed pairs of excitatory and inhibitory neurons relative to uniform pairs could be demonstrated. This phenomenon has also been described by others in unpatterned cultures [Muller, 1997] and may therefore be a feature underlying neuronal network formation in general. Based on these findings, it can be assumed that inherent features of neuronal behaviour and cellular recognition mechanisms were found in the cultured networks and appear to be undisturbed by patterned growth. At the same time, it was possible to reduce the complexity of the forming networks dramatically in a cell culture on a patterned surface. Thus, features of network architecture and synaptic connectivity could be investigated on the single cell level under highly defined conditions.

Computer aided design and finite element modeling to predict bulk mechanical properties of bone scaffolds using triply periodic minimal surfaces (progettazione assistita al calcolatore ed analisi con il metodo degli elementi finiti per predire il comportamento meccanico di scaffolds ossei creati con l'uso di superfici minime periodiche in tre direzioni)

The aim of Tissue Engineering is to develop biological substitutes that will restore lost morphological and functional features of diseased or damaged portions of organs. Recently computer-aided technology has received considerable attention in the area of tissue engineering and the advance of additive manufacture (AM) techniques has significantly improved control over the pore network architecture of tissue engineering scaffolds. To regenerate tissues more efficiently, an ideal scaffold should have appropriate porosity and pore structure. More sophisticated porous configurations with higher architectures of the pore network and scaffolding structures that mimic the intricate architecture and complexity of native organs and tissues are then required. This study adopts a macro-structural shape design approach to the production of open porous materials (Titanium foams), which utilizes spatial periodicity as a simple way to generate the models. From among various pore architectures which have been studied, this work simulated pore structure by triply-periodic minimal surfaces (TPMS) for the construction of tissue engineering scaffolds. TPMS are shown to be a versatile source of biomorphic scaffold design. A set of tissue scaffolds using the TPMS-based unit cell libraries was designed. TPMS-based Titanium foams were meant to be printed three dimensional with the relative predicted geometry, microstructure and consequently mechanical properties. Trough a finite element analysis (FEA) the mechanical properties of the designed scaffolds were determined in compression and analyzed in terms of their porosity and assemblies of unit cells. The purpose of this work was to investigate the mechanical performance of TPMS models trying to understand the best compromise between mechanical and geometrical requirements of the scaffolds. The intention was to predict the structural modulus in open porous materials via structural design of interconnected three-dimensional lattices, hence optimising geometrical properties. With the aid of FEA results, it is expected that the effective mechanical properties for the TPMS-based scaffold units can be used to design optimized scaffolds for tissue engineering applications. Regardless of the influence of fabrication method, it is desirable to calculate scaffold properties so that the effect of these properties on tissue regeneration may be better understood.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Dynamics of a minimal model of interlocked positive and negative feedback loops of transcriptional regulation by cAMP-response element binding proteins.

cAMP-response element binding (CREB) proteins are involved in transcriptional regulation in a number of cellular processes (e.g., neural plasticity and circadian rhythms). The CREB family contains activators and repressors that may interact through positive and negative feedback loops. These loops can be generated by auto- and cross-regulation of expression of CREB proteins, via CRE elements in or near their genes. Experiments suggest that such feedback loops may operate in several systems (e.g., Aplysia and rat). To understand the functional implications of such feedback loops, which are interlocked via cross-regulation of transcription, a minimal model with a positive and negative loop was developed and investigated using bifurcation analysis. Bifurcation analysis revealed diverse nonlinear dynamics (e.g., bistability and oscillations). The stability of steady states or oscillations could be changed by time delays in the synthesis of the activator (CREB1) or the repressor (CREB2). Investigation of stochastic fluctuations due to small numbers of molecules of CREB1 and CREB2 revealed a bimodal distribution of CREB molecules in the bistability region. The robustness of the stable HIGH and LOW states of CREB expression to stochastic noise differs, and a critical number of molecules was required to sustain the HIGH state for days or longer. Increasing positive feedback or decreasing negative feedback also increased the lifetime of the HIGH state, and persistence of this state may correlate with long-term memory formation. A critical number of molecules was also required to sustain robust oscillations of CREB expression. If a steady state was near a deterministic Hopf bifurcation point, stochastic resonance could induce oscillations. This comparative analysis of deterministic and stochastic dynamics not only provides insights into the possible dynamics of CREB regulatory motifs, but also demonstrates a framework for understanding other regulatory processes with similar network architecture.

Dynamics of a minimal model of interlocked positive and negative feedback loops of transcriptional regulation by cAMP-response element binding proteins.

cAMP-response element binding (CREB) proteins are involved in transcriptional regulation in a number of cellular processes (e.g., neural plasticity and circadian rhythms). The CREB family contains activators and repressors that may interact through positive and negative feedback loops. These loops can be generated by auto- and cross-regulation of expression of CREB proteins, via CRE elements in or near their genes. Experiments suggest that such feedback loops may operate in several systems (e.g., Aplysia and rat). To understand the functional implications of such feedback loops, which are interlocked via cross-regulation of transcription, a minimal model with a positive and negative loop was developed and investigated using bifurcation analysis. Bifurcation analysis revealed diverse nonlinear dynamics (e.g., bistability and oscillations). The stability of steady states or oscillations could be changed by time delays in the synthesis of the activator (CREB1) or the repressor (CREB2). Investigation of stochastic fluctuations due to small numbers of molecules of CREB1 and CREB2 revealed a bimodal distribution of CREB molecules in the bistability region. The robustness of the stable HIGH and LOW states of CREB expression to stochastic noise differs, and a critical number of molecules was required to sustain the HIGH state for days or longer. Increasing positive feedback or decreasing negative feedback also increased the lifetime of the HIGH state, and persistence of this state may correlate with long-term memory formation. A critical number of molecules was also required to sustain robust oscillations of CREB expression. If a steady state was near a deterministic Hopf bifurcation point, stochastic resonance could induce oscillations. This comparative analysis of deterministic and stochastic dynamics not only provides insights into the possible dynamics of CREB regulatory motifs, but also demonstrates a framework for understanding other regulatory processes with similar network architecture.