Scalable Intelligence for Scheduling Systems

A personalização é um aspeto chave de uma interação homem-computador efetiva. Numa era em que existe uma abundância de informação e tantas pessoas a interagir com ela, de muitas maneiras, a capacidade de se ajustar aos seus utilizadores é crucial para qualquer sistema moderno. A criação de sistemas adaptáveis é um domínio bastante complexo que necessita de métodos muito específicos para ter sucesso. No entanto, nos dias de hoje ainda não existe um modelo ou arquitetura padrão para usar nos sistemas adaptativos modernos. A principal motivação desta tese é a proposta de uma arquitetura para modelação do utilizador que seja capaz de incorporar diferentes módulos necessários para criar um sistema com inteligência escalável com técnicas de modelação. Os módulos cooperam de forma a analisar os utilizadores e caracterizar o seu comportamento, usando essa informação para fornecer uma experiência de sistema customizada que irá aumentar não só a usabilidade do sistema mas também a produtividade e conhecimento do utilizador. A arquitetura proposta é constituída por três componentes: uma unidade de informação do utilizador, uma estrutura matemática capaz de classificar os utilizadores e a técnica a usar quando se adapta o conteúdo. A unidade de informação do utilizador é responsável por conhecer os vários tipos de indivíduos que podem usar o sistema, por capturar cada detalhe de interações relevantes entre si e os seus utilizadores e também contém a base de dados que guarda essa informação. A estrutura matemática é o classificador de utilizadores, e tem como tarefa a sua análise e classificação num de três perfis: iniciado, intermédio ou avançado. Tanto as redes de Bayes como as neuronais são utilizadas, e uma explicação de como as preparar e treinar para lidar com a informação do utilizador é apresentada. Com o perfil do utilizador definido torna-se necessária uma técnica para adaptar o conteúdo do sistema. Nesta proposta, uma abordagem de iniciativa mista é apresentada tendo como base a liberdade de tanto o utilizador como o sistema controlarem a comunicação entre si. A arquitetura proposta foi desenvolvida como parte integrante do projeto ADSyS - um sistema de escalonamento dinâmico - utilizado para resolver problemas de escalonamento sujeitos a eventos dinâmicos. Possui uma complexidade elevada mesmo para utilizadores frequentes, daí a necessidade de adaptar o seu conteúdo de forma a aumentar a sua usabilidade. Com o objetivo de avaliar as contribuições deste trabalho, um estudo computacional acerca do reconhecimento dos utilizadores foi desenvolvido, tendo por base duas sessões de avaliação de usabilidade com grupos de utilizadores distintos. Foi possível concluir acerca dos benefícios na utilização de técnicas de modelação do utilizador com a arquitetura proposta.

Behavioural and neural correlates of operant conditioning in Lymnaea stagnalis : role of previous experience during development /

The freshwater mollusc Lymnaea stagnalis was utilized in this study to further the understanding of how network properties change as a result of associative learning, and to determine whether or not this plasticity is dependent on previous experience during development. The respiratory and neural correlates of operant conditioning were first determined in normally reared Lymnaea. The same procedure was then applied to differentially reared Lymnaea, that is, animals that had never experienced aerial respiration during their development. The aim was to determine whether these animals would demonstrate the same responses to the training paradigm. In normally reared animals, a behavioural reduction in aerial respiration was accompanied by numerous changes within the neural network. Specifically, I provide evidence of changes at the level of the respiratory central pattern generator and the motor output. In the differentially reared animals, there was little behavioural data to suggest learning and memory. There were, however, significant differences in the network parameters, similar to those observed in normally reared animals. This demonstrated an effect of operant conditioning on differentially reared animals. In this thesis, I have identified additional correlates of operant conditioning in normally reared animals and provide evidence of associative learning in differentially reared animals. I conclude plasticity is not dependent on previous experience, but is rather ontogenetically programmed within the neural network.

Comparison of classification ability of hyperball algorithms to neural network and k-nearest neighbour algorithms

The main focus of this thesis is to evaluate and compare Hyperbalilearning algorithm (HBL) to other learning algorithms. In this work HBL is compared to feed forward artificial neural networks using back propagation learning, K-nearest neighbor and 103 algorithms. In order to evaluate the similarity of these algorithms, we carried out three experiments using nine benchmark data sets from UCI machine learning repository. The first experiment compares HBL to other algorithms when sample size of dataset is changing. The second experiment compares HBL to other algorithms when dimensionality of data changes. The last experiment compares HBL to other algorithms according to the level of agreement to data target values. Our observations in general showed, considering classification accuracy as a measure, HBL is performing as good as most ANn variants. Additionally, we also deduced that HBL.:s classification accuracy outperforms 103's and K-nearest neighbour's for the selected data sets.

Improving sampling, optimization and feature extraction in Boltzmann machines

L’apprentissage supervisé de réseaux hiérarchiques à grande échelle connaît présentement un succès fulgurant. Malgré cette effervescence, l’apprentissage non-supervisé représente toujours, selon plusieurs chercheurs, un élément clé de l’Intelligence Artificielle, où les agents doivent apprendre à partir d’un nombre potentiellement limité de données. Cette thèse s’inscrit dans cette pensée et aborde divers sujets de recherche liés au problème d’estimation de densité par l’entremise des machines de Boltzmann (BM), modèles graphiques probabilistes au coeur de l’apprentissage profond. Nos contributions touchent les domaines de l’échantillonnage, l’estimation de fonctions de partition, l’optimisation ainsi que l’apprentissage de représentations invariantes. Cette thèse débute par l’exposition d’un nouvel algorithme d'échantillonnage adaptatif, qui ajuste (de fa ̧con automatique) la température des chaînes de Markov sous simulation, afin de maintenir une vitesse de convergence élevée tout au long de l’apprentissage. Lorsqu’utilisé dans le contexte de l’apprentissage par maximum de vraisemblance stochastique (SML), notre algorithme engendre une robustesse accrue face à la sélection du taux d’apprentissage, ainsi qu’une meilleure vitesse de convergence. Nos résultats sont présent ́es dans le domaine des BMs, mais la méthode est générale et applicable à l’apprentissage de tout modèle probabiliste exploitant l’échantillonnage par chaînes de Markov. Tandis que le gradient du maximum de vraisemblance peut-être approximé par échantillonnage, l’évaluation de la log-vraisemblance nécessite un estimé de la fonction de partition. Contrairement aux approches traditionnelles qui considèrent un modèle donné comme une boîte noire, nous proposons plutôt d’exploiter la dynamique de l’apprentissage en estimant les changements successifs de log-partition encourus à chaque mise à jour des paramètres. Le problème d’estimation est reformulé comme un problème d’inférence similaire au filtre de Kalman, mais sur un graphe bi-dimensionnel, où les dimensions correspondent aux axes du temps et au paramètre de température. Sur le thème de l’optimisation, nous présentons également un algorithme permettant d’appliquer, de manière efficace, le gradient naturel à des machines de Boltzmann comportant des milliers d’unités. Jusqu’à présent, son adoption était limitée par son haut coût computationel ainsi que sa demande en mémoire. Notre algorithme, Metric-Free Natural Gradient (MFNG), permet d’éviter le calcul explicite de la matrice d’information de Fisher (et son inverse) en exploitant un solveur linéaire combiné à un produit matrice-vecteur efficace. L’algorithme est prometteur: en terme du nombre d’évaluations de fonctions, MFNG converge plus rapidement que SML. Son implémentation demeure malheureusement inefficace en temps de calcul. Ces travaux explorent également les mécanismes sous-jacents à l’apprentissage de représentations invariantes. À cette fin, nous utilisons la famille de machines de Boltzmann restreintes “spike & slab” (ssRBM), que nous modifions afin de pouvoir modéliser des distributions binaires et parcimonieuses. Les variables latentes binaires de la ssRBM peuvent être rendues invariantes à un sous-espace vectoriel, en associant à chacune d’elles, un vecteur de variables latentes continues (dénommées “slabs”). Ceci se traduit par une invariance accrue au niveau de la représentation et un meilleur taux de classification lorsque peu de données étiquetées sont disponibles. Nous terminons cette thèse sur un sujet ambitieux: l’apprentissage de représentations pouvant séparer les facteurs de variations présents dans le signal d’entrée. Nous proposons une solution à base de ssRBM bilinéaire (avec deux groupes de facteurs latents) et formulons le problème comme l’un de “pooling” dans des sous-espaces vectoriels complémentaires.

Traitement vocal et vieillissement normal : apports comportementaux et neurofonctionnels

La voix est tout sauf un stimulus auditif ordinaire. Pour cause, elle prend son importance de manière très précoce chez l’Homme lorsque, dans l’environnement amniotique, le fœtus entend pour la toute première fois la voix de sa mère. C’est en quelque sorte par l’intermédiaire de cette voix que les premiers contacts avec le monde extérieur, mais également avec l’Autre, s’effectuent. Le statut particulier de la voix humaine perdure au fil du développement, devenant plus tard le principal médium véhiculant le langage oral, si significatif pour l’Homme. En parallèle, et de manière tout aussi adaptative, elle permet la transmission d’informations non langagières renseignant sur l’identité, l’état émotionnel mais également le statut social de chaque individu. C’est ainsi que simplement en entendant la voix d’une personne inconnue, il est généralement possible d’en extrapoler son âge, son genre, mais également d’avoir une idée assez précise de l’état émotionnel dans lequel elle se trouve. Les capacités permettant d’extraire de la voix les divers éléments informationnels qu’elle contient ne seraient toutefois pas stables au fil du temps. Ainsi, le vieillissement normal semble associé à des difficultés de traitement des informations vocales de nature langagière, mais également non langagière. De nombreuses études se sont intéressées au déclin des capacités de traitement du discours avec l’âge. Beaucoup moins de travaux ont cependant considéré les conséquences du vieillissement sur le domaine paralinguistique et, lorsque des travaux s’y sont attardés, c’est essentiellement la sphère affective qui a été investiguée. En raison de ce peu d’études, mais également de leur focus portant spécifiquement sur la sphère émotionnelle, il est extrêmement ardu de généraliser les résultats obtenus au traitement vocal général. La présente thèse s’est donc intéressée aux capacités de traitement de la voix dans le vieillissement normal. La première étude de cette thèse (Article 1) avait pour objectif d’évaluer l’impact du vieillissement normal sur les capacités comportementales de traitement paralinguistique vocal non émotionnel. Pour ce faire, une batterie informatisée composée de quatre tâches a été élaborée : la batterie d’évaluation de la perception vocale (Batterie EPV; tâches de catégorisation de genre, de discrimination de sources sonores, adaptative de discrimination et de mémorisation). Cette batterie permettait de comparer les performances d’adultes jeunes et âgés lors du traitement de stimuli vocaux et non vocaux, mais également lors du traitement de divers stimuli vocaux. Cette première étude met en évidence, pour trois des quatre tâches comportementales, des performances inférieures chez les adultes âgés et ce, malgré le contrôle statistique des contributions du déclin auditif et cognitif. Pour les aînés, le traitement de stimuli vocaux, en comparaison au traitement de stimuli non vocaux, n’était toutefois pas systématiquement inférieur à celui des jeunes adultes. Sans que les performances ne puissent être prédites par la mesure cognitive utilisée comme covariable (performances au MoCA), il appert que les demandes cognitives inhérentes aux tâches participent à ces différences intergroupes. Le second article de ce travail visait quant à lui à explorer à l’aide de l’imagerie par résonnance magnétique fonctionnelle (IRMf), l’influence du vieillissement normal sur les réseaux neuronaux sous-tendant le traitement de l’information vocale, une telle investigation n’ayant jamais été effectuée auparavant. Pour ce faire, une tâche d’écoute passive (permettant le contraste de blocs de sons vocaux et non vocaux) ainsi qu’une tâche adaptative de discrimination ont été utilisées. La tâche adaptative, basée sur une type de protocole psycoacoustique « up-down », assurait l’obtention de niveau de performance équivalent entre les deux groupes, une condition nécessaire pour la comparaison de groupe au niveau neurofonctionnel. La comparaison des adultes jeunes et âgés n’a mis en évidence aucune disparité quant au recrutement des aires répondant préférentiellement à la voix : les aires vocales temporales (AVT). Ce résultat suggère que l’âge n’affecte pas la mobilisation des aires spécialisées dans le traitement de la voix. Néanmoins, à l’extérieur des AVT et chez les aînés, le sous recrutement d’une portion du cortex auditif a été observé, en parallèle au recrutement additionnel de régions pariétale, temporale et frontale (Article 2 – Étude 1). Lors de la réalisation d’une tâche adaptative de discrimination, contrairement à ce qui était attendu, les seuils de discrimination des deux groupes d’âges étaient comparables. Pour effectuer la tâche, les participants âgés ont cependant recruté un réseau neuronal plus étendu que celui des jeunes adultes, et pour les aînés, l’activation additionnelle de régions frontale et temporale sous-tendaient la réalisation de la tâche (Article 2 - Étude 2). Les données comportementales présentées dans cette thèse suggèrent que l’effet délétère que semble avoir le vieillissement normal sur les capacités de traitement paralinguistique vocal affectif est également retrouvé lors du traitement d’informations vocales émotionnellement neutres. En parallèle, la mise en place de phénomènes de plasticité cérébrale est objectivée. Ces derniers ne toucheraient cependant pas les réseaux spécialisés dans le traitement de la voix, qui seraient recrutés de manière comparable par les adultes jeunes et âgés. Néanmoins, la tâche d’écoute passive a mis en évidence la présence, chez les aînés, du recrutement sous-optimal d’une portion du cortex auditif (gyrus temporal transverse). En parallèle, et ce pour les deux tâches, des réseaux neuronaux surnuméraires étaient sollicitées par les adultes âgés, permettant potentiellement d’assurer, chez les ainés, le maintien de performances adéquates.

Tunnel-Based Artificial Neural Network Technique to Calculate the Resonant Frequency of a Thick-Substrate Microstrip Antenna

The mathematical formulation of empirically developed formulas Jirr the calculation of the resonant frequency of a thick-substrate (h s 0.08151 A,,) microstrip antenna has been analyzed. With the use qt' tunnel-based artificial neural networks (ANNs), the resonant frequency of antennas with h satisfying the thick-substrate condition are calculated and compared with the existing experimental results and also with the simulation results obtained with the use of an IE3D software package. The artificial neural network results are in very good agreement with the experimental results

Neural Network based studies on Spectroscopic Analysis and Image Processing

International School of Photonics, Cochin University of Science and Technology

Implementation of a Neural Network Classifier for Noise Sources in the Ocean

The paper investigates the feasibility of implementing an intelligent classifier for noise sources in the ocean, with the help of artificial neural networks, using higher order spectral features. Non-linear interactions between the component frequencies of the noise data can give rise to certain phase relations called Quadratic Phase Coupling (QPC), which cannot be characterized by power spectral analysis. However, bispectral analysis, which is a higher order estimation technique, can reveal the presence of such phase couplings and provide a measure to quantify such couplings. A feed forward neural network has been trained and validated with higher order spectral features

Evolving Distributed Algorithms with Genetic Programming

Distributed systems are one of the most vital components of the economy. The most prominent example is probably the internet, a constituent element of our knowledge society. During the recent years, the number of novel network types has steadily increased. Amongst others, sensor networks, distributed systems composed of tiny computational devices with scarce resources, have emerged. The further development and heterogeneous connection of such systems imposes new requirements on the software development process. Mobile and wireless networks, for instance, have to organize themselves autonomously and must be able to react to changes in the environment and to failing nodes alike. Researching new approaches for the design of distributed algorithms may lead to methods with which these requirements can be met efficiently. In this thesis, one such method is developed, tested, and discussed in respect of its practical utility. Our new design approach for distributed algorithms is based on Genetic Programming, a member of the family of evolutionary algorithms. Evolutionary algorithms are metaheuristic optimization methods which copy principles from natural evolution. They use a population of solution candidates which they try to refine step by step in order to attain optimal values for predefined objective functions. The synthesis of an algorithm with our approach starts with an analysis step in which the wanted global behavior of the distributed system is specified. From this specification, objective functions are derived which steer a Genetic Programming process where the solution candidates are distributed programs. The objective functions rate how close these programs approximate the goal behavior in multiple randomized network simulations. The evolutionary process step by step selects the most promising solution candidates and modifies and combines them with mutation and crossover operators. This way, a description of the global behavior of a distributed system is translated automatically to programs which, if executed locally on the nodes of the system, exhibit this behavior. In our work, we test six different ways for representing distributed programs, comprising adaptations and extensions of well-known Genetic Programming methods (SGP, eSGP, and LGP), one bio-inspired approach (Fraglets), and two new program representations called Rule-based Genetic Programming (RBGP, eRBGP) designed by us. We breed programs in these representations for three well-known example problems in distributed systems: election algorithms, the distributed mutual exclusion at a critical section, and the distributed computation of the greatest common divisor of a set of numbers. Synthesizing distributed programs the evolutionary way does not necessarily lead to the envisaged results. In a detailed analysis, we discuss the problematic features which make this form of Genetic Programming particularly hard. The two Rule-based Genetic Programming approaches have been developed especially in order to mitigate these difficulties. In our experiments, at least one of them (eRBGP) turned out to be a very efficient approach and in most cases, was superior to the other representations.

Using Recurrent Networks for Dimensionality Reduction

This report explores how recurrent neural networks can be exploited for learning high-dimensional mappings. Since recurrent networks are as powerful as Turing machines, an interesting question is how recurrent networks can be used to simplify the problem of learning from examples. The main problem with learning high-dimensional functions is the curse of dimensionality which roughly states that the number of examples needed to learn a function increases exponentially with input dimension. This thesis proposes a way of avoiding this problem by using a recurrent network to decompose a high-dimensional function into many lower dimensional functions connected in a feedback loop.

Fast Learning by Bounding Likelihoods in Sigmoid Type Belief Networks

Sigmoid type belief networks, a class of probabilistic neural networks, provide a natural framework for compactly representing probabilistic information in a variety of unsupervised and supervised learning problems. Often the parameters used in these networks need to be learned from examples. Unfortunately, estimating the parameters via exact probabilistic calculations (i.e, the EM-algorithm) is intractable even for networks with fairly small numbers of hidden units. We propose to avoid the infeasibility of the E step by bounding likelihoods instead of computing them exactly. We introduce extended and complementary representations for these networks and show that the estimation of the network parameters can be made fast (reduced to quadratic optimization) by performing the estimation in either of the alternative domains. The complementary networks can be used for continuous density estimation as well.

Fully complex-valued radial basis function networks for orthogonal least squares regression

We consider a fully complex-valued radial basis function (RBF) network for regression application. The locally regularised orthogonal least squares (LROLS) algorithm with the D-optimality experimental design, originally derived for constructing parsimonious real-valued RBF network models, is extended to the fully complex-valued RBF network. Like its real-valued counterpart, the proposed algorithm aims to achieve maximised model robustness and sparsity by combining two effective and complementary approaches. The LROLS algorithm alone is capable of producing a very parsimonious model with excellent generalisation performance while the D-optimality design criterion further enhances the model efficiency and robustness. By specifying an appropriate weighting for the D-optimality cost in the combined model selecting criterion, the entire model construction procedure becomes automatic. An example of identifying a complex-valued nonlinear channel is used to illustrate the regression application of the proposed fully complex-valued RBF network.

Predictive control using feedback linearization based on dynamic neural models

This paper presents a hybrid control strategy integrating dynamic neural networks and feedback linearization into a predictive control scheme. Feedback linearization is an important nonlinear control technique which transforms a nonlinear system into a linear system using nonlinear transformations and a model of the plant. In this work, empirical models based on dynamic neural networks have been employed. Dynamic neural networks are mathematical structures described by differential equations, which can be trained to approximate general nonlinear systems. A case study based on a mixing process is presented.

A camera based digit location and recognition system for garment tracking

Garment information tracking is required for clean room garment management. In this paper, we present a camera-based robust system with implementation of Optical Character Reconition (OCR) techniques to fulfill garment label recognition. In the system, a camera is used for image capturing; an adaptive thresholding algorithm is employed to generate binary images; Connected Component Labelling (CCL) is then adopted for object detection in the binary image as a part of finding the ROI (Region of Interest); Artificial Neural Networks (ANNs) with the BP (Back Propagation) learning algorithm are used for digit recognition; and finally the system is verified by a system database. The system has been tested. The results show that it is capable of coping with variance of lighting, digit twisting, background complexity, and font orientations. The system performance with association to the digit recognition rate has met the design requirement. It has achieved real-time and error-free garment information tracking during the testing.

Neural and wavelet network models for financial distress classification

This work analyzes the use of linear discriminant models, multi-layer perceptron neural networks and wavelet networks for corporate financial distress prediction. Although simple and easy to interpret, linear models require statistical assumptions that may be unrealistic. Neural networks are able to discriminate patterns that are not linearly separable, but the large number of parameters involved in a neural model often causes generalization problems. Wavelet networks are classification models that implement nonlinear discriminant surfaces as the superposition of dilated and translated versions of a single "mother wavelet" function. In this paper, an algorithm is proposed to select dilation and translation parameters that yield a wavelet network classifier with good parsimony characteristics. The models are compared in a case study involving failed and continuing British firms in the period 1997-2000. Problems associated with over-parameterized neural networks are illustrated and the Optimal Brain Damage pruning technique is employed to obtain a parsimonious neural model. The results, supported by a re-sampling study, show that both neural and wavelet networks may be a valid alternative to classical linear discriminant models.