Music Learning Networks: Supporting the Music Learning of Adolescents

Thesis (Ph.D.)--University of Washington, 2016-08

Idiotypic Immune Networks in Mobile Robot Control

Jerne's idiotypic network theory postulates that the immune response involves inter-antibody stimulation and suppression as well as matching to antigens. The theory has proved the most popular Artificial Immune System (AIS) model for incorporation into behavior-based robotics but guidelines for implementing idiotypic selection are scarce. Furthermore, the direct effects of employing the technique have not been demonstrated in the form of a comparison with non-idiotypic systems. This paper aims to address these issues. A method for integrating an idiotypic AIS network with a Reinforcement Learning based control system (RL) is described and the mechanisms underlying antibody stimulation and suppression are explained in detail. Some hypotheses that account for the network advantage are put forward and tested using three systems with increasing idiotypic complexity. The basic RL, a simplified hybrid AIS-RL that implements idiotypic selection independently of derived concentration levels and a full hybrid AIS-RL scheme are examined. The test bed takes the form of a simulated Pioneer robot that is required to navigate through maze worlds detecting and tracking door markers.

Idiotypic Immune Networks in Mobile Robot Control

Jerne's idiotypic network theory postulates that the immune response involves inter-antibody stimulation and suppression as well as matching to antigens. The theory has proved the most popular Artificial Immune System (AIS) model for incorporation into behavior-based robotics but guidelines for implementing idiotypic selection are scarce. Furthermore, the direct effects of employing the technique have not been demonstrated in the form of a comparison with non-idiotypic systems. This paper aims to address these issues. A method for integrating an idiotypic AIS network with a Reinforcement Learning based control system (RL) is described and the mechanisms underlying antibody stimulation and suppression are explained in detail. Some hypotheses that account for the network advantage are put forward and tested using three systems with increasing idiotypic complexity. The basic RL, a simplified hybrid AIS-RL that implements idiotypic selection independently of derived concentration levels and a full hybrid AIS-RL scheme are examined. The test bed takes the form of a simulated Pioneer robot that is required to navigate through maze worlds detecting and tracking door markers.

Genetic algorithm seeding of idiotypic networks for mobile-robot navigation

Robot-control designers have begun to exploit the properties of the human immune system in order to produce dynamic systems that can adapt to complex, varying, real-world tasks. Jerne’s idiotypic-network theory has proved the most popular artificial-immune-system (AIS) method for incorporation into behaviour-based robotics, since idiotypic selection produces highly adaptive responses. However, previous efforts have mostly focused on evolving the network connections and have often worked with a single, preengineered set of behaviours, limiting variability. This paper describes a method for encoding behaviours as a variable set of attributes, and shows that when the encoding is used with a genetic algorithm (GA), multiple sets of diverse behaviours can develop naturally and rapidly, providing much greater scope for flexible behaviour-selection. The algorithm is tested extensively with a simulated e-puck robot that navigates around a maze by tracking colour. Results show that highly successful behaviour sets can be generated within about 25 minutes, and that much greater diversity can be obtained when multiple autonomous populations are used, rather than a single one.

Mimicking the behaviour of idiotypic AIS robot controllers using probabilistic systems

Previous work has shown that robot navigation systems that employ an architecture based upon the idiotypic network theory of the immune system have an advantage over control techniques that rely on reinforcement learning only. This is thought to be a result of intelligent behaviour selection on the part of the idiotypic robot. In this paper an attempt is made to imitate idiotypic dynamics by creating controllers that use reinforcement with a number of different probabilistic schemes to select robot behaviour. The aims are to show that the idiotypic system is not merely performing some kind of periodic random behaviour selection, and to try to gain further insight into the processes that govern the idiotypic mechanism. Trials are carried out using simulated Pioneer robots that undertake navigation exercises. Results show that a scheme that boosts the probability of selecting highly-ranked alternative behaviours to 50% during stall conditions comes closest to achieving the properties of the idiotypic system, but remains unable to match it in terms of all round performance.

The McSweeney's Group: Modernist Roots and Contemporary Permutations in Little Magazines

The purpose of this project centered on the influential literary magazine Timothy McSweeney’s Quarterly Concern. Using Bruno Latour’s network theory as well as the methods put forth by Robert Scholes and Clifford Wulfman to study modernist little magazines, I analyzed the influence McSweeney’s has on contemporary little magazines. I traced the connections between McSweeney’s and other paradigmatic examples of little magazines—The Believer and n+1—to show how the McSweeney’s aesthetic and business practice creates a model for more recent publications. My thesis argued that The Believer continues McSweeney’s aesthetic mission. In contrast, n+1 positioned itself against the McSweeney’s aesthetic, which indirectly created a space within the little magazines for writers, philosophers, and artists to debate the prevailing aesthetic theories of the contemporary period. The creation of this space connects these contemporary magazines back to modernist little magazines, thereby validating my decision to use the methods of Scholes and Wulfman.

La migración infantil indocumentada de Centroamérica, un desafío a la reformulación de la política migratoria de Estados Unidos. Estudio de caso de Honduras (2010-2014)

El siguiente estudio de caso analiza la influencia de la llegada de la migración infantil indocumentada centroamericana en la reformulación de la política migratoria de Estados Unidos en el período 2010-2014. Enfocándose en el caso de Honduras para dar a conocer y analizar las causas que crean las dinámicas migratorias por parte de los menores, este trabajo analiza como la llegada de estas poblaciones genera ciertos efectos en el proceso de toma de decisión de las políticas internas de los Estados Unidos. Por un lado, para resaltar las características del fenómeno migratorio, se utilizan las teorías de redes sociales y la teoría push and pull. Por otro, mediante los conceptos de Sensibilidad y Vulnerabilidad expuestos en la teoría de la Interdependencia Compleja de las Relaciones Internacionales, como también el concepto de Seguridad Societal propuesto por Barry Buzan se estudia el nivel de influencia del fenómeno infantil en el gobierno norteamericano.

A preliminary theory of dark network resilience

A crucial contemporary policy question for governments across the globe is how to cope with international crime and terrorist networks. Many such “dark” networks—that is, networks that operate covertly and illegally—display a remarkable level of resilience when faced with shocks and attacks. Based on an in-depth study of three cases (MK, the armed wing of the African National Congress in South Africa during apartheid; FARC, the Marxist guerrilla movement in Colombia; and the Liberation Tigers of Tamil Eelam, LTTE, in Sri Lanka), we present a set of propositions to outline how shocks impact dark network characteristics (resources and legitimacy) and networked capabilities (replacing actors, linkages, balancing integration and differentiation) and how these in turn affect a dark network's resilience over time. We discuss the implications of our findings for policymakers.

Enhancing enterprise social network use : a control theory study

Organizations employ Enterprise Social Networks (ESNs) (e.g., Yammer) expecting better intra-organizational communication, effective knowledge sharing and, in general, greater collaboration. Despite their similarities with Public Social Networks (PSNs) (e.g., Twitter), ESNs are struggling to gain credence with employees. This paper is part of a larger research project that investigates mechanisms to enhance employees’ engagement in the ESNs. Through the lens of Control Theory, this paper reports preliminary findings of a pilot case study aimed to propose formal and informal mechanisms that impact employees’ intrinsic and extrinsic motivations to encourage their use of ESNs. The study results highlight (i) the need to better understand employees’ extrinsic and intrinsic motivations to use Social Networks, and (ii) that unlike a PSN which acts as a hedonic system, an ESN acts as a utilitarian system, highlighting the importance of supporting intrinsic motivations in its implementation.

Introduction to the special issue on next generation network architectures, protocols, theory, systems and applications

This special issue of Networking Science focuses on Next Generation Network (NGN) that enables the deployment of access independent services over converged fixed and mobile networks. NGN is a packet-based network and uses the Internet protocol (IP) to transport the various types of traffic (voice, video, data and signalling). NGN facilitates easy adoption of distributed computing applications by providing high speed connectivity in a converged networked environment. It also makes end user devices and applications highly intelligent and efficient by empowering them with programmability and remote configuration options. However, there are a number of important challenges in provisioning next generation network technologies in a converged communication environment. Some preliminary challenges include those that relate to QoS, switching and routing, management and control, and security which must be addressed on an urgent or emergency basis. The consideration of architectural issues in the design and pro- vision of secure services for NGN deserves special attention and hence is the main theme of this special issue.

An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.

Reinforcement learning using a continuous time actor-critic framework with spiking neurons.

Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modulated spike-timing-dependent plasticity have made first steps towards bridging the gap between the two approaches, but faced two problems. First, reinforcement learning is typically formulated in a discrete framework, ill-adapted to the description of natural situations. Second, biologically plausible models of reward-modulated spike-timing-dependent plasticity require precise calculation of the reward prediction error, yet it remains to be shown how this can be computed by neurons. Here we propose a solution to these problems by extending the continuous temporal difference (TD) learning of Doya (2000) to the case of spiking neurons in an actor-critic network operating in continuous time, and with continuous state and action representations. In our model, the critic learns to predict expected future rewards in real time. Its activity, together with actual rewards, conditions the delivery of a neuromodulatory TD signal to itself and to the actor, which is responsible for action choice. In simulations, we show that such an architecture can solve a Morris water-maze-like navigation task, in a number of trials consistent with reported animal performance. We also use our model to solve the acrobot and the cartpole problems, two complex motor control tasks. Our model provides a plausible way of computing reward prediction error in the brain. Moreover, the analytically derived learning rule is consistent with experimental evidence for dopamine-modulated spike-timing-dependent plasticity.