Roteamento em Redes de Sensores Sem Fios Com Base Em Aprendizagem Por Reforço

The use of wireless sensor and actuator networks in industry has been increasing past few years, bringing multiple benefits compared to wired systems, like network flexibility and manageability. Such networks consists of a possibly large number of small and autonomous sensor and actuator devices with wireless communication capabilities. The data collected by sensors are sent directly or through intermediary nodes along the network to a base station called sink node. The data routing in this environment is an essential matter since it is strictly bounded to the energy efficiency, thus the network lifetime. This work investigates the application of a routing technique based on Reinforcement Learning s Q-Learning algorithm to a wireless sensor network by using an NS-2 simulated environment. Several metrics like energy consumption, data packet delivery rates and delays are used to validate de proposal comparing it with another solutions existing in the literature

Autonomic and embedded wireless sensor protocols for critical infrastructures

The tragic events of September 11th ushered a new era of unprecedented challenges. Our nation has to be protected from the alarming threats of adversaries. These threats exploit the nation's critical infrastructures affecting all sectors of the economy. There is the need for pervasive monitoring and decentralized control of the nation's critical infrastructures. The communications needs of monitoring and control of critical infrastructures was traditionally catered for by wired communication systems. These technologies ensured high reliability and bandwidth but are however very expensive, inflexible and do not support mobility and pervasive monitoring. The communication protocols are Ethernet-based that used contention access protocols which results in high unsuccessful transmission and delay. An emerging class of wireless networks, named embedded wireless sensor and actuator networks has potential benefits for real-time monitoring and control of critical infrastructures. The use of embedded wireless networks for monitoring and control of critical infrastructures requires secure, reliable and timely exchange of information among controllers, distributed sensors and actuators. The exchange of information is over shared wireless media. However, wireless media is highly unpredictable due to path loss, shadow fading and ambient noise. Monitoring and control applications have stringent requirements on reliability, delay and security. The primary issue addressed in this dissertation is the impact of wireless media in harsh industrial environment on the reliable and timely delivery of critical data. In the first part of the dissertation, a combined networking and information theoretic approach was adopted to determine the transmit power required to maintain a minimum wireless channel capacity for reliable data transmission. The second part described a channel-aware scheduling scheme that ensured efficient utilization of the wireless link and guaranteed delay. Various analytical evaluations and simulations are used to evaluate and validate the feasibility of the methodologies and demonstrate that the protocols achieved reliable and real-time data delivery in wireless industrial networks.

Trajectory Planning in UAV-Aided Wireless Networks Via Reinforcement Learning

This thesis is focused on the design of a flexible, dynamic and innovative telecommunication's system for future 6G applications on vehicular communications. The system is based on the development of drones acting as mobile base stations in an urban scenario to cope with the increasing traffic demand and avoid network's congestion conditions. In particular, the exploitation of Reinforcement Learning algorithms is used to let the drone learn autonomously how to behave in a scenario full of obstacles with the goal of tracking and serve the maximum number of moving vehicles, by at the same time, minimizing the energy consumed to perform its tasks. This project is an extraordinary opportunity to open the doors to a new way of applying and develop telecommunications in an urban scenario by mixing it to the rising world of the Artificial Intelligence.

User and Machine Authentication and Authorization Infrastructure for Distributed Wireless Sensor Network Testbeds

The intention of an authentication and authorization infrastructure (AAI) is to simplify and unify access to different web resources. With a single login, a user can access web applications at multiple organizations. The Shibboleth authentication and authorization infrastructure is a standards-based, open source software package for web single sign-on (SSO) across or within organizational boundaries. It allows service providers to make fine-grained authorization decisions for individual access of protected online resources. The Shibboleth system is a widely used AAI, but only supports protection of browser-based web resources. We have implemented a Shibboleth AAI extension to protect web services using Simple Object Access Protocol (SOAP). Besides user authentication for browser-based web resources, this extension also provides user and machine authentication for web service-based resources. Although implemented for a Shibboleth AAI, the architecture can be easily adapted to other AAIs.

Aversive Stimuli and Functional Networks of Fear Learning in PTSD : 734

Background: One characteristic of post traumatic stress disorder is an inability to adapt to a safe environment i.e. to change behavior when predictions of adverse outcomes are not met. Recent studies have also indicated that PTSD patients have altered pain processing, with hyperactivation of the putamen and insula to aversive stimuli (Geuze et al, 2007). The present study examined neuronal responses to aversive and predicted aversive events. Methods: Twenty-four trauma exposed non-PTSD controls and nineteen subjects with PTSD underwent fMRI imaging during a partial reinforcement fear conditioning paradigm, with a mild electric shock as the unconditioned stimuli (UCS). Three conditions were analyzed: actual presentations of the UCS, events when a UCS was expected, but omitted (CS+), and events when the UCS was neither expected nor delivered (CS-). Results: The UCS evoked significant alterations in the pain matrix consisting of the brainstem, the midbrain, the thalamus, the insula, the anterior and middle cingulate and the contralateral somatosensory cortex. PTSD subjects displayed bilaterally elevated putamen activity to the electric shock, as compared to controls. In trials when USC was expected, but omitted, significant activations were observed in the brainstem, the midbrain, the anterior insula and the anterior cingulate. PTSD subjects displayed similar activations, but also elevated activations in the amygdala and the posterior insula. Conclusions: These results indicate altered fear and safety learning in PTSD, and neuronal activations are further explored in terms of functional connectivity using psychophysiological interaction analyses.

BatNet: an implementation of a 6LoWPAN sensor and actuator network

This demo presents BatNet, a 6LoWPAN Wireless Transducer Network, in a Home Automation context. Its suitability for such application is shown by means of several performance and usability tests.

ArchWiSeN: uma estratégia baseada em modelos para desenvolvimento de aplicações para redes de sensores e atuadores sem fio

Wireless Sensor and Actuator Networks (WSAN) are a key component in Ubiquitous Computing Systems and have many applications in different knowledge domains. Programming for such networks is very hard and requires developers to know the available sensor platforms specificities, increasing the learning curve for developing WSAN applications. In this work, an MDA (Model-Driven Architecture) approach for WSAN applications development called ArchWiSeN is proposed. The goal of such approach is to facilitate the development task by providing: (i) A WSAN domain-specific language, (ii) a methodology for WSAN application development; and (iii) an MDA infrastructure composed of several software artifacts (PIM, PSMs and transformations). ArchWiSeN allows the direct contribution of domain experts in the WSAN application development without the need of specialized knowledge on WSAN platforms and, at the same time, allows network experts to manage the application requirements without the need for specific knowledge of the application domain. Furthermore, this approach also aims to enable developers to express and validate functional and non-functional requirements of the application, incorporate services offered by WSAN middleware platforms and promote reuse of the developed software artifacts. In this sense, this Thesis proposes an approach that includes all WSAN development stages for current and emerging scenarios through the proposed MDA infrastructure. An evaluation of the proposal was performed by: (i) a proof of concept encompassing three different scenarios performed with the usage of the MDA infrastructure to describe the WSAN development process using the application engineering process, (ii) a controlled experiment to assess the use of the proposed approach compared to traditional method of WSAN application development, (iii) the analysis of ArchWiSeN support of middleware services to ensure that WSAN applications using such services can achieve their requirements ; and (iv) systematic analysis of ArchWiSeN in terms of desired characteristics for MDA tool when compared with other existing MDA tools for WSAN.

Adaptive and Maladaptive Implications of Reinforcement Learning Processes: Fronto-Striatal Loops and Behavioural Correlates

That humans and animals learn from interaction with the environment is a foundational idea underlying nearly all theories of learning and intelligence. Learning that certain outcomes are associated with specific actions or stimuli (both internal and external), is at the very core of the capacity to adapt behaviour to environmental changes. In the present work, appetitive and aversive reinforcement learning paradigms have been used to investigate the fronto-striatal loops and behavioural correlates of adaptive and maladaptive reinforcement learning processes, aiming to a deeper understanding of how cortical and subcortical substrates interacts between them and with other brain systems to support learning. By combining a large variety of neuroscientific approaches, including behavioral and psychophysiological methods, EEG and neuroimaging techniques, these studies aim at clarifying and advancing the knowledge of the neural bases and computational mechanisms of reinforcement learning, both in normal and neurologically impaired population.

Design and implementation of a Reinforcement Learning framework for iOS devices

Reinforcement Learning is an increasingly popular area of Artificial Intelligence. The applications of this learning paradigm are many, but its application in mobile computing is in its infancy. This study aims to provide an overview of current Reinforcement Learning applications on mobile devices, as well as to introduce a new framework for iOS devices: Swift-RL Lib. This new Swift package allows developers to easily support and integrate two of the most common RL algorithms, Q-Learning and Deep Q-Network, in a fully customizable environment. All processes are performed on the device, without any need for remote computation. The framework was tested in different settings and evaluated through several use cases. Through an in-depth performance analysis, we show that the platform provides effective and efficient support for Reinforcement Learning for mobile applications.

LWiSSy: uma linguagem específica de domínio para modelagem de sistemas de redes de sensores e atuadores sem fio

The field of Wireless Sensor and Actuator Networks (WSAN) is fast increasing and has attracted the interest of both the research community and the industry because of several factors, such as the applicability of such networks in different application domains (aviation, civil engineering, medicine, and others). Moreover, advances in wireless communication and the reduction of hardware components size also contributed for a fast spread of these networks. However, there are still several challenges and open issues that need to be tackled in order to achieve the full potential of WSAN usage. The development of WSAN systems is one of the most relevant of these challenges considering the number of variables involved in this process. Currently, a broad range of WSAN platforms and low level programming languages are available to build WSAN systems. Thus, developers need to deal with details of different sensor platforms and low-level programming abstractions of sensor operational systems on one hand, and they also need to have specific (high level) knowledge about the distinct application domains, on the other hand. Therefore, in order to decouple the handling of these two different levels of knowledge, making easier the development process of WSAN systems, we propose LWiSSy (Domain Language for Wireless Sensor and Actuator Networks Systems), a domain specific language (DSL) for WSAN. The use of DSLs raises the abstraction level during the programming of systems and modularizes the system building in several steps. Thus, LWiSSy allows the domain experts to directly contribute in the development of WSANs without having knowledge on low level sensor platforms, and network experts to program sensor nodes to meet application requirements without having specific knowledge on the application domain. Additionally, LWiSSy enables the system decomposition in different levels of abstraction according to structural and behavioral features and granularities (network, node group and single node level programming)

Virtualización de dispositivos para servicios en la Ciudad del Futuro

El aumento de las capacidades de interconexión de dispositivos de todo tipo está suponiendo una revolución en el campo de la prestación de servicios, tanto en la cantidad como en la variedad. Esta evolución ha puesto de manifiesto la necesidad de abordar un desarrollo tecnológico sin precedentes, donde la previsión de dispositivos interconectados e interoperando entre sí y con las personas alcanza cifras del orden de los millardos. Esta idea de un mundo de cosas interconectadas ha dado lugar a una visión que se ha dado en llamar Internet de las Cosas. Un mundo donde las cosas de cualquier tipo pueden interactuar con otras cosas, incluyendo las que forman parte de redes con recurso limitados. Y esto además conduce a la creación de servicios compuestos que superan a la suma de las partes. Además de la relevancia tecnológica, esta nueva visión enlaza con la de la Ciudad del Futuro. Un concepto que recurre a la convergencia de la energía, el transporte y las tecnologías de la información y las comunicaciones para definir una forma mediante la que lograr el crecimiento sostenible y competitivo, mejorando así la calidad de vida y abriendo el gobierno de las ciudades a la participación ciudadana. En la línea de desarrollo que permite avanzar hacia la consecución de tales objetivos, este Proyecto Fin de Carrera propone una forma de virtualizar los servicios ofrecidos por la diversidad de dispositivos que van adquiriendo la capacidad de interoperar en una red. Para ello se apoya en el uso de una capa de intermediación orientada a servicios, nSOM, desarrollada en la EUITT. Sobre esta arquitectura se proponen como objetivos el diseño y desarrollo de una pasarela de servicios que haga accesibles desde la web los recursos ofrecidos en una red de sensores; el diseño y desarrollo de un registro de dispositivos y servicios en concordancia a la propuesta de arquitectura de referencia para Internet de las Cosas; y el estudio y diseño de un marco para la composición de servicios orquestados en redes de recursos limitados. Para alcanzar estos objetivos primero se abordará un estudio del estado del arte donde se profundizará en el conocimiento de la las tecnologías para la interoperatividad entre cosas, abordando los principios de las redes inalámbricas de sensores y actuadores, las arquitecturas para las comunicaciones Máquina a Máquina e Internet de las Cosas, y la visión de la Web de las Cosas. Seguidamente se tratarán las tecnologías de red y de servicios de interés, para finalizar con un breve repaso a las tecnologías para la composición de servicios. Le seguirá una descripción detallada de la arquitectura nSOM y del diseño propuesto para este proyecto. Finalmente se propondrá un escenario sobre el que se llevarán a cabo diferentes pruebas de validación. ABSTRACT. The increasing of the capabilities of all kind of devices is causing a revolution in the field of the provision of services, both in quantity and in diversity. This situation has highlighted the need to address unprecedented technological development, where the forecast of interconnected and interoperable devices between them and human beings reaches the order of billions. And these numbers go further when the connectivity of constrained networks is taken into account. This idea of an interconnected world of things has led to a vision that has been called "The Internet of Things". It’s a vision of a world where things of any kind can interact with other things, even those in the domain of a constrained network. This also leads to the creation of new composed services that exceed the sum of the parts. Besides the technological interest, this new vision relates with the one from the Smart City. A concept that uses the convergence of the energy, the transport, and the information and communication technologies to define a way to achieve sustainable and competitive growth, improving the quality of life, and opening the governance of the cities to the participation. In the development pathway to reach these goals, this Final Degree Dissertation proposes a way for the virtualization of the services offered by the variety of devices that are reaching the ability to interoperate in a network. For this it is supported by a service oriented middleware called nSOM that has been developed at EUITT. Using this architecture the goals proposed for this project are the design and development of a service gateway that makes available the resources of a sensor network through a web interface; the design and development of a Device & Service Registry according to the reference architecture proposal for the Internet of Things; and the study and design of a composition framework for orchestrated services in constrained networks. To achieve these goals this dissertation begins with a State of the Art study where the background knowledge about the technologies in use for the interoperation of things will be settled. At first it starts talking about Wireless Sensor and Actuator Networks, the architectures for Machine-to-Machine communication and Internet of Things, and also the concepts for the Web of Things vision. Next the related network and services technologies are explored, ending with a brief review of service composition technologies. Then will follow a detailed description of the nSOM architecture, and also of the proposed design for this project. Finally a scenario will be proposed where a series of validation tests will be conducted.

Gestión dinámica de redes de sensores y actuadores

Este proyecto fin de carrera tiene como finalidad el diseño y la implementación de un sistema de monitorización y gestión dinámica de redes de sensores y actuadores inalámbricos (Wireless Sensor and Actuator Networks – WSAN) en base a la información de configuración almacenada en una base de datos sobre la cual un motor de detección vigila posibles cambios. Este motor informará de los cambios a la herramienta de gestión y monitorización de la WSAN para que sean llevados a cabo en la red desplegada. Este trabajo se enmarca en otro más amplio cuya finalidad es la demostración de la posibilidad de reconfigurar dinámicamente una WSAN utilizando los mecanismos propios de las Líneas de Productos Software Dinámicos (DSPL, por sus siglas en inglés). Se ha diseñado e implementado el software que proporciona los métodos necesarios para la comunicación y actuación sobre la red de sensores y actuadores inalámbricos, además de permitir el control de cada uno de los dispositivos pertenecientes a dicha red y que los dispositivos se incorporen a dicha red de manera autónoma. El desarrollo y pruebas de este proyecto fin de carrera se ha realizado utilizando una máquina virtual sobre la que se ha configurado convenientemente una plataforma que incluye un emulador de red de sensores y actuadores de tecnología SunSpot (Solarium) y todas las herramientas de desarrollo y ejecución necesarias (entre ellas, SunSpot SDK 6.0 y NetBeans). Esta máquina virtual ejecuta un sistema operativo Unix (Ubuntu Server 12.4) y facilita el rápido despliegue de las herramientas implementadas así como la integración de las mismas en desarrollos más amplios. En esta memoria se describe todo el proceso de diseño e implementación del software desarrollado, las conclusiones obtenidas de su ejecución y una guía de usuario para su despliegue y manejo. ABSTRACT. The aim of this project is the design and implementation of a system to monitor and dynamically manage a wireless sensor and actuator network (WSAN) in consistence with the configuration information stored in a database whose changes are monitored by a so-called monitoring engine. This engine informs the management and monitoring tool about the changes, in order for these to be carried out on the deployed network. This project is a part of a broader one aimed at demonstrating the ability to dynamically reconfigure a WSAN using the mechanisms of the Dynamic Software Product Lines (DSPL). A software has been designed and implemented which provides the methods to communicate with and actuate on the WSAN. It also allows to control each of the devices, as well as their autonomous incorporation to the network. Development and testing of this project was done using a virtual machine that has a conveniently configured platform which includes a SunSpot technology WSAN emulator (Solarium) as well as all the necessary development and implementation tools (including SunSpot 6.0 SDK and NetBeans). This virtual machine runs a Unix (Ubuntu Server 12.4) operating system and makes it easy to rapidly deploy the implemented tools and to integrate them into broader developments. This document explains the whole process of designing and implementing the software, the conclusions of execution and a user's manual.

How the web of things challenges requirements engineering

As a subset of the Internet of Things (IoT), the Web of Things (WoT) shares many characteristics with wireless sensor and actuator networks (WSANs) and ubiquitous computing systems (Ubicomp). Yet to a far greater degree than the IoT, WSANs or Ubicomp, the WoT will integrate physical and information objects, necessitating a means to model and reason about a range of context types that have hitherto received little or no attention from the RE community. RE practice is only now developing the means to support WSANs and Ubicomp system development, including faltering first steps in the representation of context. We argue that these techniques will need to be developed further, with a particular focus on rich context types, if RE is to support WoT application development. © 2012 Springer-Verlag.

Engineering a search and rescue application with a wireless sensor network - based localization mechanism

The advent of Wireless Sensor Network (WSN) technologies is paving the way for a panoply of new ubiquitous computing applications, some of them with critical requirements. In the ART-WiSe framework, we are designing a two-tiered communication architecture for supporting real-time and reliable communications in WSNs. Within this context, we have been developing a test-bed application, for testing, validating and demonstrating our theoretical findings - a search&rescue/pursuit-evasion application. Basically, a WSN deployment is used to detect, localize and track a target robot and a station controls a rescuer/pursuer robot until it gets close enough to the target robot. This paper describes how this application was engineered, particularly focusing on the implementation of the localization mechanism.

Impact of Node Mobility on Routing Protocols for Wireless Sensor Networks

Wireless sensor networks monitor their surrounding environment for the occurrence of some anticipated phenomenon. Most of the research related to sensor networks considers the static deployment of sensor nodes. Mobility of sensor node can be considered as an extra dimension of complexity, which poses interesting and challenging problems. Node mobility is a very important aspect in the design of effective routing algorithm for mobile wireless networks. In this work we intent to present the impact of different mobility models on the performance of the wireless sensor networks. Routing characteristics of various routing protocols for ad-hoc network were studied considering different mobility models. Performance metrics such as end-to-end delay, throughput and routing load were considered and their variations in the case of mobility models like Freeway, RPGM were studied. This work will be useful to figure out the characteristics of routing protocols depending on the mobility patterns of sensors