Sharing Human-Generated Observations by Integrating HMI and the Semantic Sensor Web

Current “Internet of Things” concepts point to a future where connected objects gather meaningful information about their environment and share it with other objects and people. In particular, objects embedding Human Machine Interaction (HMI), such as mobile devices and, increasingly, connected vehicles, home appliances, urban interactive infrastructures, etc., may not only be conceived as sources of sensor information, but, through interaction with their users, they can also produce highly valuable context-aware human-generated observations. We believe that the great promise offered by combining and sharing all of the different sources of information available can be realized through the integration of HMI and Semantic Sensor Web technologies. This paper presents a technological framework that harmonizes two of the most influential HMI and Sensor Web initiatives: the W3C’s Multimodal Architecture and Interfaces (MMI) and the Open Geospatial Consortium (OGC) Sensor Web Enablement (SWE) with its semantic extension, respectively. Although the proposed framework is general enough to be applied in a variety of connected objects integrating HMI, a particular development is presented for a connected car scenario where drivers’ observations about the traffic or their environment are shared across the Semantic Sensor Web. For implementation and evaluation purposes an on-board OSGi (Open Services Gateway Initiative) architecture was built, integrating several available HMI, Sensor Web and Semantic Web technologies. A technical performance test and a conceptual validation of the scenario with potential users are reported, with results suggesting the approach is sound

PEPSI: Privacy-Enhanced Participatory Sensing Infrastructure.

Participatory Sensing combines the ubiquity of mobile phones with sensing capabilities of Wireless Sensor Networks. It targets pervasive collection of information, e.g., temperature, traffic conditions, or health-related data. As users produce measurements from their mobile devices, voluntary participation becomes essential. However, a number of privacy concerns -- due to the personal information conveyed by data reports -- hinder large-scale deployment of participatory sensing applications. Prior work on privacy protection, for participatory sensing, has often relayed on unrealistic assumptions and with no provably-secure guarantees. The goal of this project is to introduce PEPSI: a Privacy-Enhanced Participatory Sensing Infrastructure. We explore realistic architectural assumptions and a minimal set of (formal) privacy requirements, aiming at protecting privacy of both data producers and consumers. We design a solution that attains privacy guarantees with provable security at very low additional computational cost and almost no extra communication overhead.

Ultra Low Power FPGA-Based Architecture for Wake-up Radio in Wireless Sensor Networks

In this paper the capabilities of ultra low power FPGAs to implement Wake-up Radios (WuR) for ultra low energy Wireless Sensor Networks (WSNs) are analyzed. The main goal is to evaluate the utilization of very low power configurable devices to take advantage of their speed, flexibility and low power consumption instead of the more common approaches based on ASICs or microcontrollers. In this context, energy efficiency is a key aspect, considering that usually the instant power consumption is considered a figure of merit, more than the total energy consumed by the application.

Wake up Radio Architecture for Wireless Sensor Networks Using an Ultra Low Power FPGA

In this paper an implementation of a Wake up Radio(WuR) with addressing capabilities based on an ultra low power FPGA for ultra low energy Wireless Sensor Networks (WSNs) is proposed. The main goal is to evaluate the utilization of very low power configurable devices to take advantage of their speed, flexibility and low power consumption instead of the traditional approaches based on ASICs or microcontrollers, for communication frame decoding and communication data control.

Using the Own Flexibility of a Climbing Robot as a Double Force Sensor

Force sensors are used when interaction tasks are carried out by robots in general, and by climbing robots in particular. If the mechanics and electronics systems are contained inside the own robot, the robot becomes portable without external control. Commercial force sensors cannot be used due to limited space and weight. By selecting the links material with appropriate stiffness and placing strain gauges on the structure, the own robot flexibility can be used such as force sensor. Thus, forces applied on the robot tip can be measured without additional external devices. Only gauges and small internal electronic converters are necessary. This paper illustrates the proposed algorithm to achieve these measurements. Additionally, experimental results are presented.

Ultra high frequency thin film saw devices

Durante los últimos años el flujo de datos en la transmisión que tiene lugar en los sistemas de comunicación ha aumentado considerablemente de forma que día a día se requieren más aplicaciones trabajando en un rango de frecuencias muy alto (3-30 GHz). Muchos de estos sistemas de comunicación incluyen dispositivos de onda acústica superficial (SAW) y por tanto se hace necesario el aumento de frecuencia a la que éstos trabajan. Pero este incremento de frecuencia de los dispositivos SAW no sólo es utilizado en los sistemas de comunicación, varios tipos de sensores, por ejemplo, aumentan su sensibilidad cuando la frecuencia a la que trabajan también lo hace. Tradicionalmente los dispositivos SAW se han fabricado sobre cuarzo, LiNbO3 y LiTaO3 principalmente. Sin embargo la principal limitación de estos materiales es su velocidad SAW. Además, debido a la alta temperatura a la que se depositan no pueden ser integrados en la tecnología de fabricación CMOS. El uso de la tecnología de capa delgada, en la que un material piezoeléctrico es depositado sobre un substrato, se está utilizando en las últimas décadas para incrementar la velocidad SAW de la estructura y poder obtener dispositivos trabajando en el rango de frecuencias requerido en la actualidad. Por otra parte, esta tecnología podría ser integrada en el proceso de fabricación CMOS. Durante esta tesis nos hemos centrado en la fabricación de dispositivos SAW trabajando a muy alta frecuencia. Para ello, utilizando la tecnología de capa delgada, hemos utilizado la estructura nitruro de aluminio (AlN) sobre diamante que permite conseguir velocidades SAW del sustrato que no se pueden alcanzar con otros materiales. El depósito de AlN se realizó mediante sputtering reactivo. Durante esta tesis se han realizado diferentes experimentos para optimizar dicho depósito de forma que se han obtenido los parámetros óptimos para los cuales se pueden obtener capas de AlN de alta calidad sobre cualquier tipo de sustrato. Además todo el proceso se realizó a baja temperatura para que el procesado de estos dispositivos pueda ser compatible con la tecnología CMOS. Una vez optimizada la estructura AlN/diamante, mediante litografía por haz de electrones se fabricaron resonadores SAW de tamaño nanométrico que sumado a la alta velocidad resultante de la combinación AlN/diamante nos ha permitido obtener dispositivos trabajando en el rango de 10-28 GHz con un alto factor de calidad y rechazo fuera de la banda. Estás frecuencias y prestaciones no han sido alcanzadas por el momento en resonadores de este tipo. Por otra parte, se han utilizado estos dispositivos para fabricar sensores de presión de alta sensibilidad. Estos dispositivos son afectados altamente por los cambios de temperatura. Se realizó también un exhaustivo estudio de cómo se comportan en temperatura estos resonadores, entre -250ºC y 250ºC (rango de temperaturas no estudiado hasta el momento) diferenciándose dos regiones una a muy baja temperatura en la que el dispositivo muestra un coeficiente de retraso en frecuencia (TCF) relativamente bajo y otra a partir de los -100ºC en la que el TCF es similar al observado en la bibliografía. Por tanto, durante esta tesis se ha optimizado el depósito de AlN sobre diamante para que sea compatible con la tecnología CMOS y permita el procesado de dispositivos trabajando a muy alta frecuencia con altas prestaciones para comunicaciones y sensores. ABSTRACT The increasing volume of information in data transmission systems results in a growing demand of applications working in the super-high-frequency band (3–30 GHz). Most of these systems work with surface acoustic wave (SAW) devices and thus there is a necessity of increasing their resonance frequency. Moreover, sensor application includes this kind of devices. The sensitivity of them is proportional with its frequency. Traditionally, quartz, LiNbO3 and LiTaO3 have been used in the fabrication of SAW devices. These materials suffer from a variety of limitations and in particular they have low SAW velocity as well as being incompatible with the CMOS technology. In order to overcome these problems, thin film technology, where a piezoelectric material is deposited on top of a substrate, has been used during the last decades. The piezoelectric/substrate structure allows to reach the frequencies required nowadays and could be compatible with the mass electronic production CMOS technology. This thesis work focuses on the fabrication of SAW devices working in the super-high-frequency range. Thin film technology has been used in order to get it, especially aluminum nitride (AlN) deposited by reactive sputtering on diamond has been used to increase the SAW velocity. Different experiments were carried out to optimize the parameters for the deposit of high quality AlN on any kind of substrates. In addition, the system was optimized under low temperature and thus this process is CMOS compatible. Once the AlN/diamond was optimized, thanks to the used e-beam lithography, nanometric SAW resonators were fabricated. The combination of the structure and the size of the devices allow the fabrication of devices working in the range of 10-28 GHz with a high quality factor and out of band rejection. These high performances and frequencies have not been reached so far for this kind of devices. Moreover, these devices have been used as high sensitivity pressure sensors. They are affected by temperature changes and thus a wide temperature range (-250ºC to 250ºC) study was done. From this study two regions were observed. At very low temperature, the temperature coefficient of frequency (TCF) is low. From -100ºC upwards the TCF is similar to the one appearing in the literature. Therefore, during this thesis work, the sputtering of AlN on diamond substrates was optimized for the CMOS compatible fabrication of high frequency and high performance SAW devices for communication and sensor application.

High precision pressure sensors based on SAW devices in the GHz range

In this paper, an AlN/free-standing nanocrystalline diamond (NCD) system is proposed in order to process high frequency surface acoustic wave (SAW) resonators for sensing applications. The main problem of synthetic diamond is its high surface roughness that worsens the sputtered AlN quality and hence the device response. In order to study the feasibility of this structure, AlN films from 150 nm up to 1200 nm thick have been deposited on free-standing NCD. We have then analysed the influence of the AlN layer thickness on its crystal quality and device response. Optimized thin films of 300 nm have been used to fabricate of one-port SAW resonators operating in the 10–14 GHz frequency range. A SAW based sensor pressure with a sensibility of 0.33 MHz/bar has been fabricated.

A practical approach for outdoors distributed target localization in wireless sensor networks

Wireless sensor networks are posed as the new communication paradigm where the use of small, low-complexity, and low-power devices is preferred over costly centralized systems. The spectra of potential applications of sensor networks is very wide, ranging from monitoring, surveillance, and localization, among others. Localization is a key application in sensor networks and the use of simple, efficient, and distributed algorithms is of paramount practical importance. Combining convex optimization tools with consensus algorithms we propose a distributed localization algorithm for scenarios where received signal strength indicator readings are used. We approach the localization problem by formulating an alternative problem that uses distance estimates locally computed at each node. The formulated problem is solved by a relaxed version using semidefinite relaxation technique. Conditions under which the relaxed problem yields to the same solution as the original problem are given and a distributed consensusbased implementation of the algorithm is proposed based on an augmented Lagrangian approach and primaldual decomposition methods. Although suboptimal, the proposed approach is very suitable for its implementation in real sensor networks, i.e., it is scalable, robust against node failures and requires only local communication among neighboring nodes. Simulation results show that running an additional local search around the found solution can yield performance close to the maximum likelihood estimate.

Lifetime maximization for beamforming applications in wireless sensor networks

Energy efficiency is a major design issue in the context of Wireless Sensor Networks (WSN). If data is to be sent to a far-away base station, collaborative beamforming by the sensors may help to dis- tribute the load among the nodes and reduce fast battery depletion. However, collaborative beamforming techniques are far from opti- mality and in many cases may be wasting more power than required. In this contribution we consider the issue of energy efficiency in beamforming applications. Using a convex optimization framework, we propose the design of a virtual beamformer that maximizes the network's lifetime while satisfying a pre-specified Quality of Service (QoS) requirement. A distributed consensus-based algorithm for the computation of the optimal beamformer is also provided

Implementation of the disruption predictor APODIS in JET Real Time Network using the MARTe framework

Disruptions in tokamaks devices are unavoidable, and they can have a significant impact on machine integrity. So it is very important have mechanisms to predict this phenomenon. Disruption prediction is a very complex task, not only because it is a multi-dimensional problem, but also because in order to be effective, it has to detect well in advance the actual disruptive event, in order to be able to use successful mitigation strategies. With these constraints in mind a real-time disruption predictor has been developed to be used in JET tokamak. The predictor has been designed to run in the Multithreaded Application Real-Time executor (MARTe) framework. The predictor ?Advanced Predictor Of DISruptions? (APODIS) is based on Support Vector Machine (SVM).

Structural health monitoring network system with wireless communications inside closed aerospace structures

Structural Health Monitoring (SHM) requires integrated "all in one" electronic devices capable of performing analysis of structural integrity and on-board damage detection in aircraft?s structures. PAMELA III (Phased Array Monitoring for Enhanced Life Assessment, version III) SHM embedded system is an example of this device type. This equipment is capable of generating excitation signals to be applied to an array of integrated piezoelectric Phased Array (PhA) transducers stuck to aircraft structure, acquiring the response signals, and carrying out the advanced signal processing to obtain SHM maps. PAMELA III is connected with a host computer in order to receive the configuration parameters and sending the obtained SHM maps, alarms and so on. This host can communicate with PAMELA III through an Ethernet interface. To avoid the use of wires where necessary, it is possible to add Wi-Fi capabilities to PAMELA III, connecting a Wi-Fi node working as a bridge, and to establish a wireless communication between PAMELA III and the host. However, in a real aircraft scenario, several PAMELA III devices must work together inside closed structures. In this situation, it is not possible for all PAMELA III devices to establish a wireless communication directly with the host, due to the signal attenuation caused by the different obstacles of the aircraft structure. To provide communication among all PAMELA III devices and the host, a wireless mesh network (WMN) system has been implemented inside a closed aluminum wingbox. In a WMN, as long as a node is connected to at least one other node, it will have full connectivity to the entire network because each mesh node forwards packets to other nodes in the network as required. Mesh protocols automatically determine the best route through the network and can dynamically reconfigure the network if a link drops out. The advantages and disadvantages on the use of a wireless mesh network system inside closed aerospace structures are discussed.

Smart parking service based on wireless sensor networks

n this paper, we present the design and implementation of a prototype system of Smart Parking Services based on Wireless Sensor Networks (WSNs) that allows vehicle drivers to effectively find the free parking places. The proposed scheme consists of wireless sensor networks, embedded web-server, central web-server and mobile phone application. In the system, low-cost wireless sensors networks modules are deployed into each parking slot equipped with one sensor node. The state of the parking slot is detected by sensor node and is reported periodically to embedded web-server via the deployed wireless sensor networks. This information is sent to central web-server using Wi-Fi networks in real-time, and also the vehicle driver can find vacant parking lots using standard mobile devices.

Wake-up architecture for Wireless sensor nodes based on ultra low power FPGA

In this work a novel wake-up architecture for wireless sensor nodes based on ultra low power FPGA is presented. A simple wake up messaging mechanism for data gathering applications is proposed. The main goal of this work is to evaluate the utilization of low power configurable devices to take advantage of their speed, flexibility and low power consumption compared with traditional approaches, based on ASICs or microcontrollers, for frame decoding and data control. A test bed based on infrared communications has been built to validate the messaging mechanism and the processing architecture.

The Influence of Communication Range on Connectivity for Resilient Wireless Sensor Networks Using a Probabilistic Approach.

Wireless sensor networks (WSNs) consist of thousands of nodes that need to communicate with each other. However, it is possible that some nodes are isolated from other nodes due to limited communication range. This paper focuses on the influence of communication range on the probability that all nodes are connected under two conditions, respectively: (1) all nodes have the same communication range, and (2) communication range of each node is a random variable. In the former case, this work proves that, for 0menor queepsmenor quee^(-1) , if the probability of the network being connected is 0.36eps , by means of increasing communication range by constant C(eps) , the probability of network being connected is at least 1-eps. Explicit function C(eps) is given. It turns out that, once the network is connected, it also makes the WSNs resilient against nodes failure. In the latter case, this paper proposes that the network connection probability is modeled as Cox process. The change of network connection probability with respect to distribution parameters and resilience performance is presented. Finally, a method to decide the distribution parameters of node communication range in order to satisfy a given network connection probability is developed.

Design and control of multi-finger haptic devices for dexterous manipulation

En la interacción con el entorno que nos rodea durante nuestra vida diaria (utilizar un cepillo de dientes, abrir puertas, utilizar el teléfono móvil, etc.) y en situaciones profesionales (intervenciones médicas, procesos de producción, etc.), típicamente realizamos manipulaciones avanzadas que incluyen la utilización de los dedos de ambas manos. De esta forma el desarrollo de métodos de interacción háptica multi-dedo dan lugar a interfaces hombre-máquina más naturales y realistas. No obstante, la mayoría de interfaces hápticas disponibles en el mercado están basadas en interacciones con un solo punto de contacto; esto puede ser suficiente para la exploración o palpación del entorno pero no permite la realización de tareas más avanzadas como agarres. En esta tesis, se investiga el diseño mecánico, control y aplicaciones de dispositivos hápticos modulares con capacidad de reflexión de fuerzas en los dedos índice, corazón y pulgar del usuario. El diseño mecánico de la interfaz diseñada, ha sido optimizado con funciones multi-objetivo para conseguir una baja inercia, un amplio espacio de trabajo, alta manipulabilidad y reflexión de fuerzas superiores a 3 N en el espacio de trabajo. El ancho de banda y la rigidez del dispositivo se han evaluado mediante simulación y experimentación real. Una de las áreas más importantes en el diseño de estos dispositivos es el efector final, ya que es la parte que está en contacto con el usuario. Durante este trabajo se ha diseñado un dedal de bajo peso, adaptable a diferentes usuarios que, mediante la incorporación de sensores de contacto, permite estimar fuerzas normales y tangenciales durante la interacción con entornos reales y virtuales. Para el diseño de la arquitectura de control, se estudiaron los principales requisitos para estos dispositivos. Entre estos, cabe destacar la adquisición, procesado e intercambio a través de internet de numerosas señales de control e instrumentación; la computación de equaciones matemáticas incluyendo la cinemática directa e inversa, jacobiana, algoritmos de detección de agarres, etc. Todos estos componentes deben calcularse en tiempo real garantizando una frecuencia mínima de 1 KHz. Además, se describen sistemas para manipulación de precisión virtual y remota; así como el diseño de un método denominado "desacoplo cinemático iterativo" para computar la cinemática inversa de robots y la comparación con otros métodos actuales. Para entender la importancia de la interacción multimodal, se ha llevado a cabo un estudio para comprobar qué estímulos sensoriales se correlacionan con tiempos de respuesta más rápidos y de mayor precisión. Estos experimentos se desarrollaron en colaboración con neurocientíficos del instituto Technion Israel Institute of Technology. Comparando los tiempos de respuesta en la interacción unimodal (auditiva, visual y háptica) con combinaciones bimodales y trimodales de los mismos, se demuestra que el movimiento sincronizado de los dedos para generar respuestas de agarre se basa principalmente en la percepción háptica. La ventaja en el tiempo de procesamiento de los estímulos hápticos, sugiere que los entornos virtuales que incluyen esta componente sensorial generan mejores contingencias motoras y mejoran la credibilidad de los eventos. Se concluye que, los sistemas que incluyen percepción háptica dotan a los usuarios de más tiempo en las etapas cognitivas para rellenar información de forma creativa y formar una experiencia más rica. Una aplicación interesante de los dispositivos hápticos es el diseño de nuevos simuladores que permitan entrenar habilidades manuales en el sector médico. En colaboración con fisioterapeutas de Griffith University en Australia, se desarrolló un simulador que permite realizar ejercicios de rehabilitación de la mano. Las propiedades de rigidez no lineales de la articulación metacarpofalange del dedo índice se estimaron mediante la utilización del efector final diseñado. Estos parámetros, se han implementado en un escenario que simula el comportamiento de la mano humana y que permite la interacción háptica a través de esta interfaz. Las aplicaciones potenciales de este simulador están relacionadas con entrenamiento y educación de estudiantes de fisioterapia. En esta tesis, se han desarrollado nuevos métodos que permiten el control simultáneo de robots y manos robóticas en la interacción con entornos reales. El espacio de trabajo alcanzable por el dispositivo háptico, se extiende mediante el cambio de modo de control automático entre posición y velocidad. Además, estos métodos permiten reconocer el gesto del usuario durante las primeras etapas de aproximación al objeto para su agarre. Mediante experimentos de manipulación avanzada de objetos con un manipulador y diferentes manos robóticas, se muestra que el tiempo en realizar una tarea se reduce y que el sistema permite la realización de la tarea con precisión. Este trabajo, es el resultado de una colaboración con investigadores de Harvard BioRobotics Laboratory. ABSTRACT When we interact with the environment in our daily life (using a toothbrush, opening doors, using cell-phones, etc.), or in professional situations (medical interventions, manufacturing processes, etc.) we typically perform dexterous manipulations that involve multiple fingers and palm for both hands. Therefore, multi-Finger haptic methods can provide a realistic and natural human-machine interface to enhance immersion when interacting with simulated or remote environments. Most commercial devices allow haptic interaction with only one contact point, which may be sufficient for some exploration or palpation tasks but are not enough to perform advanced object manipulations such as grasping. In this thesis, I investigate the mechanical design, control and applications of a modular haptic device that can provide force feedback to the index, thumb and middle fingers of the user. The designed mechanical device is optimized with a multi-objective design function to achieve a low inertia, a large workspace, manipulability, and force-feedback of up to 3 N within the workspace; the bandwidth and rigidity for the device is assessed through simulation and real experimentation. One of the most important areas when designing haptic devices is the end-effector, since it is in contact with the user. In this thesis the design and evaluation of a thimble-like, lightweight, user-adaptable, and cost-effective device that incorporates four contact force sensors is described. This design allows estimation of the forces applied by a user during manipulation of virtual and real objects. The design of a real-time, modular control architecture for multi-finger haptic interaction is described. Requirements for control of multi-finger haptic devices are explored. Moreover, a large number of signals have to be acquired, processed, sent over the network and mathematical computations such as device direct and inverse kinematics, jacobian, grasp detection algorithms, etc. have to be calculated in Real Time to assure the required high fidelity for the haptic interaction. The Hardware control architecture has different modules and consists of an FPGA for the low-level controller and a RT controller for managing all the complex calculations (jacobian, kinematics, etc.); this provides a compact and scalable solution for the required high computation capabilities assuring a correct frequency rate for the control loop of 1 kHz. A set-up for dexterous virtual and real manipulation is described. Moreover, a new algorithm named the iterative kinematic decoupling method was implemented to solve the inverse kinematics of a robotic manipulator. In order to understand the importance of multi-modal interaction including haptics, a subject study was carried out to look for sensory stimuli that correlate with fast response time and enhanced accuracy. This experiment was carried out in collaboration with neuro-scientists from Technion Israel Institute of Technology. By comparing the grasping response times in unimodal (auditory, visual, and haptic) events with the response times in events with bimodal and trimodal combinations. It is concluded that in grasping tasks the synchronized motion of the fingers to generate the grasping response relies on haptic cues. This processing-speed advantage of haptic cues suggests that multimodalhaptic virtual environments are superior in generating motor contingencies, enhancing the plausibility of events. Applications that include haptics provide users with more time at the cognitive stages to fill in missing information creatively and form a richer experience. A major application of haptic devices is the design of new simulators to train manual skills for the medical sector. In collaboration with physical therapists from Griffith University in Australia, we developed a simulator to allow hand rehabilitation manipulations. First, the non-linear stiffness properties of the metacarpophalangeal joint of the index finger were estimated by using the designed end-effector; these parameters are implemented in a scenario that simulates the behavior of the human hand and that allows haptic interaction through the designed haptic device. The potential application of this work is related to educational and medical training purposes. In this thesis, new methods to simultaneously control the position and orientation of a robotic manipulator and the grasp of a robotic hand when interacting with large real environments are studied. The reachable workspace is extended by automatically switching between rate and position control modes. Moreover, the human hand gesture is recognized by reading the relative movements of the index, thumb and middle fingers of the user during the early stages of the approximation-to-the-object phase and then mapped to the robotic hand actuators. These methods are validated to perform dexterous manipulation of objects with a robotic manipulator, and different robotic hands. This work is the result of a research collaboration with researchers from the Harvard BioRobotics Laboratory. The developed experiments show that the overall task time is reduced and that the developed methods allow for full dexterity and correct completion of dexterous manipulations.