Adaptive Sleep-Wake Control using Reinforcement Learning in Sensor Networks

The aim in this paper is to allocate the `sleep time' of the individual sensors in an intrusion detection application so that the energy consumption from the sensors is reduced, while keeping the tracking error to a minimum. We propose two novel reinforcement learning (RL) based algorithms that attempt to minimize a certain long-run average cost objective. Both our algorithms incorporate feature-based representations to handle the curse of dimensionality associated with the underlying partially-observable Markov decision process (POMDP). Further, the feature selection scheme used in our algorithms intelligently manages the energy cost and tracking cost factors, which in turn assists the search for the optimal sleeping policy. We also extend these algorithms to a setting where the intruder's mobility model is not known by incorporating a stochastic iterative scheme for estimating the mobility model. The simulation results on a synthetic 2-d network setting are encouraging.

Design and Testing of a Control System for Supply Side Electrical Load Management

India's energy demand is increasing rapidly with the intensive growth of economy. The electricity demand in India exceeded the availability, both in terms of base load energy and peak availability. The efficient use of energy source and its conversion and utilizations are the viable alternatives available to the utilities or industry. There are essentially two approaches to electrical energy management. First at the supply / utility end (Supply Side Management or SSM) and the other at the consumer end (Demand Side Management or DSM). This work is based on Supply Side Management (SSM) protocol and consists of design, fabrication and testing of a control device that will be able to automatically regulate the power flow to an individual consumer's premise. This control device can monitor the overuse of electricity (above the connected load or contracted demand) by the individual consumers. The present project work specially emphasizes on contract demand of every consumer and tries to reduce the use beyond the contract demand. This control unit design includes both software and hardware work and designed for 0.5 kW contract demand. The device is tested in laboratory and reveals its potential use in the field.

Multi-agent Reinforcement Learning for Traffic Signal Control

Optimal control of traffic lights at junctions or traffic signal control (TSC) is essential for reducing the average delay experienced by the road users amidst the rapid increase in the usage of vehicles. In this paper, we formulate the TSC problem as a discounted cost Markov decision process (MDP) and apply multi-agent reinforcement learning (MARL) algorithms to obtain dynamic TSC policies. We model each traffic signal junction as an independent agent. An agent decides the signal duration of its phases in a round-robin (RR) manner using multi-agent Q-learning with either is an element of-greedy or UCB 3] based exploration strategies. It updates its Q-factors based on the cost feedback signal received from its neighbouring agents. This feedback signal can be easily constructed and is shown to be effective in minimizing the average delay of the vehicles in the network. We show through simulations over VISSIM that our algorithms perform significantly better than both the standard fixed signal timing (FST) algorithm and the saturation balancing (SAT) algorithm 15] over two real road networks.

Control of Large-size SiC Growth Process Using Modeling tool

Large size bulk silicon carbide (SiC) crystals are commonly grown by the physical vapor transport (PVT) method. The PVT growth of SiC crystals involves sublimation and condensation, chemical reactions, stoichiometry, mass transport, induced thermal stress, as well as defect and micropipes generation and propagation. The quality and polytype of as-grown SiC crystals are related to the temperature distribution inside the growth chamber during the growth process, it is critical to predict the temperature distribution from the measured temperatures outside the crucible by pyrometers. A radio-frequency induction-heating furnace was used for the growth of large-size SiC crystals by the PVT method in the present study. Modeling and simulation have been used to develop the SiC growth process and to improve the SiC crystal quality. Parameters such as the temperature measured at the top of crucible, temperature measured at the bottom of the crucible, and inert gas pressure are used to control the SiC growth process. By measuring the temperatures at the top and bottom of the crucible, the temperatures inside the crucible were predicted with the help of modeling tool. SiC crystals of 6H polytype were obtained and characterized by the Raman scattering spectroscopy and SEM, and crystals of few millimeter size grown inside the crucible were found without micropipes. Expansion of the crystals were also performed with the help of modeling and simulation.

Control computerizado mediante Kinect

[ES]En el presente trabajo se tratará de exponer el método de trabajo, el desarrollo y los posibles usos de este dispositivo como un método de control en un marco de uso industrial orientado a la economía

Desarrollo de un sistema de control remoto utilizando EPICS

[ES] El trabajo se centra en el estudio de un Freeware de sistemas distribuidos llamado EPICS, para la monitorización de un motor de paso y el uso de una cámara. Este proyecto consta de tres partes. Un primer capítulo está dedicado a introducir los conceptos básicos de EPICS que serán necesarios para el desarrollo del proyecto en su totalidad. Tales como la estructura de un sistema estándar EPICS, los conceptos de Process Variable, IOC, database, device support y driver support. También se hablará de las herramientas complementarias utilizadas para llevar a cabo el trabajo, como Python, C, C++ o AsynDriver. Una segunda parte dónde se describe en detalle el proceso llevado a cabo para crear las distintas aplicaciones para el motor y la cámara. En una tercera, se describe el proceso de como se han hecho los distintos entornos gráficos para cada una de las aplicaciones.