Reinforcement learning with average cost for adaptive control of traffic lights at intersections

We propose for the first time two reinforcement learning algorithms with function approximation for average cost adaptive control of traffic lights. One of these algorithms is a version of Q-learning with function approximation while the other is a policy gradient actor-critic algorithm that incorporates multi-timescale stochastic approximation. We show performance comparisons on various network settings of these algorithms with a range of fixed timing algorithms, as well as a Q-learning algorithm with full state representation that we also implement. We observe that whereas (as expected) on a two-junction corridor, the full state representation algorithm shows the best results, this algorithm is not implementable on larger road networks. The algorithm PG-AC-TLC that we propose is seen to show the best overall performance.

Average variation L2-stability criteria for time-varying feedback systems-A unified approach

The problem of developing L2-stability criteria for feedback systems with a single time-varying gain, which impose average variation constraints on the gain is treated. A unified approach is presented which facilitates the development of such average variation criteria for both linear and nonlinear systems. The stability criteria derived here are shown to be more general than the existing results.

New Insights into Optimal Discrete Rate Adaptation for Average Power Constrained Single and Multi-Node Systems

The throughput-optimal discrete-rate adaptation policy, when nodes are subject to constraints on the average power and bit error rate, is governed by a power control parameter, for which a closed-form characterization has remained an open problem. The parameter is essential in determining the rate adaptation thresholds and the transmit rate and power at any time, and ensuring adherence to the power constraint. We derive novel insightful bounds and approximations that characterize the power control parameter and the throughput in closed-form. The results are comprehensive as they apply to the general class of Nakagami-m (m >= 1) fading channels, which includes Rayleigh fading, uncoded and coded modulation, and single and multi-node systems with selection. The results are appealing as they are provably tight in the asymptotic large average power regime, and are designed and verified to be accurate even for smaller average powers.

Ultrasound modulation of coherent light in a multiple-scattering medium: experimental verification of nonzero average phase carried by light

We demonstrate the phase fluctuation introduced by oscillation of scattering centers in the focal volume of an ultrasound transducer in an optical tomography experiment has a nonzero mean. The conditions to be met for the above are: (i) the frequency of the ultrasound should be in the vicinity of the most dominant natural frequency of vibration of the ultrasound focal volume, (ii) the corresponding acoustic wavelength should be much larger than l(n)*, a modified transport mean-free-path applicable for phase decorrelation and (iii) the focal volume of the ultrasound transducer should not be larger than 4 - 5 times (l(n)*)(3). We demonstrate through simulations that as the ratio of the ultrasound focal volume to (l(n)*)(3) increases, the average of the phase fluctuation decreases and becomes zero when the focal volume becomes greater than around 4(l(n)*)(3); and through simulations and experiments that as the acoustic frequency increases from 100 Hz to 1 MHz, the average phase decreases to zero. Through experiments done in chicken breast we show that the average phase increases from around 110 degrees to 130 degrees when the background medium is changed from water to glycerol, indicating that the average of the phase fluctuation can be used to sense changes in refractive index deep within tissue.

A weighted average-based external clock synchronisation protocol for wireless sensor networks

Wireless Sensor Networks (WSNs) have many application scenarios where external clock synchronisation may be required because a WSN may consist of components which are not connected to each other. In this paper, we first propose a novel weighted average-based internal clock synchronisation (WICS) protocol, which synchronises all the clocks of a WSN with the clock of a reference node periodically. Based on this protocol, we then propose our weighted average-based external clock synchronisation (WECS) protocol. We have analysed the proposed protocols for maximum synchronisation error and shown that it is always upper bounded. Extensive simulation studies of the proposed protocols have been carried out using Castalia simulator. Simulation results validate our above theoretical claim and also show that the proposed protocols perform better in comparison to other protocols in terms of synchronisation accuracy. A prototype implementation of the WICS protocol using a few TelosB motes also validates the above conclusions.

A weighted average based external clock synchronization protocol for wireless sensor networks

Clock synchronization is an extremely important requirement of wireless sensor networks(WSNs). There are many application scenarios such as weather monitoring and forecasting etc. where external clock synchronization may be required because WSN itself may consists of components which are not connected to each other. A usual approach for external clock synchronization in WSNs is to synchronize the clock of a reference node with an external source such as UTC, and the remaining nodes synchronize with the reference node using an internal clock synchronization protocol. In order to provide highly accurate time, both the offset and the drift rate of each clock with respect to reference node are estimated from time to time, and these are used for getting correct time from local clock reading. A problem with this approach is that it is difficult to estimate the offset of a clock with respect to the reference node when drift rate of clocks varies over a period of time. In this paper, we first propose a novel internal clock synchronization protocol based on weighted averaging technique, which synchronizes all the clocks of a WSN to a reference node periodically. We call this protocol weighted average based internal clock synchronization(WICS) protocol. Based on this protocol, we then propose our weighted average based external clock synchronization(WECS) protocol. We have analyzed the proposed protocols for maximum synchronization error and shown that it is always upper bounded. Extensive simulation studies of the proposed protocols have been carried out using Castalia simulator. Simulation results validate our theoretical claim that the maximum synchronization error is always upper bounded and also show that the proposed protocols perform better in comparison to other protocols in terms of synchronization accuracy. A prototype implementation of the proposed internal clock synchronization protocol using a few TelosB motes also validates our claim.

Tradeoff of average power and average delay for a point-to-point link with fading

We consider a discrete time system with packets arriving randomly at rate lambda per slot to a fading point-to-point link, for which the transmitter can control the number of packets served in a slot by varying the transmit power. We provide an asymptotic characterization of the minimum average delay of the packets, when average transmitter power is a small positive quantity V more than the minimum average power required for queue stability. We show that the minimum average delay will grow either as log (1/V) or 1/V when V down arrow 0, for certain sets of values of lambda. These sets are determined by the distribution of fading gain, the maximum number of packets which can be transmitted in a slot, and the assumed transmit power function, as a function of the fading gain and the number of packets transmitted. We identify a case where the above behaviour of the tradeoff differs from that obtained from a previously considered model, in which the random queue length process is assumed to evolve on the non-negative real line.

Tradeoff of average service cost and average delay for the state dependent M/M/1 queue

The optimal tradeoff between average service cost rate and average delay, is addressed for a M/M/1 queueing model with queue-length dependent service rates, chosen from a finite set. We provide an asymptotic characterization of the minimum average delay, when the average service cost rate is a small positive quantity V more than the minimum average service cost rate required for stability. We show that depending on the value of the arrival rate, the assumed service cost rate function, and the possible values of the service rates, the minimum average delay either a) increases only to a finite value, b) increases without bound as log(1/V), or c) increases without bound as 1/V, when V down arrow 0. We apply the analysis to a flow-level resource allocation model for a wireless downlink. We also investigate the asymptotic tradeoff for a sequence of policies which are obtained from an approximate fluid model for the M/M/1 queue.

SEP-optimal antenna selection for average interference constrained underlay cognitive radios

In the underlay mode of cognitive radio, secondary users can transmit when the primary is transmitting, but under tight interference constraints, which limit the secondary system performance. Antenna selection (AS)-based multiple antenna techniques, which require less hardware and yet exploit spatial diversity, help improve the secondary system performance. In this paper, we develop the optimal transmit AS rule that minimizes the symbol error probability (SEP) of an average interference-constrained secondary system that operates in the underlay mode. We show that the optimal rule is a non-linear function of the power gains of the channels from secondary transmit antenna to primary receiver and secondary transmit antenna to secondary receive antenna. The optimal rule is different from the several ad hoc rules that have been proposed in the literature. We also propose a closed-form, tractable variant of the optimal rule and analyze its SEP. Several results are presented to compare the performance of the closed-form rule with the ad hoc rules, and interesting inter-relationships among them are brought out.

Fiber Bragg Grating Sensor Instrumentation for Parking Space Occupancy Management

This work aims at providing an effective parking management system by reducing the drivers' searching time for vacant car-parking space, in turn improving the traffic flow in the car park areas. This is achieved by the use of Fiber Bragg Grating Sensor (FBG) sensor instrumentation in vehicle parking management system. Present work involves embedding an array of FBG sensors underground in the parking space, then determining the strain changes on the FBG sensor due to load applied by the vehicle parked in the parking space, occupancy of the parking space is determined. To validate the FBG sensor parking management system, three most common cases have been considered. This closed loop FBG parking management system can give real-time feed-back to space-guidance display board helping the driver in maneuvering the vehicle to the appropriate parking space. The proposed technique offers optimized usage of parking space for the various segments of cars and also facilitates in a conjoined automated billing system, as compared to conventional method of parking systems.

Comparative Evaluation of Switching and Average Models of a DC-DC Boost Converter for Real-Time Simulation

This paper presents a comparative evaluation of the average and switching models of a dc-dc boost converter from the point of view of real-time simulation. Both the models are used to simulate the converter in real-time on a Field Programmable Gate Array (FPGA) platform. The converter is considered to function over a wide range of operating conditions, and could do transition between continuous conduction mode (CCM) and discontinuous conduction mode (DCM). While the average model is known to be computationally efficient from the perspective of off-line simulation, the same is shown here to consume more logical resources than the switching model for real-time simulation of the dc-dc converter. Further, evaluation of the boundary condition between CCM and DCM is found to be the main reason for the increased consumption of resources by the average model.

Local structural disorder and its influence on the average global structure and polar properties in Na0.5Bi0.5TiO3

Na0.5Bi0.5TiO3 (NBT) and its derivatives have prompted a great surge in interest owing to their potential as lead-free piezoelectrics. In spite of five decades since its discovery, there is still a lack of clarity on crucial issues such as the origin of significant dielectric relaxation at room temperature, structural factors influencing its depoling, and the status of the recently proposed monoclinic (Cc) structure vis-a-vis the nanosized structural heterogeneities. In this work, these issues are resolved by comparative analysis of local and global structures on poled and unpoled NBT specimens using electron, x-ray, and neutron diffraction in conjunction with first-principles calculation, dielectric, ferroelectric, and piezoelectric measurements. The reported global monoclinic (Cc) distortion is shown not to correspond to the thermodynamic equilibrium state at room temperature. The global monocliniclike appearance rather owes its origin to the presence of local structural and strain heterogeneities. Poling removes the structural inhomogeneities and establishes a long-range rhombohedral distortion. In the process the system gets irreversibly transformed from a nonergodic relaxor to a normal ferroelectric state. The thermal depoling is shown to be associated with the onset of incompatible in-phase tilted octahedral regions in the field-stabilized long range rhombohedral distortion.

Zero-Sum Stochastic Games with Partial Information and Average Payoff

We consider a discrete time partially observable zero-sum stochastic game with average payoff criterion. We study the game using an equivalent completely observable game. We show that the game has a value and also we present a pair of optimal strategies for both the players.

Average Modelling of a Voltage Source Inverter with Dead-Time in a Synchronous Reference Frame

Voltage source inverters are an integral part of renewable power sources and smart grid systems. Computationally efficient and fairly accurate models for the voltage source inverter are required to carry out extensive simulation studies on complex power networks. Accuracy requires that the effect of dead-time be incorporated in the inverter model. The dead-time is essentially a short delay introduced between the gating pulses to the complementary switches in an inverter leg for the safety of power devices. As the modern voltage source inverters switch at fairly high frequencies, the dead-time significantly influences the output fundamental voltage. Dead-time also causes low-frequency harmonic distortion and is hence important from a power quality perspective. This paper studies the dead-time effect in a synchronous dq reference frame, since dynamic studies and controller design are typically carried out in this frame of reference. For the sake of computational efficiency, average models are derived, incorporating the dead-time effect, in both RYB and dq reference frames. The average models are shown to consume less computation time than their corresponding switching models, the accuracies of the models being comparable. The proposed average synchronous reference frame model, including effect of dead-time, is validated through experimental results.

Structural, Magnetic, and Electrical Properties of Microwave-Sintered Cr3+-Doped Sr Hexaferrites

SrCrxFe12-xO19 (x = 0.0, 0.1, 0.3, 0.5, 0.7, 0.9) hexaferrites were prepared by a microwave-hydrothermal method and subsequently sintered at 950 degrees C for 90 min using the microwave sintering method. The results show that, with increasing Cr3+ content, the lattice parameters changed anisotropically. The average grain sizes of sintered samples were in the range of 280 nm to 660 nm. The saturation magnetization systematically decreased with increasing Cr3+ doping, but the coercivity values increased. The electrical resistivity (log rho) decreased linearly with increasing temperature up to a certain temperature known as the transition temperature (T-c), and T-c decreased with further increase (x>0.5) of the Cr3+ content. This decrease in log rho and the activation energy (E-g) is due to electron hopping and occupancy of doped ions at different lattice sites. We found that the dielectric constant and dielectric loss for all the samples decreased with the Cr3+ content. The structural, magnetic, and electrical properties of Cr3+-doped SrFe12O19 hexaferrites have thus been investigated.