Discrete-Time Controlled Markov Processes with Average Cost Criterion: A Survey

This work is a survey of the average cost control problem for discrete-time Markov processes. The authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades. The exposition ranges from finite to Borel state and action spaces and includes a variety of methodologies to find and characterize optimal policies. The authors have included a brief historical perspective of the research efforts in this area and have compiled a substantial yet not exhaustive bibliography. The authors have also identified several important questions that are still open to investigation.

Housing the skilled migrants in the Helsinki Metropolitan Area : International attractiveness of housing, neighbourhoods and the city region

The aim of this thesis is to examine the skilled migrants’ satisfaction with the Helsinki Metropolitan Area. The examination is executed on three scales: housing, neighbourhoods and the city region. Specific focus is on the built environment and how it meets the needs of the migrants. The empirical data is formed of 25 semi-structured interviews with skilled migrants and additionally 5 expert interviews. Skilled and educated workforce is an increasingly important resource in the new economy, and cities are competing globally for talented workers. With aging population and a need to develop its innovational structure, the Helsinki Metropolitan Area needs migrant workforce. It has been stated that quality of place is a central factor for skilled migrants when choosing where to settle, and from this perspective their satisfaction with the region is significant. In housing, the skilled migrants found the price-quality ratio and the general sizes of apartments inadequate. The housing market is difficult for the migrants to approach, since they often do not speak Finnish and there are prejudices towards foreigners. The general quality of housing was rated well. On the neighbourhood level, the skilled migrants had settled in residential areas which are also preferred by the Finnish skilled workers. While the migrants showed suburban orientation in their settlement patterns, they were not concentrated in the suburban areas which host large shares of traditional immigrant groups. Migrants were usually satisfied with their neighbourhoods; however, part of the suburban dwellers were unsatisfied with the services and social life in their neighbourhoods. Considering the level of the city region, the most challenging feature for the skilled migrants was the social life. The migrants felt that the social environment is homogeneous and difficult to approach. The physical environment was generally rated well, the most appreciated features being public transportation, human scale of the Metropolitan Helsinki, cleanliness, and the urban nature. Urban culture and services were seen good for the city region’s size, but lacking in international comparison.

Average lattices and aperiodic structures

Statistically averaged lattices provide a common basis to understand the diffraction properties of structures displaying deviations from regular crystal structures. An average lattice is defined and examples are given in one and two dimensions along with their diffraction patterns. The absence of periodicity in reciprocal space corresponding to aperiodic structures is shown to arise out of different projected spacings that are irrationally related, when the grid points are projected along the chosen coordinate axes. It is shown that the projected length scales are important factors which determine the existence or absence of observable periodicity in the diffraction pattern more than the sequence of arrangement.

Strong ground motion at Bhuj city during the Kutch earthquake

In the absence of near field strong motion records, the level of ground motion during the devastating 26 January 2001 earthquake has to be found by indirect means. For the city of Bhuj, three broad band velocity time histories have been recorded by India Meteorological Department. In this paper these data are processed to obtain an estimate of strong ground motion at Bhuj. It is estimated that the peak ground acceleration at Bhuj was of the order of 0.38 g. Ground motion in the surrounding region is indirectly found using available spectral response recorder (SRR) data. These instrument-based results are compared with analytical results obtained from a half-space regional model.

Trends in consanguineous marriage in Karnataka, South India, 1980–89

Analysis of data on 106,848 marriages in the cities of Bangalore and Mysore, South India, between 1980 and 1989 showed that levels of consanguineous marriage varied between cities through time and by religion. The average coefficient of inbreeding was higher in Bangalore (F = 0·0339) than in Mysore (F = 0·0203), principally reflecting large-scale, post-Independence rural migration into Bangalore. Although there was some evidence of a decline in consanguineous marriages in Mysore, there was no convincing support in either city for earlier projections of a rapid reduction in the popularity of unions between close biological relatives.

Seismic Hazard and Microzonation Studies using Geographic Information System - A Case Study of Bangalore city

Seismic hazard and microzonation of cities enable to characterize the potential seismic areas that need to be taken into account when designing new structures or retrofitting the existing ones. Study of seismic hazard and preparation of geotechnical microzonation maps has been attempted using Geographical Information System (GIS). GIS will provide an effective solution for integrating different layers of information thus providing a useful input for city planning and in particular input to earthquake resistant design of structures in an area. Seismic hazard is the study of expected earthquake ground motions at any point on the earth. Microzonation is the process of sub division of region in to number of zones based on the earthquake effects in the local scale. Seismic microzonation is the process of estimating response of soil layers under earthquake excitation and thus the variation of ground motion characteristic on the ground surface. For the seismic microzonation, geotechnical site characterization need to be assessed at local scale (micro level), which is further used to assess of the site response and liquefaction susceptibility of the sites. Seismotectonic atlas of the area having a radius of 350km around Bangalore has been prepared with all the seismogenic sources and historic earthquake events (a catalogue of about 1400 events since 1906). We have attempted to carryout the site characterization of Bangalore by collating conventional geotechnical boreholes data (about 900 borehole data with depth) and integrated in GIS. 3-D subsurface model of Bangalore prepared using GIS is shown in Figure 1.Further, Shear wave velocity survey based on geophysical method at about 60 locations in the city has been carried out in 220 square Kms area. Site response and local site effects have been evaluated using 1-dimensional ground response analysis. Spatial variability of soil overburden depths, ground surface Peak Ground Acceleration’s(PGA), spectral acceleration for different frequencies, liquefaction susceptibility have been mapped in the 220 sq km area using GIS.ArcInfo software has been used for this purpose. These maps can be used for the city planning and risk & vulnerability studies. Figure 2 shows a map of peak ground acceleration at rock level for Bangalore city. Microtremor experiments were jointly carried out with NGRI scientists at about 55 locations in the city and the predominant frequency of the overburden soil columns were evaluated.

Fault-Tolerant Average Execution Time Optimization for General Purpose Multi-Processor System-on Chips

Fault-tolerance is due to the semiconductor technology development important, not only for safety-critical systems but also for general-purpose (non-safety critical) systems. However, instead of guaranteeing that deadlines always are met, it is for general-purpose systems important to minimize the average execution time (AET) while ensuring fault-tolerance. For a given job and a soft (transient) error probability, we define mathematical formulas for AET that includes bus communication overhead for both voting (active replication) and rollback-recovery with checkpointing (RRC). And, for a given multi-processor system-on-chip (MPSoC), we define integer linear programming (ILP) models that minimize AET including bus communication overhead when: (1) selecting the number of checkpoints when using RRC, (2) finding the number of processors and job-to-processor assignment when using voting, and (3) defining fault-tolerance scheme (voting or RRC) per job and defining its usage for each job. Experiments demonstrate significant savings in AET.

An optimal weighted-average congestion based pricing scheme for enhanced QoS, , LNCS 4882, 2007

Pricing is an effective tool to control congestion and achieve quality of service (QoS) provisioning for multiple differentiated levels of service. In this paper, we consider the problem of pricing for congestion control in the case of a network of nodes under a single service class and multiple queues, and present a multi-layered pricing scheme. We propose an algorithm for finding the optimal state dependent price levels for individual queues, at each node. The pricing policy used depends on a weighted average queue length at each node. This helps in reducing frequent price variations and is in the spirit of the random early detection (RED) mechanism used in TCP/IP networks. We observe in our numerical results a considerable improvement in performance using our scheme over that of a recently proposed related scheme in terms of both throughput and delay performance. In particular, our approach exhibits a throughput improvement in the range of 34 to 69 percent in all cases studied (over all routes) over the above scheme.

Reinforcement learning with average cost for adaptive control of traffic lights at intersections

We propose for the first time two reinforcement learning algorithms with function approximation for average cost adaptive control of traffic lights. One of these algorithms is a version of Q-learning with function approximation while the other is a policy gradient actor-critic algorithm that incorporates multi-timescale stochastic approximation. We show performance comparisons on various network settings of these algorithms with a range of fixed timing algorithms, as well as a Q-learning algorithm with full state representation that we also implement. We observe that whereas (as expected) on a two-junction corridor, the full state representation algorithm shows the best results, this algorithm is not implementable on larger road networks. The algorithm PG-AC-TLC that we propose is seen to show the best overall performance.

Average variation L2-stability criteria for time-varying feedback systems-A unified approach

The problem of developing L2-stability criteria for feedback systems with a single time-varying gain, which impose average variation constraints on the gain is treated. A unified approach is presented which facilitates the development of such average variation criteria for both linear and nonlinear systems. The stability criteria derived here are shown to be more general than the existing results.

New Insights into Optimal Discrete Rate Adaptation for Average Power Constrained Single and Multi-Node Systems

The throughput-optimal discrete-rate adaptation policy, when nodes are subject to constraints on the average power and bit error rate, is governed by a power control parameter, for which a closed-form characterization has remained an open problem. The parameter is essential in determining the rate adaptation thresholds and the transmit rate and power at any time, and ensuring adherence to the power constraint. We derive novel insightful bounds and approximations that characterize the power control parameter and the throughput in closed-form. The results are comprehensive as they apply to the general class of Nakagami-m (m >= 1) fading channels, which includes Rayleigh fading, uncoded and coded modulation, and single and multi-node systems with selection. The results are appealing as they are provably tight in the asymptotic large average power regime, and are designed and verified to be accurate even for smaller average powers.

Ultrasound modulation of coherent light in a multiple-scattering medium: experimental verification of nonzero average phase carried by light

We demonstrate the phase fluctuation introduced by oscillation of scattering centers in the focal volume of an ultrasound transducer in an optical tomography experiment has a nonzero mean. The conditions to be met for the above are: (i) the frequency of the ultrasound should be in the vicinity of the most dominant natural frequency of vibration of the ultrasound focal volume, (ii) the corresponding acoustic wavelength should be much larger than l(n)*, a modified transport mean-free-path applicable for phase decorrelation and (iii) the focal volume of the ultrasound transducer should not be larger than 4 - 5 times (l(n)*)(3). We demonstrate through simulations that as the ratio of the ultrasound focal volume to (l(n)*)(3) increases, the average of the phase fluctuation decreases and becomes zero when the focal volume becomes greater than around 4(l(n)*)(3); and through simulations and experiments that as the acoustic frequency increases from 100 Hz to 1 MHz, the average phase decreases to zero. Through experiments done in chicken breast we show that the average phase increases from around 110 degrees to 130 degrees when the background medium is changed from water to glycerol, indicating that the average of the phase fluctuation can be used to sense changes in refractive index deep within tissue.