The actor-critic algorithm as multi-time-scale stochastic approximation

The actor-critic algorithm of Barto and others for simulation-based optimization of Markov decision processes is cast as a two time Scale stochastic approximation. Convergence analysis, approximation issues and an example are studied.

Process-kinetics in facing targets sputtering of multi-component oxide thin films

The kinetics of the processes in facing targets sputtering of multicomponent oxide films is presented. The novel configuration of the process exhibits an enhanced ionization efficiency. Discharge diagnostics performed using optical emission spectroscopy revealed strong dependence of plasma parameters on process conditions. Numerical simulation based on thermalization and diffusion of sputtered atoms has been performed to estimate the transport efficiency in off-axis mode. Composition, structure and epitaxial quality of YBa2Cu3O7-x films prepared was found to be strongly dependent on atomic flux ratios (of Cu/Y and Ba/Y) arriving at the substrate, resputtering effect and phase stability of YBa2Cu3O7-x These studies have been shown to be useful in understanding the complex processes that occur in sputtering of multicomponent films. (C) 1999 Elsevier Science S.A. All rights reserved.

Accelerating multi-core simulators

Simulation is an important means of evaluating new microarchitectures. With the invention of multi-core (CMP) platforms, simulators are becoming larger and more complex. However, with the availability of CMPs with larger caches and higher operating frequency, the wall clock time required for simulating an application has become comparatively shorter. Reducing this simulation time further is a great challenge, especially in the case of multi-threaded workload due to indeterminacy introduced due to simultaneously executing various threads. In this paper, we propose a technique for speeding multi-core simulation. The model of the processor core and cache are replaced with functional models, to achieve speedup. A timed Petri net model is used to estimate the execution time of the processor and the memory access latencies are estimated using hit/miss information obtained from the functional model of the cache. This model can be used to predict performance of data parallel applications or multiprogramming workload on CMP platform with various cache hierarchies and shared bus interconnect. The error in estimation of the execution time of an application is within 6%. The speedup achieved ranges between an average of 2x--4x over the cycle accurate simulator.

On optimal timer-based distributed selection for rate-adaptive multi-user diversity systems

We develop an optimal, distributed, and low feedback timer-based selection scheme to enable next generation rate-adaptive wireless systems to exploit multi-user diversity. In our scheme, each user sets a timer depending on its signal to noise ratio (SNR) and transmits a small packet to identify itself when its timer expires. When the SNR-to-timer mapping is monotone non-decreasing, timers of users with better SNRs expire earlier. Thus, the base station (BS) simply selects the first user whose timer expiry it can detect, and transmits data to it at as high a rate as reliably possible. However, timers that expire too close to one another cannot be detected by the BS due to collisions. We characterize in detail the structure of the SNR-to-timer mapping that optimally handles these collisions to maximize the average data rate. We prove that the optimal timer values take only a discrete set of values, and that the rate adaptation policy strongly influences the optimal scheme's structure. The optimal average rate is very close to that of ideal selection in which the BS always selects highest rate user, and is much higher than that of the popular, but ad hoc, timer schemes considered in the literature.

DMT of multi-hop cooperative networks

Some basic results that help in determining the Diversity-Multiplexing Tradeoff (DMT) of cooperative multihop networks are first identified. As examples, the maximum achievable diversity gain is shown to equal the min-cut between source and sink, whereas the maximal multiplexing gain is shown to equal the minimum rank of the matrix characterizing the MIMO channel appearing across a cut in the network. Two multi-hop generalizations of the two-hop network are then considered, namely layered networks as well as a class of networks introduced here and termed as K-parallel-path (KPP) networks. The DMT of KPP networks is characterized for K > 3. It is shown that a linear DMT between the maximum diversity dmax and the maximum multiplexing gain of 1 is achievable for fully-connected, layered networks. Explicit coding schemes achieving the DMT that make use of cyclic-division-algebra-based distributed space-time codes underlie the above results. Two key implications of the results in the paper are that the half-duplex constraint does not entail any rate loss for a large class of cooperative networks and that simple, amplify-and-forward protocols are often sufficient to attain the optimal DMT.

Twisted aromatics, 9-anthryl and 1-pyrenyl terpyridines organize into novel multi-directional 'ladder-like' motifs in the solid state

9-Anthryl and 1-pyrenyl terpyridines (1 and 2, respectively), key precursors for the design of novel fluorescent sensors have been synthesized and characterized by H-1 NMR, mass spectroscopy and X-ray crystallography. Twisted molecular conformations for each 1 and 2 were observed in their single crystal structures. Energy minimization calculations for the 1 and 2 using the semi-empirical AM1 method show that the 'twisted' conformation is intrinsic to these systems. We observe interconnected networks of edge-to-face CH...pi interactions, which appear to be cooperative in nature, in each of the crystal structures. The two twisted molecules, although having differently shaped polyaromatic hydrocarbon substituents, show similar patterns of edge-to-face CH...pi interactions.The presently described systems comprise of two aromatic surfaces that are almost orthogonal to each other. This twisted or orthogonal nature of the molecules leads to the formation of interesting multi-directional ladder like supramolecular organizations. A combination of edge-to-face and face-to-face packing modes helps to stabilize these motifs. The ladder like architecture in 1 is helical in nature. (C) 2002 Published by Elsevier Science B.V.

Multi-Armed Bandit Mechanisms for Multi-Slot Sponsored Searth Auctions

In pay-per click sponsored search auctions which are currently extensively used by search engines, the auction for a keyword involves a certain number of advertisers (say k) competing for available slots (say m) to display their ads. This auction is typically conducted for a number of rounds (say T). There are click probabilities mu_ij associated with agent-slot pairs. The search engine's goal is to maximize social welfare, for example, the sum of values of the advertisers. The search engine does not know the true value of an advertiser for a click to her ad and also does not know the click probabilities mu_ij s. A key problem for the search engine therefore is to learn these during the T rounds of the auction and also to ensure that the auction mechanism is truthful. Mechanisms for addressing such learning and incentives issues have recently been introduced and would be referred to as multi-armed-bandit (MAB) mechanisms. When m = 1,characterizations for truthful MAB mechanisms are available in the literature and it has been shown that the regret for such mechanisms will be O(T^{2/3}). In this paper, we seek to derive a characterization in the realistic but nontrivial general case when m > 1 and obtain several interesting results.

Strategies for Rescheduling Tightly-Coupled Parallel Applications in Multi-Cluster Grids

As computational Grids are increasingly used for executing long running multi-phase parallel applications, it is important to develop efficient rescheduling frameworks that adapt application execution in response to resource and application dynamics. In this paper, three strategies or algorithms have been developed for deciding when and where to reschedule parallel applications that execute on multi-cluster Grids. The algorithms derive rescheduling plans that consist of potential points in application execution for rescheduling and schedules of resources for application execution between two consecutive rescheduling points. Using large number of simulations, it is shown that the rescheduling plans developed by the algorithms can lead to large decrease in application execution times when compared to executions without rescheduling on dynamic Grid resources. The rescheduling plans generated by the algorithms are also shown to be competitive when compared to the near-optimal plans generated by brute-force methods. Of the algorithms, genetic algorithm yielded the most efficient rescheduling plans with 9-12% smaller average execution times than the other algorithms.

Analytical solutions for protective filters based on soil-retention and permeability criteria with respect to the phenomenon of soil boiling

For the successful performance of a granular filter medium, existing design guidelines, which are based on the particle size distribution (PSD) characteristics of the base soil and filter medium, require two contradictory conditions to be satisfied, viz., soil retention and permeability. In spite of the wider applicability of these guidelines, it is well recognized that (i) they are applicable to a particular range of soils tested in the laboratory, (ii) the design procedures do not include performance-based selection criteria, and (iii) there are no means to establish the sensitivity of the important variables influencing performance. In the present work, analytical solutions are developed to obtain a factor of safety with respect to soil-retention and permeability criteria for a base soil - filter medium system subjected to a soil boiling condition. The proposed analytical solutions take into consideration relevant geotechnical properties such as void ratio, permeability, dry unit weight, effective friction angle, shape and size of soil particles, seepage discharge, and existing hydraulic gradient. The solution is validated through example applications and experimental results, and it is established that it can be used successfully in the selection as well as design of granular filters and can be applied to all types of base soils.