Discrete-Time Controlled Markov Processes with Average Cost Criterion: A Survey

This work is a survey of the average cost control problem for discrete-time Markov processes. The authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades. The exposition ranges from finite to Borel state and action spaces and includes a variety of methodologies to find and characterize optimal policies. The authors have included a brief historical perspective of the research efforts in this area and have compiled a substantial yet not exhaustive bibliography. The authors have also identified several important questions that are still open to investigation.

A discrete state perspective of manipulator workspaces Vorstellungen von diskreten zuständen von manipulator-arbeitsräumen

The finite resolution of joint drives or sensors imparts a discrete nature to the joints of a manipulator. Because of this an arbitrary point in the workspace cannot be reached without error even in ideal mechanical environment. This paper investigates the effect of this discrete nature of the joints on the accuracy of performance of a manipulator and develops a method to select the joint states to reach a point with least error. It is shown that the configuration leading to least error cannot, in general, be found from configuration space, especially when there is large variation in the link lengths or joint resolutions or both. The anomaly becomes severe when the gross motion of the end-effector approaches the local resolution of the workspace. The paper also shows how to distinguish two workspaces which may be identical so far as the boundary points are concerned, taking the joint resolutions into account. Finally, the concepts have been extended to define continuous space global and local performance indices for general multi degree of freedom manipulators.

Active vibration suppression of beams with discrete actuators using optimal dynamic inversion

Most of the structural elements like beams, cables etc. are flexible and should be modeled as distributed parameter systems (DPS) to represent the reality better. For large structures, the usual approach of 'modal representation' is not an accurate representation. Moreover, for excessive vibrations (possibly due to strong wind, earthquake etc.), external power source (controller) is needed to suppress it, as the natural damping of these structures is usually small. In this paper, we propose to use a recently developed optimal dynamic inversion technique to design a set of discrete controllers for this purpose. We assume that the control force to the structure is applied through finite number of actuators, which are located at predefined locations in the spatial domain. The method used in this paper determines control forces directly from the partial differential equation (PDE) model of the system. The formulation has better practical significance, both because it leads to a closed form solution of the controller (hence avoids computational issues) as well as because a set of discrete actuators along the spatial domain can be implemented with relative ease (as compared to a continuous actuator)

Instability, intermittency, and multiscaling in discrete growth models of kinetic roughening

We show by numerical simulations that discretized versions of commonly studied continuum nonlinear growth equations (such as the Kardar-Parisi-Zhangequation and the Lai-Das Sarma-Villain equation) and related atomistic models of epitaxial growth have a generic instability in which isolated pillars (or grooves) on an otherwise flat interface grow in time when their height (or depth) exceeds a critical value. Depending on the details of the model, the instability found in the discretized version may or may not be present in the truly continuum growth equation, indicating that the behavior of discretized nonlinear growth equations may be very different from that of their continuum counterparts. This instability can be controlled either by the introduction of higher-order nonlinear terms with appropriate coefficients or by restricting the growth of pillars (or grooves) by other means. A number of such ''controlled instability'' models are studied by simulation. For appropriate choice of the parameters used for controlling the instability, these models exhibit intermittent behavior, characterized by multiexponent scaling of height fluctuations, over the time interval during which the instability is active. The behavior found in this regime is very similar to the ''turbulent'' behavior observed in recent simulations of several one- and two-dimensional atomistic models of epitaxial growth.

Some algorithms for discrete time queues with finite capacity

We consider a discrete time queue with finite capacity and i.i.d. and Markov modulated arrivals, Efficient algorithms are developed to calculate the moments and the distributions of the first time to overflow and the regeneration length, Results are extended to the multiserver queue. Some illustrative numerical examples are provided.

Fluorescence and visual sensing of nitroaromatic explosives using electron rich discrete fluorophores

Several pi-electron rich fluorescent aromatic compounds containing trimethylsilylethynyl functionality have been synthesized by employing Sonogashira coupling reaction and they were characterized fully by NMR (H-1, C-13)/IR spectroscopy. Incorporation of bulky trimethylsilylethynyl groups on the peripheral of the fluorophores prevents self-quenching of the initial intensity through pi-pi interaction and thereby maintains the spectroscopic stability in solution. These compounds showed fluorescence behavior in chloroform solution and were used as selective fluorescence sensors for the detection of electron deficient nitroaromatics. All these fluorophores showed the largest quenching response with high selectivity for nitroaromatics among the various electron deficient aromatic compounds tested. Quantitative analysis of the fluorescence titration profile of 9,10-bis(trimethylsilylethynyl) anthracene with picric acid provided evidence that this particular fluorophore detects picric acid even at ppb level. A sharp visual detection of 2,4,6-trinitrotoluene was observed upon subjecting 1,3,6,8-tetrakis (trimethylsilylethynyl) pyrene fluorophore to increasing quantities of 2,4,6-trinitrotoluene in chloroform. Furthermore, thin film of the fluorophores was made by spin coating of a solution of 1.0 x 10(-3) M in chloroform or dichloromethane on a quartz plate and was used for the detection of vapors of nitroaromatics at room temperature. The vapor-phase sensing experiments suggested that the sensing process is reproducible and quite selective for nitroaromatic compounds. Selective fluorescence quenching response including a sharp visual color change for nitroaromatics makes these fluorophores as promising fluorescence sensory materials for nitroaromatic compounds (NAC) with a detection limit of even ppb level as judged with picric acid.

Precoding with X-codes to increase capacity with discrete input alphabets

We consider Gaussian multiple-input multiple-output (MIMO) channels with discrete input alphabets. We propose a non-diagonal precoder based on X-Codes in to increase the mutual information. The MIMO channel is transformed into a set of parallel subchannels using Singular Value Decomposition (SVD) and X-codes are then used to pair the subchannels. X-Codes are fully characterized by the pairings and the 2 × 2 real rotation matrices for each pair (parameterized with a single angle). This precoding structure enables to express the total mutual information as a sum of the mutual information of all the pairs. The problem of finding the optimal precoder with the above structure, which maximizes the total mutual information, is equivalent to i) optimizing the rotation angle and the power allocation within each pair and ii) finding the optimal pairing and power allocation among the pairs. It is shown that the mutual information achieved with the proposed pairing scheme is very close to that achieved with the optimal precoder by Cruz et al., and significantly better than mercury/waterfilling strategy by Lozano et al.. Our approach greatly simplifies both the precoder optimization and the detection complexity, making it suitable for practical applications.

Precoding by Pairing Subchannels to Increase MIMO Capacity With Discrete Input Alphabets

We consider Gaussian multiple-input multiple-output (MIMO) channels with discrete input alphabets. We propose a non-diagonal precoder based on the X-Codes in 1] to increase the mutual information. The MIMO channel is transformed into a set of parallel subchannels using singular value decomposition (SVD) and X-Codes are then used to pair the subchannels. X-Codes are fully characterized by the pairings and a 2 x 2 real rotation matrix for each pair (parameterized with a single angle). This precoding structure enables us to express the total mutual information as a sum of the mutual information of all the pairs. The problem of finding the optimal precoder with the above structure, which maximizes the total mutual information, is solved by: i) optimizing the rotation angle and the power allocation within each pair and ii) finding the optimal pairing and power allocation among the pairs. It is shown that the mutual information achieved with the proposed pairing scheme is very close to that achieved with the optimal precoder by Cruz et al., and is significantly better than Mercury/waterfilling strategy by Lozano et al. Our approach greatly simplifies both the precoder optimization and the detection complexity, making it suitable for practical applications.

Localisation of discrete change in a transformer winding: a network-function-loci approach

An entirely different approach for localisation of winding deformation based on terminal measurements is presented. Within the context of this study, winding deformation means, a discrete and specific change externally imposed at a particular position on the winding. The proposed method is based on pre-computing and plotting the complex network-function loci e.g. driving-point impedance (DPI)] at a selected frequency, for a meaningful range of values for each element (increasing and decreasing) of the ladder network which represents the winding. This loci diagram is called the nomogram. After introducing a discrete change, amplitude and phase of DPI are measured. By plotting this single measurement on the nomogram, it is possible to estimate the location and identify the extent of change. In contrast to the existing approach, the proposed method is fast, non-iterative and yields reasonably good localisation. Experimental results for actual transformer windings (interleaved and continuous disc) are presented.

PL due to Discrete Gap Levels in Some Chalcogenide Glasses- A Configurational Coordinate Diagram Illustration

Photoluminescence (PL) studies were carried out on a-Se and a few Ge20Se80−xBix and Ge20Se70−xBixTe10 bulk glassy semiconductors at 4.2 K with Ar+ laser as excitation source. While a-Se and samples with lesser at% of Bi show fine structured PL with a large Stokes shift, samples with higher at% of Bi did not show any detectable PL. The investigations show at least three radiative recombination transitions. Features extracted by deconvoluting the experimental spectra show that the discrete gap levels associated with the inherent coordination defects are involved in the PL transitions. Absence of PL in samples with higher Bi at% are explained on the basis of nonradiative transition mechanisms. Overall PL mechanism involving gap levels in chalcogenide glasses is illustrated with the help of a configurational coordinate diagram.

A Nash Bargaining Approach to Retention Enhancing Bid Optimization in Sponsored Search Auctions with Discrete Bids

Bid optimization is now becoming quite popular in sponsored search auctions on the Web. Given a keyword and the maximum willingness to pay of each advertiser interested in the keyword, the bid optimizer generates a profile of bids for the advertisers with the objective of maximizing customer retention without compromising the revenue of the search engine. In this paper, we present a bid optimization algorithm that is based on a Nash bargaining model where the first player is the search engine and the second player is a virtual agent representing all the bidders. We make the realistic assumption that each bidder specifies a maximum willingness to pay values and a discrete, finite set of bid values. We show that the Nash bargaining solution for this problem always lies on a certain edge of the convex hull such that one end point of the edge is the vector of maximum willingness to pay of all the bidders. We show that the other endpoint of this edge can be computed as a solution of a linear programming problem. We also show how the solution can be transformed to a bid profile of the advertisers.