An Actor-Critic Algorithm for Finite Horizon Markov Decision Processes

We develop a simulation based algorithm for finite horizon Markov decision processes with finite state and finite action space. Illustrative numerical experiments with the proposed algorithm are shown for problems in flow control of communication networks and capacity switching in semiconductor fabrication.

LMS versus Wiener filter for a decision feedback equalizer

In this paper we study an LMS-DFE. We use the ODE framework to show that the LMS-DFE attractors are close to the true DFE Wiener filter (designed considering the decision errors) at high SNR. Therefore, via LMS one can obtain a computationally efficient way to obtain the true DFE Wiener filter under high SNR. We also provide examples to show that the DFE filter so obtained can significantly outperform the usual DFE Wiener filter (designed assuming perfect decisions) at all practical SNRs. In fact, the performance improvement is very significant even at high SNRs (up to 50%), where the popular Wiener filter designed with perfect decisions, is believed to be closer to the optimal one.

Spatial Decision Support System for Assessing Micro, Mini and Small Hydel Potential

This study describes the design and implementation of DSS for assessment of Mini, Micro and Small Schemes. The design links a set of modelling, manipulation, spatial analyses and display tools to a structured database that has the facility to store both observed and simulated data. The main hypothesis is that this tool can be used to form a core of practical methodology that will result in more resilient in less time and can be used by decision-making bodies to assess the impacts of various scenarios (e.g.: changes in land use pattern) and to review, cost and benefits of decisions to be made. It also offers means of entering, accessing and interpreting the information for the purpose of sound decision making. Thus, the overall objective of this DSS is the development of set of tools aimed at transforming data into information and aid decisions at different scales.

Decision support system for regional electricity planning

Electricity appears to be the energy carrier of choice for modern economics since growth in electricity has outpaced growth in the demand for fuels. A decision maker (DM) for accurate and efficient decisions in electricity distribution requires the sector wise and location wise electricity consumption information to predict the requirement of electricity. In this regard, an interactive computer-based Decision Support System (DSS) has been developed to compile, analyse and present the data at disaggregated levels for regional energy planning. This helps in providing the precise information needed to make timely decisions related to transmission and distribution planning leading to increased efficiency and productivity. This paper discusses the design and implementation of a DSS, which facilitates to analyse the consumption of electricity at various hierarchical levels (division, taluk, sub division, feeder) for selected periods. This DSS is validated with the data of transmission and distribution systems of Kolar district in Karnataka State, India.

Bounds on Probability of Error in Decision Feedback Equalizers

Evaluation of the probability of error in decision feedback equalizers is difficult due to the presence of a hard limiter in the feedback path. This paper derives the upper and lower bounds on the probability of a single error and multiple error patterns. The bounds are fairly tight. The bounds can also be used to select proper tap gains of the equalizer.

Decision tree design and applications in speech processing

The design and operation of the minimum cost classifier, where the total cost is the sum of the measurement cost and the classification cost, is computationally complex. Noting the difficulties associated with this approach, decision tree design directly from a set of labelled samples is proposed in this paper. The feature space is first partitioned to transform the problem to one of discrete features. The resulting problem is solved by a dynamic programming algorithm over an explicitly ordered state space of all outcomes of all feature subsets. The solution procedure is very general and is applicable to any minimum cost pattern classification problem in which each feature has a finite number of outcomes. These techniques are applied to (i) voiced, unvoiced, and silence classification of speech, and (ii) spoken vowel recognition. The resulting decision trees are operationally very efficient and yield attractive classification accuracies.

Validation of Decision Tables Used in Process Control

Process control rules may be specified using decision tables. Such a specification is superior when logical decisions to be taken in control dominate. In this paper we give a method of detecting redundancies, incompleteness, and contradictions in such specifications. Using such a technique thus ensures the validity of the specifications.

Comparative statistical aging studies on an oil-pressboard insulation model

Accelerated aging experiments have been conducted on a representative oil-pressboard insulation model to investigate the effect of constant and sequential stresses on the PD behavior using a built-in phase resolved partial discharge analyzer. A cycle of the applied voltage starting from the zero of the positive half cycle was divided into 16 equal phase windows (Φ1 to Φ16) and partial discharge (PD) magnitude distribution in each phase was determined. Based on the experimental results, three stages of aging mechanism were identified. Gumbel's extreme value distribution of the largest element was used to model the first stage of aging process. Second and subsequent stages were modeled using two-parameter Weibull distribution. Spearman's non-parametric rank correlation test statistic and Kolmogrov-Smirnov two sample test were used to relate the aging process of each phase with the corresponding process of the full cycle. To bring out clearly the effect of stress level, its duration and test procedure on the distribution parameters and hence of the aging process, non-parametric ANOVA techniques like Kruskal-Wallis and Fisher's LSD multiple comparison tests were used. Results of the analysis show that two phases (Φ13 and Φ14) near the vicinity of the negative voltage peak were found to contribute significantly to the aging process and their aging mechanism also correlated well with that of the corresponding full cycle mechanism. Attempts have been made to relate these results with the published work of other workers

Geometric Decision Tree

In this paper, we present a new algorithm for learning oblique decision trees. Most of the current decision tree algorithms rely on impurity measures to assess the goodness of hyperplanes at each node while learning a decision tree in top-down fashion. These impurity measures do not properly capture the geometric structures in the data. Motivated by this, our algorithm uses a strategy for assessing the hyperplanes in such a way that the geometric structure in the data is taken into account. At each node of the decision tree, we find the clustering hyperplanes for both the classes and use their angle bisectors as the split rule at that node. We show through empirical studies that this idea leads to small decision trees and better performance. We also present some analysis to show that the angle bisectors of clustering hyperplanes that we use as the split rules at each node are solutions of an interesting optimization problem and hence argue that this is a principled method of learning a decision tree.

Statistical properties of entropy-consuming fluctuations in jammed states of laponite suspensions: Fluctuation relations and generalized Gumbel distribution

We report a universal large deviation behavior of spatially averaged global injected power just before the rejuvenation of the jammed state formed by an aging suspension of laponite clay under an applied stress. The probability distribution function (PDF) of these entropy consuming strongly non-Gaussian fluctuations follow an universal large deviation functional form described by the generalized Gumbel (GG) distribution like many other equilibrium and nonequilibrium systems with high degree of correlations but do not obey the Gallavotti-Cohen steady-state fluctuation relation (SSFR). However, far from the unjamming transition (for smaller applied stresses) SSFR is satisfied for both Gaussian as well as non-Gaussian PDF. The observed slow variation of the mean shear rate with system size supports a recent theoretical prediction for observing GG distribution.

Estimation of the shear transformation zone size fin a bulk metallic glass through statistical a analysis of the first pop-in stresses during spherical nanoindentation

The size of the shear transformation zone (STZ) that initiates the elastic to plastic transition in a Zr-based bulk metallic glass was estimated by conducting a statistical analysis of the first pop-in event during spherical nanoindentation. A series of experiments led us to a successful description of the distribution of shear strength for the transition and its dependence on the loading rate. From the activation volume determined by statistical analysis the STZ size was estimated based on a cooperative shearing model. (C) 2012 Acta Materialia Inc. Published by Elsevier Ltd. All rights reserved.

An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.