An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.

Stochastic optimization for adaptive labor staffing in service systems

Service systems are labor intensive. Further, the workload tends to vary greatly with time. Adapting the staffing levels to the workloads in such systems is nontrivial due to a large number of parameters and operational variations, but crucial for business objectives such as minimal labor inventory. One of the central challenges is to optimize the staffing while maintaining system steady-state and compliance to aggregate SLA constraints. We formulate this problem as a parametrized constrained Markov process and propose a novel stochastic optimization algorithm for solving it. Our algorithm is a multi-timescale stochastic approximation scheme that incorporates a SPSA based algorithm for ‘primal descent' and couples it with a ‘dual ascent' scheme for the Lagrange multipliers. We validate this optimization scheme on five real-life service systems and compare it with a state-of-the-art optimization tool-kit OptQuest. Being two orders of magnitude faster than OptQuest, our scheme is particularly suitable for adaptive labor staffing. Also, we observe that it guarantees convergence and finds better solutions than OptQuest in many cases.

Loop transformations: convexity, pruning, and optimization

High-level loop transformations are a key instrument in mapping computational kernels to effectively exploit the resources in modern processor architectures. Nevertheless, selecting required compositions of loop transformations to achieve this remains a significantly challenging task; current compilers may be off by orders of magnitude in performance compared to hand-optimized programs. To address this fundamental challenge, we first present a convex characterization of all distinct, semantics-preserving, multidimensional affine transformations. We then bring together algebraic, algorithmic, and performance analysis results to design a tractable optimization algorithm over this highly expressive space. Our framework has been implemented and validated experimentally on a representative set of benchmarks running on state-of-the-art multi-core platforms.

Adaptive power optimization of on-chip SNUCA cache on tiled chip multicore architecture using remap policy

Advances in technology have increased the number of cores and size of caches present on chip multicore platforms(CMPs). As a result, leakage power consumption of on-chip caches has already become a major power consuming component of the memory subsystem. We propose to reduce leakage power consumption in static nonuniform cache architecture(SNUCA) on a tiled CMP by dynamically varying the number of cache slices used and switching off unused cache slices. A cache slice in a tile includes all cache banks present in that tile. Switched-off cache slices are remapped considering the communication costs to reduce cache usage with minimal impact on execution time. This saves leakage power consumption in switched-off L2 cache slices. On an average, there map policy achieves 41% and 49% higher EDP savings compared to static and dynamic NUCA (DNUCA) cache policies on a scalable tiled CMP, respectively.

Quasi-based hierarchical clustering for land cover mapping using satellite images

This paper presents an improved hierarchical clustering algorithm for land cover mapping problem using quasi-random distribution. Initially, Niche Particle Swarm Optimization (NPSO) with pseudo/quasi-random distribution is used for splitting the data into number of cluster centers by satisfying Bayesian Information Criteria (BIC). Themain objective is to search and locate the best possible number of cluster and its centers. NPSO which highly depends on the initial distribution of particles in search space is not been exploited to its full potential. In this study, we have compared more uniformly distributed quasi-random with pseudo-random distribution with NPSO for splitting data set. Here to generate quasi-random distribution, Faure method has been used. Performance of previously proposed methods namely K-means, Mean Shift Clustering (MSC) and NPSO with pseudo-random is compared with the proposed approach - NPSO with quasi distribution(Faure). These algorithms are used on synthetic data set and multi-spectral satellite image (Landsat 7 thematic mapper). From the result obtained we conclude that use of quasi-random sequence with NPSO for hierarchical clustering algorithm results in a more accurate data classification.

Simulation of length-preserving motions of flexible one dimensional oObjects using optimization

During the motion of one dimensional flexible objects such as ropes, chains, etc., the assumption of constant length is realistic. Moreover,their motion appears to be naturally minimizing some abstract distance measure, wherein the disturbance at one end gradually dies down along the curve defining the object. This paper presents purely kinematic strategies for deriving length-preserving transformations of flexible objects that minimize appropriate ‘motion’. The strategies involve sequential and overall optimization of the motion derived using variational calculus. Numerical simulations are performed for the motion of a planar curve and results show stable converging behavior for single-step infinitesimal and finite perturbations 1 as well as multi-step perturbations. Additionally, our generalized approach provides different intuitive motions for various problem-specific measures of motion, one of which is shown to converge to the conventional tractrix-based solution. Simulation results for arbitrary shapes and excitations are also included.

Multiobjective discrete particle swarm optimization for multisensor image alignment

A new technique is proposed for multisensor image registration by matching the features using discrete particle swarm optimization (DPSO). The feature points are first extracted from the reference and sensed image using improved Harris corner detector available in the literature. From the extracted corner points, DPSO finds the three corresponding points in the sensed and reference images using multiobjective optimization of distance and angle conditions through objective switching technique. By this, the global best matched points are obtained which are used to evaluate the affine transformation for the sensed image. The performance of the image registration is evaluated and concluded that the proposed approach is efficient.

Integrated sustainable irrigation planning with multiobjective Fuzzy optimization approach

Multiobjective fuzzy methodology is applied to a case study of Khadakwasla complex irrigation project located near Pune city of Maharashtra State, India. Three objectives, namely, maximization of net benefits, crop production and labour employment are considered. Effect of reuse of wastewater on the planning scenario is also studied. Three membership functions, namely, nonlinear, hyperbolic and exponential are analyzed for multiobjective fuzzy optimization. In the present study, objective functions are considered as fuzzy in nature whereas inflows are considered as dependable. It is concluded that exponential and hyperbolic membership functions provided similar cropping pattern for most of the situations whereas nonlinear membership functions provided different cropping pattern. However, in all the three cases, irrigation intensities are more than the existing irrigation intensity.

Optimal synthesis of adjustable planar four-bar crank-rocker type mechanisms for approximate multi-path generation

This paper deals with an optimization based method for synthesis of adjustable planar four-bar, crank-rocker mechanisms. For multiple different and desired paths to be traced by a point on the coupler, a two stage method first determines the parameters of the possible driving dyads. Then the remaining mechanism parameters are determined in the second stage where a least-squares based circle-fitting procedure is used. Compared to existing formulations, the optimization method uses less number of design variables. Two numerical examples demonstrate the effectiveness of the proposed synthesis method. (C) 2013 Elsevier Ltd. All rights reserved.

Optimal capacity relay node placement in a multi-hop network on a line

We use information theoretic achievable rate formulas for the multi-relay channel to study the problem of optimal placement of relay nodes along the straight line joining a source node and a destination node. The achievable rate formulas that we utilize are for full-duplex radios at the relays and decode-and-forward relaying. For the single relay case, and individual power constraints at the source node and the relay node, we provide explicit formulas for the optimal relay location and the optimal power allocation to the source-relay channel, for the exponential and the power-law path-loss channel models. For the multiple relay case, we consider exponential path-loss and a total power constraint over the source and the relays, and derive an optimization problem, the solution of which provides the optimal relay locations. Numerical results suggest that at low attenuation the relays are mostly clustered close to the source in order to be able to cooperate among themselves, whereas at high attenuation they are uniformly placed and work as repeaters. We also prove that a constant rate independent of the attenuation in the network can be achieved by placing a large enough number of relay nodes uniformly between the source and the destination, under the exponential path-loss model with total power constraint.

Role of melt convection on optimization of PCM-based heat sink under cyclic heat load

In this article, we study the thermal performance of phase-change material (PCM)-based heat sinks under cyclic heat load and subjected to melt convection. Plate fin type heat sinks made of aluminum and filled with PCM are considered in this study. The heat sink is heated from the bottom. For a prescribed value of heat flux, design of such a heat sink can be optimized with respect to its geometry, with the objective of minimizing the temperature rise during heating and ensuring complete solidification of PCM at the end of the cooling period for a given cycle. For given length and base plate thickness of a heat sink, a genetic algorithm (GA)-based optimization is carried out with respect to geometrical variables such as fin thickness, fin height, and the number of fins. The thermal performance of the heat sink for a given set of parameters is evaluated using an enthalpy-based heat transfer model, which provides the necessary data for the optimization algorithm. The effect of melt convection is studied by taking two cases, one without melt convection (conduction regime) and the other with convection. The results show that melt convection alters the results of geometrical optimization.

Cooperative Communication of UAV to Perform Multi-task Using Nature Inspired Techniques

In this paper, the approach for assigning cooperative communication of Uninhabited Aerial Vehicles (UAV) to perform multiple tasks on multiple targets is posed as a combinatorial optimization problem. The multiple task such as classification, attack and verification of target using UAV is employed using nature inspired techniques such as Artificial Immune System (AIS), Particle Swarm Optimization (PSO) and Virtual Bee Algorithm (VBA). The nature inspired techniques have an advantage over classical combinatorial optimization methods like prohibitive computational complexity to solve this NP-hard problem. Using the algorithms we find the best sequence in which to attack and destroy the targets while minimizing the total distance traveled or the maximum distance traveled by an UAV. The performance analysis of the UAV to classify, attack and verify the target is evaluated using AIS, PSO and VBA.

The relationship between classification of multi-domain proteins using an alignment-free approach and their functions: a case study with immunoglobulins

Establishing functional relationships between multi-domain protein sequences is a non-trivial task. Traditionally, delineating functional assignment and relationships of proteins requires domain assignments as a prerequisite. This process is sensitive to alignment quality and domain definitions. In multi-domain proteins due to multiple reasons, the quality of alignments is poor. We report the correspondence between the classification of proteins represented as full-length gene products and their functions. Our approach differs fundamentally from traditional methods in not performing the classification at the level of domains. Our method is based on an alignment free local matching scores (LMS) computation at the amino-acid sequence level followed by hierarchical clustering. As there are no gold standards for full-length protein sequence classification, we resorted to Gene Ontology and domain-architecture based similarity measures to assess our classification. The final clusters obtained using LMS show high functional and domain architectural similarities. Comparison of the current method with alignment based approaches at both domain and full-length protein showed superiority of the LMS scores. Using this method we have recreated objective relationships among different protein kinase sub-families and also classified immunoglobulin containing proteins where sub-family definitions do not exist currently. This method can be applied to any set of protein sequences and hence will be instrumental in analysis of large numbers of full-length protein sequences.

Automatic Data Allocation and Buffer Management for Multi-GPU Machines

Multi-GPU machines are being increasingly used in high-performance computing. Each GPU in such a machine has its own memory and does not share the address space either with the host CPU or other GPUs. Hence, applications utilizing multiple GPUs have to manually allocate and manage data on each GPU. Existing works that propose to automate data allocations for GPUs have limitations and inefficiencies in terms of allocation sizes, exploiting reuse, transfer costs, and scalability. We propose a scalable and fully automatic data allocation and buffer management scheme for affine loop nests on multi-GPU machines. We call it the Bounding-Box-based Memory Manager (BBMM). BBMM can perform at runtime, during standard set operations like union, intersection, and difference, finding subset and superset relations on hyperrectangular regions of array data (bounding boxes). It uses these operations along with some compiler assistance to identify, allocate, and manage data required by applications in terms of disjoint bounding boxes. This allows it to (1) allocate exactly or nearly as much data as is required by computations running on each GPU, (2) efficiently track buffer allocations and hence maximize data reuse across tiles and minimize data transfer overhead, and (3) and as a result, maximize utilization of the combined memory on multi-GPU machines. BBMM can work with any choice of parallelizing transformations, computation placement, and scheduling schemes, whether static or dynamic. Experiments run on a four-GPU machine with various scientific programs showed that BBMM reduces data allocations on each GPU by up to 75% compared to current allocation schemes, yields performance of at least 88% of manually written code, and allows excellent weak scaling.

Selection and thermodynamic analysis of a turbocharger for a producer gas-fuelled multi-cylinder engine

The current work addresses the use of producer gas, a bio-derived gaseous alternative fuel, in engines designed for natural gas, derived from diesel engine frames. Impact of the use of producer gas on the general engine performance with specific focus on turbo-charging is addressed. The operation of a particular engine frame with diesel, natural gas and producer gas indicates that the peak load achieved is highest with diesel fuel (in compression ignition mode) followed by natural gas and producer gas (both in spark ignite mode). Detailed analysis of the engine power de-rating on fuelling with natural gas and producer gas indicates that the change in compression ratio (migration from compression to spark ignited mode), difference in mixture calorific value and turbocharger mismatch are the primary contributing factors. The largest de-rating occurs due to turbocharger mismatch. Turbocharger selection and optimization is identified as the strategy to recover the non-thermodynamic power loss, identified as the recovery potential (the loss due to mixture calorific value and turbocharger mismatch) on operating the engine with a fuel different from the base fuel. A turbocharged after-cooled six cylinder, 5.9 l, 90 kWe (diesel rating) engine (12.2 bar BMEP) is available commercially as a naturally aspirated natural gas engine delivering a peak load of 44.0 kWe (6.0 bar BMEP). The engine delivers a load of 27.3 kWe with producer gas under naturally aspirated mode. On charge boosting the engine with a turbocharger similar in configuration to the diesel engine turbocharger, the peak load delivered with producer gas is 36 kWe (4.8 bar BMEP) indicating a de-rating of about 60% over the baseline diesel mode. Estimation of knock limited peak load for producer gas-fuelled operation on the engine frame using a Wiebe function-based zero-dimensional code indicates a knock limited peak load of 76 kWe, indicating the potential to recover about 40 kWe. As a part of the recovery strategy, optimizing the ignition timing for maximum brake torque based on both spark sweep tests and established combustion descriptors and engine-turbocharger matching for producer gas-fuelled operation resulted in a knock limited peak load of 72.8 kWe (9.9 bar BMEP) at a compressor pressure ratio of 2.30. The de-rating of about 17.0 kWe compared to diesel rating is attributed to the reduction in compression ratio. With load recovery, the specific biomass consumption reduces from 1.2 kg/kWh to 1.0 kg/kWh, an improvement of over 16% while the engine thermal efficiency increases from 28% to 32%. The thermodynamic analysis of the compressor and the turbine indicates an isentropic efficiency of 74.5% and 73%, respectively.