Solving MDPs using two-timescale simulated annealing with multiplicative weights

We develop extensions of the Simulated Annealing with Multiplicative Weights (SAMW) algorithm that proposed a method of solution of Finite-Horizon Markov Decision Processes (FH-MDPs). The extensions developed are in three directions: a) Use of the dynamic programming principle in the policy update step of SAMW b) A two-timescale actor-critic algorithm that uses simulated transitions alone, and c) Extending the algorithm to the infinite-horizon discounted-reward scenario. In particular, a) reduces the storage required from exponential to linear in the number of actions per stage-state pair. On the faster timescale, a 'critic' recursion performs policy evaluation while on the slower timescale an 'actor' recursion performs policy improvement using SAMW. We give a proof outlining convergence w.p. 1 and show experimental results on two settings: semiconductor fabrication and flow control in communication networks.

An Efficient Simulated Annealing With a Valid Solution Mechanism For TDMA Brodcast Scheduling problem

This paper presents an efficient Simulated Annealing with valid solution mechanism for finding an optimum conflict-free transmission schedule for a broadcast radio network. This is known as a Broadcast Scheduling Problem (BSP) and shown as an NP-complete problem, in earlier studies. Because of this NP-complete nature, earlier studies used genetic algorithms, mean field annealing, neural networks, factor graph and sum product algorithm, and sequential vertex coloring algorithm to obtain the solution. In our study, a valid solution mechanism is included in simulated annealing. Because of this inclusion, we are able to achieve better results even for networks with 100 nodes and 300 links. The results obtained using our methodology is compared with all the other earlier solution methods.

Synthesis of workspaces of planar manipulators with arbitrary topology using shape representation and simulated annealing

This paper presents a general methodology for the synthesis of the external boundary of the workspaces of a planar manipulator with arbitrary topology. Both the desired workspace and the manipulator workspaces are identified by their boundaries and are treated as simple closed polygons. The paper introduces the concept of best match configuration and shows that the corresponding transformation can be obtained by using the concept of shape normalization available in image processing literature. Introduction of the concept of shape in workspace synthesis allows highly accurate synthesis with fewer numbers of design variables. This paper uses a new global property based vector representation for the shape of the workspaces which is computationally efficient because six out of the seven elements of this vector are obtained as a by-product of the shape normalization procedure. The synthesis of workspaces is formulated as an optimization problem where the distance between the shape vector of the desired workspace and that of the workspace of the manipulator at hand are minimized by changing the dimensional parameters of the manipulator. In view of the irregular nature of the error manifold, the statistical optimization procedure of simulated annealing has been used. A number of worked-out examples illustrate the generality and efficiency of the present method. (C) 1998 Elsevier Science Ltd. All rights reserved.

Solving a part classification problem using simulated annealing-like hybrid

Part classification and coding is still considered as laborious and time-consuming exercise. Keeping in view, the crucial role, which it plays, in developing automated CAPP systems, the attempts have been made in this article to automate a few elements of this exercise using a shape analysis model. In this study, a 24-vector directional template is contemplated to represent the feature elements of the parts (candidate and prototype). Various transformation processes such as deformation, straightening, bypassing, insertion and deletion are embedded in the proposed simulated annealing (SA)-like hybrid algorithm to match the candidate part with their prototype. For a candidate part, searching its matching prototype from the information data is computationally expensive and requires large search space. However, the proposed SA-like hybrid algorithm for solving the part classification problem considerably minimizes the search space and ensures early convergence of the solution. The application of the proposed approach is illustrated by an example part. The proposed approach is applied for the classification of 100 candidate parts and their prototypes to demonstrate the effectiveness of the algorithm. (C) 2003 Elsevier Science Ltd. All rights reserved.

Location Area Planning Using Simulated Annealing With A New Solution Representation

Location area planning problem is to partition the cellular/mobile network into location areas with the objective of minimizing the total cost. This partitioning problem is a difficult combinatorial optimization problem. In this paper, we use the simulated annealing with a new solution representation. In our method, we can automatically generate different number of location areas using Compact Index (CI) to obtain the optimal/best partitions. We compare the results obtained in our method with the earlier results available in literature. We show that our methodology is able to perform better than earlier methods.

Real-World Extensions To Scheduling Algorithms Based On Lagrangian Relaxation

Recently, efficient scheduling algorithms based on Lagrangian relaxation have been proposed for scheduling parallel machine systems and job shops. In this article, we develop real-world extensions to these scheduling methods. In the first part of the paper, we consider the problem of scheduling single operation jobs on parallel identical machines and extend the methodology to handle multiple classes of jobs, taking into account setup times and setup costs, The proposed methodology uses Lagrangian relaxation and simulated annealing in a hybrid framework, In the second part of the paper, we consider a Lagrangian relaxation based method for scheduling job shops and extend it to obtain a scheduling methodology for a real-world flexible manufacturing system with centralized material handling.

Hybrid algorithms for power system unit commitment

Four hybrid algorithms has been developed for the solution of the unit commitment problem. They use simulated annealing as one of the constituent techniques, and produce lower cost schedules; two of them have less overhead than other soft computing techniques. They are also more robust to the choice of parameters. A special technique avoids the generating of infeasible schedules, and thus reduces computation time.

Minimizing total weighted tardiness on a batch-processing machine with non-agreeable release times and due dates

This study considers the scheduling problem observed in the burn-in operation of semiconductor final testing, where jobs are associated with release times, due dates, processing times, sizes, and non-agreeable release times and due dates. The burn-in oven is modeled as a batch-processing machine which can process a batch of several jobs as long as the total sizes of the jobs do not exceed the machine capacity and the processing time of a batch is equal to the longest time among all the jobs in the batch. Due to the importance of on-time delivery in semiconductor manufacturing, the objective measure of this problem is to minimize total weighted tardiness. We have formulated the scheduling problem into an integer linear programming model and empirically show its computational intractability. Due to the computational intractability, we propose a few simple greedy heuristic algorithms and meta-heuristic algorithm, simulated annealing (SA). A series of computational experiments are conducted to evaluate the performance of the proposed heuristic algorithms in comparison with exact solution on various small-size problem instances and in comparison with estimated optimal solution on various real-life large size problem instances. The computational results show that the SA algorithm, with initial solution obtained using our own proposed greedy heuristic algorithm, consistently finds a robust solution in a reasonable amount of computation time.

Pickup and delivery problem using metaheuristics techniques

Dial-a-ride problem (DARP) is an optimization problem which deals with the minimization of the cost of the provided service where the customers are provided a door-to-door service based on their requests. This optimization model presented in earlier studies, is considered in this study. Due to the non-linear nature of the objective function the traditional optimization methods are plagued with the problem of converging to a local minima. To overcome this pitfall we use metaheuristics namely Simulated Annealing (SA), Particle Swarm Optimization (PSO), Genetic Algorithm (GA) and Artificial Immune System (AIS). From the results obtained, we conclude that Artificial Immune System method effectively tackles this optimization problem by providing us with optimal solutions. Crown Copyright (C) 2011 Published by Elsevier Ltd. All rights reserved.

Experiments on three-dimensional wire-frame object recognition

Some experimental results on the recognition of three-dimensional wire-frame objects are presented. In order to overcome the limitations of a recent model, which employs radial basis functions-based neural networks, we have proposed a hybrid learning system for object recognition, featuring: an optimization strategy (simulated annealing) in order to avoid local minima of an energy functional; and an appropriate choice of centers of the units. Further, in an attempt to achieve improved generalization ability, and to reduce the time for training, we invoke the principle of self-organization which utilises an unsupervised learning algorithm.

A stochastic dynamical system for image segmentation

Image segmentation is formulated as a stochastic process whose invariant distribution is concentrated at points of the desired region. By choosing multiple seed points, different regions can be segmented. The algorithm is based on the theory of time-homogeneous Markov chains and has been largely motivated by the technique of simulated annealing. The method proposed here has been found to perform well on real-world clean as well as noisy images while being computationally far less expensive than stochastic optimisation techniques

Climate response to physiological forcing of carbon dioxide simulated by the coupled Community Atmosphere Model (CAM3.1) and Community Land Model (CLM3.0)

Increasing concentrations of atmospheric CO2 decrease stomatal conductance of plants and thus suppress canopy transpiration. The climate response to this CO2-physiological forcing is investigated using the Community Atmosphere Model version 3.1 coupled to Community Land Model version 3.0. In response to the physiological effect of doubling CO2, simulations show a decrease in canopy transpiration of 8%, a mean warming of 0.1K over the land surface, and negligible changes in the hydrological cycle. These climate responses are much smaller than what were found in previous modeling studies. This is largely a result of unrealistic partitioning of evapotranspiration in our model control simulation with a greatly underestimated contribution from canopy transpiration and overestimated contributions from canopy and soil evaporation. This study highlights the importance of a realistic simulation of the hydrological cycle, especially the individual components of evapotranspiration, in reducing the uncertainty in our estimation of climatic response to CO2-physiological forcing. Citation: Cao, L., G. Bala, K. Caldeira, R. Nemani, and G.Ban-Weiss (2009), Climate response to physiological forcing of carbon dioxide simulated by the coupled Community Atmosphere Model (CAM3.1) and Community Land Model (CLM3.0).

NOx Reduction and Desorption Studies Under Electric Discharge Plasma Using a Simulated Gas Mixture: A Case Study on the Effect of Corona Electrodes

In this study, reduction and desorption of oxides of nitrogen (NOx) were conducted using an electrical discharge plasma technique. The study was carried out using a simulated gas mixture to explore the possibility of re-generation of used adsorbents by a nonthermal plasma desorption technique. Three different types of corona electrodes, namely, pipe, helical wire, and straight wire, were used for analyzing their effectiveness in NOx reduction/desorption. The pipe-type corona electrode exhibited a nitric oxide (NO) conversion of 50%, which is 1.5 times that of the straight-wire-type electrode at an energy density of 175J/L. The helical-wire-type corona electrode exhibited a NOx desorption efficiency almost 4 times that of the pipe-type electrode,indicating the possibility that corona-generated species play a crucial role in desorption.

Annealing Behavior of Isotactic Polybutene-1

The annealing behavior of isotactic polybutene-1 (PB-1) has been studied by differential scanning calorimetry and wide-angle x-ray diffraction. On annealing at 110°C, PB-1 yields thick crystals melting at -140°C which are mainly of Form I. An increase in the heat of fusion (ΔHf) and crystallinity is found for annealing times up to 12 h at 110°C; at longer times these properties decrease with increasing annealing time. The increases in ΔHf and crystallinity are attributed to increases in the lamellar thickness in the chain direction and in crystal perfection, and subsequent decreases to degradation of the polymer.