The Borkar-Meyn theorem for asynchronous stochastic approximations

In this paper, we give a generalization of a result by Borkar and Meyn (2000) 1], on the stability and convergence of synchronous-update stochastic approximation algorithms, to the case of asynchronous stochastic approximations with delays. We then describe an interesting application of the result to asynchronous distributed temporal difference (TD) learning with function approximation and delays. (C) 2011 Elsevier B.V. All rights reserved.

Modeling of nonreacting and reacting turbulent spray jets using a fully stochastic separated flow approach

Numerical modeling of several turbulent nonreacting and reacting spray jets is carried out using a fully stochastic separated flow (FSSF) approach. As is widely used, the carrier-phase is considered in an Eulerian framework, while the dispersed phase is tracked in a Lagrangian framework following the stochastic separated flow (SSF) model. Various interactions between the two phases are taken into account by means of two-way coupling. Spray evaporation is described using a thermal model with an infinite conductivity in the liquid phase. The gas-phase turbulence terms are closed using the k-epsilon model. A novel mixture fraction based approach is used to stochastically model the fluctuating temperature and composition in the gas phase and these are then used to refine the estimates of the heat and mass transfer rates between the droplets and the surrounding gas-phase. In classical SSF (CSSF) methods, stochastic fluctuations of only the gas-phase velocity are modeled. Successful implementation of the FSSF approach to turbulent nonreacting and reacting spray jets is demonstrated. Results are compared against experimental measurements as well as with predictions using the CSSF approach for both nonreacting and reacting spray jets. The FSSF approach shows little difference from the CSSF predictions for nonreacting spray jets but differences are significant for reacting spray jets. In general, the FSSF approach gives good predictions of the flame length and structure but further improvements in modeling may be needed to improve the accuracy of some details of the Predictions. (C) 2011 The Combustion Institute. Published by Elsevier Inc. All rights reserved.

Doubly spectral stochastic finite element method (DSSFEM) for random field problems

Uncertainties in complex dynamic systems play an important role in the prediction of a dynamic response in the mid- and high-frequency ranges. For distributed parameter systems, parametric uncertainties can be represented by random fields leading to stochastic partial differential equations. Over the past two decades, the spectral stochastic finite-element method has been developed to discretize the random fields and solve such problems. On the other hand, for deterministic distributed parameter linear dynamic systems, the spectral finite-element method has been developed to efficiently solve the problem in the frequency domain. In spite of the fact that both approaches use spectral decomposition (one for the random fields and the other for the dynamic displacement fields), very little overlap between them has been reported in literature. In this paper, these two spectral techniques are unified with the aim that the unified approach would outperform any of the spectral methods considered on their own. An exponential autocorrelation function for the random fields, a frequency-dependent stochastic element stiffness, and mass matrices are derived for the axial and bending vibration of rods. Closed-form exact expressions are derived by using the Karhunen-Loève expansion. Numerical examples are given to illustrate the unified spectral approach.

Computationally Efficient Sub-optimal Mid Course Guidance Using Model Predictive Static Programming (MPSP)

For a homing interceptor, suitable initial condition must be achieved by mid course guidance scheme for its maximum effectiveness. To achieve desired end goal of any mid course guidance scheme, two point boundary value problem must be solved online with all realistic constrain. A Newly developed computationally efficient technique named as MPSP (Model Predictive Static Programming) is utilized in this paper for obtaining suboptimal solution of optimal mid course guidance. Time to go uncertainty is avoided in this formulation by making use of desired position where midcourse guidance terminate and terminal guidance takes over. A suitable approach angle towards desired point also can be specified in this guidance law formulation. This feature makes this law particularly attractive because warhead effectiveness issue can be indirectly solved in mid course phase.

On Optimal Performance in Mobile Ad-Hoc Networks

In this paper we are concerned with finding the maximum throughput that a mobile ad hoc network can support. Even when nodes are stationary, the problem of determining the capacity region has long been known to be NP-hard. Mobility introduces an additional dimension of complexity because nodes now also have to decide when they should initiate route discovery. Since route discovery involves communication and computation overhead, it should not be invoked very often. On the other hand, mobility implies that routes are bound to become stale resulting in sub-optimal performance if routes are not updated. We attempt to gain some understanding of these effects by considering a simple one-dimensional network model. The simplicity of our model allows us to use stochastic dynamic programming (SDP) to find the maximum possible network throughput with ideal routing and medium access control (MAC) scheduling. Using the optimal value as a benchmark, we also propose and evaluate the performance of a simple threshold-based heuristic. Unlike the optimal policy which requires considerable state information, the heuristic is very simple to implement and is not overly sensitive to the threshold value used. We find empirical conditions for our heuristic to be near-optimal as well as network scenarios when our simple heuristic does not perform very well. We provide extensive numerical and simulation results for different parameter settings of our model.

Register Allocation and Optimal Spill code Scheduling in Software Pipelined Loops using 0-1 Integer Linear Programming Formulation

In achieving higher instruction level parallelism, software pipelining increases the register pressure in the loop. The usefulness of the generated schedule may be restricted to cases where the register pressure is less than the available number of registers. Spill instructions need to be introduced otherwise. But scheduling these spill instructions in the compact schedule is a difficult task. Several heuristics have been proposed to schedule spill code. These heuristics may generate more spill code than necessary, and scheduling them may necessitate increasing the initiation interval. We model the problem of register allocation with spill code generation and scheduling in software pipelined loops as a 0-1 integer linear program. The formulation minimizes the increase in initiation interval (II) by optimally placing spill code and simultaneously minimizes the amount of spill code produced. To the best of our knowledge, this is the first integrated formulation for register allocation, optimal spill code generation and scheduling for software pipelined loops. The proposed formulation performs better than the existing heuristics by preventing an increase in II in 11.11% of the loops and generating 18.48% less spill code on average among the loops extracted from Perfect Club and SPEC benchmarks with a moderate increase in compilation time.

Stochastic Algorithms for Discrete Parameter Simulation Optimization

We present two efficient discrete parameter simulation optimization (DPSO) algorithms for the long-run average cost objective. One of these algorithms uses the smoothed functional approximation (SFA) procedure, while the other is based on simultaneous perturbation stochastic approximation (SPSA). The use of SFA for DPSO had not been proposed previously in the literature. Further, both algorithms adopt an interesting technique of random projections that we present here for the first time. We give a proof of convergence of our algorithms. Next, we present detailed numerical experiments on a problem of admission control with dependent service times. We consider two different settings involving parameter sets that have moderate and large sizes, respectively. On the first setting, we also show performance comparisons with the well-studied optimal computing budget allocation (OCBA) algorithm and also the equal allocation algorithm. Note to Practitioners-Even though SPSA and SFA have been devised in the literature for continuous optimization problems, our results indicate that they can be powerful techniques even when they are adapted to discrete optimization settings. OCBA is widely recognized as one of the most powerful methods for discrete optimization when the parameter sets are of small or moderate size. On a setting involving a parameter set of size 100, we observe that when the computing budget is small, both SPSA and OCBA show similar performance and are better in comparison to SFA, however, as the computing budget is increased, SPSA and SFA show better performance than OCBA. Both our algorithms also show good performance when the parameter set has a size of 10(8). SFA is seen to show the best overall performance. Unlike most other DPSO algorithms in the literature, an advantage with our algorithms is that they are easily implementable regardless of the size of the parameter sets and show good performance in both scenarios.

Minimization of Constraint Violation in Fuzzy Multiobjective Programming

Fuzzy multiobjective programming for a deterministic case involves maximizing the minimum goal satisfaction level among conflicting goals of different stakeholders using Max-min approach. Uncertainty due to randomness in a fuzzy multiobjective programming may be addressed by modifying the constraints using probabilistic inequality (e.g., Chebyshev’s inequality) or by addition of new constraints using statistical moments (e.g., skewness). Such modifications may result in the reduction of the optimal value of the system performance. In the present study, a methodology is developed to allow some violation in the newly added and modified constraints, and then minimizing the violation of those constraints with the objective of maximizing the minimum goal satisfaction level. Fuzzy goal programming is used to solve the multiobjective model. The proposed methodology is demonstrated with an application in the field of Waste Load Allocation (WLA) in a river system.