Stochastic Approximation Algorithms for Constrained Optimization via Simulation

We develop four algorithms for simulation-based optimization under multiple inequality constraints. Both the cost and the constraint functions are considered to be long-run averages of certain state-dependent single-stage functions. We pose the problem in the simulation optimization framework by using the Lagrange multiplier method. Two of our algorithms estimate only the gradient of the Lagrangian, while the other two estimate both the gradient and the Hessian of it. In the process, we also develop various new estimators for the gradient and Hessian. All our algorithms use two simulations each. Two of these algorithms are based on the smoothed functional (SF) technique, while the other two are based on the simultaneous perturbation stochastic approximation (SPSA) method. We prove the convergence of our algorithms and show numerical experiments on a setting involving an open Jackson network. The Newton-based SF algorithm is seen to show the best overall performance.

An actor-critic algorithm with function approximation for discounted cost constrained Markov decision processes

We develop in this article the first actor-critic reinforcement learning algorithm with function approximation for a problem of control under multiple inequality constraints. We consider the infinite horizon discounted cost framework in which both the objective and the constraint functions are suitable expected policy-dependent discounted sums of certain sample path functions. We apply the Lagrange multiplier method to handle the inequality constraints. Our algorithm makes use of multi-timescale stochastic approximation and incorporates a temporal difference (TD) critic and an actor that makes a gradient search in the space of policy parameters using efficient simultaneous perturbation stochastic approximation (SPSA) gradient estimates. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal policy. (C) 2010 Elsevier B.V. All rights reserved.

Effect of the valine-threonine constraint on the dynamics of the proline helix — A molecular dynamics study

Proline residues in helices play an important role in the structure of proteins. The proline residue introduces a kink in the helix which varies from about 5-degrees to 50-degrees. The presence of other residues such as threonine or valine near the proline region can influence the flexibility exhibited by the kinked helix, which can have an important biological role. In the present paper, the constraint introduced by threonine and valine on a proline helix is investigated by molecular dynamics studies. The systems considered am (1) a poly-alanine helix with threonine-proline residues (TP) and (2) a poly-alanine helix with valine-threonine-proline residues (VTP), in the middle. Molecular dynamics simulations are carried out on these two systems for 500 ps. The results are analyzed in terms of structural transitions, bend-related parameters and sidechain orientations.

Suboptimal Midcourse Guidance of Interceptors for High-Speed Targets with Alignment Angle Constraint

Using the recently developed computationally efficient model predictive static programming and a closely related model predictive spread control concept, two nonlinear suboptimal midcourse guidance laws are presented in this paper for interceptors engaging against incoming high-speed ballistic missiles. The guidance laws are primarily based on nonlinear optimal control theory, and hence imbed effective trajectory optimization concepts into the guidance laws. Apart from being energy efficient by minimizing the control usage throughout the trajectory (minimum control usage leads to minimum turning, and hence leads to minimum induced drag), both of these laws enforce desired alignment constraints in both elevation and azimuth in a hard-constraint sense. This good alignment during midcourse is expected to enhance the effectiveness of the terminal guidance substantially. Both point mass as well as six-degree-of-freedom simulation results (with a realistic inner-loop autopilot based on dynamic inversion) are presented in this paper, which clearly shows the effectiveness of the proposed guidance laws. It has also been observed that, even with different perturbations of missile parameters, the performance of guidance is satisfactory. A comparison study, with the vector explicit guidance scheme proposed earlier in the literature, also shows that the newly proposed model-predictive-static-programming-based and model-predictive-spread-control-based guidance schemes lead to lesser lateral acceleration demand and lesser velocity loss during engagement.

Unified approach to the constraint counting theory of glasses

An approach to the constraint counting theory of glasses is applied to many glass systems which include an oxide, chalcohalide, and chalcogenides. In this, shifting of the percolation threshold due to noncovalent bonding interactions in a basically covalent network and other recent extensions of the theory appear natural. This is particularly insightful and reveals that the chemical threshold signifies another structural transition along with the rigidity percolation threshold, thus unifying these two seemingly disparate toplogical concepts. [S0163-1829(99)11441-3].

Implications of the half-duplex constraint on relay-aided cooperation using rateless codes

The half-duplex constraint, which mandates that a cooperative relay cannot transmit and receive simultaneously, considerably simplifies the demands made on the hardware and signal processing capabilities of a relay. However, the very inability of a relay to transmit and receive simultaneously leads to a potential under-utilization of time and bandwidth resources available to the system. We analyze the impact of the half-duplex constraint on the throughput of a cooperative relay system that uses rateless codes to harness spatial diversity and efficiently transmit information from a source to a destination. We derive closed-form expressions for the throughput of the system, and show that as the number of relays increases, the throughput approaches that of a system that uses more sophisticated full-duplex nodes. Thus, half-duplex nodes are well suited for cooperation using rateless codes despite the simplicity of both the cooperation protocol and the relays.

Constraint loss under dynamic loading in rate independent plastic solids

The objectives of this paper are to examine the loss of crack tip constraint in dynamically loaded fracture specimens and to assess whether it can lead to enhancement in the fracture toughness at high loading rates which has been observed in several experimental studies. To this end, 2-D plane strain finite element analyses of single edge notched (tension) specimen and three point bend specimen subjected to time varying loads are performed. The material is assumed to obey the small strain J(2) flow theory of plasticity with rate independent behaviour. The results demonstrate that a valid J-Q field exists under dynamic loading irrespective of the crack length and specimen geometry. Further, the constraint parameter Q becomes strongly negative at high loading rates, particularly in deeply cracked specimens. The variation of dynamic fracture toughness K-dc with stress intensity rate K for cleavage cracking is predicted using a simple critical stress criterion. It is found that inertia-driven constraint loss can substantially enhance K-dc for (K) over dot > 10(5) MPa rootm/s.

Robust Formulations for Handling Uncertainty in Kernel Matrices

We study the problem of uncertainty in the entries of the Kernel matrix, arising in SVM formulation. Using Chance Constraint Programming and a novel large deviation inequality we derive a formulation which is robust to such noise. The resulting formulation applies when the noise is Gaussian, or has finite support. The formulation in general is non-convex, but in several cases of interest it reduces to a convex program. The problem of uncertainty in kernel matrix is motivated from the real world problem of classifying proteins when the structures are provided with some uncertainty. The formulation derived here naturally incorporates such uncertainty in a principled manner leading to significant improvements over the state of the art. 1.

Effect of lattice orientation on crack tip constraint in ductile single crystals

In this work, the effect of lattice orientation on the fields prevailing near a notch tip is investigated pertaining to various constraint levels in FCC single crystals. A modified boundary layer formulation is employed and numerical solutions under mode I, plane strain conditions are generated by assuming an elastic-perfectly plastic FCC single crystal. The analysis is carried out corresponding to different lattice orientations with respect to the notch line. It is found that the near-tip deformation field, especially the development of kink or slip shear bands is sensitive to the constraint level. The stress distribution and the size and shape of the plastic zone near the notch tip are also strongly influenced by the level of T-stress. The present results clearly establish that ductile single crystal fracture geometries would progressively lose crack tip constraint as the T-stress becomes more negative irrespective of lattice orientation. Also, the near-tip field for a range of constraint levels can be characterized by two-parameters such as K-T or J-Q as in isotropic plastic solids.

Realizing a flexible constraint length Viterbi decoder for software radio on a de Bruijn interconnection network

Building flexible constraint length Viterbi decoders requires us to be able to realize de Bruijn networks of various sizes on the physically provided interconnection network. This paper considers the case when the physical network is itself a de Bruijn network and presents a scalable technique for realizing any n-node de Bruijn network on an N-node de Bruijn network, where n < N. The technique ensures that the length of the longest path realized on the network is minimized and that each physical connection is utilized to send only one data item, both of which are desirable in order to reduce the hardware complexity of the network and to obtain the best possible performance.

Nonlinear Model Predictive Spread Acceleration Guidance with Impact Angle Constraint for Stationary Targets

A new technique named as model predictive spread acceleration guidance (MPSAG) is proposed in this paper. It combines nonlinear model predictive control and spread acceleration guidance philosophies. This technique is then used to design a nonlinear suboptimal guidance law for a constant speed missile against stationary target with impact angle constraint. MPSAG technique can be applied to a class of nonlinear problems, which leads to a closed form solution of the lateral acceleration (latax) history update. Guidance command assumed is the lateral acceleration (latax), applied normal to the velocity vector. The new guidance law is validated by considering the nonlinear kinematics with both lag-free as well as first order autopilot delay. The simulation results show that the proposed technique is quite promising to come up with a nonlinear guidance law that leads to both very small miss distance as well as the desired impact angle.

Nonlinear Model Predictive Spread Acceleration Guidance with Impact Angle Constraint for Slow-moving Targets

A new technique named as model predictive spread acceleration guidance (MPSAG) is proposed in this paper. It combines nonlinear model predictive control and spread acceleration guidance philosophies. This technique is then used to design a nonlinear suboptimal guidance law for a constant speed missile against stationary target with impact angle constraint. MPSAG technique can be applied to a class of nonlinear problems, which leads to a closed form solution of the lateral acceleration (latax) history update. Guidance command assumed is the lateral acceleration (latax), applied normal to the velocity vector. The new guidance law is validated by considering the nonlinear kinematics with both lag-free as well as first order autopilot delay. The simulation results show that the proposed technique is quite promising to come up with a nonlinear guidance law that leads to both very small miss distance as well as the desired impact angle.

Nonlinear Suboptimal Guidance with Impact Angle Constraint for Slow Moving Targets in 1-D Using MPSP

Using a recently developed method named as model predictive static programming (MPSP), a nonlinear suboptimal guidance law for a constant speed missile against a slow moving target with impact angle constraint is proposed. In this paper MPSP technique leads to a closed form solution of the latax history update for the given problem. Guidance command is the latax,which is normal to the missile velocity and the terminal constraints are miss distance and impact angle. The new guidance law is validated by considering the nonlinear kinematics with both lag-free and first order autopilot delay.

Clustering Based Large Margin Classification: A Scalable Approach using SOCP Formulation

This paper presents a novel Second Order Cone Programming (SOCP) formulation for large scale binary classification tasks. Assuming that the class conditional densities are mixture distributions, where each component of the mixture has a spherical covariance, the second order statistics of the components can be estimated efficiently using clustering algorithms like BIRCH. For each cluster, the second order moments are used to derive a second order cone constraint via a Chebyshev-Cantelli inequality. This constraint ensures that any data point in the cluster is classified correctly with a high probability. This leads to a large margin SOCP formulation whose size depends on the number of clusters rather than the number of training data points. Hence, the proposed formulation scales well for large datasets when compared to the state-of-the-art classifiers, Support Vector Machines (SVMs). Experiments on real world and synthetic datasets show that the proposed algorithm outperforms SVM solvers in terms of training time and achieves similar accuracies.