On extremal solutions to stochastic control problems, II

The set of attainable laws of the joint state-control process of a controlled diffusion is analyzed from a convex analytic viewpoint. Various equivalence relations depending on one-dimensional marginals thereof are defined on this set and the corresponding equivalence classes are studied.

Microstructural control in hot working of IN-718 superalloy using processing map

The hot-working characteristics of IN-718 are studied in the temperature range 900 °C to 1200 °C and strain rate range 0.001 to 100 s−1 using hot compression tests. Processing maps for hot working are developed on the basis of the strain-rate sensitivity variations with temperature and strain rate and interpreted using a dynamic materials model. The map exhibits two domains of dynamic recrystallization (DRX): one occurring at 950 °C and 0.001 s−1 with an efficiency of power dissipation of 37 pct and the other at 1200 °C and 0.1 s−1 with an efficiency of 40 pct. Dynamic recrystallization in the former domain is nucleated by the δ(Ni3Nb) precipitates and results in fine-grained microstructure. In the high-temperature DRX domain, carbides dissolve in the matrix and make interstitial carbon atoms available for increasing the rate of dislocation generation for DRX nucleation. It is recommended that IN-718 may be hot-forged initially at 1200 °C and 0.1 s−1 and finish-forged at 950 °C and 0.001 s−1 so that fine-grained structure may be achieved. The available forging practice validates these results from processing maps. At temperatures lower than 1000 °C and strain rates higher than 1 s−1 the material exhibits adiabatic shear bands. Also, at temperatures higher than 1150°C and strain rates more than 1s−1, IN-718 exhibits intercrystalline cracking. Both these regimes may be avoided in hotworking IN-718.

Microstructural control in hot working of IN-718 superalloy using processing map

The hot-working characteristics of IN-718 are studied in the temperature range 900 degrees C to 1200 degrees C and strain rate range 0.001 to 100 s(-1) using hot compression tests. Processing maps for hot working are developed on the basis of the strain-rate sensitivity variations with temperature and strain rate and interpreted using a dynamic materials model. The map exhibits two domains of dynamic recrystallization (DRX): one occurring at 950 degrees C and 0.001 s(-1) with an efficiency of power dissipation of 37 pct and the other at 1200 degrees C and 0.1 s(-1) with an efficiency of 40 pct. Dynamic recrystallization in the former domain is nucleated by the delta(Ni3Nb) precipitates and results in fine-grained microstructure. In the high-temperature DRX domain, carbides dissolve in the matrix and make interstitial carbon atoms available for increasing the rate of dislocation generation for DRX nucleation. It is recommended that IN-718 may be hot-forged initially at 1200 degrees C and 0.1 s(-1) and finish-forged at 950 degrees C and 0.001 s(-1) so that fine-grained structure may be achieved. The available forging practice validates these results from processing maps. At temperatures lower than 1000 degrees C and strain rates higher than 1 s(-1), the material exhibits adiabatic shear bands. Also, at temperatures higher than 1150 degrees C and strain rates more than 1 s(-1), IN-718 exhibits intercrystalline cracking. Both these regimes may be avoided in hot-working IN-718.

A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems

Even though dynamic programming offers an optimal control solution in a state feedback form, the method is overwhelmed by computational and storage requirements. Approximate dynamic programming implemented with an Adaptive Critic (AC) neural network structure has evolved as a powerful alternative technique that obviates the need for excessive computations and storage requirements in solving optimal control problems. In this paper, an improvement to the AC architecture, called the �Single Network Adaptive Critic (SNAC)� is presented. This approach is applicable to a wide class of nonlinear systems where the optimal control (stationary) equation can be explicitly expressed in terms of the state and costate variables. The selection of this terminology is guided by the fact that it eliminates the use of one neural network (namely the action network) that is part of a typical dual network AC setup. As a consequence, the SNAC architecture offers three potential advantages: a simpler architecture, lesser computational load and elimination of the approximation error associated with the eliminated network. In order to demonstrate these benefits and the control synthesis technique using SNAC, two problems have been solved with the AC and SNAC approaches and their computational performances are compared. One of these problems is a real-life Micro-Electro-Mechanical-system (MEMS) problem, which demonstrates that the SNAC technique is applicable to complex engineering systems.

Time-resolved fluorescence studies on covalently linked porphyrin�nitroarene complexes. Conformational control of photoinduced electron transfer reactions

Time-resolved fluorescence studies were carried out on a series of free-base and zinc(II) derivatives of meso-tetraphenylporphyrins covalently linked to either 1,3-dinitrobenzene (DNB) or 1,3,5-trinitrobenzene (TNB) acceptor units. These acceptor units were linked at different sites (at the ortho, meta or para positions of one of the phenyl groups of meso-tetraphenylporphyrin) to the donor porphyrins such that the resulting isomeric intramolecular donor-acceptor complexes exhibit different centre-to-centre (ctc) distances and relative orientations. Biexponential fluorescence decay profiles observed for several of these covalently linked complexes were rationalized in terms of the presence of ''closed'' and ''extended'' conformers. Detailed analyses of the fluorescence decay data have provided a comprehensive understanding of the photoinduced electron transfer (PET) reactions occurring in systems containing zinc(II) porphyrin donors. It is observed that although DNB-linked zinc(II) complexes follow the trends predicted for the efficiency of PET with respect to donor-acceptor distance, the TNB-linked zinc(II) porphyrins exhibit a behaviour which is dictated by steric effects. Similarly, although the thermodynamic criteria predict a greater efficiency of charge separation in TNB-linked complexes compared with DNB-linked complexes, the reverse trend observed has been attributed to orientational effects. In the complexes containing free-base porphyrin donors, PET is expected to be less efficient from a thermodynamic viewpoint. In a few of these cases, fluorescence quenching seems to occur by parallel mechanisms other than PET.

An optimal dynamic inversion approach for controlling a class of one-dimensional nonlinear distributed parameter systems

Combining the principles of dynamic inversion and optimization theory, a new approach is presented for stable control of a class of one-dimensional nonlinear distributed parameter systems, assuming the availability a continuous actuator in the spatial domain. Unlike the existing approximate-then-design and design-then-approximate techniques, here there is no need of any approximation either of the system dynamics or of the resulting controller. Rather, the control synthesis approach is fairly straight-forward and simple. The controller formulation has more elegance because we can prove the convergence of the controller to its steady state value. To demonstrate the potential of the proposed technique, a real-life temperature control problem for a heat transfer application is solved. It has been demonstrated that a desired temperature profile can be achieved starting from any arbitrary initial temperature profile.

The determination of spatial pattern in Dictyostelium discoideum

Free-living amoebae of the cellular slime mould Dictyostelium discoideum aggregate when starved and give rise to a long and thin multicellular structure, the slug. The slug resembles a metazoan embryo, and as with other embryos it is possible to specify a fate map. In the case of Dictyostelium discoideum the map is especially simple: cells in the anterior fifth of the slug die and form a stalk while the majority of those in the posterior differentiate into spores. The genesis of this anterior-posterior distinction is the subject of our review. In particular, we ask: what are the relative roles of individual pre-aggregative predispositions and post-aggregative position in determining cell fate? We review the literature on the subject and conclude that both factors are important. Variations in nutritional status, or in cell cycle phase at starvation, can bias the probability that an amoeba differentiates into a stalk cell or a spore. On the other hand, isolates, or slug fragments, consisting of only prestalk cells or only prespore cells can regulate so as to result in a normal range of both cell types. We identify three levels of control, each being responsible for guiding patterning in normal development: (i) 'coin tossing', whereby a cell autonomously exhibits a preference for developing along either the stalk or the spore pathway with relative probabilities that can be influenced by the environment; (ii) 'chemical kinetics', whereby prestalk and prespore cells originate from undifferentiated amoebae on a probabilistic basis but, having originated, interact (e.g. via positive and negative feedbacks), and the interaction influences the possibility of conversion of one cell type into the other, and (iii) 'positional information', in which the spatial distribution of morphogens in the slug influences the pathway of differentiation. In the case of possibilities (i) and (ii), sorting out of like cell types leads to the final spatial pattern. In the case of possibility (iii), the pattern arises in situ.

Reinforcement Learning With Function Approximation for Traffic Signal Control

We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e. g., the work of Abdulhai et al., on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhai et al. and Cools et al., as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of Abdulhai et al. We observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.

Risk-sensitive optimal control for Markov decision processes with monotone cost

The existence of an optimal feedback law is established for the risk-sensitive optimal control problem with denumerable state space. The main assumptions imposed are irreducibility and a near monotonicity condition on the one-step cost function. A solution can be found constructively using either value iteration or policy iteration under suitable conditions on initial feedback law.

Numerical Modelling of the Observed Diurnal Cycle of Indian Monsoon Rainfall

We have analysed the diurnal cycle of rainfall over the Indian region (10S-35N, 60E-100E) using both satellite and in-situ data, and found many interesting features associated with this fundamental, yet under-explored, mode of variability. Since there is a distinct and strong diurnal mode of variability associated with the Indian summer monsoon rainfall, we evaluate the ability of the Weather Research and Forecasting Model (WRF) to simulate the observed diurnal rainfall characteristics. The model (at 54km grid-spacing) is integrated for the month of July, 2006, since this period was particularly favourable for the study of diurnal cycle. We first evaluate the sensitivity of the model to the prescribed sea surface temperature (SST), by using two different SST datasets, namely, Final Analyses (FNL) and Real-time Global (RTG). It was found that with RTG SST the rainfall simulation over central India (CI) was significantly better than that with FNL. On the other hand, over the Bay of Bengal (BoB), rainfall simulated with FNL was marginally better than with RTG. However, the overall performance of RTG SST was found to be better than FNL, and hence it was used for further model simulations. Next, we investigated the role of the convective parameterization scheme on the simulation of diurnal cycle of rainfall. We found that the Kain-Fritsch (KF) scheme performs significantly better than Betts-Miller-Janjić (BMJ) and Grell-Devenyi schemes. We also studied the impact of other physical parameterizations, namely, microphysics, boundary layer, land surface, and the radiation parameterization, on the simulation of diurnal cycle of rainfall, and identified the “best” model configuration. We used this configuration of the “best” model to perform a sensitivity study on the role of various convective components used in the KF scheme. In particular, we studied the role of convective downdrafts, convective timescale, and feedback fraction, on the simulated diurnal cycle of rainfall. The “best” model simulations, in general, show a good agreement with observations. Specifically, (i) Over CI, the simulated diurnal rainfall peak is at 1430 IST, in comparison to the observed 1430-1730 IST peak; (ii) Over Western Ghats and Burmese mountains, the model simulates a diurnal rainfall peak at 1430 IST, as opposed to the observed peak of 1430-1730 IST; (iii) Over Sumatra, both model and observations show a diurnal peak at 1730 IST; (iv) The observed southward propagating diurnal rainfall bands over BoB are weakly simulated by WRF. Besides the diurnal cycle of rainfall, the mean spatial pattern of total rainfall and its partitioning between the convective and stratiform components, are also well simulated. The “best” model configuration was used to conduct two nested simulations with one-way, three-level nesting (54-18-6km) over CI and BoB. While, the 54km and 18km simulations were conducted for the whole of July, 2006, the 6km simulation was carried out for the period 18 - 24 July, 2006. The results of our coarse- and fine-scale numerical simulations of the diurnal cycle of monsoon rainfall will be discussed.

Viscoelastic phenomenology based structure assignment for closed-loop vibration control of a beam with sensors and actuators

In this paper we incorporate a novel approach to synthesize a class of closed-loop feedback control, based on the variational structure assignment. Properties of a viscoelastic system are used to design an active feedback controller for an undamped structural system with distributed sensor, actuator and controller. Wave dispersion properties of onedimensional beam system have been studied. Efficiency of the chosen viscoelastic model in enhancing damping and stability properties of one-dimensional viscoelastic bar have been analyzed. The variational structure is projected on a solution space of a closed-loop system involving a weakly damped structure with distributed sensor and actuator with controller. These assign the phenomenology based internal strain rate damping parameter of a viscoelastic system to the usual elastic structure but with active control. In the formulation a model of cantilever beam with non-collocated actuator and sensor has been considered. The formulation leads to the matrix identification problem of two dynamic stiffness matrices. The method has been simplified to obtain control system gains for the free vibration control of a cantilever beam system with collocated actuator-sensor, using quadratic optimal control and pole-placement methods.

Robustness Study of a Dynamic Inversion Control Law for a High Performance Aircraft

In the design of °ight control system modeling uncertainties in the form of param-eter variations is one of the major problems. It is even more critical for high performance aircrafts,since such aircrafts are purposefully designed unstable to enhance their performance (especially ma-neuverability). Hence the °ight control system needs to be quite e®ective in both assuring accurate tracking of pilot commands, while simultaneously assuring overall stability of the aircraft. In addi-tion, the control system must also be su±ciently robust to cater for possible parameter variations and inaccuracies . The primary aim of this paper is to carry out a robustness study of a dynamic inversion based nonlinear control design for a high performance aircraft, which has been developed recently [1].

Decentralized control of linear multivariable systems

A strongly connected decentralized control system may be made single channel controllable and observable with respect to any channel by decentralized feedbacks. It is noted here that the system example considered by Corfmat and Morse to illustrate this fact is already single channel controllable and observable, with respect to one of the channels. An alternate example which fits into the situation is presented in this item.

Active Vibration Suppression of One-dimensional Nonlinear Structures Using Optimal Dynamic Inversion

A flexible robot arm can be modeled as an Euler-Bernoulli beam which are infinite degrees of freedom (DOF) system. Proper control is needed to track the desired motion of a robotic arm. The infinite number of DOF of beams are reduced to finite number for controller implementation, which brings in error (due to their distributed nature). Therefore, to represent reality better distributed parameter systems (DPS) should be controlled using the systems partial differential equation (PDE) directly. In this paper, we propose to use a recently developed optimal dynamic inversion technique to design a controller to suppress nonlinear vibration of a beam. The method used in this paper determines control forces directly from the PDE model of the system. The formulation has better practical significance, because it leads to a closed form solution of the controller (hence avoids computational issues).

Reinforcement learning with average cost for adaptive control of traffic lights at intersections

We propose for the first time two reinforcement learning algorithms with function approximation for average cost adaptive control of traffic lights. One of these algorithms is a version of Q-learning with function approximation while the other is a policy gradient actor-critic algorithm that incorporates multi-timescale stochastic approximation. We show performance comparisons on various network settings of these algorithms with a range of fixed timing algorithms, as well as a Q-learning algorithm with full state representation that we also implement. We observe that whereas (as expected) on a two-junction corridor, the full state representation algorithm shows the best results, this algorithm is not implementable on larger road networks. The algorithm PG-AC-TLC that we propose is seen to show the best overall performance.