Probabilistic DHP adaptive critic for nonlinear stochastic control systems

Following the recently developed algorithms for fully probabilistic control design for general dynamic stochastic systems (Herzallah & Káarnáy, 2011; Kárný, 1996), this paper presents the solution to the probabilistic dual heuristic programming (DHP) adaptive critic method (Herzallah & Káarnáy, 2011) and randomized control algorithm for stochastic nonlinear dynamical systems. The purpose of the randomized control input design is to make the joint probability density function of the closed loop system as close as possible to a predetermined ideal joint probability density function. This paper completes the previous work (Herzallah & Kárnáy, 2011; Kárný, 1996) by formulating and solving the fully probabilistic control design problem on the more general case of nonlinear stochastic discrete time systems. A simulated example is used to demonstrate the use of the algorithm and encouraging results have been obtained.

Comparison between conventional and stochastic pinning control

Theoretical developments on pinning control of complex dynamical networks have mainly focused on the deterministic versions of the model dynamics. However, the dynamical behavior of most real networks is often affected by stochastic noise components. In this paper the pinning control of a stochastic version of the coupled map lattice network with spatiotemporal characteristics is studied. The control of these complex dynamical networks have functional uncertainty which should be considered when calculating stabilizing control signals. Two feedback control methods are considered: the conventional feedback control and modified stochastic feedback control. It is shown that the typically-used conventional control method suffers from the ignorance of model uncertainty leading to a reduction and potentially a collapse in the control efficiency. Numerical verification of the main result is provided for a chaotic coupled map lattice network. © 2011 IEEE.

Stochastic control strategies and adaptive critic methods

Adaptive critic methods have common roots as generalizations of dynamic programming for neural reinforcement learning approaches. Since they approximate the dynamic programming solutions, they are potentially suitable for learning in noisy, nonlinear and nonstationary environments. In this study, a novel probabilistic dual heuristic programming (DHP) based adaptive critic controller is proposed. Distinct to current approaches, the proposed probabilistic (DHP) adaptive critic method takes uncertainties of forward model and inverse controller into consideration. Therefore, it is suitable for deterministic and stochastic control problems characterized by functional uncertainty. Theoretical development of the proposed method is validated by analytically evaluating the correct value of the cost function which satisfies the Bellman equation in a linear quadratic control problem. The target value of the critic network is then calculated and shown to be equal to the analytically derived correct value.

Generalised Riccati solution and pinning control of complex stochastic networks

This paper considers the global synchronisation of a stochastic version of coupled map lattices networks through an innovative stochastic adaptive linear quadratic pinning control methodology. In a stochastic network, each state receives only noisy measurement of its neighbours' states. For such networks we derive a generalised Riccati solution that quantifies and incorporates uncertainty of the forward dynamics and inverse controller in the derivation of the stochastic optimal control law. The generalised Riccati solution is derived using the Lyapunov approach. A probabilistic approximation type algorithm is employed to estimate the conditional distributions of the state and inverse controller from historical data and quantifying model uncertainties. The theoretical derivation is complemented by its validation on a set of representative examples.

State-owned enterprises, exporting and productivity in China:a stochastic dominance approach

A popular explanation for China's rapid economic growth in recent years has been the dramatic increase in the number of private domestic- and foreign-owned firms and a decline in the state-owned sector. However, recent evidence suggests that China's state-owned enterprises (SOEs) are in fact stronger than ever. In this paper, we examine over 78,000 manufacturing firms between 2002 and 2006 to investigate the relationship between ownership structure and the degree of firm-level exposure to export markets and firm-level productivity. Using a conditional stochastic dominance approach, we reveal that although our results largely adhere to prior expectations, the performance of SOEs differs markedly between those that export and those that supply the domestic market only. It appears that China's internationally focused SOEs have become formidable global competitors. © 2013 John Wiley & Sons Ltd.

Non-stationary stochastic inventory lot-sizing with emission and service level constraints in a carbon cap-and-trade system

Firms worldwide are taking major initiatives to reduce the carbon footprint of their supply chains in response to the growing governmental and consumer pressures. In real life, these supply chains face stochastic and non-stationary demand but most of the studies on inventory lot-sizing problem with emission concerns consider deterministic demand. In this paper, we study the inventory lot-sizing problem under non-stationary stochastic demand condition with emission and cycle service level constraints considering carbon cap-and-trade regulatory mechanism. Using a mixed integer linear programming model, this paper aims to investigate the effects of emission parameters, product- and system-related features on the supply chain performance through extensive computational experiments to cover general type business settings and not a specific scenario. Results show that cycle service level and demand coefficient of variation have significant impacts on total cost and emission irrespective of level of demand variability while the impact of product's demand pattern is significant only at lower level of demand variability. Finally, results also show that increasing value of carbon price reduces total cost, total emission and total inventory and the scope of emission reduction by increasing carbon price is greater at higher levels of cycle service level and demand coefficient of variation. The analysis of results helps supply chain managers to take right decision in different demand and service level situations.

Dynamics of cholesteric liquid crystals in the presence of a random magnetic field:stochastic dynamics of cholesteric liquid crystal

Based on dynamic renormalization group techniques, this letter analyzes the effects of external stochastic perturbations on the dynamical properties of cholesteric liquid crystals, studied in presence of a random magnetic field. Our analysis quantifies the nature of the temperature dependence of the dynamics; the results also highlight a hitherto unexplored regime in cholesteric liquid crystal dynamics. We show that stochastic fluctuations drive the system to a second-ordered Kosterlitz-Thouless phase transition point, eventually leading to a Kardar-Parisi-Zhang (KPZ) universality class. The results go beyond quasi-first order mean-field theories, and provides the first theoretical understanding of a KPZ phase in distorted nematic liquid crystal dynamics.

Stochastic calculations for fibre Raman amplifiers with randomly varying birefringence

For the first time for the model of real-world forward-pumped fibre Raman amplifier with the randomly varying birefringence, the stochastic calculations have been done numerically based on the Kloeden-Platen-Schurz algorithm. The results obtained for the averaged gain and gain fluctuations as a function of polarization mode dispersion (PMD) parameter agree quantitatively with the results of previously developed analytical model. Simultaneously, the direct numerical simulations demonstrate an increased stochastisation (maximum in averaged gain variation) within the region of the polarization mode dispersion parameter of 0.1÷0.3 ps/km1/2. The results give an insight into margins of applicability of a generic multi-scale technique widely used to derive coupled Manakov equations and allow generalizing analytic model with accounting for pump depletion, group-delay dispersion and Kerr-nonlinearity that is of great interest for development of the high-transmission-rates optical networks.

Stochastic anti-resonance in a fibre Raman amplifier

Stochastic anti-resonance, that is resonant enhancement of randomness caused by polarization mode beatings, is analyzed both numerically and analytically on an example of fibre Raman amplifier with randomly varying birefringence. As a result of such anti-resonance, the polarization mode dispersion growth causes an escape of the signal state of polarization from a metastable state corresponding to the pulling of the signal to the pump state of polarization.This phenomenon reveals itself in abrupt growth of gain fluctuations as well as in dropping of Hurst parameter and Kramers length characterizing long memory in a system and noise induced escape from the polarization pulling state. The results based on analytical multiscale averaging technique agree perfectly with the numerical data obtained by direct numerical simulations of underlying stochastic differential equations. This challenging outcome would allow replacing the cumbersome numerical simulations for real-world extra-long high-speed communication systems.

Dynamics of spatial heterogeneity in a landfill with interacting phase densities:a stochastic analysis

A landfill represents a complex and dynamically evolving structure that can be stochastically perturbed by exogenous factors. Both thermodynamic (equilibrium) and time varying (non-steady state) properties of a landfill are affected by spatially heterogenous and nonlinear subprocesses that combine with constraining initial and boundary conditions arising from the associated surroundings. While multiple approaches have been made to model landfill statistics by incorporating spatially dependent parameters on the one hand (data based approach) and continuum dynamical mass-balance equations on the other (equation based modelling), practically no attempt has been made to amalgamate these two approaches while also incorporating inherent stochastically induced fluctuations affecting the process overall. In this article, we will implement a minimalist scheme of modelling the time evolution of a realistic three dimensional landfill through a reaction-diffusion based approach, focusing on the coupled interactions of four key variables - solid mass density, hydrolysed mass density, acetogenic mass density and methanogenic mass density, that themselves are stochastically affected by fluctuations, coupled with diffusive relaxation of the individual densities, in ambient surroundings. Our results indicate that close to the linearly stable limit, the large time steady state properties, arising out of a series of complex coupled interactions between the stochastically driven variables, are scarcely affected by the biochemical growth-decay statistics. Our results clearly show that an equilibrium landfill structure is primarily determined by the solid and hydrolysed mass densities only rendering the other variables as statistically "irrelevant" in this (large time) asymptotic limit. The other major implication of incorporation of stochasticity in the landfill evolution dynamics is in the hugely reduced production times of the plants that are now approximately 20-30 years instead of the previous deterministic model predictions of 50 years and above. The predictions from this stochastic model are in conformity with available experimental observations.