A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems

Even though dynamic programming offers an optimal control solution in a state feedback form, the method is overwhelmed by computational and storage requirements. Approximate dynamic programming implemented with an Adaptive Critic (AC) neural network structure has evolved as a powerful alternative technique that obviates the need for excessive computations and storage requirements in solving optimal control problems. In this paper, an improvement to the AC architecture, called the �Single Network Adaptive Critic (SNAC)� is presented. This approach is applicable to a wide class of nonlinear systems where the optimal control (stationary) equation can be explicitly expressed in terms of the state and costate variables. The selection of this terminology is guided by the fact that it eliminates the use of one neural network (namely the action network) that is part of a typical dual network AC setup. As a consequence, the SNAC architecture offers three potential advantages: a simpler architecture, lesser computational load and elimination of the approximation error associated with the eliminated network. In order to demonstrate these benefits and the control synthesis technique using SNAC, two problems have been solved with the AC and SNAC approaches and their computational performances are compared. One of these problems is a real-life Micro-Electro-Mechanical-system (MEMS) problem, which demonstrates that the SNAC technique is applicable to complex engineering systems.

Adaptive optimal tuning of a general class of stable LTI systems with restricted inputs

The problem addressed is one of model reference adaptive control (MRAC) of asymptotically stable plants of unknown order with zeros located anywhere in the s-plane except at the origin. The reference model is also asymptotically stable and lacking zero(s) at s = 0. The control law is to be specified only in terms of the inputs to and outputs of the plant and the reference model. For inputs from a class of functions that approach a non-zero constant, the problem is formulated in an optimal control framework. By successive refinements of the sub-optimal laws proposed here, two schemes are finally design-ed. These schemes are characterized by boundedness, convergence and optimality. Simplicity and total time-domain implementation are the additional striking features. Simulations to demonstrate the efficacy of the control schemes are presented.

Optimal management of beaver population using a reduced-order distributed parameter model and single network adaptive critics

Beavers are often found to be in conflict with human interests by creating nuisances like building dams on flowing water (leading to flooding), blocking irrigation canals, cutting down timbers, etc. At the same time they contribute to raising water tables, increased vegetation, etc. Consequently, maintaining an optimal beaver population is beneficial. Because of their diffusion externality (due to migratory nature), strategies based on lumped parameter models are often ineffective. Using a distributed parameter model for beaver population that accounts for their spatial and temporal behavior, an optimal control (trapping) strategy is presented in this paper that leads to a desired distribution of the animal density in a region in the long run. The optimal control solution presented, imbeds the solution for a large number of initial conditions (i.e., it has a feedback form), which is otherwise nontrivial to obtain. The solution obtained can be used in real-time by a nonexpert in control theory since it involves only using the neural networks trained offline. Proper orthogonal decomposition-based basis function design followed by their use in a Galerkin projection has been incorporated in the solution process as a model reduction technique. Optimal solutions are obtained through a "single network adaptive critic" (SNAC) neural-network architecture.

An integrated model for optimal reservoir operation for irrigation of multiple crops

An integrated model is developed, based on seasonal inputs of reservoir inflow and rainfall in the irrigated area, to determine the optimal reservoir release policies and irrigation allocations to multiple crops. The model is conceptually made up of two modules, Module 1 is an intraseasonal allocation model to maximize the sum of relative yields of all crops, for a given state of the system, using linear programming (LP). The module takes into account reservoir storage continuity, soil moisture balance, and crop root growth with time. Module 2 is a seasonal allocation model to derive the steady state reservoir operating policy using stochastic dynamic programming (SDP). Reservoir storage, seasonal inflow, and seasonal rainfall are the state variables in the SDP. The objective in SDP is to maximize the expected sum of relative yields of all crops in a year. The results of module 1 and the transition probabilities of seasonal inflow and rainfall form the input for module 2. The use of seasonal inputs coupled with the LP-SDP solution strategy in the present formulation facilitates in relaxing the limitations of an earlier study, while affecting additional improvements. The model is applied to an existing reservoir in Karnataka State, India.

Active vibration suppression of beams with discrete actuators using optimal dynamic inversion

Most of the structural elements like beams, cables etc. are flexible and should be modeled as distributed parameter systems (DPS) to represent the reality better. For large structures, the usual approach of 'modal representation' is not an accurate representation. Moreover, for excessive vibrations (possibly due to strong wind, earthquake etc.), external power source (controller) is needed to suppress it, as the natural damping of these structures is usually small. In this paper, we propose to use a recently developed optimal dynamic inversion technique to design a set of discrete controllers for this purpose. We assume that the control force to the structure is applied through finite number of actuators, which are located at predefined locations in the spatial domain. The method used in this paper determines control forces directly from the partial differential equation (PDE) model of the system. The formulation has better practical significance, both because it leads to a closed form solution of the controller (hence avoids computational issues) as well as because a set of discrete actuators along the spatial domain can be implemented with relative ease (as compared to a continuous actuator)

An optimal dynamic inversion approach for controlling a class of one-dimensional nonlinear distributed parameter systems

Combining the principles of dynamic inversion and optimization theory, a new approach is presented for stable control of a class of one-dimensional nonlinear distributed parameter systems, assuming the availability a continuous actuator in the spatial domain. Unlike the existing approximate-then-design and design-then-approximate techniques, here there is no need of any approximation either of the system dynamics or of the resulting controller. Rather, the control synthesis approach is fairly straight-forward and simple. The controller formulation has more elegance because we can prove the convergence of the controller to its steady state value. To demonstrate the potential of the proposed technique, a real-life temperature control problem for a heat transfer application is solved. It has been demonstrated that a desired temperature profile can be achieved starting from any arbitrary initial temperature profile.

Genetic regulation of flower development

Flower development provides a model system to study mechanisms that govern pattern formation in plants. Most flowers consist of four organ types that are present in a specific order from the periphery to the centre of the flower. Reviewed here are studies on flower development in two model species: Arabidopsis thaliana and Antirrhinum majus that focus on the molecular genetic analysis of homeotic mutations affecting pattern formation in the flower. Based on these studies a model was proposed that explains how three classes of regulatory genes can together control the development of the correct pattern of organs in the flower. The universality of the basic tenets of the model is apparent from the analysis of the homologues of the Arabidopsis genes from other plant species

On generating optimal signal probabilities for random tests: A genetic approach

Genetic Algorithms are robust search and optimization techniques. A Genetic Algorithm based approach for determining the optimal input distributions for generating random test vectors is proposed in the paper. A cost function based on the COP testability measure for determining the efficacy of the input distributions is discussed, A brief overview of Genetic Algorithms (GAs) and the specific details of our implementation are described. Experimental results based on ISCAS-85 benchmark circuits are presented. The performance pf our GA-based approach is compared with previous results. While the GA generates more efficient input distributions than the previous methods which are based on gradient descent search, the overheads of the GA in computing the input distributions are larger. To account for the relatively quick convergence of the gradient descent methods, we analyze the landscape of the COP-based cost function. We prove that the cost function is unimodal in the search space. This feature makes the cost function amenable to optimization by gradient-descent techniques as compared to random search methods such as Genetic Algorithms.

Optimal control of arrivals to queues with delayed queue length information

We consider discrete-time versions of two classical problems in the optimal control of admission to a queueing system: i) optimal routing of arrivals to two parallel queues and ii) optimal acceptance/rejection of arrivals to a single queue. We extend the formulation of these problems to permit a k step delay in the observation of the queue lengths by the controller. For geometric inter-arrival times and geometric service times the problems are formulated as controlled Markov chains with expected total discounted cost as the minimization objective. For problem i) we show that when k = 1, the optimal policy is to allocate an arrival to the queue with the smaller expected queue length (JSEQ: Join the Shortest Expected Queue). We also show that for this problem, for k greater than or equal to 2, JSEQ is not optimal. For problem ii) we show that when k = 1, the optimal policy is a threshold policy. There are, however, two thresholds m(0) greater than or equal to m(1) > 0, such that mo is used when the previous action was to reject, and mi is used when the previous action was to accept.

Secretion of endocrine signals by the primate embryo during the peri-implantation period

Development of preimplantation embryos and blastocyst implantation are critical early events in the establishment of pregnancy. In primates, embryonic signals, secreted during the peri-implantation period, are believed to play a major role in the regulation of embryonic differentiation and implantation. However, only limited progress has been made in the molecular and functional characterization of embryonic signals, partly due to severe paucity of primate embryos and the lack of optimal culture conditions to obtain viable embryo development. Two embryonic (endocrine) secretions, i.e. chorionic gonadotrophin (CG) and gonadotrophin releasing hormone (GnRH) are being studied. This article reviews the current status of knowledge on the recovery and culture of embryos, their secretion of CG, GnRH and other potential endocrine signals and their regulation and physiological role(s) during the peri-implantation period in primates, including humans.

The precision of regulation in Dictyostelium discoideum: implications for cell-type proportioning in the absence of spatial pattern

We have made careful counts of the exact number of spore, stalk and basal disc cells in small fruiting bodies of Dictyostelium discoideum (undifferentiated amoebae are found only rarely and on average their fraction is 4.96 x 10(-4)). (i) Within aggregates of a given size, the relative apportioning of amoebae to the main cell types occurs with a remarkable degree of precision. In most cases the coefficient of variation (c.v.) in the mean fraction of cells that form spores is within 4.86%. The contribution of stalk and basal disc cells is highly variable when considered separately (c.v.'s upto 25% and 100%, respectively), but markedly less so when considered together. Calculations based on theoretical models indicate that purely cell-autonomous specification of cell, fate cannot account for die observed accuracy of proportioning. Cell-autonomous determination to a prestalk or prespore condition followed by cell type interconversion, and stabilised by feedbacks, suffices to explain the measured accuracy. (ii) The fraction of amoebae that differentiates into spores increases monotonically with the total number of cells. This fraction rises from an average of 73.6% for total cell numbers below 30 and reaches 86.0% for cell numbers between 170 and 200 (it remains steady thereafter at around 86%). Correspondingly, the fraction of amoebae differentiating into stalk or basal disc decreases viith total size. These trends are in accordance with evolutionary expectations and imply that a mechanism for sensing the overall size of the aggregate also plays an essential role in the determination of cell-type proportions.

In vitro and in vivo regulation of assimilatory nitrite reductase from Candida utilis

The nitrate assimilation pathway in Candida utilis, as in other assimilatory organisms, is mediated by two enzymes: nitrate reductase and nitrite reductase. Purified nitrite reductase has been shown to be a heterodimer consisting of 58- and 66-kDa subunits. In the present study, nitrite reductase was found to be capable of utilising both NADH and NADPH as electron donors. FAD, which is an essential coenzyme, stabilised the enzyme during the purification process. The enzyme was modified by cysteine modifiers, and the inactivation could be reversed by thiol reagents. One cysteine was demonstrated to be essential for the enzymatic activity. In vitro, the enzyme was inactivated by ammonium salts, the end product of the path way, proving that the enzyme is assimilatory in function. In vivo, the enzyme was induced by nitrate and repressed by ammonium ions. During induction and repression, the levels of nitrite reductase mRNA, protein, and enzyme activity were modulated together, which indicated that the primary level of regulation of this enzyme was at the transcriptional level. When the enzyme was incubated with ammonium salts in vitro or when the enzyme was assayed in cells grown with the same salts as the source of nitrogen, the residual enzymatic activities were similar. Thus, a study of the in vitro inactivation can give a clue to understanding the mechanism of in vivo regulation of nitrite reductase in Candida utilis.

Regulation of progesterone biosynthesis in the human placenta by estradiol 17 beta and progesterone

Ex vivo addition of estradiol 17 beta to first trimester or term human placental minces caused a significant increase in the quantity of progesterone produced. Addition of an aromatase inhibitor, CGS 16949 A or the estrogen receptor antagonist, ICI 182780, significantly inhibited progesterone production confirming the role of estradiol 17 beta in the regulation of progesterone synthesis in human placenta. RU 486 and ZK 98299, which are antagonists of progesterone receptor, significantly modulated progesterone synthesis in the human placenta but exhibited paradoxical effects on the first trimester and term placenta We conclude that progesterone synthesis in the human placenta is regulated by estradiol 17 beta and progesterone. This is the first report providing evidence for autoregulation of progesterone synthesis in the human placenta.