Control of Agent Swarms using Generalized Centroidal Cyclic Pursuit Laws

One of the major tasks in swarm intelligence is to design decentralized but homogenoeus strategies to enable controlling the behaviour of swarms of agents. It has been shown in the literature that the point of convergence and motion of a swarm of autonomous mobile agents can be controlled by using cyclic pursuit laws. In cyclic pursuit, there exists a predefined cyclic connection between agents and each agent pursues the next agent in the cycle. In this paper we generalize this idea to a case where an agent pursues a point which is the weighted average of the positions of the remaining agents. This point correspond to a particular pursuit sequence. Using this concept of centroidal cyclic pursuit, the behavior of the agents is analyzed such that, by suitably selecting the agents' gain, the rendezvous point of the agents can be controlled, directed linear motion of the agents can be achieved, and the trajectories of the agents can be changed by switching between the pursuit sequences keeping some of the behaviors of the agents invariant. Simulation experiments are given to support the analytical proofs.

Relationship between the controllability grammian and closed-loop eigenvalues: the single input case

The controllability grammian is important in many control applications. Given a set of closed-loop eigenvalues the corresponding controllability grammian can be obtained by computing the controller which assigns the eigenvalues and then by solving the Lyapunov equation that defines the grammian. The relationship between the controllability grammian, resulting from state feedback, and the closed-loop eigenvalues of a single input linear time invariant (LTI) system is obtained. The proposed methodology does not require the computation of the controller that assigns the specified eigenvalues. The closed-loop system matrix is obtained from the knowledge of the open-loop system matrix, control influence matrix and the specified closed-loop eigenvalues. Knowing the closed-loop system matrix, the grammian is then obtained from the solution of the Lyapunov equation that defines it. Finally the proposed idea is extended to find the state covariance matrix for a specified set of closed-loop eigenvalues (without computing the controller), due to impulsive input in the disturbance channel and to solve the eigenvalue assignment problem for the single input case.

Matrix method for eigenstructure assignment - The multi-input case with application

The eigenvalue and eigenstructure assignment procedure has found application in a wide variety of control problems. In this paper a method for assigning eigenstructure to a linear time invariant multi-input system is proposed. The algorithm determines a matrix that has eigenvalues and eigenvectors at the desired locations. It is obtained from the knowledge of the open-loop system and the desired eigenstructure. Solution of the matrix equation, involving unknown controller gams, open-loop system matrices, and desired eigenvalues and eigenvectors, results hi the state feedback controller. The proposed algorithm requires the closed-loop eigenvalues to be different from those of the open-loop case. This apparent constraint can easily be overcome by a negligible shift in the values. Application of the procedure is illustrated through the offset control of a satellite supported, from an orbiting platform, by a flexible tether.

Matrix method for eigenvalue assignment: The single input case

The eigenvalue assignment/pole placement procedure has found application in a wide variety of control problems. The associated literature is rather extensive with a number of techniques discussed to that end. In this paper a method for assigning eigenvalues to a Linear Time Invariant (LTI) single input system is proposed. The algorithm determines a matrix, which has eigenvalues at the desired locations. It is obtained from the knowledge of the open-loop system and the desired eigenvalues. Solution of the matrix equation, involving unknown controller gains, open-loop system matrices and desired eigenvalues, results in the state feedback controller. The proposed algorithm requires the closed-loop eigenvalues to be different from those of the open-loop case. This apparent constraint is easily overcome by a negligible shift in the values. Two examples are considered to verify the proposed algorithm. The first one pertains to the in-plane libration of a Tethered Satellite System (TSS) while the second is concerned with control of the short period dynamics of a flexible airplane. Finally, the method is extended to determine the Controllability Grammian, corresponding to the specified closed-loop eigenvalues, without computing the controller gains.

Matrix method for eigenstructure assignment - The multi-input case with application

The eigenvalue and eigenstructure assignment procedure has found application in a wide variety of control problems. In this paper a method for assigning eigenstructure to a Linear time invariant multi-input system is proposed. The algorithm determines a matrix that has eigenvalues and eigenvectors at the desired locations. It is obtained from the knowledge of the open-loop system and the desired eigenstructure. solution of the matrix equation, involving unknown controller gains, open-loop system matrices, and desired eigenvalues and eigenvectors, results in the state feedback controller. The proposed algorithm requires the closed-loop eigenvalues to be different from those of the open-loop case. This apparent constraint can easily be overcome by a negligible shift in the values. Application of the procedure is illustrated through the offset control of a satellite supported, from an orbiting platform, by a flexible tether,

Matrix method for eigenvalue assignment: The single input case

The eigenvalue assignment/pole placement procedure has found application in a wide variety of control problems. The associated literature is rather extensive with a number of techniques discussed to that end. In this paper a method for assigning eigenvalues to a Linear Time Invariant (LTI) single input system is proposed. The algorithm determines a matrix, which has eigenvalues at the desired locations. It is obtained from the knowledge of the open-loop system and the desired eigenvalues. Solution of the matrix equation, involving unknown controller gains, open-loop system matrices and desired eigenvalues, results in the state feedback controller. The proposed algorithm requires the closed-loop eigenvalues to be different from those of the open-loop case. This apparent constraint is easily overcome by a negligible shift in the values. Two examples are considered to verify the proposed algorithm. The first one pertains to the in-plane libration of a Tethered Satellite System (TSS) while the second is concerned with control of the short period dynamics of a flexible airplane. Finally, the method is extended to determine the Controllability Grammian, corresponding to the specified closed-loop eigenvalues, without computing the controller gains.

Robust/Optimal Temperature Profile Control of a High-Speed Aerospace Vehicle Using Neural Networks

An approximate dynamic programming (ADP)-based suboptimal neurocontroller to obtain desired temperature for a high-speed aerospace vehicle is synthesized in this paper. A I-D distributed parameter model of a fin is developed from basic thermal physics principles. "Snapshot" solutions of the dynamics are generated with a simple dynamic inversion-based feedback controller. Empirical basis functions are designed using the "proper orthogonal decomposition" (POD) technique and the snapshot solutions. A low-order nonlinear lumped parameter system to characterize the infinite dimensional system is obtained by carrying out a Galerkin projection. An ADP-based neurocontroller with a dual heuristic programming (DHP) formulation is obtained with a single-network-adaptive-critic (SNAC) controller for this approximate nonlinear model. Actual control in the original domain is calculated with the same POD basis functions through a reverse mapping. Further contribution of this paper includes development of an online robust neurocontroller to account for unmodeled dynamics and parametric uncertainties inherent in such a complex dynamic system. A neural network (NN) weight update rule that guarantees boundedness of the weights and relaxes the need for persistence of excitation (PE) condition is presented. Simulation studies show that in a fairly extensive but compact domain, any desired temperature profile can be achieved starting from any initial temperature profile. Therefore, the ADP and NN-based controllers appear to have the potential to become controller synthesis tools for nonlinear distributed parameter systems.

A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems

Even though dynamic programming offers an optimal control solution in a state feedback form, the method is overwhelmed by computational and storage requirements. Approximate dynamic programming implemented with an Adaptive Critic (AC) neural network structure has evolved as a powerful alternative technique that obviates the need for excessive computations and storage requirements in solving optimal control problems. In this paper, an improvement to the AC architecture, called the �Single Network Adaptive Critic (SNAC)� is presented. This approach is applicable to a wide class of nonlinear systems where the optimal control (stationary) equation can be explicitly expressed in terms of the state and costate variables. The selection of this terminology is guided by the fact that it eliminates the use of one neural network (namely the action network) that is part of a typical dual network AC setup. As a consequence, the SNAC architecture offers three potential advantages: a simpler architecture, lesser computational load and elimination of the approximation error associated with the eliminated network. In order to demonstrate these benefits and the control synthesis technique using SNAC, two problems have been solved with the AC and SNAC approaches and their computational performances are compared. One of these problems is a real-life Micro-Electro-Mechanical-system (MEMS) problem, which demonstrates that the SNAC technique is applicable to complex engineering systems.

Reduced Order Suboptimal Control Design for a Class of Nonlinear Distributed Parameter Systems Using POD and /spl thetas/-D Techniques

A new computational tool is presented in this paper for suboptimal control design of a class of nonlinear distributed parameter systems. First proper orthogonal decomposition based problem-oriented basis functions are designed, which are then used in a Galerkin projection to come up with a low-order lumped parameter approximation. Next, a suboptimal controller is designed using the emerging /spl thetas/-D technique for lumped parameter systems. This time domain sub-optimal control solution is then mapped back to the distributed domain using the same basis functions, which essentially leads to a closed form solution for the controller in a state feedback form. Numerical results for a real-life nonlinear temperature control problem indicate that the proposed method holds promise as a good suboptimal control design technique for distributed parameter systems.

Clan-based evolutionary approach for solving control problems

This paper presents a dan-based evolutionary approach for solving control problems. Three selected control problems, viz. linear-quadratic, harvest, and push-cart problems, are solved using the proposed approach. Results are compared with those of the evolutionary programming (EP) approach. In most of the cases, the proposed approach is successful in obtaining (near) optimal solutions for these selected problems.

Reinforcement Learning With Function Approximation for Traffic Signal Control

We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e. g., the work of Abdulhai et al., on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhai et al. and Cools et al., as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of Abdulhai et al. We observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.

Performance analysis of single-symbol maximum likelihood decodable linear STBCs

Performance of space-time block codes can be improved using the coordinate interleaving of the input symbols from rotated M-ary phase shift keying (MPSK) and M-ary quadrature amplitude modulation (MQAM) constellations. This paper is on the performance analysis of coordinate-interleaved space-time codes, which are a subset of single-symbol maximum likelihood decodable linear space-time block codes, for wireless multiple antenna terminals. The analytical and simulation results show that full diversity is achievable. Using the equivalent single-input single-output model, simple expressions for the average bit error rates are derived over flat uncorrelated Rayleigh fading channels. Optimum rotation angles are found by finding the minimum of the average bit error rate curves.

Risk-sensitive optimal control for Markov decision processes with monotone cost

The existence of an optimal feedback law is established for the risk-sensitive optimal control problem with denumerable state space. The main assumptions imposed are irreducibility and a near monotonicity condition on the one-step cost function. A solution can be found constructively using either value iteration or policy iteration under suitable conditions on initial feedback law.

An adaptive cascade compensator for a wide class of stable minimum or nonminimum phase plants through time moment estimation

Model Reference Adaptive Control (MRAC) of a wide repertoire of stable Linear Time Invariant (LTI) systems is addressed here. Even an upper bound on the order of the finite-dimensional system is unavailable. Further, the unknown plant is permitted to have both minimum phase and nonminimum phase zeros. Model following with reference to a completely specified reference model excited by a class of piecewise continuous bounded signals is the goal. The problem is approached by taking recourse to the time moments representation of an LTI system. The treatment here is confined to Single-Input Single-Output (SISO) systems. The adaptive controller is built upon an on-line scheme for time moment estimation of a system given no more than its input and output. As a first step, a cascade compensator is devised. The primary contribution lies in developing a unified framework to eventually address with more finesse the problem of adaptive control of a large family of plants allowed to be minimum or nonminimum phase. Thus, the scheme presented in this paper is confined to lay the basis for more refined compensators-cascade, feedback and both-initially for SISO systems and progressively for Multi-Input Multi-Output (MIMO) systems. Simulations are presented.