On the Convergence of Two-Stage Iterative Processes for Solving Linear Equations

This paper considers two-stage iterative processes for solving the linear system $Af = b$. The outer iteration is defined by $Mf^{k + 1} = Nf^k + b$, where $M$ is a nonsingular matrix such that $M - N = A$. At each stage $f^{k + 1} $ is computed approximately using an inner iteration process to solve $Mv = Nf^k + b$ for $v$. At the $k$th outer iteration, $p_k $ inner iterations are performed. It is shown that this procedure converges if $p_k \geqq P$ for some $P$ provided that the inner iteration is convergent and that the outer process would converge if $f^{k + 1} $ were determined exactly at every step. Convergence is also proved under more specialized conditions, and for the procedure where $p_k = p$ for all $k$, an estimate for $p$ is obtained which optimizes the convergence rate. Examples are given for systems arising from the numerical solution of elliptic partial differential equations and numerical results are presented.

Model selection approaches for non-linear system identification: a review

The identification of non-linear systems using only observed finite datasets has become a mature research area over the last two decades. A class of linear-in-the-parameter models with universal approximation capabilities have been intensively studied and widely used due to the availability of many linear-learning algorithms and their inherent convergence conditions. This article presents a systematic overview of basic research on model selection approaches for linear-in-the-parameter models. One of the fundamental problems in non-linear system identification is to find the minimal model with the best model generalisation performance from observational data only. The important concepts in achieving good model generalisation used in various non-linear system-identification algorithms are first reviewed, including Bayesian parameter regularisation and models selective criteria based on the cross validation and experimental design. A significant advance in machine learning has been the development of the support vector machine as a means for identifying kernel models based on the structural risk minimisation principle. The developments on the convex optimisation-based model construction algorithms including the support vector regression algorithms are outlined. Input selection algorithms and on-line system identification algorithms are also included in this review. Finally, some industrial applications of non-linear models are discussed.

A critique of neural networks for discrete-time linear control

This paper discusses the use of multi-layer perceptron networks for linear or linearizable, adaptive feedback.control schemes in a discrete-time environment. A close look is taken at the model structure selected and the extent of the resulting parametrization. A comparison is made with standard, non-perceptron algorithms, e.g. self-tuning control, and it is shown how gross over-parametrization can occur in the neural network case. Because of the resultant heavy computational burden and poor controller convergence, a strong case is made against the use of neural networks for discrete-time linear control.

Optimal control of a class of discrete–continuous non-linear systems — decomposition and hierarchical structure

A technique is derived for solving a non-linear optimal control problem by iterating on a sequence of simplified problems in linear quadratic form. The technique is designed to achieve the correct solution of the original non-linear optimal control problem in spite of these simplifications. A mixed approach with a discrete performance index and continuous state variable system description is used as the basis of the design, and it is shown how the global problem can be decomposed into local sub-system problems and a co-ordinator within a hierarchical framework. An analysis of the optimality and convergence properties of the algorithm is presented and the effectiveness of the technique is demonstrated using a simulation example with a non-separable performance index.

Plane wave approximation in linear elasticity

We consider the approximation of solutions of the time-harmonic linear elastic wave equation by linear combinations of plane waves. We prove algebraic orders of convergence both with respect to the dimension of the approximating space and to the diameter of the domain. The error is measured in Sobolev norms and the constants in the estimates explicitly depend on the problem wavenumber. The obtained estimates can be used in the h- and p-convergence analysis of wave-based finite element schemes.

Approximate Gauss-Newton methods for nonlinear least squares problems

The Gauss–Newton algorithm is an iterative method regularly used for solving nonlinear least squares problems. It is particularly well suited to the treatment of very large scale variational data assimilation problems that arise in atmosphere and ocean forecasting. The procedure consists of a sequence of linear least squares approximations to the nonlinear problem, each of which is solved by an “inner” direct or iterative process. In comparison with Newton’s method and its variants, the algorithm is attractive because it does not require the evaluation of second-order derivatives in the Hessian of the objective function. In practice the exact Gauss–Newton method is too expensive to apply operationally in meteorological forecasting, and various approximations are made in order to reduce computational costs and to solve the problems in real time. Here we investigate the effects on the convergence of the Gauss–Newton method of two types of approximation used commonly in data assimilation. First, we examine “truncated” Gauss–Newton methods where the inner linear least squares problem is not solved exactly, and second, we examine “perturbed” Gauss–Newton methods where the true linearized inner problem is approximated by a simplified, or perturbed, linear least squares problem. We give conditions ensuring that the truncated and perturbed Gauss–Newton methods converge and also derive rates of convergence for the iterations. The results are illustrated by a simple numerical example. A practical application to the problem of data assimilation in a typical meteorological system is presented.

On the convergence of the no response test

The no response test is a new scheme in inverse problems for partial differential equations which was recently proposed in [D. R. Luke and R. Potthast, SIAM J. Appl. Math., 63 (2003), pp. 1292–1312] in the framework of inverse acoustic scattering problems. The main idea of the scheme is to construct special probing waves which are small on some test domain. Then the response for these waves is constructed. If the response is small, the unknown object is assumed to be a subset of the test domain. The response is constructed from one, several, or many particular solutions of the problem under consideration. In this paper, we investigate the convergence of the no response test for the reconstruction information about inclusions D from the Cauchy values of solutions to the Helmholtz equation on an outer surface $\partial\Omega$ with $\overline{D} \subset \Omega$. We show that the one‐wave no response test provides a criterion to test the analytic extensibility of a field. In particular, we investigate the construction of approximations for the set of singular points $N(u)$ of the total fields u from one given pair of Cauchy data. Thus, the no response test solves a particular version of the classical Cauchy problem. Also, if an infinite number of fields is given, we prove that a multifield version of the no response test reconstructs the unknown inclusion D. This is the first convergence analysis which could be achieved for the no response test.

Linear viscoelasticity from molecular dynamics simulation of entangled polymers

The linear viscoelastic (LVE) spectrum is one of the primary fingerprints of polymer solutions and melts, carrying information about most relaxation processes in the system. Many single chain theories and models start with predicting the LVE spectrum to validate their assumptions. However, until now, no reliable linear stress relaxation data were available from simulations of multichain systems. In this work, we propose a new efficient way to calculate a wide variety of correlation functions and mean-square displacements during simulations without significant additional CPU cost. Using this method, we calculate stress−stress autocorrelation functions for a simple bead−spring model of polymer melt for a wide range of chain lengths, densities, temperatures, and chain stiffnesses. The obtained stress−stress autocorrelation functions were compared with the single chain slip−spring model in order to obtain entanglement related parameters, such as the plateau modulus or the molecular weight between entanglements. Then, the dependence of the plateau modulus on the packing length is discussed. We have also identified three different contributions to the stress relaxation:  bond length relaxation, colloidal and polymeric. Their dependence on the density and the temperature is demonstrated for short unentangled systems without inertia.

Linear amplification of marginally neutral baroclinic waves

Baroclinic wave development is investigated for unstable parallel shear flows in the limit of vanishing normal-mode growth rate. This development is described in terms of the propagation and interaction mechanisms of two coherent structures, called counter-propagating Rossby waves (CRWs). It is shown that, in this limit of vanishing normal-mode growth rate, arbitrary initial conditions produce sustained linear amplification of the marginally neutral normal mode (mNM). This linear excitation of the mNM is subsequently interpreted in terms of a resonance phenomenon. Moreover, while the mathematical character of the normal-mode problem changes abruptly as the bifurcation point in the dispersion diagram is encountered and crossed, it is shown that from an initial-value viewpoint, this transition is smooth. Consequently, the resonance interpretation remains relevant (albeit for a finite time) for wavenumbers slightly different from the ones defining cut-off points. The results are further applied to a two-layer version of the classic Eady model in which the upper rigid lid has been replaced by a simple stratosphere.