Improving ultimate convergence of an augmented Lagrangian method

Optimization methods that employ the classical Powell-Hestenes-Rockafellar augmented Lagrangian are useful tools for solving nonlinear programming problems. Their reputation decreased in the last 10 years due to the comparative success of interior-point Newtonian algorithms, which are asymptotically faster. In this research, a combination of both approaches is evaluated. The idea is to produce a competitive method, being more robust and efficient than its `pure` counterparts for critical problems. Moreover, an additional hybrid algorithm is defined, in which the interior-point method is replaced by the Newtonian resolution of a Karush-Kuhn-Tucker (KKT) system identified by the augmented Lagrangian algorithm. The software used in this work is freely available through the Tango Project web page:http://www.ime.usp.br/similar to egbirgin/tango/.

Uniform Convergence of the Newton Method for Aubin Continuous Maps

* This work was supported by National Science Foundation grant DMS 9404431.

The Graves Theorem Revisited II: Robust Convergence of the Newton Method

AMS subject classification: 65J15, 47H04, 90C30.

On the proximal Landweber Newton method for a class of nonsmooth convex problems

We consider a class of nonsmooth convex optimization problems where the objective function is a convex differentiable function regularized by the sum of the group reproducing kernel norm and (Formula presented.)-norm of the problem variables. This class of problems has many applications in variable selections such as the group LASSO and sparse group LASSO. In this paper, we propose a proximal Landweber Newton method for this class of convex optimization problems, and carry out the convergence and computational complexity analysis for this method. Theoretical analysis and numerical results show that the proposed algorithm is promising.

Fixed Points of Closed and Compact Composite Sequences of Operators and Projectors in a Class of Banach Spaces

Some results on fixed points related to the contractive compositions of bounded operators in a class of complete metric spaces which can be also considered as Banach's spaces are discussed through the paper. The class of composite operators under study can include, in particular, sequences of projection operators under, in general, oblique projective operators. In this paper we are concerned with composite operators which include sequences of pairs of contractive operators involving, in general, oblique projection operators. The results are generalized to sequences of, in general, nonconstant bounded closed operators which can have bounded, closed, and compact limit operators, such that the relevant composite sequences are also compact operators. It is proven that in both cases, Banach contraction principle guarantees the existence of unique fixed points under contractive conditions.

Fast automatic two-stage nonlinear model identification based on the extreme learning machine

It is convenient and effective to solve nonlinear problems with a model that has a linear-in-the-parameters (LITP) structure. However, the nonlinear parameters (e.g. the width of Gaussian function) of each model term needs to be pre-determined either from expert experience or through exhaustive search. An alternative approach is to optimize them by a gradient-based technique (e.g. Newton’s method). Unfortunately, all of these methods still need a lot of computations. Recently, the extreme learning machine (ELM) has shown its advantages in terms of fast learning from data, but the sparsity of the constructed model cannot be guaranteed. This paper proposes a novel algorithm for automatic construction of a nonlinear system model based on the extreme learning machine. This is achieved by effectively integrating the ELM and leave-one-out (LOO) cross validation with our two-stage stepwise construction procedure [1]. The main objective is to improve the compactness and generalization capability of the model constructed by the ELM method. Numerical analysis shows that the proposed algorithm only involves about half of the computation of orthogonal least squares (OLS) based method. Simulation examples are included to confirm the efficacy and superiority of the proposed technique.

Approximate Gauss-Newton methods for nonlinear least squares problems

The Gauss–Newton algorithm is an iterative method regularly used for solving nonlinear least squares problems. It is particularly well suited to the treatment of very large scale variational data assimilation problems that arise in atmosphere and ocean forecasting. The procedure consists of a sequence of linear least squares approximations to the nonlinear problem, each of which is solved by an “inner” direct or iterative process. In comparison with Newton’s method and its variants, the algorithm is attractive because it does not require the evaluation of second-order derivatives in the Hessian of the objective function. In practice the exact Gauss–Newton method is too expensive to apply operationally in meteorological forecasting, and various approximations are made in order to reduce computational costs and to solve the problems in real time. Here we investigate the effects on the convergence of the Gauss–Newton method of two types of approximation used commonly in data assimilation. First, we examine “truncated” Gauss–Newton methods where the inner linear least squares problem is not solved exactly, and second, we examine “perturbed” Gauss–Newton methods where the true linearized inner problem is approximated by a simplified, or perturbed, linear least squares problem. We give conditions ensuring that the truncated and perturbed Gauss–Newton methods converge and also derive rates of convergence for the iterations. The results are illustrated by a simple numerical example. A practical application to the problem of data assimilation in a typical meteorological system is presented.

Aplicação do método de Newton desacoplado para o fluxo de carga continuado

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Solução numérica das equações de Navier-Stokes em um canal-tipo estenose usando métodos compactos e não compacos de alta ordem

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Proposição de uma técnica de parametrização geométrica para o fluxo de carga continuado

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Desenvolvimento de metodologias para a localização de intruso em ambientes indoor

O presente trabalho propõe metodologias para detectar a presença e localizar um intruso em ambientes indoor, 2-D e 3-D, sendo que neste último, utiliza-se um sistema cooperativo de antenas e, em ambos os casos, o sistema é baseado em radares multiestáticos. Para obter uma alta resolução, o radar opera com pulsos UWB, que possuem amplitude espectral máxima em 1 GHz para ambientes 2-D e, pulsos de banda larga com frequências entre 200 MHz e 500 MHz para ambientes 3-D. A estimativa de localização, para os ambientes bidimensionais, é feita pela técnica de otimização Enxame de Partículas - PSO (Particle Swarm Optimization), pelo método de Newton com eliminação de Gauss e pelo método dos mínimos quadrados com eliminação de Gauss. Para o ambiente tridimensional, foi desenvolvida uma metodologia vetorial que estima uma possível região de localização do intruso. Para a simulação das ondas eletromagnéticas se utiliza o método numérico FDTD (Diferenças Finitas no Domínio do Tempo) associado à técnica de absorção UPML (Uniaxial Perfectly Matched Layer) com o objetivo de truncar o domínio de análise simulando uma propagação ao infinito. Para a análise do ambiente em 2-D foi desenvolvido o ACOR-UWB-2-D e para o ambiente 3-D foi utilizado o software LANE SAGS.

HIGH PERFORMANCE, LOW COST SUBSPACE DECOMPOSITION AND POLYNOMIAL ROOTING FOR REAL TIME DIRECTION OF ARRIVAL ESTIMATION: ANALYSIS AND IMPLEMENTATION

This thesis develops high performance real-time signal processing modules for direction of arrival (DOA) estimation for localization systems. It proposes highly parallel algorithms for performing subspace decomposition and polynomial rooting, which are otherwise traditionally implemented using sequential algorithms. The proposed algorithms address the emerging need for real-time localization for a wide range of applications. As the antenna array size increases, the complexity of signal processing algorithms increases, making it increasingly difficult to satisfy the real-time constraints. This thesis addresses real-time implementation by proposing parallel algorithms, that maintain considerable improvement over traditional algorithms, especially for systems with larger number of antenna array elements. Singular value decomposition (SVD) and polynomial rooting are two computationally complex steps and act as the bottleneck to achieving real-time performance. The proposed algorithms are suitable for implementation on field programmable gated arrays (FPGAs), single instruction multiple data (SIMD) hardware or application specific integrated chips (ASICs), which offer large number of processing elements that can be exploited for parallel processing. The designs proposed in this thesis are modular, easily expandable and easy to implement. Firstly, this thesis proposes a fast converging SVD algorithm. The proposed method reduces the number of iterations it takes to converge to correct singular values, thus achieving closer to real-time performance. A general algorithm and a modular system design are provided making it easy for designers to replicate and extend the design to larger matrix sizes. Moreover, the method is highly parallel, which can be exploited in various hardware platforms mentioned earlier. A fixed point implementation of proposed SVD algorithm is presented. The FPGA design is pipelined to the maximum extent to increase the maximum achievable frequency of operation. The system was developed with the objective of achieving high throughput. Various modern cores available in FPGAs were used to maximize the performance and details of these modules are presented in detail. Finally, a parallel polynomial rooting technique based on Newton’s method applicable exclusively to root-MUSIC polynomials is proposed. Unique characteristics of root-MUSIC polynomial’s complex dynamics were exploited to derive this polynomial rooting method. The technique exhibits parallelism and converges to the desired root within fixed number of iterations, making this suitable for polynomial rooting of large degree polynomials. We believe this is the first time that complex dynamics of root-MUSIC polynomial were analyzed to propose an algorithm. In all, the thesis addresses two major bottlenecks in a direction of arrival estimation system, by providing simple, high throughput, parallel algorithms.

An approximate method for the solution of an influence function foil rolling model

Fleck and Johnson (Int. J. Mech. Sci. 29 (1987) 507) and Fleck et al. (Proc. Inst. Mech. Eng. 206 (1992) 119) have developed foil rolling models which allow for large deformations in the roll profile, including the possibility that the rolls flatten completely. However, these models require computationally expensive iterative solution techniques. A new approach to the approximate solution of the Fleck et al. (1992) Influence Function Model has been developed using both analytic and approximation techniques. The numerical difficulties arising from solving an integral equation in the flattened region have been reduced by applying an Inverse Hilbert Transform to get an analytic expression for the pressure. The method described in this paper is applicable to cases where there is or there is not a flat region.