Proximal methods for nonlinear programming: double regularization and inexact subproblems

This paper describes the first phase of a project attempting to construct an efficient general-purpose nonlinear optimizer using an augmented Lagrangian outer loop with a relative error criterion, and an inner loop employing a state-of-the art conjugate gradient solver. The outer loop can also employ double regularized proximal kernels, a fairly recent theoretical development that leads to fully smooth subproblems. We first enhance the existing theory to show that our approach is globally convergent in both the primal and dual spaces when applied to convex problems. We then present an extensive computational evaluation using the CUTE test set, showing that some aspects of our approach are promising, but some are not. These conclusions in turn lead to additional computational experiments suggesting where to next focus our theoretical and computational efforts.

Influence diagnostics in nonlinear mixed-effects elliptical models

In this work we propose and analyze nonlinear elliptical models for longitudinal data, which represent an alternative to gaussian models in the cases of heavy tails, for instance. The elliptical distributions may help to control the influence of the observations in the parameter estimates by naturally attributing different weights for each case. We consider random effects to introduce the within-group correlation and work with the marginal model without requiring numerical integration. An iterative algorithm to obtain maximum likelihood estimates for the parameters is presented, as well as diagnostic results based on residual distances and local influence [Cook, D., 1986. Assessment of local influence. journal of the Royal Statistical Society - Series B 48 (2), 133-169; Cook D., 1987. Influence assessment. journal of Applied Statistics 14 (2),117-131; Escobar, L.A., Meeker, W.Q., 1992, Assessing influence in regression analysis with censored data, Biometrics 48, 507-528]. As numerical illustration, we apply the obtained results to a kinetics longitudinal data set presented in [Vonesh, E.F., Carter, R.L., 1992. Mixed-effects nonlinear regression for unbalanced repeated measures. Biometrics 48, 1-17], which was analyzed under the assumption of normality. (C) 2009 Elsevier B.V. All rights reserved.

Orthogonal packing of identical rectangles within isotropic convex regions

A mixed integer continuous nonlinear model and a solution method for the problem of orthogonally packing identical rectangles within an arbitrary convex region are introduced in the present work. The convex region is assumed to be made of an isotropic material in such a way that arbitrary rotations of the items, preserving the orthogonality constraint, are allowed. The solution method is based on a combination of branch and bound and active-set strategies for bound-constrained minimization of smooth functions. Numerical results show the reliability of the presented approach. (C) 2010 Elsevier Ltd. All rights reserved.

Exact penalties for variational inequalities with applications to nonlinear complementarity problems

In this paper, we present a new reformulation of the KKT system associated to a variational inequality as a semismooth equation. The reformulation is derived from the concept of differentiable exact penalties for nonlinear programming. The best theoretical results are presented for nonlinear complementarity problems, where simple, verifiable, conditions ensure that the penalty is exact. We close the paper with some preliminary computational tests on the use of a semismooth Newton method to solve the equation derived from the new reformulation. We also compare its performance with the Newton method applied to classical reformulations based on the Fischer-Burmeister function and on the minimum. The new reformulation combines the best features of the classical ones, being as easy to solve as the reformulation that uses the Fischer-Burmeister function while requiring as few Newton steps as the one that is based on the minimum.

Second-order negative-curvature methods for box-constrained and general constrained optimization

A Nonlinear Programming algorithm that converges to second-order stationary points is introduced in this paper. The main tool is a second-order negative-curvature method for box-constrained minimization of a certain class of functions that do not possess continuous second derivatives. This method is used to define an Augmented Lagrangian algorithm of PHR (Powell-Hestenes-Rockafellar) type. Convergence proofs under weak constraint qualifications are given. Numerical examples showing that the new method converges to second-order stationary points in situations in which first-order methods fail are exhibited.

Outer Trust-Region Method for Constrained Optimization

Given an algorithm A for solving some mathematical problem based on the iterative solution of simpler subproblems, an outer trust-region (OTR) modification of A is the result of adding a trust-region constraint to each subproblem. The trust-region size is adaptively updated according to the behavior of crucial variables. The new subproblems should not be more complex than the original ones, and the convergence properties of the OTR algorithm should be the same as those of Algorithm A. In the present work, the OTR approach is exploited in connection with the ""greediness phenomenon"" of nonlinear programming. Convergence results for an OTR version of an augmented Lagrangian method for nonconvex constrained optimization are proved, and numerical experiments are presented.

Structured minimal-memory inexact quasi-Newton method and secant preconditioners for augmented Lagrangian optimization

Augmented Lagrangian methods for large-scale optimization usually require efficient algorithms for minimization with box constraints. On the other hand, active-set box-constraint methods employ unconstrained optimization algorithms for minimization inside the faces of the box. Several approaches may be employed for computing internal search directions in the large-scale case. In this paper a minimal-memory quasi-Newton approach with secant preconditioners is proposed, taking into account the structure of Augmented Lagrangians that come from the popular Powell-Hestenes-Rockafellar scheme. A combined algorithm, that uses the quasi-Newton formula or a truncated-Newton procedure, depending on the presence of active constraints in the penalty-Lagrangian function, is also suggested. Numerical experiments using the Cute collection are presented.

Improving ultimate convergence of an augmented Lagrangian method

Optimization methods that employ the classical Powell-Hestenes-Rockafellar augmented Lagrangian are useful tools for solving nonlinear programming problems. Their reputation decreased in the last 10 years due to the comparative success of interior-point Newtonian algorithms, which are asymptotically faster. In this research, a combination of both approaches is evaluated. The idea is to produce a competitive method, being more robust and efficient than its `pure` counterparts for critical problems. Moreover, an additional hybrid algorithm is defined, in which the interior-point method is replaced by the Newtonian resolution of a Karush-Kuhn-Tucker (KKT) system identified by the augmented Lagrangian algorithm. The software used in this work is freely available through the Tango Project web page:http://www.ime.usp.br/similar to egbirgin/tango/.

Augmented Lagrangian methods under the constant positive linear dependence constraint qualification

Two Augmented Lagrangian algorithms for solving KKT systems are introduced. The algorithms differ in the way in which penalty parameters are updated. Possibly infeasible accumulation points are characterized. It is proved that feasible limit points that satisfy the Constant Positive Linear Dependence constraint qualification are KKT solutions. Boundedness of the penalty parameters is proved under suitable assumptions. Numerical experiments are presented.

A note on influence diagnostics in nonlinear mixed-effects elliptical models

This paper provides general matrix formulas for computing the score function, the (expected and observed) Fisher information and the A matrices (required for the assessment of local influence) for a quite general model which includes the one proposed by Russo et al. (2009). Additionally, we also present an expression for the generalized leverage on fixed and random effects. The matrix formulation has notational advantages, since despite the complexity of the postulated model, all general formulas are compact, clear and have nice forms. (C) 2010 Elsevier B.V. All rights reserved.

Bayesian nonlinear regression models with scale mixtures of skew-normal distributions: Estimation and case influence diagnostics

The purpose of this paper is to develop a Bayesian analysis for nonlinear regression models under scale mixtures of skew-normal distributions. This novel class of models provides a useful generalization of the symmetrical nonlinear regression models since the error distributions cover both skewness and heavy-tailed distributions such as the skew-t, skew-slash and the skew-contaminated normal distributions. The main advantage of these class of distributions is that they have a nice hierarchical representation that allows the implementation of Markov chain Monte Carlo (MCMC) methods to simulate samples from the joint posterior distribution. In order to examine the robust aspects of this flexible class, against outlying and influential observations, we present a Bayesian case deletion influence diagnostics based on the Kullback-Leibler divergence. Further, some discussions on the model selection criteria are given. The newly developed procedures are illustrated considering two simulations study, and a real data previously analyzed under normal and skew-normal nonlinear regression models. (C) 2010 Elsevier B.V. All rights reserved.

Improved testing inference in mixed linear models

Mixed linear models are commonly used in repeated measures studies. They account for the dependence amongst observations obtained from the same experimental unit. Often, the number of observations is small, and it is thus important to use inference strategies that incorporate small sample corrections. In this paper, we develop modified versions of the likelihood ratio test for fixed effects inference in mixed linear models. In particular, we derive a Bartlett correction to such a test, and also to a test obtained from a modified profile likelihood function. Our results generalize those in [Zucker, D.M., Lieberman, O., Manor, O., 2000. Improved small sample inference in the mixed linear model: Bartlett correction and adjusted likelihood. Journal of the Royal Statistical Society B, 62,827-838] by allowing the parameter of interest to be vector-valued. Additionally, our Bartlett corrections allow for random effects nonlinear covariance matrix structure. We report simulation results which show that the proposed tests display superior finite sample behavior relative to the standard likelihood ratio test. An application is also presented and discussed. (C) 2008 Elsevier B.V. All rights reserved.

Solvatochromism in Binary Mixtures: First Report on a Solvation Free Energy Relationship between Solvent Exchange Equilibrium Constants and the Properties of the Medium

We have employed UV-vis spectroscopy in order to investigate details of the solvation of six solvatochromic indicators, hereafter designated as ""probes"", namely, 2,6-diphenyl-4-(2,4,6-triphenylpyridinium-1-yl) phenolate (RB); 4-[(E)-2-(1-methylpyridinium-4-yl)ethenyl] phenolate, MePM; 1-methylquinolinium-8-olate, QB; 2-bromo-4-[(E)-2-(1-methylpyridinium-4-yl)ethenyl] phenolate, MePMBr, 2,6-dichloro-4-(2,4,6-triphenylpyridinium-1-yl) phenolate (WB); and 2,6-dibromo-4-[(E)-2-(1-methylpyridinium-4-yl)ethenyl] phenolate, MePMBr,, respectively. These can be divided into three pairs, each includes two probes of similar pK(a) in water and different lipophilicity. Solvation has been studied in binary mixtures, BMs, of water, W, with 12 protic organic solvents, S, including mono- and bifunctional alcohols (2-alkoxyethanoles, unsaturated and chlorinated alcohols). Each medium was treated as a mixture of S, W, and a complex solvent, S-W, formed by hydrogen bonding. Values of lambda(max) (of the probe intramolecular charge transfer) were converted into empirical polarity scales, E(T)(probe) in kcal/mol, whose values were correlated with the effective mole fraction of water in the medium, chi w(effective). This correlation furnished three equilibrium constants for the exchange of solvents in the probe solvation shell; phi(W/S) (W substitutes S): phi(S-W/W) (S-W substitutes W), and phi(S-W/S) (S-W substitutes S), respectively. The values of these constants depend on the physicochemical properties of the probe and the medium. We tested, for the first time, the applicability of a new solvation free energy relationship: phi = constant + a alpha(BM) + b beta(BM) + s(pi*(BM) + d delta) + p log P(BM), where a, b, s, and p are regression coefficients alpha(BM), beta(BM), and pi*(BM) are solvatochromic parameters of the BM, delta is a correction term for pi*, and log P is an empirical scale of lipophilicity. Correlations were carried out with two-, three-, and four-medium descriptors. In all cases, three descriptors gave satisfactory correlations; use of four parameters gave only a marginal increase of the goodness of fit. For phi(W/S), the most important descriptor was found to be the lipophilicity of the medium; for phi(S-W/W) and phi(S-W/S), solvent basicity is either statistically relevant or is the most important descriptor. These responses are different from those of E(T)(probe) of many solvatochromic indicators in pure solvents, where the importance of solvent basicity is usually marginal, and can be neglected.

Thermo-solvatochromism of merocyanine polarity probes - What are the consequences of increasing probe lipophilicity through annelation?

The question raised in the title has been answered by comparing the solvatochromism of two series of polarity probes, the lipophilicities of which were increased either by increasing the length of an alkyl group (R) attached to a fixed pyridine-based structure or through annelation (i.e., by fusing benzene rings onto a central pyridine-based structure). The following novel solvatochromic probes were synthesized: 2,6-dibromo-4-[(E)-2-(1-methylquinolinium-4-yl)ethenyl]-phenolate (MeQMBr(2)) and 2,6-dibromo-4-[(E)-2-(1-methyl-acridinium-4- yl) ethenyl)]phenolate (MeAMBr(2) The solvatochromic behavior of these probes, along with that of 2,6dibromo-4-[(E)-2-(1-methylpyridinium-4-yl)ethenyl]phenol-ate(MePMBr(2)) was analyzed in terms of increasing probe lipophilicity, through annelation. Values of the empirical solvent polarity scale [E(T)(MePMBr(2))] in kcalmol(-1) correlated linearly with ET(30), the corresponding values for the extensively employed probe 2,6-diphenyl-4-(2,4,6-triphenylpyridinium-1-yl)phenolate (RB). On the other hand, the nonlinear correlations of ET(MeQMBr(2)) or ET(MeAMBr(2)) with E(T)(30) are described by second-order polynomials. Possible reasons for this behavior include: i) self-aggregation of the probe, ii) photoinduced cis/trans isomerization of the dye, and iii) probe structure- and solvent-dependent contributions of the quinonoid and zwitterionic limiting formulas to the ground and excited states of the probe. We show that mechanisms (i) and (ii) are not operative under the experimental conditions employed; experimental evidence (NMR) and theoretical calculations are presented to support the conjecture that the length of the central ethenylic bond in the dye increases in the order MeAMBr(2) > MeQMBr(2) > MePMBr(2), That is, the contribution of the zwitterionic limiting formula predominates for the latter probe, as is also the case for RB, this being the reason for the observed linear correlation between the ET(MePMBr2) and the ET(30) scales. The effect of increasing probe lipophilicity on solvatochromic behavior therefore depends on the strategy employed. Increasing the length of R affects solvatochromism much less than annelation, because the former structural change hardly perturbs the energy of the intramolecular charge-transfer transition responsible for solvatochromism. The thermo-solvatochromic behavior (effect of temperature on solvatochromism) of the three probes was studied in mixtures of water with propanol and/or with DMSO. The solvation model used explicitly considers the presence of three ""species"" in the system: bulk solution and probe solvation shell [namely, water (W), organic solvent (Solv)], and solvent-water hydrogen-bonded aggregate (Solv-W). For aqueous propanol, the probe is efficiently solvated by Solv-W; the strong interaction of DMSO with W drastically decreases the efficiency of Solv-W in solvating the probe, relative to its precursor solvents. Temperature increases resulted in desolvation of the probes, due to the concomitant reduction in the structured characters of the components of the binary mixtures.