Approximate Gauss-Newton methods for nonlinear least squares problems

The Gauss–Newton algorithm is an iterative method regularly used for solving nonlinear least squares problems. It is particularly well suited to the treatment of very large scale variational data assimilation problems that arise in atmosphere and ocean forecasting. The procedure consists of a sequence of linear least squares approximations to the nonlinear problem, each of which is solved by an “inner” direct or iterative process. In comparison with Newton’s method and its variants, the algorithm is attractive because it does not require the evaluation of second-order derivatives in the Hessian of the objective function. In practice the exact Gauss–Newton method is too expensive to apply operationally in meteorological forecasting, and various approximations are made in order to reduce computational costs and to solve the problems in real time. Here we investigate the effects on the convergence of the Gauss–Newton method of two types of approximation used commonly in data assimilation. First, we examine “truncated” Gauss–Newton methods where the inner linear least squares problem is not solved exactly, and second, we examine “perturbed” Gauss–Newton methods where the true linearized inner problem is approximated by a simplified, or perturbed, linear least squares problem. We give conditions ensuring that the truncated and perturbed Gauss–Newton methods converge and also derive rates of convergence for the iterations. The results are illustrated by a simple numerical example. A practical application to the problem of data assimilation in a typical meteorological system is presented.

Optimal discrimination and classification of THz spectra in the wavelet domain

In rapid scan Fourier transform spectrometry, we show that the noise in the wavelet coefficients resulting from the filter bank decomposition of the complex insertion loss function is linearly related to the noise power in the sample interferogram by a noise amplification factor. By maximizing an objective function composed of the power of the wavelet coefficients divided by the noise amplification factor, optimal feature extraction in the wavelet domain is performed. The performance of a classifier based on the output of a filter bank is shown to be considerably better than that of an Euclidean distance classifier in the original spectral domain. An optimization procedure results in a further improvement of the wavelet classifier. The procedure is suitable for enhancing the contrast or classifying spectra acquired by either continuous wave or THz transient spectrometers as well as for increasing the dynamic range of THz imaging systems. (C) 2003 Optical Society of America.

A minimum approximate-BER beamforming approach for PSK modulated wireless systems

A beamforming algorithm is introduced based on the general objective function that approximates the bit error rate for the wireless systems with binary phase shift keying and quadrature phase shift keying modulation schemes. The proposed minimum approximate bit error rate (ABER) beamforming approach does not rely on the Gaussian assumption of the channel noise. Therefore, this approach is also applicable when the channel noise is non-Gaussian. The simulation results show that the proposed minimum ABER solution improves the standard minimum mean squares error beamforming solution, in terms of a smaller achievable system's bit error rate.

Application of the entropy technique and genetic algorithms to construction site layout planning of medium-size projects

Genetic algorithms (GAs) have been introduced into site layout planning as reported in a number of studies. In these studies, the objective functions were defined so as to employ the GAs in searching for the optimal site layout. However, few studies have been carried out to investigate the actual closeness of relationships between site facilities; it is these relationships that ultimately govern the site layout. This study has determined that the underlying factors of site layout planning for medium-size projects include work flow, personnel flow, safety and environment, and personal preferences. By finding the weightings on these factors and the corresponding closeness indices between each facility, a closeness relationship has been deduced. Two contemporary mathematical approaches - fuzzy logic theory and an entropy measure - were adopted in finding these results in order to minimize the uncertainty and vagueness of the collected data and improve the quality of the information. GAs were then applied to searching for the optimal site layout in a medium-size government project using the GeneHunter software. The objective function involved minimizing the total travel distance. An optimal layout was obtained within a short time. This reveals that the application of GA to site layout planning is highly promising and efficient.

Conditioning and preconditioning of the variational data assimilation problem

Numerical weather prediction (NWP) centres use numerical models of the atmospheric flow to forecast future weather states from an estimate of the current state. Variational data assimilation (VAR) is used commonly to determine an optimal state estimate that miminizes the errors between observations of the dynamical system and model predictions of the flow. The rate of convergence of the VAR scheme and the sensitivity of the solution to errors in the data are dependent on the condition number of the Hessian of the variational least-squares objective function. The traditional formulation of VAR is ill-conditioned and hence leads to slow convergence and an inaccurate solution. In practice, operational NWP centres precondition the system via a control variable transform to reduce the condition number of the Hessian. In this paper we investigate the conditioning of VAR for a single, periodic, spatially-distributed state variable. We present theoretical bounds on the condition number of the original and preconditioned Hessians and hence demonstrate the improvement produced by the preconditioning. We also investigate theoretically the effect of observation position and error variance on the preconditioned system and show that the problem becomes more ill-conditioned with increasingly dense and accurate observations. Finally, we confirm the theoretical results in an operational setting by giving experimental results from the Met Office variational system.

When do export subsidies have a redistributional role? Reply

Our differences are three. The first arises from the belief that "... a nonzero value for the optimally chosen policy instrument implies that the instrument is efficient for redistribution" (Alston, Smith, and Vercammen, p. 543, paragraph 3). Consider the two equations: (1) o* = f(P3) and (2) = -f(3) ++r h* (a, P3) representing the solution to the problem of maximizing weighted, Marshallian surplus using, simultaneously, a per-unit border intervention, 9, and a per-unit domestic intervention, wr. In the solution, parameter ot denotes the weight applied to producer surplus; parameter p denotes the weight applied to government revenues; consumer surplus is implicitly weighted one; and the country in question is small in the sense that it is unable to affect world price by any of its domestic adjustments (see the Appendix). Details of the forms of the functions f((P) and h(ot, p) are easily derived, but what matters in the context of Alston, Smith, and Vercammen's Comment is: Redistributivep referencest hatf avorp roducers are consistent with higher values "alpha," and whereas the optimal domestic intervention, 7r*, has both "alpha and beta effects," the optimal border intervention, r*, has only a "beta effect,"-it does not have a redistributional role. Garth Holloway is reader in agricultural economics and statistics, Department of Agricultural and Food Economics, School of Agriculture, Policy, and Development, University of Reading. The author is very grateful to Xavier Irz, Bhavani Shankar, Chittur Srinivasan, Colin Thirtle, and Richard Tiffin for their comments and their wisdom; and to Mario Mazzochi, Marinos Tsigas, and Cal Turvey for their scholarship, including help in tracking down a fairly complete collection of the papers that cite Alston and Hurd. They are not responsible for any errors or omissions. Note, in equation (1), that the border intervention is positive whenever a distortion exists because 8 > 0 implies 3 - 1 + 8 > 1 and, thus, f((P) > 0 (see Appendix). Using Alston, Smith, and Vercammen's definition, the instrument is now "efficient," and therefore has a redistributive role. But now, suppose that the distortion is removed so that 3 - 1 + 8 = 1, 8 = 0, and consequently the border intervention is zero. According to Alston, Smith, and Vercammen, the instrument is now "inefficient" and has no redistributive role. The reader will note that this thought experiment has said nothing about supporting farm incomes, and so has nothing whatsoever to do with efficient redistribution. Of course, the definition is false. It follows that a domestic distortion arising from the "excess-burden argument" 3 = 1 + 8, 8 > 0 does not make an export subsidy "efficient." The export subsidy, having only a "beta effect," does not have a redistributional role. The second disagreement emerges from the comment that Holloway "... uses an idiosyncratic definition of the relevant objective function of the government (Alston, Smith, and Vercammen, p. 543, paragraph 2)." The objective function that generates equations (1) and (2) (see the Appendix) is the same as the objective function used by Gardner (1995) when he first questioned Alston, Carter, and Smith's claim that a "domestic distortion can make a border intervention efficient in transferring surplus from consumers and taxpayers to farmers." The objective function used by Gardner (1995) is the same objective function used in the contributions that precede it and thus defines the literature on the debate about borderversus- domestic intervention (Streeten; Yeh; Paarlberg 1984, 1985; Orden; Gardner 1985). The objective function in the latter literature is the same as the one implied in another literature that originates from Wallace and includes most notably Gardner (1983), but also Alston and Hurd. Amer. J. Agr. Econ. 86(2) (May 2004): 549-552 Copyright 2004 American Agricultural Economics Association This content downloaded on Tue, 15 Jan 2013 07:58:41 AM All use subject to JSTOR Terms and Conditions 550 May 2004 Amer. J. Agr. Econ. The objective function in Holloway is this same objective function-it is, of course, Marshallian surplus.1 The third disagreement concerns scholarship. The Comment does not seem to be cognizant of several important papers, especially Bhagwati and Ramaswami, and Bhagwati, both of which precede Corden (1974, 1997); but also Lipsey and Lancaster, and Moschini and Sckokai; one important aspect of Alston and Hurd; and one extremely important result in Holloway. This oversight has some unfortunate repercussions. First, it misdirects to the wrong origins of intellectual property. Second, it misleads about the appropriateness of some welfare calculations. Third, it prevents Alston, Smith, and Vercammen from linking a finding in Holloway (pp. 242-43) with an old theorem (Lipsey and Lancaster) that settles the controversy (Alston, Carter, and Smith 1993, 1995; Gardner 1995; and, presently, Alston, Smith, and Vercammen) about the efficiency of border intervention in the presence of domestic distortions.

Reassessing the interaction between investment and tenure uncertainty

A diverse body of empirical literature recognizes that investment can influence tenure security, yet this phenomenon has rarely been examined analytically. This paper develops a theoretical model that demonstrates explicitly conditions under which the probability of eviction is endogenous to investment undertaken on illegally encroached land. By accommodating explicitly the government's objective function and its ability to commit credibly to an eviction policy, the model reveals why both those farmers who under-invest, and those who raise their investment levels to improve tenure security, may be behaving rationally. Indeed, both types of behaviour are accommodated within a single model.

Towards optimizing the selection of neurons in affordable neural networks

This paper considers variations of a neuron pool selection method known as Affordable Neural Network (AfNN). A saliency measure, based on the second derivative of the objective function is proposed to assess the ability of a trained AfNN to provide neuronal redundancy. The discrepancies between the various affordability variants are explained by correlating unique sub group selections with relevant saliency variations. Overall this study shows that the method in which neurons are selected from a pool is more relevant to how salient individual neurons are, than how often a particular neuron is used during training. The findings herein are relevant to not only providing an analogy to brain function but, also, in optimizing the way a neural network using the affordability method is trained.

Conditioning of the weak-constraint variational data assimilation problem for numerical weather prediction

4-Dimensional Variational Data Assimilation (4DVAR) assimilates observations through the minimisation of a least-squares objective function, which is constrained by the model flow. We refer to 4DVAR as strong-constraint 4DVAR (sc4DVAR) in this thesis as it assumes the model is perfect. Relaxing this assumption gives rise to weak-constraint 4DVAR (wc4DVAR), leading to a different minimisation problem with more degrees of freedom. We consider two wc4DVAR formulations in this thesis, the model error formulation and state estimation formulation. The 4DVAR objective function is traditionally solved using gradient-based iterative methods. The principle method used in Numerical Weather Prediction today is the Gauss-Newton approach. This method introduces a linearised `inner-loop' objective function, which upon convergence, updates the solution of the non-linear `outer-loop' objective function. This requires many evaluations of the objective function and its gradient, which emphasises the importance of the Hessian. The eigenvalues and eigenvectors of the Hessian provide insight into the degree of convexity of the objective function, while also indicating the difficulty one may encounter while iterative solving 4DVAR. The condition number of the Hessian is an appropriate measure for the sensitivity of the problem to input data. The condition number can also indicate the rate of convergence and solution accuracy of the minimisation algorithm. This thesis investigates the sensitivity of the solution process minimising both wc4DVAR objective functions to the internal assimilation parameters composing the problem. We gain insight into these sensitivities by bounding the condition number of the Hessians of both objective functions. We also precondition the model error objective function and show improved convergence. We show that both formulations' sensitivities are related to error variance balance, assimilation window length and correlation length-scales using the bounds. We further demonstrate this through numerical experiments on the condition number and data assimilation experiments using linear and non-linear chaotic toy models.

Conditioning and preconditioning of the optimal state estimation problem

Optimal state estimation is a method that requires minimising a weighted, nonlinear, least-squares objective function in order to obtain the best estimate of the current state of a dynamical system. Often the minimisation is non-trivial due to the large scale of the problem, the relative sparsity of the observations and the nonlinearity of the objective function. To simplify the problem the solution is often found via a sequence of linearised objective functions. The condition number of the Hessian of the linearised problem is an important indicator of the convergence rate of the minimisation and the expected accuracy of the solution. In the standard formulation the convergence is slow, indicating an ill-conditioned objective function. A transformation to different variables is often used to ameliorate the conditioning of the Hessian by changing, or preconditioning, the Hessian. There is only sparse information in the literature for describing the causes of ill-conditioning of the optimal state estimation problem and explaining the effect of preconditioning on the condition number. This paper derives descriptive theoretical bounds on the condition number of both the unpreconditioned and preconditioned system in order to better understand the conditioning of the problem. We use these bounds to explain why the standard objective function is often ill-conditioned and why a standard preconditioning reduces the condition number. We also use the bounds on the preconditioned Hessian to understand the main factors that affect the conditioning of the system. We illustrate the results with simple numerical experiments.

The multiple team formation problem using sociometry

The Team Formation problem (TFP) has become a well-known problem in the OR literature over the last few years. In this problem, the allocation of multiple individuals that match a required set of skills as a group must be chosen to maximise one or several social positive attributes. Speci�cally, the aim of the current research is two-fold. First, two new dimensions of the TFP are added by considering multiple projects and fractions of people's dedication. This new problem is named the Multiple Team Formation Problem (MTFP). Second, an optimization model consisting in a quadratic objective function, linear constraints and integer variables is proposed for the problem. The optimization model is solved by three algorithms: a Constraint Programming approach provided by a commercial solver, a Local Search heuristic and a Variable Neighbourhood Search metaheuristic. These three algorithms constitute the first attempt to solve the MTFP, being a variable neighbourhood local search metaheuristic the most effi�cient in almost all cases. Applications of this problem commonly appear in real-life situations, particularly with the current and ongoing development of social network analysis. Therefore, this work opens multiple paths for future research.

Autopoietic organization of firm: an illustration for the construction industry

Generally poor productivity, delays, low profitability and exceeded budgets are Common problems in modern construction management, however it seems that a basic obstacle lies far deeper in the understanding of a firm's fundamental mission, its existence. The main objective of this paper therefore is to examine the operational living of a construction firm and by doing that to reveal the key problem or the solution for a construction firm - its organization. A firm as a social system in which interactions between its constitutive components (employees) are surordinated to its maintenance (keeping a system alive) is an autopoietic social system. Two domains of external perturbations are uncovered to which a construction firm has to adapt (market driven and project driven perturbations). Constructed conceptual model of an autopoietic organization is based upon two necessary and sufficient operational domains that a firm has to create in order to become an autopoietic, adaptive social system. The first one is a domain of interactions between employees and other operationally external systems, which is representing an idea-generating domain of interactions. The second is employee's autonomous operational domain, which embodies employee's autonomy and individuality and represents a necessary condition for the establishment of an idea-generating domain. Finally, it is recognized that interactions within these four domains keep a construction firm alive.

Use of 16S rRNA-targeted oligonucleotide probes to investigate function and phylogeny of sulphate-reducing bacteria and methanogenic archaea in a UK estuary

Sulphate-reducing bacteria (SRB) and methanogenic archaea (MA) are important anaerobic terminal oxidisers of organic matter. However, we have little knowledge about the distribution and types of SRB and MA in the environment or the functional role they play in situ. Here we have utilised sediment slurry microcosms amended with ecologically significant substrates, including acetate and hydrogen, and specific functional inhibitors, to identify the important SRB and MA groups in two contrasting sites on a UK estuary. Substrate and inhibitor additions had significant effects on methane production and on acetate and sulphate consumption in the slurries. By using specific 16S-targeted oligonucleotide probes we were able to link specific SRB and MA groups to the use of the added substrates. Acetate consumption in the freshwater-dominated sediments was mediated by Methanosarcinales under low-sulphate conditions and Desulfobacter under the high-sulphate conditions that simulated a tidal incursion. In the marine-dominated sediments, acetate consumption was linked to Desulfobacter. Addition of trimethylamine, a non-competitive substrate for methanogenesis, led to a large increase in Methanosarcinales signal in marine slurries. Desulfobulbus was linked to non-sulphate-dependent H-2 consumption in the freshwater sediments. The addition of sulphate to freshwater sediments inhibited methane production and reduced signal from probes targeted to Methanosarcinales and Methanomicrobiales, while the addition of molybdate to marine sediments inhibited Desulfobulbus and Desulfobacterium. These data complement our understanding of the ecophysiology of the organisms detected and make a firm connection between the capabilities of species, as observed in the laboratory, to their roles in the environment. (C) 2003 Federation of European Microbiological Societies. Published by Elsevier Science B.V. All rights reserved.

Modulation of the fecal microflora profile and immune function by a novel trans-galactooligosaccharide mixture (B-GOS) in healthy elderly volunteers

Background: Aging is associated with reduced numbers of beneficial colonic bifidobacteria and impaired immunity. Galactooligosaccharides (GOSs) stimulate the growth of bifidobacteria in younger adults, but little is known about their effects in the elderly and their immunomodulatory capacity. Objective: We assessed the effect of a prebiotic GOS mixture (B-GOS) on immune function and fecal microflora composition in healthy elderly subjects. Design: In a double-blind, placebo-controlled, crossover study, 44 elderly subjects were randomly assigned to receive either a placebo or the B-GOS treatment (5.5 g/d). Subjects consumed the treatments for 10 wk, and then went through a 4-wk washout period, before switching to the other treatment for the final 10 wk. Blood and fecal samples were collected at the beginning, middle (5 wk), and end of the test period. Predominant bacterial groups were quantified, and phagocytosis, natural killer (NK) cell activity, cytokine production, plasma cholesterol, and HDL cholesterol were measured. Results: B-GOS significantly increased the numbers of beneficial bacteria, especially bifidobacteria, at the expense of less beneficial groups compared with the baseline and placebo. Significant increases in phagocytosis, NK cell activity, and the production of antiinflammatory cytokine interleukin-10 (IL-10) and significant reduction in the production of proinflammatory cytokines (IL-6, IL-1 beta , and tumor necrosis factor-alpha) were also observed. B-GOS exerted no effects on total cholesterol or HDL-cholesterol production, however. Conclusions: B-GOS administration to healthy elderly persons resulted in positive effects on both the microflora composition and the immune response. Therefore, B-GOS may be a useful dietary candidate for the enhancement of gastrointestinal health and immune function in elderly persons. Am J Clin Nutr 2008; 88: 1438-46.

Effects of cis-9,trans-11 and trans-1 0,cis-12 conjugated linoleic acid on immune cell function in healthy humans

Background: Animal studies have suggested that conjugated linoleic acid (CLA), a natural component of ruminant meat and dairy products, may confer beneficial effects on health. However, little information on the effects of CLA on immune function is available, especially in humans. Furthermore, the effects of individual isomers of CLA have not been adequately investigated. Objective: This study investigated the effects of supplementing the diet with 3 doses of highly enriched cis-9,trans-11 CLA (0.59, 1.19, and 2.38 g/d) or trans-10,cis-12 CLA (0.63, 1.26, and 2.52 g/d) on immune outcomes in healthy humans. Design: The study had a randomized, double-blind, crossover design. Healthy men consumed 1, 2, and 4 capsules sequentially that contained 80% of either cis-9,trans-11 CLA or trans-10,cis-12 CLA for consecutive 8-wk periods. This regimen was followed by a 6-wk washout and a crossover to the other isomer. Results: Both CLA isomers decreased mitogen-induced T lymphocyte activation in a dose-dependent manner. There was a significant negative correlation between mitogen-induced T lymphocyte activation and the proportions of both cis-9,trans-11 CLA and trans-10,cis-12 CLA in peripheral blood mononuclear cell lipids. However, CLA did not affect lymphocyte subpopulations or serum concentrations of C-reactive protein and did not have any consistent effects on ex vivo cytokine production. Conclusion: CLA supplementation results in a dose-dependent reduction in the mitogen-induced activation of T lymphocytes. The effects of cis-9,trans-l I CLA and trans-10,cis-12 CLA were similar, and there was a negative correlation between mitogen-induced T lymphocyte activation and the cis-9,trans-11 CLA and trans-10,cis-12 CLA contents of mononuclear cells.