Revisiting optimization algorithms for maximum likelihood estimation

Parmi les méthodes d’estimation de paramètres de loi de probabilité en statistique, le maximum de vraisemblance est une des techniques les plus populaires, comme, sous des conditions l´egères, les estimateurs ainsi produits sont consistants et asymptotiquement efficaces. Les problèmes de maximum de vraisemblance peuvent être traités comme des problèmes de programmation non linéaires, éventuellement non convexe, pour lesquels deux grandes classes de méthodes de résolution sont les techniques de région de confiance et les méthodes de recherche linéaire. En outre, il est possible d’exploiter la structure de ces problèmes pour tenter d’accélerer la convergence de ces méthodes, sous certaines hypothèses. Dans ce travail, nous revisitons certaines approches classiques ou récemment d´eveloppées en optimisation non linéaire, dans le contexte particulier de l’estimation de maximum de vraisemblance. Nous développons également de nouveaux algorithmes pour résoudre ce problème, reconsidérant différentes techniques d’approximation de hessiens, et proposons de nouvelles méthodes de calcul de pas, en particulier dans le cadre des algorithmes de recherche linéaire. Il s’agit notamment d’algorithmes nous permettant de changer d’approximation de hessien et d’adapter la longueur du pas dans une direction de recherche fixée. Finalement, nous évaluons l’efficacité numérique des méthodes proposées dans le cadre de l’estimation de modèles de choix discrets, en particulier les modèles logit mélangés.

Étude du maximum et des hauts points de la marche aléatoire branchante inhomogène et du champ libre gaussien inhomogène

Voir la bibliographie du mémoire pour les références du résumé. See the thesis`s bibliography for the references in the summary.

Maximum mass of neutron stars

We determine the structure of neutron stars within a Brueckner-Hartree-Fock approach based on realistic nucleon-nucleon, nucleon-hyperon, and hyperon-hyperon interactions. Our results indicate rather low maximum masses below 1.4 solar masses. This feature is insensitive to the nucleonic part of the EOS due to a strong compensation mechanism caused by the appearance of hyperons and represents thus strong evidence for the presence of nonbaryonic "quark" matter in the interior of heavy stars.

Maximum Entropy Discrimination

We present a general framework for discriminative estimation based on the maximum entropy principle and its extensions. All calculations involve distributions over structures and/or parameters rather than specific settings and reduce to relative entropy projections. This holds even when the data is not separable within the chosen parametric class, in the context of anomaly detection rather than classification, or when the labels in the training set are uncertain or incomplete. Support vector machines are naturally subsumed under this class and we provide several extensions. We are also able to estimate exactly and efficiently discriminative distributions over tree structures of class-conditional models within this framework. Preliminary experimental results are indicative of the potential in these techniques.

Biologically Plausible Neural Circuits for Realization of Maximum Operations

Object recognition in the visual cortex is based on a hierarchical architecture, in which specialized brain regions along the ventral pathway extract object features of increasing levels of complexity, accompanied by greater invariance in stimulus size, position, and orientation. Recent theoretical studies postulate a non-linear pooling function, such as the maximum (MAX) operation could be fundamental in achieving such invariance. In this paper, we are concerned with neurally plausible mechanisms that may be involved in realizing the MAX operation. Four canonical circuits are proposed, each based on neural mechanisms that have been previously discussed in the context of cortical processing. Through simulations and mathematical analysis, we examine the relative performance and robustness of these mechanisms. We derive experimentally verifiable predictions for each circuit and discuss their respective physiological considerations.

Are the maximum hardness and minimum polarizability principles always obeyed in nontotally symmetric vibrations?

An overview is given on a study which showed that not only in chemical reactions but also in the favorable case of nontotally symmetric vibrations where the chemical and external potentials keep approximately constant, the generalized maximum hardness principle (GMHP) and generalized minimum polarizability principle (GMPP) may not be obeyed. A method that allows an accurate determination of the nontotally symmetric molecular distortions with more marked GMPP or anti-GMPP character through diagonalization of the polarizability Hessian matrix is introduced

A comparison between maximum length sequence and conventional auditory brainstem responses as a screening tool for auditory sensitivity

This paper compares conventional auditory brainstem response tests (ABRs) and Maximum Length Sequence auditory brainstem response tests (MLS ABRs). The results found that the faster MLS ABRs could prove an accurate screening tool for auditory sensitivity.

Comparison of speech processor MAP minimum (T) and maximum (C) levels with psychophysical thresholds and maximum acceptable loudness levels in users of the Nucleus 22 cochlear implant system

This paper discusses the Nucleus 22 cochlear implant.

Estimating the spatial scales of regionalized variables by nested sampling, hierarchical analysis of variance and residual maximum likelihood

The variogram is essential for local estimation and mapping of any variable by kriging. The variogram itself must usually be estimated from sample data. The sampling density is a compromise between precision and cost, but it must be sufficiently dense to encompass the principal spatial sources of variance. A nested, multi-stage, sampling with separating distances increasing in geometric progression from stage to stage will do that. The data may then be analyzed by a hierarchical analysis of variance to estimate the components of variance for every stage, and hence lag. By accumulating the components starting from the shortest lag one obtains a rough variogram for modest effort. For balanced designs the analysis of variance is optimal; for unbalanced ones, however, these estimators are not necessarily the best, and the analysis by residual maximum likelihood (REML) will usually be preferable. The paper summarizes the underlying theory and illustrates its application with data from three surveys, one in which the design had four stages and was balanced and two implemented with unbalanced designs to economize when there were more stages. A Fortran program is available for the analysis of variance, and code for the REML analysis is listed in the paper. (c) 2005 Elsevier Ltd. All rights reserved.

Sediment magnetic properties of glacial till deposited since the Little Ice Age maximum for selected glaciers at Svartisen and Okstindan, northern Norway

Samples of glacial till deposited since the Little Ice Age (LIA) maximum by two glaciers, North Bogbre at Svartisen and Corneliussen-breen at Okstindan, northern Norway, were obtained from transects running from the current glacier snout to the LIA (c. AD 1750) limit. The samples were analysed to determine their sediment magnetic properties, which display considerable variability. Significant trends in some magnetic parameters are evident with distance from the glacier margin and hence length of subaerial exposure. Magnetic susceptibility (X) decreases away from the contemporary snout, perhaps due to the weathering of ferrimagnetic minerals into antiferromagnetic forms, although this trend is generally not statistically significant. Trends in the ratios of soft IRM/hard IRM which are statistically significant support this hypothesis, suggesting that antiferromagnetic minerals are increasing relative to ferrimagnetic minerals towards the LIA maximum. Backfield ratios (IRM -100 mT/SIRM) also display a significant and strong trend towards magnetically harder behaviour with proximity to the LIA maximum. Thus, by employing a chronosequence approach, it may be possible to use sediment magnetics data as a tool for reconstructing glacier retreat in areas where more traditional techniques, such as lichenometry, are not applicable.

Comparing sampling needs for variograms of soil properties computed by the method of moments and residual maximum likelihood

It has been generally accepted that the method of moments (MoM) variogram, which has been widely applied in soil science, requires about 100 sites at an appropriate interval apart to describe the variation adequately. This sample size is often larger than can be afforded for soil surveys of agricultural fields or contaminated sites. Furthermore, it might be a much larger sample size than is needed where the scale of variation is large. A possible alternative in such situations is the residual maximum likelihood (REML) variogram because fewer data appear to be required. The REML method is parametric and is considered reliable where there is trend in the data because it is based on generalized increments that filter trend out and only the covariance parameters are estimated. Previous research has suggested that fewer data are needed to compute a reliable variogram using a maximum likelihood approach such as REML, however, the results can vary according to the nature of the spatial variation. There remain issues to examine: how many fewer data can be used, how should the sampling sites be distributed over the site of interest, and how do different degrees of spatial variation affect the data requirements? The soil of four field sites of different size, physiography, parent material and soil type was sampled intensively, and MoM and REML variograms were calculated for clay content. The data were then sub-sampled to give different sample sizes and distributions of sites and the variograms were computed again. The model parameters for the sets of variograms for each site were used for cross-validation. Predictions based on REML variograms were generally more accurate than those from MoM variograms with fewer than 100 sampling sites. A sample size of around 50 sites at an appropriate distance apart, possibly determined from variograms of ancillary data, appears adequate to compute REML variograms for kriging soil properties for precision agriculture and contaminated sites. (C) 2007 Elsevier B.V. All rights reserved.

Estimating the spatial scale of herbicide and soil interactions by nested sampling, hierarchical analysis of variance and residual maximum likelihood

An unbalanced nested sampling design was used to investigate the spatial scale of soil and herbicide interactions at the field scale. A hierarchical analysis of variance based on residual maximum likelihood (REML) was used to analyse the data and provide a first estimate of the variogram. Soil samples were taken at 108 locations at a range of separating distances in a 9 ha field to explore small and medium scale spatial variation. Soil organic matter content, pH, particle size distribution, microbial biomass and the degradation and sorption of the herbicide, isoproturon, were determined for each soil sample. A large proportion of the spatial variation in isoproturon degradation and sorption occurred at sampling intervals less than 60 m, however, the sampling design did not resolve the variation present at scales greater than this. A sampling interval of 20-25 m should ensure that the main spatial structures are identified for isoproturon degradation rate and sorption without too great a loss of information in this field.