Revisiting optimization algorithms for maximum likelihood estimation

Parmi les méthodes d’estimation de paramètres de loi de probabilité en statistique, le maximum de vraisemblance est une des techniques les plus populaires, comme, sous des conditions l´egères, les estimateurs ainsi produits sont consistants et asymptotiquement efficaces. Les problèmes de maximum de vraisemblance peuvent être traités comme des problèmes de programmation non linéaires, éventuellement non convexe, pour lesquels deux grandes classes de méthodes de résolution sont les techniques de région de confiance et les méthodes de recherche linéaire. En outre, il est possible d’exploiter la structure de ces problèmes pour tenter d’accélerer la convergence de ces méthodes, sous certaines hypothèses. Dans ce travail, nous revisitons certaines approches classiques ou récemment d´eveloppées en optimisation non linéaire, dans le contexte particulier de l’estimation de maximum de vraisemblance. Nous développons également de nouveaux algorithmes pour résoudre ce problème, reconsidérant différentes techniques d’approximation de hessiens, et proposons de nouvelles méthodes de calcul de pas, en particulier dans le cadre des algorithmes de recherche linéaire. Il s’agit notamment d’algorithmes nous permettant de changer d’approximation de hessien et d’adapter la longueur du pas dans une direction de recherche fixée. Finalement, nous évaluons l’efficacité numérique des méthodes proposées dans le cadre de l’estimation de modèles de choix discrets, en particulier les modèles logit mélangés.

Essays on numerically efficient inference in nonlinear and non-Gaussian state space models, and commodity market analysis.

The first two articles build procedures to simulate vector of univariate states and estimate parameters in nonlinear and non Gaussian state space models. We propose state space speci fications that offer more flexibility in modeling dynamic relationship with latent variables. Our procedures are extension of the HESSIAN method of McCausland[2012]. Thus, they use approximation of the posterior density of the vector of states that allow to : simulate directly from the state vector posterior distribution, to simulate the states vector in one bloc and jointly with the vector of parameters, and to not allow data augmentation. These properties allow to build posterior simulators with very high relative numerical efficiency. Generic, they open a new path in nonlinear and non Gaussian state space analysis with limited contribution of the modeler. The third article is an essay in commodity market analysis. Private firms coexist with farmers' cooperatives in commodity markets in subsaharan african countries. The private firms have the biggest market share while some theoretical models predict they disappearance once confronted to farmers cooperatives. Elsewhere, some empirical studies and observations link cooperative incidence in a region with interpersonal trust, and thus to farmers trust toward cooperatives. We propose a model that sustain these empirical facts. A model where the cooperative reputation is a leading factor determining the market equilibrium of a price competition between a cooperative and a private firm

Staffing Optimization with Chance Constraints in Call Centers

Les centres d’appels sont des éléments clés de presque n’importe quelle grande organisation. Le problème de gestion du travail a reçu beaucoup d’attention dans la littérature. Une formulation typique se base sur des mesures de performance sur un horizon infini, et le problème d’affectation d’agents est habituellement résolu en combinant des méthodes d’optimisation et de simulation. Dans cette thèse, nous considérons un problème d’affection d’agents pour des centres d’appels soumis a des contraintes en probabilité. Nous introduisons une formulation qui exige que les contraintes de qualité de service (QoS) soient satisfaites avec une forte probabilité, et définissons une approximation de ce problème par moyenne échantillonnale dans un cadre de compétences multiples. Nous établissons la convergence de la solution du problème approximatif vers celle du problème initial quand la taille de l’échantillon croit. Pour le cas particulier où tous les agents ont toutes les compétences (un seul groupe d’agents), nous concevons trois méthodes d’optimisation basées sur la simulation pour le problème de moyenne échantillonnale. Étant donné un niveau initial de personnel, nous augmentons le nombre d’agents pour les périodes où les contraintes sont violées, et nous diminuons le nombre d’agents pour les périodes telles que les contraintes soient toujours satisfaites après cette réduction. Des expériences numériques sont menées sur plusieurs modèles de centre d’appels à faible occupation, au cours desquelles les algorithmes donnent de bonnes solutions, i.e. la plupart des contraintes en probabilité sont satisfaites, et nous ne pouvons pas réduire le personnel dans une période donnée sont introduire de violation de contraintes. Un avantage de ces algorithmes, par rapport à d’autres méthodes, est la facilité d’implémentation.

Improving sampling, optimization and feature extraction in Boltzmann machines

L’apprentissage supervisé de réseaux hiérarchiques à grande échelle connaît présentement un succès fulgurant. Malgré cette effervescence, l’apprentissage non-supervisé représente toujours, selon plusieurs chercheurs, un élément clé de l’Intelligence Artificielle, où les agents doivent apprendre à partir d’un nombre potentiellement limité de données. Cette thèse s’inscrit dans cette pensée et aborde divers sujets de recherche liés au problème d’estimation de densité par l’entremise des machines de Boltzmann (BM), modèles graphiques probabilistes au coeur de l’apprentissage profond. Nos contributions touchent les domaines de l’échantillonnage, l’estimation de fonctions de partition, l’optimisation ainsi que l’apprentissage de représentations invariantes. Cette thèse débute par l’exposition d’un nouvel algorithme d'échantillonnage adaptatif, qui ajuste (de fa ̧con automatique) la température des chaînes de Markov sous simulation, afin de maintenir une vitesse de convergence élevée tout au long de l’apprentissage. Lorsqu’utilisé dans le contexte de l’apprentissage par maximum de vraisemblance stochastique (SML), notre algorithme engendre une robustesse accrue face à la sélection du taux d’apprentissage, ainsi qu’une meilleure vitesse de convergence. Nos résultats sont présent ́es dans le domaine des BMs, mais la méthode est générale et applicable à l’apprentissage de tout modèle probabiliste exploitant l’échantillonnage par chaînes de Markov. Tandis que le gradient du maximum de vraisemblance peut-être approximé par échantillonnage, l’évaluation de la log-vraisemblance nécessite un estimé de la fonction de partition. Contrairement aux approches traditionnelles qui considèrent un modèle donné comme une boîte noire, nous proposons plutôt d’exploiter la dynamique de l’apprentissage en estimant les changements successifs de log-partition encourus à chaque mise à jour des paramètres. Le problème d’estimation est reformulé comme un problème d’inférence similaire au filtre de Kalman, mais sur un graphe bi-dimensionnel, où les dimensions correspondent aux axes du temps et au paramètre de température. Sur le thème de l’optimisation, nous présentons également un algorithme permettant d’appliquer, de manière efficace, le gradient naturel à des machines de Boltzmann comportant des milliers d’unités. Jusqu’à présent, son adoption était limitée par son haut coût computationel ainsi que sa demande en mémoire. Notre algorithme, Metric-Free Natural Gradient (MFNG), permet d’éviter le calcul explicite de la matrice d’information de Fisher (et son inverse) en exploitant un solveur linéaire combiné à un produit matrice-vecteur efficace. L’algorithme est prometteur: en terme du nombre d’évaluations de fonctions, MFNG converge plus rapidement que SML. Son implémentation demeure malheureusement inefficace en temps de calcul. Ces travaux explorent également les mécanismes sous-jacents à l’apprentissage de représentations invariantes. À cette fin, nous utilisons la famille de machines de Boltzmann restreintes “spike & slab” (ssRBM), que nous modifions afin de pouvoir modéliser des distributions binaires et parcimonieuses. Les variables latentes binaires de la ssRBM peuvent être rendues invariantes à un sous-espace vectoriel, en associant à chacune d’elles, un vecteur de variables latentes continues (dénommées “slabs”). Ceci se traduit par une invariance accrue au niveau de la représentation et un meilleur taux de classification lorsque peu de données étiquetées sont disponibles. Nous terminons cette thèse sur un sujet ambitieux: l’apprentissage de représentations pouvant séparer les facteurs de variations présents dans le signal d’entrée. Nous proposons une solution à base de ssRBM bilinéaire (avec deux groupes de facteurs latents) et formulons le problème comme l’un de “pooling” dans des sous-espaces vectoriels complémentaires.

Transformations quasi-conformes de maillages volumiques et applications en infographie

La modélisation géométrique est importante autant en infographie qu'en ingénierie. Notre capacité à représenter l'information géométrique fixe les limites et la facilité avec laquelle on manipule les objets 3D. Une de ces représentations géométriques est le maillage volumique, formé de polyèdres assemblés de sorte à approcher une forme désirée. Certaines applications, tels que le placage de textures et le remaillage, ont avantage à déformer le maillage vers un domaine plus régulier pour faciliter le traitement. On dit qu'une déformation est \emph{quasi-conforme} si elle borne la distorsion. Cette thèse porte sur l’étude et le développement d'algorithmes de déformation quasi-conforme de maillages volumiques. Nous étudions ces types de déformations parce qu’elles offrent de bonnes propriétés de préservation de l’aspect local d’un solide et qu’elles ont été peu étudiées dans le contexte de l’informatique graphique, contrairement à leurs pendants 2D. Cette recherche tente de généraliser aux volumes des concepts bien maitrisés pour la déformation de surfaces. Premièrement, nous présentons une approche linéaire de la quasi-conformité. Nous développons une méthode déformant l’objet vers son domaine paramétrique par une méthode des moindres carrés linéaires. Cette méthode est simple d'implémentation et rapide d'exécution, mais n'est qu'une approximation de la quasi-conformité car elle ne borne pas la distorsion. Deuxièmement, nous remédions à ce problème par une approche non linéaire basée sur les positions des sommets. Nous développons une technique déformant le domaine paramétrique vers le solide par une méthode des moindres carrés non linéaires. La non-linéarité permet l’inclusion de contraintes garantissant l’injectivité de la déformation. De plus, la déformation du domaine paramétrique au lieu de l’objet lui-même permet l’utilisation de domaines plus généraux. Troisièmement, nous présentons une approche non linéaire basée sur les angles dièdres. Cette méthode définit la déformation du solide par les angles dièdres au lieu des positions des sommets du maillage. Ce changement de variables permet une expression naturelle des bornes de distorsion de la déformation. Nous présentons quelques applications de cette nouvelle approche dont la paramétrisation, l'interpolation, l'optimisation et la compression de maillages tétraédriques.

STUDIES ON THE STRESS FLUCTUATIONS IN SHEARED STOKESIAN SUSPENSIONS USING CHAOS THEORY AND NONLINEAR DYNAMICS

The thesis report results obtained from a detailed analysis of the fluctuations of the rheological parameters viz. shear and normal stresses, simulated by means of the Stokesian Dynamics method, of a macroscopically homogeneous sheared suspension of neutrally buoyant non-Brownian suspension of identical spheres in the Couette gap between two parallel walls in the limit of vanishingly small Reynolds numbers using the tools of non-linear dynamics and chaos theory for a range of particle concentration and Couette gaps. The thesis used the tools of nonlinear dynamics and chaos theory viz. average mutual information, space-time separation plots, visual recurrence analysis, principal component analysis, false nearest-neighbor technique, correlation integrals, computation of Lyapunov exponents for a range of area fraction of particles and for different Couette gaps. The thesis observed that one stress component can be predicted using another stress component at the same area fraction. This implies a type of synchronization of one stress component with another stress component. This finding suggests us to further analysis of the synchronization of stress components with another stress component at the same or different area fraction of particles. The different model equations of stress components for different area fraction of particles hints at the possible existence a general formula for stress fluctuations with area fraction of particle as a parameter

Optimization of pH and Direct Imaging Conditions of Complexed Methylene Blue Sensitized Poly(vinyl chloride) Films

Significant results of our experimental investigations on the dependence of pH on real time transmission characteristics on recording media fabricated by doping PVC with complexed methylene blue are presented. The optimum pH value for faster bleaching was found to be 4×5. In typical applications, the illumination from one side, normal to the surface of this material, initiates a chemical sequence that records the incident light pattern in the polymer. Thus direct imaging can be successfully done on this sample. The recorded letters were very legible with good contrast and no scattering centres. Diffraction efficiency measurements were also carried out on this material.

Plasma Polymerised Organic Thin Films-A Study on the Structural, Electrical and Nonlinear Optical Properties for Possible Applications

This proposed thesis is entitled “Plasma Polymerised Organic Thin Films: A study on the Structural, Electrical, and Nonlinear Optical Properties for Possible Applications. Polymers and polymer based materials find enormous applications in the realm of electronics and optoelectronics. They are employed as both active and passive components in making various devices. Enormous research activities are going on in this area for the last three decades or so, and many useful contributions are made quite accidentally. Conducting polymers is such a discovery, and eversince the discovery of conducting polyacetylene, a new branch of science itself has emerged in the form of synthetic metals. Conducting polymers are useful materials for many applications like polymer displays, high density data storage, polymer FETs, polymer LEDs, photo voltaic devices and electrochemical cells. With the emergence of molecular electronics and its potential in finding useful applications, organic thin films are receiving an unusual attention by scientists and engineers alike. This is evident from the vast literature pertaining to this field appearing in various journals. Recently, computer aided design of organic molecules have added further impetus to the ongoing research activities in this area. Polymers, especially, conducting polymers can be prepared both in the bulk and in the thinfilm form. However, many applications necessitate that they are grown in the thin film form either as free standing or on appropriate substrates. As far as their bulk counterparts are concerned, they can be prepared by various polymerisation techniques such as chemical routes and electrochemical means. A survey of the literature reveals that polymers like polyaniline, polypyrrole, polythiophene, have been investigated with a view to studying their structural electrical and optical properties. Among the various alternate techniques employed for the preparation of polymer thin films, the method of plasma polymerisation needs special attention in this context. The technique of plasma polymerisation is an inexpensive method and often requires very less infra structure. This method includes the employment of ac, rf, dc, microwave and pulsed sources. They produce pinhole free homogeneous films on appropriate substrates under controlled conditions. In conventional plasma polymerisation set up, the monomer is fed into an evacuated chamber and an ac/rf/dc/ w/pulsed discharge is created which enables the monomer species to dissociate, leading to the formation of polymer thin films. However, it has been found that the structure and hence the properties exhibited by plasma polymerized thin films are quite different from that of their counterparts produced by other thin film preparation techniques such as electrochemical deposition or spin coating. The properties of these thin films can be tuned only if the interrelationship between the structure and other properties are understood from a fundamental point of view. So very often, a through evaluation of the various properties is a pre-requisite for tailoring the properties of the thin films for applications. It has been found that conjugation is a necessary condition for enhancing the conductivity of polymer thin films. RF technique of plasma polymerisation is an excellent tool to induce conjugation and this modifies the electrical properties too. Both oxidative and reductive doping can be employed to modify the electrical properties of the polymer thin films for various applications. This is where organic thin films based on polymers scored over inorganic thin films, where in large area devices can be fabricated with organic semiconductors which is difficult to achieve by inorganic materials. For such applications, a variety of polymers have been synthesized such as polyaniline, polythiophene, polypyrrole etc. There are newer polymers added to this family every now and then. There are many virgin areas where plasma polymers are yet to make a foray namely low-k dielectrics or as potential nonlinear optical materials such as optical limiters. There are also many materials which are not been prepared by the method of plasma polymerisation. Some of the materials which are not been dealt with are phenyl hydrazine and tea tree oil. The advantage of employing organic extracts like tea tree oil monomers as precursors for making plasma polymers is that there can be value addition to the already existing uses and possibility exists in converting them to electronic grade materials, especially semiconductors and optically active materials for photonic applications. One of the major motivations of this study is to synthesize plasma polymer thin films based on aniline, phenyl hydrazine, pyrrole, tea tree oil and eucalyptus oil by employing both rf and ac plasma polymerisation techniques. This will be carried out with the objective of growing thin films on various substrates such as glass, quartz and indium tin oxide (ITO) coated glass. There are various properties namely structural, electrical, dielectric permittivity, nonlinear optical properties which are to be evaluated to establish the relationship with the structure and the other properties. Special emphasis will be laid in evaluating the optical parameters like refractive index (n), extinction coefficient (k), the real and imaginary components of dielectric constant and the optical transition energies of the polymer thin films from the spectroscopic ellipsometric studies. Apart from evaluating these physical constants, it is also possible to predict whether a material exhibit nonlinear optical properties by ellipsometric investigations. So further studies using open aperture z-scan technique in order to evaluate the nonlinear optical properties of a few selected samples which are potential nonlinear optical materials is another objective of the present study. It will be another endeavour to offer an appropriate explanation for the nonlinear optical properties displayed by these films. Doping of plasma polymers is found to modify both the electrical conductivity and optical properties. Iodine is found to modify the properties of the polymer thin films. However insitu iodine doping is tricky and the film often looses its stability because of the escape of iodine. An appropriate insitu technique of doping will be developed to dope iodine in to the plasma polymerized thin films. Doping of polymer thin films with iodine results in improved and modified optical and electrical properties. However it requires tools like FTIR and UV-Vis-NIR spectroscopy to elucidate the structural and optical modifications imparted to the polymer films. This will be attempted here to establish the role of iodine in the modification of the properties exhibited by the films

Optimization of pH and Direct Imaging Conditions of Complexed Methylene Blue Sensitized Poly(vinyl chloride) Films

Significant results of our experimental investigations on the dependence of pH on real time transmission characteristics on recording media fabricated by doping PVC with complexed methylene blue are presented. The optimum pH value for faster bleaching was found to be 4×5. In typical applications, the illumination from one side, normal to the surface of this material, initiates a chemical sequence that records the incident light pattern in the polymer. Thus direct imaging can be successfully done on this sample. The recorded letters were very legible with good contrast and no scattering centres. Diffraction efficiency measurements were also carried out on this material.

Optimization of pH and direct imaging conditions of complexed methylene blue sensitized poly(vinyl chloride) films

Significant results of our experimental investigations on the dependence of pH on real time transmission characteristics on recording media fabricated by doping PVC with complexed methylene blue are presented. The optimum pH value for faster bleaching was found to be 4 . 5. In typical applications, the illumination from one side, normal to the surface of this material, initiates a chemical sequence that records the incident light pattern in the polymer. Thus direct imaging can be successfully done on this sample. The recorded letters were very legible with good contrast and no scattering centres. Diffraction efficiency measurements were also carried out on this material.

Nonlinear Dynamics of Semiconductor LasersControl and Synchronization of Chaos:

Nonlinear dynamics of laser systems has become an interesting area of research in recent times. Lasers are good examples of nonlinear dissipative systems showing many kinds of nonlinear phenomena such as chaos, multistability and quasiperiodicity. The study of these phenomena in lasers has fundamental scientific importance since the investigations on these effects reveal many interesting features of nonlinear effects in practical systems. Further, the understanding of the instabilities in lasers is helpful in detecting and controlling such effects. Chaos is one of the most interesting phenomena shown by nonlinear deterministic systems. It is found that, like many nonlinear dissipative systems, lasers also show chaos for certain ranges of parameters. Many investigations on laser chaos have been done in the last two decades. The earlier studies in this field were concentrated on the dynamical aspects of laser chaos. However, recent developments in this area mainly belong to the control and synchronization of chaos. A number of attempts have been reported in controlling or suppressing chaos in lasers since lasers are the practical systems aimed to operated in stable or periodic mode. On the other hand, laser chaos has been found to be applicable in high speed secure communication based on synchronization of chaos. Thus, chaos in laser systems has technological importance also. Semiconductor lasers are most applicable in the fields of optical communications among various kinds of laser due to many reasons such as their compactness, reliability modest cost and the opportunity of direct current modulation. They show chaos and other instabilities under various physical conditions such as direct modulation and optical or optoelectronic feedback. It is desirable for semiconductor lasers to have stable and regular operation. Thus, the understanding of chaos and other instabilities in semiconductor lasers and their xi control is highly important in photonics. We address the problem of controlling chaos produced by direct modulation of laser diodes. We consider the delay feedback control methods for this purpose and study their performance using numerical simulation. Besides the control of chaos, control of other nonlinear effects such as quasiperiodicity and bistability using delay feedback methods are also investigated. A number of secure communication schemes based on synchronization of chaos semiconductor lasers have been successfully demonstrated theoretically and experimentally. The current investigations in these field include the study of practical issues on the implementations of such encryption schemes. We theoretically study the issues such as channel delay, phase mismatch and frequency detuning on the synchronization of chaos in directly modulated laser diodes. It would be helpful for designing and implementing chaotic encryption schemes using synchronization of chaos in modulated semiconductor laser

Ultrasonic measurement of the elastic constants of selected nonlinear optical /semiorganic crystals with orthorhombic symmetry

Ultrasonic is a good tool to investigate the elastic properties of crystals. It enables one to determine all the elastic constants, Poisson’s ratios, volume compressibility and bulk modulus of crystals from velocity measurements. It also enables one to demonstrate the anisotropy of elastic properties by plotting sections of the surfaces of phase velocity, slowness, group velocity, Young’s modulus and linear compressibility along the a-b, b-c and a-c planes. They also help one to understand more about phonon amplification and help to interpret various phenomena associated with ultrasonic wave propagation, thermal conductivity, phonon transport etc. Study of nonlinear optical crystals is very important from an application point of view. Hundreds of new NLO materials are synthesized to meet the requirements for various applications. Inorganic, organic and organometallic or semiorganic classes of compounds have been studied for several reasons. Semiorganic compounds have some advantages over their inorganic and inorganic counterparts with regard to their mechanical properties. High damage resistance, high melting point, good transparency and non-hygroscopy are some of the basic requirements for a material to be suitable for device fabrication. New NLO materials are being synthesized and investigation of the mechanical and elastic properties of these crystals is very important to test the suitability of these materials for technological applications

Spectral and nonlinear optical characterization of ZnO nanocomposites

The spectral and nonlinear optical characteristics of nano ZnO and its composites are investigated. The fluorescence behaviour of nano colloids of ZnO has been studied as a function of the excitation wavelength and there is a red shift in emission peak with excitation wavelength. Apart from the observation of the reported ultra violet and green emissions, our results reveal that additional blue emissions at 420 nm and 490 nm are developed with increasing particle size. Systematic studies on nano ZnO have indicated the presence of luminescence due to excitonic emissions when excited with 255 nm as well as significant contribution from surface defect states when excited with 325 nm. In the weak confinement regime, the third-order optical susceptibility χ(3) increases with increasing particle size (R) and annealing temperature (T) and a R2 and T2.5 dependence of χ(3) is obtained for nano ZnO. ZnO nanocolloids exhibit induced absorption whereas the self assembled films of ZnO exhibit saturable absorption due to saturation of linear absorption of ZnO defect states and electronic effects. ZnO nanocomposites exhibit negative nonlinear index of refraction which can be attributed to two photon absorption followed by weak free carrier absorption. The increase of the third-order nonlinearity in the composites can be attributed to the enhancement of exciton oscillator strength. The nonlinear response of ZnO nanocomposites is wavelength dependent and switching from induced absorption to saturable absorption has been observed at resonant wavelengths. Such a change-over is related to the interplay of plasmon/exciton band bleach and optical limiting mechanisms. This study is important in identifying the spectral range and the composition over which the nonlinear material acts as an optical limiter. ZnO based nanocomposites are potential materials for enhanced and tunable light emission and for the development of nonlinear optical devices with a relatively small optical limiting threshold.