Improving sampling, optimization and feature extraction in Boltzmann machines

L’apprentissage supervisé de réseaux hiérarchiques à grande échelle connaît présentement un succès fulgurant. Malgré cette effervescence, l’apprentissage non-supervisé représente toujours, selon plusieurs chercheurs, un élément clé de l’Intelligence Artificielle, où les agents doivent apprendre à partir d’un nombre potentiellement limité de données. Cette thèse s’inscrit dans cette pensée et aborde divers sujets de recherche liés au problème d’estimation de densité par l’entremise des machines de Boltzmann (BM), modèles graphiques probabilistes au coeur de l’apprentissage profond. Nos contributions touchent les domaines de l’échantillonnage, l’estimation de fonctions de partition, l’optimisation ainsi que l’apprentissage de représentations invariantes. Cette thèse débute par l’exposition d’un nouvel algorithme d'échantillonnage adaptatif, qui ajuste (de fa ̧con automatique) la température des chaînes de Markov sous simulation, afin de maintenir une vitesse de convergence élevée tout au long de l’apprentissage. Lorsqu’utilisé dans le contexte de l’apprentissage par maximum de vraisemblance stochastique (SML), notre algorithme engendre une robustesse accrue face à la sélection du taux d’apprentissage, ainsi qu’une meilleure vitesse de convergence. Nos résultats sont présent ́es dans le domaine des BMs, mais la méthode est générale et applicable à l’apprentissage de tout modèle probabiliste exploitant l’échantillonnage par chaînes de Markov. Tandis que le gradient du maximum de vraisemblance peut-être approximé par échantillonnage, l’évaluation de la log-vraisemblance nécessite un estimé de la fonction de partition. Contrairement aux approches traditionnelles qui considèrent un modèle donné comme une boîte noire, nous proposons plutôt d’exploiter la dynamique de l’apprentissage en estimant les changements successifs de log-partition encourus à chaque mise à jour des paramètres. Le problème d’estimation est reformulé comme un problème d’inférence similaire au filtre de Kalman, mais sur un graphe bi-dimensionnel, où les dimensions correspondent aux axes du temps et au paramètre de température. Sur le thème de l’optimisation, nous présentons également un algorithme permettant d’appliquer, de manière efficace, le gradient naturel à des machines de Boltzmann comportant des milliers d’unités. Jusqu’à présent, son adoption était limitée par son haut coût computationel ainsi que sa demande en mémoire. Notre algorithme, Metric-Free Natural Gradient (MFNG), permet d’éviter le calcul explicite de la matrice d’information de Fisher (et son inverse) en exploitant un solveur linéaire combiné à un produit matrice-vecteur efficace. L’algorithme est prometteur: en terme du nombre d’évaluations de fonctions, MFNG converge plus rapidement que SML. Son implémentation demeure malheureusement inefficace en temps de calcul. Ces travaux explorent également les mécanismes sous-jacents à l’apprentissage de représentations invariantes. À cette fin, nous utilisons la famille de machines de Boltzmann restreintes “spike & slab” (ssRBM), que nous modifions afin de pouvoir modéliser des distributions binaires et parcimonieuses. Les variables latentes binaires de la ssRBM peuvent être rendues invariantes à un sous-espace vectoriel, en associant à chacune d’elles, un vecteur de variables latentes continues (dénommées “slabs”). Ceci se traduit par une invariance accrue au niveau de la représentation et un meilleur taux de classification lorsque peu de données étiquetées sont disponibles. Nous terminons cette thèse sur un sujet ambitieux: l’apprentissage de représentations pouvant séparer les facteurs de variations présents dans le signal d’entrée. Nous proposons une solution à base de ssRBM bilinéaire (avec deux groupes de facteurs latents) et formulons le problème comme l’un de “pooling” dans des sous-espaces vectoriels complémentaires.

Dielectric Behavior of Natural Rubber Composites in Microwave Fields

In this article, we report the preparation of conducting natural rubber (NR) with polyaniline (Pani). NR was made into a conductive material by the compounding of NR with Pani in powder form. NR latex was made into a conductive material by the in situ polymerization of aniline in the presence of NR latex. Different compositions of Pani- NR semi-interpenetrating networks were prepared, and the dielectric properties of all of the samples were determined in microwave frequencies. The cavity perturbation techpique was used for this study. A HP8510 vector network analyzer with a rectangular cavity resonator was used for this study. S bands 2-4 GHz in frequency were used. Thermal studies were also carried out with thermogravimetric analysis and differential scanning calorimetry.

Dielectric Behavior of Natural Rubber Composites in Microwave Fields

In this article, we report the preparation of conducting natural rubber (NR) with polyaniline (Pani). NR was made into a conductive material by the compounding of NR with Pani in powder form. NR latex was made into a conductive material by the in situ polymerization of aniline in the presence of NR latex. Different compositions of Pani- NR semi-interpenetrating networks were prepared, and the dielectric properties of all of the samples were determined in microwave frequencies. The cavity perturbation techpique was used for this study. A HP8510 vector network analyzer with a rectangular cavity resonator was used for this study. S bands 2-4 GHz in frequency were used. Thermal studies were also carried out with thermogravimetric analysis and differential scanning calorimetry.

Synthesis and Characterisation of Thermoplastic Ionomers based on Natural Rubber

Chemically modified novel thermo-reversible zinc sulphonated ionomers based on natural rubber (NR), radiation induced styrene grafted natural rubber (RI-SGNR), and chemically induced styrene grafted natural rubber (CI-SGNR) were synthesized using acetyl sulphate/zinc acetate reagent system. Evidence for the attachment of sulphonate groups has been furnished by FTIR spectra. which was supplanted by FTNMR results. Estimation of the zinc sulphonate group was done using spectroscopic techniques such as XRFS and ICPAES. The TGA results prove improvement in the therrno-oxidative stability of the modified natural rubber. Both DSC and DMTA studies show that the incorporation of the ionic groups affect the thermal transition of the base polymer. Retention of the improved physical properties of the novel ionomers even after three repeated cycles of mastication and molding at 120 degree C may be considered as the evidence for the reprocessabiJity of the ionomer. Effect of both particulate (carbon black. silica & zinc stearate) and fibrous fillers (nylon & glass) on the properties of the radiation induced styrene grafted natural rubber ionomer has been evaluated. Incorporation of HAF carbon black results in maximum improvement in physical properties. Silica reinforces the backbone chain and weakens the ionic associations. Zinc stearate plays the dual role of reinforcement and ptasticization. The nylon and glass filled lonorner compounds show good improvement in the physical properties in comparison with the neat ionomer. Dispersion and adhesion of the fillers in the ionomer matrix has been amply supported by their SEM micrographs. Microwave probing of the electrical behavior of the 26.5 ZnSRISGNR ionomer reveals that the maximum relative complex conductivity and the complex permittivity appear at the frequency of 2.6 GHz. The complex conductivity of the base polymer increases from 1.8x 10.12 S/cm to 3.3xlO·4 S/cm. Influence of fillers on the dielectric constant and conductivity of the new ionic thermoplastic elastomer has been studied. The ionomer I nylon compound shows the highest microwave conductivity. Use of the 26.5 ZnS-RISGNR ionomer as a compatibilizer for obtaining the technologically compatible blends from the immiscible SBR/NBR system has been verified. The heat fugitive ionic cross-linked natural rubber may be, therefore, useful as an alternative to vulcanized rubber and thermoplastic elastomer

Energy Management in Wireless Sensor Networks: A Crosslayer,Channel Adaptive Approach Towards Performance Optimization

The proliferation of wireless sensor networks in a large spectrum of applications had been spurered by the rapid advances in MEMS(micro-electro mechanical systems )based sensor technology coupled with low power,Low cost digital signal processors and radio frequency circuits.A sensor network is composed of thousands of low cost and portable devices bearing large sensing computing and wireless communication capabilities. This large collection of tiny sensors can form a robust data computing and communication distributed system for automated information gathering and distributed sensing.The main attractive feature is that such a sensor network can be deployed in remote areas.Since the sensor node is battery powered,all the sensor nodes should collaborate together to form a fault tolerant network so as toprovide an efficient utilization of precious network resources like wireless channel,memory and battery capacity.The most crucial constraint is the energy consumption which has become the prime challenge for the design of long lived sensor nodes.

A Numerically efficient power optimization scheme for coded OFDM systems in achieving minimum frame error rate

Coded OFDM is a transmission technique that is used in many practical communication systems. In a coded OFDM system, source data are coded, interleaved and multiplexed for transmission over many frequency sub-channels. In a conventional coded OFDM system, the transmission power of each subcarrier is the same regardless of the channel condition. However, some subcarrier can suffer deep fading with multi-paths and the power allocated to the faded subcarrier is likely to be wasted. In this paper, we compute the FER and BER bounds of a coded OFDM system given as convex functions for a given channel coder, inter-leaver and channel response. The power optimization is shown to be a convex optimization problem that can be solved numerically with great efficiency. With the proposed power optimization scheme, near-optimum power allocation for a given coded OFDM system and channel response to minimize FER or BER under a constant transmission power constraint is obtained

Lógica en la toma de decisiones de inversión en el sector de extracción de petróleo y gas natural

Este documento se centra en la presentación de información y análisis de la misma a la hora de establecer la manera en que empresas del sector de extracción de gas natural y generación de energía a base de dicho recurso, toman decisiones en cuanto a inversión, centrándose en la lógica que usan a la hora de emprender este proceso. Esto debido a la constante necesidad de establecer procesos que permitan tomar decisiones más acertadas, incluyendo todas las herramientas posibles para tal fin. La lógica es una de estas herramientas, pues permite encadenar factores con el fin de obtener resultados positivos. Por tal razón, se hace importante conocer el uso de esta herramienta, teniendo en cuentas de qué manera y en que contextos es usada. Con el fin de tener una mayor orientación, este estudio estará centrado en un sector específico, el cual es el de la extracción de petróleo y gas natural. Lo anterior entendiendo la necesidad existente de fundamentación teórica que permita establecer de manera clara la forma apropiada de tomar decisiones en un sector tan diverso y complejo como lo es el mencionado. El contexto empresarial actual exige una visión global, no basada en la lógica lineal causal que hoy se tiene como referencia. El sector de extracción de petróleo y gas natural es un ejemplo particular en cuanto a la manera en cuanto se toman decisiones en inversión, puesto que en su mayoría son empresas de capital intensivo, las cuales mantienen un flujo elevado de recursos monetarios.

Optimal management of natural resources. Accounting for heterogeneity

Dynamic optimization methods have become increasingly important over the last years in economics. Within the dynamic optimization techniques employed, optimal control has emerged as the most powerful tool for the theoretical economic analysis. However, there is the need to advance further and take account that many dynamic economic processes are, in addition, dependent on some other parameter different than time. One can think of relaxing the assumption of a representative (homogeneous) agent in macro- and micro-economic applications allowing for heterogeneity among the agents. For instance, the optimal adaptation and diffusion of a new technology over time, may depend on the age of the person that adopted the new technology. Therefore, the economic models must take account of heterogeneity conditions within the dynamic framework. This thesis intends to accomplish two goals. The first goal is to analyze and revise existing environmental policies that focus on defining the optimal management of natural resources over time, by taking account of the heterogeneity of environmental conditions. Thus, the thesis makes a policy orientated contribution in the field of environmental policy by defining the necessary changes to transform an environmental policy based on the assumption of homogeneity into an environmental policy which takes account of heterogeneity. As a result the newly defined environmental policy will be more efficient and likely also politically more acceptable since it is tailored more specifically to the heterogeneous environmental conditions. Additionally to its policy orientated contribution, this thesis aims making a methodological contribution by applying a new optimization technique for solving problems where the control variables depend on two or more arguments --- the so-called two-stage solution approach ---, and by applying a numerical method --- the Escalator Boxcar Train Method --- for solving distributed optimal control problems, i.e., problems where the state variables, in addition to the control variables, depend on two or more arguments. Chapter 2 presents a theoretical framework to determine optimal resource allocation over time for the production of a good by heterogeneous producers, who generate a stock externalit and derives government policies to modify the behavior of competitive producers in order to achieve optimality. Chapter 3 illustrates the method in a more specific context, and integrates the aspects of quality and time, presenting a theoretical model that allows to determine the socially optimal outcome over time and space for the problem of waterlogging in irrigated agricultural production. Chapter 4 of this thesis concentrates on forestry resources and analyses the optimal selective-logging regime of a size-distributed forest.

A benchmark method for global optimization problems in structure determination from powder diffraction data

Quasi-Newton-Raphson minimization and conjugate gradient minimization have been used to solve the crystal structures of famotidine form B and capsaicin from X-ray powder diffraction data and characterize the chi(2) agreement surfaces. One million quasi-Newton-Raphson minimizations found the famotidine global minimum with a frequency of ca 1 in 5000 and the capsaicin global minimum with a frequency of ca 1 in 10 000. These results, which are corroborated by conjugate gradient minimization, demonstrate the existence of numerous pathways from some of the highest points on these chi(2) agreement surfaces to the respective global minima, which are passable using only downhill moves. This important observation has significant ramifications for the development of improved structure determination algorithms.

Can the study of natural vegetation succession assist in the control of soil erosion on abandoned croplands on the Loess Plateau, China?

In the Loess Plateau, China, arable cultivation of slope lands is common and associated with serious soil erosion. Planting trees or grass may control erosion, but planted species may consume more soil water and can threaten long-term ecosystem sustainability. Natural vegetation succession is an alternative ecological solution to restore degraded land, but there is a time cost, given that the establishment of natural vegetation, adequate to prevent soil erosion, is a longer process than planting. The aims of this study were to identify the environmental factors controlling the type of vegetation established on abandoned cropland and to identify candidate species that might be sown soon after abandonment to accelerate vegetation succession and establishment of natural vegetation to prevent soil erosion. A field survey of thirty-three 2 × 2–m plots was carried out in July 2003, recording age since abandonment, vegetation cover, and frequency of species together with major environmental and soil variables. Data were analyzed using correspondence analysis, classification tree analysis, and species response curves. Four vegetation types were identified and the data analysis confirmed the importance of time since abandonment, total P, and soil water in controlling the type of vegetation established. Among the dominant species in the three late-successional vegetation types, the most appropriate candidates for accelerating and directing vegetation succession were King Ranch bluestem (Bothriochloa ischaemum) and Lespedeza davurica (Leguminosae). These species possess combinations of the following characteristics: tolerance of low water and nutrient availability, fibrous root system and strong lateral vegetative spread, and a persistent seed bank.

Optimization of apodization functions in terahertz transient spectrometry

A quadratic programming optimization procedure for designing asymmetric apodization windows tailored to the shape of time-domain sample waveforms recorded using a terahertz transient spectrometer is proposed. By artificially degrading the waveforms, the performance of the designed window in both the time and the frequency domains is compared with that of conventional rectangular, triangular (Mertz), and Hamming windows. Examples of window optimization assuming Gaussian functions as the building elements of the apodization window are provided. The formulation is sufficiently general to accommodate other basis functions. (C) 2007 Optical Society of America

Temporal-envelope constancy of speech in rooms and the perceptual weighting of frequency bands

Three experiments measured constancy in speech perception, using natural-speech messages or noise-band vocoder versions of them. The eight vocoder-bands had equally log-spaced center-frequencies and the shapes of corresponding “auditory” filters. Consequently, the bands had the temporal envelopes that arise in these auditory filters when the speech is played. The “sir” or “stir” test-words were distinguished by degrees of amplitude modulation, and played in the context; “next you’ll get _ to click on.” Listeners identified test-words appropriately, even in the vocoder conditions where the speech had a “noise-like” quality. Constancy was assessed by comparing the identification of test-words with low or high levels of room reflections across conditions where the context had either a low or a high level of reflections. Constancy was obtained with both the natural and the vocoded speech, indicating that the effect arises through temporal-envelope processing. Two further experiments assessed perceptual weighting of the different bands, both in the test word and in the context. The resulting weighting functions both increase monotonically with frequency, following the spectral characteristics of the test-word’s [s]. It is suggested that these two weighting functions are similar because they both come about through the perceptual grouping of the test-word’s bands.

Exploring the interplay between natural decadal variability and anthropogenic climate change in summer rainfall over China. Part I: Observational evidence

Summer rainfall over China has experienced substantial variability on longer time scales during the last century, and the question remains whether this is due to natural, internal variability or is part of the emerging signal of anthropogenic climate change. Using the best available observations over China, the decadal variability and recent trends in summer rainfall are investigated with the emphasis on changes in the seasonal evolution and on the temporal characteristics of daily rainfall. The possible relationships with global warming are reassessed. Substantial decadal variability in summer rainfall has been confirmed during the period 1958–2008; this is not unique to this period but is also seen in the earlier decades of the twentieth century. Two dominant patterns of decadal variability have been identified that contribute substantially to the recent trend of southern flooding and northern drought. Natural decadal variability appears to dominate in general but in the cases of rainfall intensity and the frequency of rainfall days, particularly light rain days, then the dominant EOFs have a rather different character, being of one sign over most of China, and having principal components (PCs) that appear more trendlike. The increasing intensity of rainfall throughout China and the decrease in light rainfall days, particularly in the north, could at least partially be of anthropogenic origin, both global and regional, linked to increased greenhouse gases and increased aerosols.

Single-carrier frequency domain equalisation for hammerstein communication systems using complex-valued neural networks

Single-carrier (SC) block transmission with frequency-domain equalisation (FDE) offers a viable transmission technology for combating the adverse effects of long dispersive channels encountered in high-rate broadband wireless communication systems. However, for high bandwidthefficiency and high power-efficiency systems, the channel can generally be modelled by the Hammerstein system that includes the nonlinear distortion effects of the high power amplifier (HPA) at transmitter. For such nonlinear Hammerstein channels, the standard SC-FDE scheme no longer works. This paper advocates a complex-valued (CV) B-spline neural network based nonlinear SC-FDE scheme for Hammerstein channels. Specifically, We model the nonlinear HPA, which represents the CV static nonlinearity of the Hammerstein channel, by a CV B-spline neural network, and we develop two efficient alternating least squares schemes for estimating the parameters of the Hammerstein channel, including both the channel impulse response coefficients and the parameters of the CV B-spline model. We also use another CV B-spline neural network to model the inversion of the nonlinear HPA, and the parameters of this inverting B-spline model can easily be estimated using the standard least squares algorithm based on the pseudo training data obtained as a natural byproduct of the Hammerstein channel identification. Equalisation of the SC Hammerstein channel can then be accomplished by the usual one-tap linear equalisation in frequency domain as well as the inverse B-spline neural network model obtained in time domain. Extensive simulation results are included to demonstrate the effectiveness of our nonlinear SC-FDE scheme for Hammerstein channels.

High-frequency water quality monitoring in an urban catchment: hydrochemical dynamics, primary production and implications for the Water Framework Directive

This paper describes the hydrochemistry of a lowland, urbanised river-system, The Cut in England, using in situ sub-daily sampling. The Cut receives effluent discharges from four major sewage treatment works serving around 190,000 people. These discharges consist largely of treated water, originally abstracted from the River Thames and returned via the water supply network, substantially increasing the natural flow. The hourly water quality data were supplemented by weekly manual sampling with laboratory analysis to check the hourly data and measure further determinands. Mean phosphorus and nitrate concentrations were very high, breaching standards set by EU legislation. Though 56% of the catchment area is agricultural, the hydrochemical dynamics were significantly impacted by effluent discharges which accounted for approximately 50% of the annual P catchment input loads and, on average, 59% of river flow at the monitoring point. Diurnal dissolved oxygen data demonstrated high in-stream productivity. From a comparison of high frequency and conventional monitoring data, it is inferred that much of the primary production was dominated by benthic algae, largely diatoms. Despite the high productivity and nutrient concentrations, the river water did not become anoxic and major phytoplankton blooms were not observed. The strong diurnal and annual variation observed showed that assessments of water quality made under the Water Framework Directive (WFD) are sensitive to the time and season of sampling. It is recommended that specific sampling time windows be specified for each determinand, and that WFD targets should be applied in combination to help identify periods of greatest ecological risk. This article is protected by copyright. All rights reserved.