Incremental natural-gradient actor-critic algorithms

We present four new reinforcement learning algorithms based on actor-critic and natural-gradient ideas, and provide their convergence proofs. Actor-critic rein- forcement learning methods are online approximations to policy iteration in which the value-function parameters are estimated using temporal difference learning and the policy parameters are updated by stochastic gradient descent. Methods based on policy gradients in this way are of special interest because of their com- patibility with function approximation methods, which are needed to handle large or infinite state spaces. The use of temporal difference learning in this way is of interest because in many applications it dramatically reduces the variance of the gradient estimates. The use of the natural gradient is of interest because it can produce better conditioned parameterizations and has been shown to further re- duce variance in some cases. Our results extend prior two-timescale convergence results for actor-critic methods by Konda and Tsitsiklis by using temporal differ- ence learning in the actor and by incorporating natural gradients, and they extend prior empirical studies of natural actor-critic methods by Peters, Vijayakumar and Schaal by providing the first convergence proofs and the first fully incremental algorithms.

A comparison of three methods for downscaling daily precipitation in the Punjab region

Many downscaling techniques have been developed in the past few years for projection of station-scale hydrological variables from large-scale atmospheric variables simulated by general circulation models (GCMs) to assess the hydrological impacts of climate change. This article compares the performances of three downscaling methods, viz. conditional random field (CRF), K-nearest neighbour (KNN) and support vector machine (SVM) methods in downscaling precipitation in the Punjab region of India, belonging to the monsoon regime. The CRF model is a recently developed method for downscaling hydrological variables in a probabilistic framework, while the SVM model is a popular machine learning tool useful in terms of its ability to generalize and capture nonlinear relationships between predictors and predictand. The KNN model is an analogue-type method that queries days similar to a given feature vector from the training data and classifies future days by random sampling from a weighted set of K closest training examples. The models are applied for downscaling monsoon (June to September) daily precipitation at six locations in Punjab. Model performances with respect to reproduction of various statistics such as dry and wet spell length distributions, daily rainfall distribution, and intersite correlations are examined. It is found that the CRF and KNN models perform slightly better than the SVM model in reproducing most daily rainfall statistics. These models are then used to project future precipitation at the six locations. Output from the Canadian global climate model (CGCM3) GCM for three scenarios, viz. A1B, A2, and B1 is used for projection of future precipitation. The projections show a change in probability density functions of daily rainfall amount and changes in the wet and dry spell distributions of daily precipitation. Copyright (C) 2011 John Wiley & Sons, Ltd.

Support Vector Machine Approach to Downscale Precipitation in Climate Change Scenarios

Concern over changes in global climate has increased in recent years with improvement in understanding of atmospheric dynamics and growth in evidence of climate link to long‐term variability in hydrologic records. Climate impact studies rely on climate change information at fine spatial resolution. Towards this, the past decade has witnessed significant progress in development of downscaling models to cascade the climate information provided by General Circulation Models (GCMs) at coarse spatial resolution to the scale relevant for hydrologic studies. While a plethora of downscaling models have been applied successfully to mid‐latitude regions, a few studies are available on tropical regions where the atmosphere is known to have more complex behavior. In this paper, a support vector machine (SVM) approach is proposed for statistical downscaling to interpret climate change signals provided by GCMs over tropical regions of India. Climate variables affecting spatio‐temporal variation of precipitation at each meteorological sub‐division of India are identified. Following this, cluster analysis is applied on climate data to identify the wet and dry seasons in each year. The data pertaining to climate variables and precipitation of each meteorological sub‐division is then used to develop SVM based downscaling model for each season. Subsequently, the SVM based downscaling model is applied to future climate predictions from the second generation Coupled Global Climate Model (CGCM2) to assess the impact of climate change on hydrological inputs to the meteorological sub‐divisions. The results obtained from the SVM downscaling model are then analyzed to assess the impact of climate change on precipitation over India.

Application of composition gradient solid electrolytes in sensors and thermodynamic measurements

New compos~tiong radient solid electrolytes are developed which have application in high temperature solid state galvanic sensors and provide a new tool for thermodynamic measurements. The electrolyte consists oi a solid solution between two ionic conductors with a common mobile ion and spatial variation in composition of otber coxup nents. Incorporation of the composite electrolyte in sensors permits the use oi dissimilar gas electrodes. It is demonsuated, both experimentall y and theoretically, that the composition gradient of the relativeiy immobile species does not give rise to a diffusion potential.The emi of a cell is determined by the activity of the mobile species at the two eiectrodes. The thermodynamic properties of solid solutions can be measured using the gradient solid electrolyte. The experimental stuay is based on model systems A?(COj)x(S04)l-x (A=Na,K),where S \.aria across the electrolyte. The functionally gradient solid electrolytes used for activity measurements consist of pure carbonate at one ena and the solid solution under stuav at the other. The identical vaiues of activity, obtained h m t hree different modes of operation of the ceil. indicate unit transport number for the ddi metal ion in the graciient electrolyte. Tlle activities in the solid solutions exhibit moderate positive deviations from Raoult 's law.

On the role of cloud adjustment time scale in simulating precipitation with Relaxed Arakawa-Schubert convection scheme

The precipitation by Relaxed Arakawa-Schubert cumulus parameterization in a General Circulation Model (GCM) is sensitive to the choice of relaxation parameter or specified cloud adjustment time scale. In the present study, we examine sensitivity of simulated precipitation to the choice of cloud adjustment time scale (tau(adj)) over different parts of the tropics using National Center for Environmental Prediction (NCEP) Seasonal Forecast Model (SFM) during June-September. The results show that a single specified value of tau(adj) performs best only over a particular region and different values are preferred over different parts of the world. To find a relation between tau(adj) and cloud depth (convective activity) we choose six regions over the tropics. Based on the observed relation between outgoing long-wave radiation and tau(adj), we propose a linear cloud-type dependent relaxation parameter to be used in the model. The simulations over most parts of the tropics show improved results due to this newly formulated cloud-type dependent relaxation parameter.

Temperature gradient induced phase transitions and morphological changes in diamond thin film

The diamond films were deposited onto a wurtzite gallium nitride (GaN) thin film substrate using hot-filament chemical vapor deposition (HFCVD). During the film deposition a lateral temperature gradient was imposed across the substrate by inclining the substrate. As grown films predominantly showed the hexagonal phase, when no inclination was applied to the substrate. Tilting the substrate with respect to the heating filament by 6 degrees imposed a lateral temperature gradient across the substrate, which induced the formation of a cubic diamond phase. Diamond grains were predominantly oriented in the (100) direction. However, a further increase in the substrate tilt angle to 12 degrees, resulted in grains oriented in the (111) direction. The growth rate and hence the morphology of diamond grains varied along the inclined substrate. The present study focuses on the measurements of dominant phase formation and crystal orientation with varying substrate inclination using orientation-imaging microscopy (OIM). This technique enables direct examination of individual diamond grains and their crystallographic orientation. (C) 2012 Elsevier B.V. All rights reserved.

Effect of Grain Size on Structural and Magnetic Properties of CuFe2O4 Nanograins Synthesized by Chemical Co-Precipitation

CuFe2O4 nanograins have been prepared by the chemical co-precipitation technique and calcined in the temperature range of 200-1200 degrees C for 3 h. A wide range of grain sizes has been observed in this sintering temperature range, which has been determined to be 4 to 56 nm. Formation of ferrite has also been confirmed by FTIR measurement through the presence of wide band near 600 and 430 cm(-1) for the samples in the as-dried condition. Systematic variation of wave number has been observed with the variation of the calcination temperature. B-H loops exhibit transition from superparamagnetic to ferrimagnetic state above the calcination temperature of 900 degrees C. Coercivity of the samples at lower calcination temperature of 900 degrees C reduces significantly and tends towards zero coercivity, which is suggestive of superparamagnetic transition for the samples sintered below this temperature. Frequency spectrum of the real and imaginary part of complex initial permeability have been measured for the samples calcined at different temperature, which shows wide range of frequency stability. Curie temperature, T-c has been measured from temperature dependence initial permeability at a fixed frequency of 100 kHz. Although there is small variation of T-c with sintering temperature, the reduction of permeability with temperature drastically reduce for lower sintering temperature, which is in conformity with the change of B-H loops with the variation of sintering temperatures.

Synthesis of BiFeO by carbonate precipitation

Magnetoelectric multiferroic BiFeO3 (BFO) was synthesized by a simple carbonate precipitation technique of metal nitrate solutions. X-ray powder diffraction and thermo-gravimetric analysis (TGA) revealed that the precipitate consists of an intimate mixture of crystalline bismuth carbonate and an amorphous hydroxide of iron. The precipitate yielded BiFeO3 at an optimal calcination temperature of similar to 560A degrees C. Energy dispersive X-ray (EDX) analysis showed 1:1 ratio between Bi and Fe in the oxide. X-ray photoelectron spectroscopy (XPS) studies confirmed that Fe to be in +3 oxidation states both in the precipitated powder and BiFeO3. The synthesized BFO exhibits a very weak ferromagnetic correlation at room temperature and the degree of which increases slightly on cooling down to 10 K suggesting alteration in the long range spatial modulation of the spins arrangement as compared to the bulk BiFeO3.

Monsoon sensitivity to aerosol direct radiative forcing in the community atmosphere model

Aerosol forcing remains a dominant uncertainty in climate studies. The impact of aerosol direct radiative forcing on Indian monsoon is extremely complex and is strongly dependent on the model, aerosol distribution and characteristics specified in the model, modelling strategy employed as well as on spatial and temporal scales. The present study investigates (i) the aerosol direct radiative forcing impact on mean Indian summer monsoon when a combination of quasi-realistic mean annual cycles of scattering and absorbing aerosols derived from an aerosol transport model constrained with satellite observed Aerosol Optical Depth (AOD) is prescribed, (ii) the dominant feedback mechanism behind the simulated impact of all-aerosol direct radiative forcing on monsoon and (iii) the relative impacts of absorbing and scattering aerosols on mean Indian summer monsoon. We have used CAM3, an atmospheric GCM (AGCM) that has a comprehensive treatment of the aerosol-radiation interaction. This AGCM has been used to perform climate simulations with three different representations of aerosol direct radiative forcing due to the total, scattering aerosols and black carbon aerosols. We have also conducted experiments without any aerosol forcing. Aerosol direct impact due to scattering aerosols causes significant reduction in summer monsoon precipitation over India with a tendency for southward shift of Tropical Convergence Zones (TCZs) over the Indian region. Aerosol forcing reduces surface solar absorption over the primary rainbelt region of India and reduces the surface and lower tropospheric temperatures. Concurrent warming of the lower atmosphere over the warm oceanic region in the south reduces the land-ocean temperature contrast and weakens the monsoon overturning circulation and the advection of moisture into the landmass. This increases atmospheric convective stability, and decreases convection, clouds, precipitation and associated latent heat release. Our analysis reveals a defining negative moisture-advection feedback that acts as an internal damping mechanism spinning down the regional hydrological cycle and leading to significant circulation changes in response to external radiative forcing perturbations. When total aerosol loading (both absorbing and scattering aerosols) is prescribed, dust and black carbon aerosols are found to cause significant atmospheric heating over the monsoon region but the aerosol-induced weakening of meridional lower tropospheric temperature gradient (leading to weaker summer monsoon rainfall) more than offsets the increase in summer-time rainfall resulting from the atmospheric heating effect of absorbing aerosols, leading to a net decrease of summer monsoon rainfall. Further, we have carried out climate simulations with globally constant AODs and also with the constant AODs over the extended Indian region replaced by realistic AODs. Regional aerosol radiative forcing perturbations over the Indian region is found to have impact not only over the region of loading but over remote tropical regions as well. This warrants the need to prescribe realistic aerosol properties in strategic regions such as India in order to accurately assess the aerosol impact.

On the basin-scale detection and attribution of human-induced climate change in monsoon precipitation and streamflow

Detecting and quantifying the presence of human-induced climate change in regional hydrology is important for studying the impacts of such changes on the water resources systems as well as for reliable future projections and policy making for adaptation. In this article a formal fingerprint-based detection and attribution analysis has been attempted to study the changes in the observed monsoon precipitation and streamflow in the rain-fed Mahanadi River Basin in India, considering the variability across different climate models. This is achieved through the use of observations, several climate model runs, a principal component analysis and regression based statistical downscaling technique, and a Genetic Programming based rainfall-runoff model. It is found that the decreases in observed hydrological variables across the second half of the 20th century lie outside the range that is expected from natural internal variability of climate alone at 95% statistical confidence level, for most of the climate models considered. For several climate models, such changes are consistent with those expected from anthropogenic emissions of greenhouse gases. However, unequivocal attribution to human-induced climate change cannot be claimed across all the climate models and uncertainties in our detection procedure, arising out of various sources including the use of models, cannot be ruled out. Changes in solar irradiance and volcanic activities are considered as other plausible natural external causes of climate change. Time evolution of the anthropogenic climate change ``signal'' in the hydrological observations, above the natural internal climate variability ``noise'' shows that the detection of the signal is achieved earlier in streamflow as compared to precipitation for most of the climate models, suggesting larger impacts of human-induced climate change on streamflow than precipitation at the river basin scale.

Physically Based, Hydrologic Model Results Based on Three Precipitation Products

The main objective of the study is to examine the accuracy of and differences among simulated streamflows driven by rainfall estimates from a network of 22 rain gauges spread over a 2,170 km2 watershed, NEXRAD Stage III radar data, and Tropical Rainfall Measuring Mission (TRMM) 3B42 satellite data. The Gridded Surface Subsurface Hydrologic Analysis (GSSHA), a physically based, distributed parameter, grid-structured, hydrologic model, was used to simulate the June-2002 flooding event in the Upper Guadalupe River watershed in south central Texas. There were significant differences between the rainfall fields estimated by the three types of measurement technologies. These differences resulted in even larger differences in the simulated hydrologic response of the watershed. In general, simulations driven by radar rainfall yielded better results than those driven by satellite or rain-gauge estimates. This study also presents an overview of effects of land cover changes on runoff and stream discharge. The results demonstrate that, for major rainfall events similar to the 2002 event, the effect of urbanization on the watershed in the past two decades would not have made any significant effect on the hydrologic response. The effect of urbanization on the hydrologic response increases as the size of the rainfall event decreases.

Effect of compressibility on the velocity gradient tensor at free shear flow boundaries

Analyses of the invariants of the velocity gradient ten- sor were performed on flow fields obtained by DNS of compressible plane mixing layers at convective Mach num- bers Mc=0:15 and 1.1. Joint pdfs of the 2nd and 3rd invariants were examined at turbulent/nonturbulent (T/NT) boundaries—defined as surfaces where the local vorticity first exceeds a threshold fraction of the maximum of the mean vorticity. By increasing the threshold from very small lev-els, the boundary points were moved closer into the turbulent region, and the effects on the pdfs of the invariants were ob-served. Generally, T/NT boundaries are in sheet-like regions at both Mach numbers. At the higher Mach number a distinct lobe appears in the joint pdf isolines which has not been ob-served/reported before. A connection to the delayed entrain-ment and reduced growth rate of the higher Mach number flow is proposed.

Bilateral smoothing of gradient vector field and application to image segmentation

Medical image segmentation finds application in computer-aided diagnosis, computer-guided surgery, measuring tissue volumes, locating tumors, and pathologies. One approach to segmentation is to use active contours or snakes. Active contours start from an initialization (often manually specified) and are guided by image-dependent forces to the object boundary. Snakes may also be guided by gradient vector fields associated with an image. The first main result in this direction is that of Xu and Prince, who proposed the notion of gradient vector flow (GVF), which is computed iteratively. We propose a new formalism to compute the vector flow based on the notion of bilateral filtering of the gradient field associated with the edge map - we refer to it as the bilateral vector flow (BVF). The range kernel definition that we employ is different from the one employed in the standard Gaussian bilateral filter. The advantage of the BVF formalism is that smooth gradient vector flow fields with enhanced edge information can be computed noniteratively. The quality of image segmentation turned out to be on par with that obtained using the GVF and in some cases better than the GVF.